Wave Walker DSP

DSP Algorithms for RF Systems

Trending

Buy the Book!

DSP for Beginners: Simple Explanations for Complex Numbers! The second edition includes a new chapter on complex sinusoids.

Central Limit Theorem: Fact or Fiction?
August 30, 2021

Table of Contents

Introduction

Where were you when you first heard about the Central Limit Theorem (CLT)?

If you are like me, you heard of the CLT in the classroom as a justification for the assumption of Gaussian noise in the RF environment. The CLT states that the summation of independent random variables (RV) approaches a Gaussian distribution with an increasing number of random variables.

The CLT is convenient in simplifying mathematics, but when and how should it be used? Under what circumstances is it valid?

Let’s take a brief look at how well three different random variables follow the CLT.

More blog posts on DSP Math:

Uniform Random Variable

Let us start with a simple case, a uniform random variable on [-1, +1]. Figure 1 shows the estimated probability density function (PDF) for a uniform RVs, the summation of two uniform RVs, and three uniform RVs. In each case the PDF is normalized to have unit variance.

With only two RVs the PDF is triangular in shape and is approaching a Gaussian distribution, and at only three RVs the CLT has taken effect and the distribution is indistinguishable from a Gaussian distribution. The continuous nature of the underlying distribution allows for much faster convergence of the CLT.

Figure 1: Uniform random variables are summed in order to approximate a Gaussian random variable.
Figure 1: Uniform random variables are summed in order to approximate a Gaussian random variable.

Discrete Random Variable

Consider a discrete random variable which approximates BPSK symbols, {-1, +1}. Figure 2 shows that many more RVs are needed to approximate a Gaussian distribution. The triangular distribution be seen at 10 RVs, an outline of a Gaussian distribution can be seen at 100 RVs, but it takes 1000 RVs for Gaussian approximation to be valid. The discrete nature of the distribution requires substantially more RVs for the CLT to apply than the continuous uniform distribution.

Figure 2: The discrete RV requires many more terms in order to converge with the Central Limit Theorem.
Figure 2: The discrete RV requires many more terms in order to converge with the Central Limit Theorem.

Filtered BPSK Symbols

Let’s take it one step further: filtered BPSK symbols with a random center frequency offset (CFO). Figure 3 shows the distribution has elements from both a continuous and discrete distributions. The discrete terms dominate the distribution for a small number of RVs but comes to approximate the Gaussian better as the number increases.

Figure 3: Filtered BPSK symbols have a distribution that has both continuous and discrete elements.
Figure 3: Filtered BPSK symbols have a distribution that has both continuous and discrete elements.

Conclusion

The CLT is a useful tool in simplifying some distributions to a Gaussian approximation but how quickly it converges depends on the underlying distributions as well as the number of random variables in the summation. Be careful to avoid applying it to a situation in which the CLT may not apply or the approximation may not as good as desired.

More blog posts on DSP Math:

Leave a Reply

God, the Lord, is my strength; He makes my feet like the deer's; He makes me tread on my high places. Habakkuk 3:19
For everything there is a season, and a time for every matter under heaven. A time to cast away stones, and a time to gather stones together. A time to embrace, and a time to refrain from embracing. Ecclesiastes 3:1,5
The earth was without form and void, and darkness was over the face of the deep. And the Spirit of God was hovering over the face of the waters. Genesis 1:2
Behold, I am toward God as you are; I too was pinched off from a piece of clay. Job 33:6
Enter His gates with thanksgiving, and His courts with praise! Give thanks to Him; bless His name! Psalm 100:4
Lift up your hands to the holy place and bless the Lord! Psalm 134:2
Blessed is the man who trusts in the Lord, whose trust is the Lord. He is like a tree planted by water, that sends out its roots by the stream, and does not fear when heat comes, for its leaves remain green, and is not anxious in the year of drought, for it does not cease to bear fruit. Jeremiah 17:7-8
He said to him, “You shall love the Lord your God with all your heart and with all your soul and with all your mind. This is the great and first commandment. And a second is like it: You shall love your neighbor as yourself. On these two commandments depend all the Law and the Prophets.” Matthew 22:37-39
Then He said to me, “Prophesy over these bones, and say to them, O dry bones, hear the word of the Lord. Thus says the Lord God to these bones: Behold, I will cause breath to enter you, and you shall live." Ezekiel 37:4-5
Riches do not profit in the day of wrath, but righteousness delivers from death. Proverbs 11:4
The angel of the Lord appeared to him in a flame of fire out of the midst of a bush. He looked, and behold, the bush was burning, yet it was not consumed. And Moses said, “I will turn aside to see this great sight, why the bush is not burned.” When the Lord saw that he turned aside to see, God called to him out of the bush, “Moses, Moses!” And he said, “Here I am.” Exodus 3:2-3
Daniel answered and said: “Blessed be the name of God forever and ever, to whom belong wisdom and might. He changes times and seasons; He removes kings and sets up kings; He gives wisdom to the wise and knowledge to those who have understanding." Daniel 2:20-21
Now the Lord is the Spirit, and where the Spirit of the Lord is, there is freedom. 2 Corinthians 3:17
Previous slide
Next slide

This website participates in the Amazon Associates program. As an Amazon Associate I earn from qualifying purchases.

© 2021-2024 Wave Walker DSP