Intuition Behind Independent and Identically Distributed (i.i.d) Random Variables

Data Formation and Random Variables

Data is often collected by sampling from a pool of measurable sets of numerical values. Each sample in this pool can be thought of as originating from a specific distribution that characterizes the properties of the dataset. In statistical terms, we represent each sample as a random variable, which is a cornerstone concept in probability theory and statistics. A random variable is a function that maps an outcome, such as the height of a person, to a real number on the number line:  X(Height)→R.  The importance of this mapping allows us to apply math to numerical terms to gain insights into these data sets. Specifically, understanding random variables often involves using calculus for operations like integration and differentiation to analyze probability density functions.

Empirical Distribution

To empirically understand the distribution of human heights, we might sample from a reasonably large pool of candidates, say 30 people. We measure their heights, round the measurements to two decimal places, and plot the frequencies of each height on a histogram. In this histogram, the x-axis represents the height, and the y-axis represents the frequency of each height bin. A 'bin' refers to a specific range of heights and in this example, we can assume that the bin width is 10 cm due to rounding.

Identically Distributed Random Variables

In our example, each person in the sample pool serves as an individual random variable, and each random variable takes a value from a known distribution—namely, the distribution of human heights. Since all the sampled heights must conform to this known distribution, we say that the random variables are identically distributed.

Independence

Independence is a cornerstone concept in probability theory. It means that the outcome of one random variable provides no information about the outcome of another. Mathematically, this is expressed as

P(X1=1.8 m and X2=1.7 m)=P(X1)×P(X2). In essence, the joint probability X1 and X2 of having specific heights is the product of their individual probabilities.

Importance of I.I.D Assumptions

The i.i.d assumption simplifies many statistical analyses. For instance, if we want to understand the joint distribution of all these random variables (i.e., a multivariate distribution), we can simply multiply their individual distributions together due to their independence and identical distribution.

Let's bring all the concepts together in the histogram as shown above:

• Random Variables: Each bar in the histogram can be considered a set of random variables with heights falling within the range represented by that bar.

• Distribution: The overall shape of the histogram represents the distribution of human heights, in this case, approximated by a normal distribution.

• Independence: The heights of individuals in each bar are independent of the heights in other bars, assuming the data is i.i.d.

• Identically Distributed: The assumption here is that all heights are sampled from the same underlying distribution, which is why the bars follow a specific shape (in this case, a bell curve).

Summary and Conclusion:

• Data Formation: Data is collected by sampling from sets of numerical values, each represented as a random variable.

• Random Variables: A random variable maps an outcome (e.g., height of a person) to a real number on the number line.

• Empirical Distribution: To understand a distribution (like human height), a large sample pool is used. Heights are measured, rounded, and then plotted on a histogram. The 'bins' on the histogram represent ranges of heights.

• Identically Distributed: In the example of human heights, each person serves as an individual random variable with values from the known distribution of human heights. Since all individuals follow this distribution, the random variables are identically distributed.

• Independence: The concept implies that the outcome of one random variable doesn't affect another. Mathematically, the joint probability of two events can be represented as the product of their individual probabilities.

• I.I.D Assumptions: The assumptions of independence and identical distribution (i.i.d) simplify statistical analyses. For instance, the joint distribution of multiple i.i.d random variables can be easily computed by multiplying their individual distributions.

In essence, the i.i.d assumptions simplify statistical analysis and have practical applications in real-world examples. The key takeaway is to understand the interconnectedness between measurable sets, random variables, distributions, and independence. Grasping these concepts paves the way for effectively utilizing the powerful simplifications that come with i.i.d random variables.