The Expected Value and Variance of Discrete Random Variables

Summary

This video delves into the concepts of expectation and variance for discrete random variables. It explains the probability distribution of a discrete variable and how to calculate its expected value (mean) and variance using specific formulas. The expected value is described as the theoretical mean, not always a practically possible value, while the variance measures how spread out the values are around the mean. Through a detailed example involving a biased coin, the video demonstrates how these concepts apply in practice and discusses the law of large numbers in the context of simulations.

Highlights

• Learn the difference between expected value and sample mean - they're not always the same! 🎯
• Discover how variance tells the story of data spread around the mean. 🔍
• See how theoretical formulas translate to practical applications with a fun coin example. 🪙
• Appreciate the nifty math trick that simplifies variance calculations. 🧙‍♂️
• Take a peek at how large scale simulations bridge the gap between theory and reality. 🌉

Key Takeaways

• Expected value is the theoretical mean of a random variable, not always a probable outcome! 🎲
• Variance measures how much the values of a random variable vary from the mean. 📉
• Understanding the relationship between expected value and variance can simplify calculations. 🧮
• The law of large numbers shows that with a large sample size, the average approaches the expected value. 📊
• Simulations can demonstrate how theoretical principles hold up in practice. 🎮

Overview

The video by jbstatistics unpacks the often misunderstood concepts of expectation and variance for discrete random variables. It starts with the basics, explaining the components of a probability distribution and how they relate to the notion of a random variable's expected value. While the term 'expected' might suggest it as a common outcome, it's actually a representation of the theoretical mean of all possible outcomes, calculated by the weighted average of values.

Continuing with variance, the video highlights its role in quantifying the variability of a random variable's outcomes. This is calculated using the expectation of deviations from the mean squared, a process illustrated through the formula X minus μ squared times its probability. An alternative, mathematical relationship is also shared, showing how it simplifies variance computations by focusing on the expectation of the square of X.

Through the engaging example of flipping a biased coin, viewers can see how these statistical concepts play out in real-world scenarios. It illustrates the expected value, variance, and the effects of the law of large numbers. Using a simulation of a million coin flips, the video reveals how simulated averages nearly align with expected theoretical values, bringing abstract mathematical concepts into tangible reality.

Chapters

• 00:00 - 01:30: Introduction to Expectation and Variance This chapter introduces the concepts of expectation and variance related to discrete random variables. It starts with an example of a discrete probability distribution for a random variable X. Explanation is provided that a discrete probability distribution includes all potential values of X and their associated probabilities. The random variable is denoted by capital X, while lowercase x represents the values it can assume.
• 01:30 - 03:00: Expected Value and its Calculation This chapter discusses the concept of expected value in the context of probability and distribution of random variables. It introduces the notion of expected value or expectation, explaining it as the theoretical mean of a random variable's distribution. The chapter clarifies that this theoretical mean is derived from the probabilities of the variable's possible values, not from sample data.
• 03:00 - 04:30: Expectation of Functions The chapter titled 'Expectation of Functions' explains the concept of the expectation of a random variable X, also known as the mean of the random variable or the mean of the probability distribution. It highlights that the expectation is represented by the Greek letter mu and cautions that the terms 'expected value' and 'expectation' can be misleading, as they suggest most likely value, which is often not the case. The chapter emphasizes that the expected value is often not even a possible value of the random variable.
• 04:30 - 08:00: Variance and Its Calculation The chapter titled 'Variance and Its Calculation' focuses on understanding and calculating the expected value of random variables. It distinguishes between the approach used for discrete random variables and continuous random variables, highlighting that the methods for each differ. The transcript provided discusses the calculation of expected value specifically for discrete random variables, mentioning the formula which involves multiplying each possible value with its probability. The need for adjustments when dealing with continuous random variables is also noted, indicating a more complex approach for those cases.
• 08:00 - 11:00: Example with Novelty Coin The chapter delves into the concept of expected value for a random variable, X, explaining it as a weighted average of X's values, with weights being the probabilities of X occurring. It further expands on calculating expectations of functions of X, exemplified by functions like X cubed or the square root of X.
• 11:00 - 14:30: Simulation and Law of Large Numbers The chapter introduces the concept of calculating the expectation of a function G of X, explaining it as a weighted average where each value is multiplied by the probability of its occurrence. This concept extends to calculating the variance of X, often denoted as Sigma squared.
• 14:30 - 16:30: Expectation of Squared Values In this chapter, the expectation of squared values is discussed, specifically focusing on the variance of a random variable X. The variance is defined as the expectation of the square of the distance of X from its mean (mu), providing a measure of variability in X. For a discrete random variable, the formula used involves taking the square of X minus mu, multiplying it by the probability of X, and summing this over all possible values of X. This provides a conceptual understanding of variance calculation.
• 16:30 - 18:00: Calculating Variance Step-by-Step The chapter 'Calculating Variance Step-by-Step' explains a useful relationship in mathematics for calculating variance. It explores the equation where the expectation of (X - mu)^2 is equivalent to the expectation of X^2 minus the square of the expectation of X. It further explains that since the expected value of X and mu are the same, the equation can also be expressed as the expectation of X^2 minus mu. This relationship simplifies the calculation of variance.
• 18:00 - 19:30: Summary and Conclusion The chapter "Summary and Conclusion" discusses the significance and applications of a certain mathematical relationship, emphasizing its utility in calculations and mathematical proofs. Although the reasoning behind this relationship is not immediately clear, it can be demonstrated mathematically. The chapter concludes by setting up a practical example involving the purchase of a novelty coin to calculate some quantities.

The Expected Value and Variance of Discrete Random Variables Transcription

• 00:00 - 00:30 let's investigate the expectation and variance of discrete random variables here's an example of a probability distribution of a discrete random variable X a discrete probability distribution is a listing of all possible values of the random variable X and their probabilities of occurring capital X represents the random variable X and lowercase X represents values the random variable X can take on here this particular random variable takes on the
• 00:30 - 01:00 values 0 1 & 2 and those values have these probabilities of occurring we might want to know some characteristics of the distribution of X such as its mean and variance the expected value or expectation of a random variable is the theoretical mean of the random variable it is not based on sample data it is the theoretical mean of a distribution the notation for the expected value of a
• 01:00 - 01:30 random variable X is e of X it's simply the mean of the random variable or equivalently the mean of the random variables probability distribution and we also represent the mean with the Greek letter mu in some ways it's a bit unfortunate that we use the terms expected value and expectation as they might be a bit misleading the expectation is a mean it does not mean the most likely value and in fact a lot of the time the expected value is not even a possible value of the random
• 01:30 - 02:00 variable it is simply the theoretical mean of the random variable how we calculate the expected value differs between discrete and continuous random variables for the rest of this video I'll be working with the expected value of discrete random variables some adjustments will need to be made when we discuss continuous random variables to calculate the expected value of a discrete random variable X we use this formula we multiply each possible value
• 02:00 - 02:30 of x by its probability of occurring and add that up over all X this means that the expected value of a random variable X is a weighted average of the values of X we weight them by their probabilities of occurring we can also calculate the expectation of a function of X using a similar approach here G of X represents a function of X for example G of X might be X cubed or the square root of x to
• 02:30 - 03:00 calculate the expectation of a function G of X we multiply the value of that function by the probability that value occurs and add that up over all possible values of X here again that's a weighted average of the values of the function weighted by their probability of occurring we'll use that notion to calculate the variance of X we often use Sigma squared to represent the variance
• 03:00 - 03:30 of a random variable the variance of a random variable X is defined to be the expectation of X minus mu squared this can be thought of as the expectation of the squared distance of X from its mean and that's a measure of how much variability there is in X and to calculate that for a discrete random variable we use this formula we take X minus mu squared multiply it by its probability of occurring and add that up over all X conceptually that's what the
• 03:30 - 04:00 variance is but it turns out there is a very useful relationship that can often help to make the calculations a little bit easier in practice mathematically it can be shown that the expectation of X minus mu squared is equal to the expectation of the square of X minus the square of the expectation of X since the expected value of X and mu are the same thing we could also write this as the expectation of the square of X minus mu
• 04:00 - 04:30 squared this relationship comes in very handy in some calculations and it's also very useful in some mathematical proofs and derivations it's not obvious just by looking at it why this relationship holds it's not too tough to show mathematically but I'll save that for another time let's calculate these quantities for an example suppose you bought a novelty coin that
• 04:30 - 05:00 has a probability of 0.6 of coming up heads when flipped let X represent the number of heads when this coin is tossed twice here's the probability distribution of X it's the same one I showed at the start of this video if you don't know where this comes from I work this out in my introduction to discrete random variables video suppose we want to calculate the expectation of the random variable X in other words how many heads will we get on average it doesn't make sense to take the ordinary
• 05:00 - 05:30 average of 0 1 & 2 since that wouldn't be fair to the values that have a greater probability of occurring if the values were all equally likely then the expectation formula simplifies to the ordinary average of these three values but here 0 1 & 2 are not equally likely and we need to use the expectation formula here's the formula for the expectation of X and I'm also going to plot the probability distribution with the three values of X on the x axis and
• 05:30 - 06:00 their probabilities of occurring on the y axis to calculate the expectation we multiply each value by its probability of occurring and add up all those terms here that's 0 times 0.16 plus 1 times point 4 8 plus 2 times 0.36 and if we carry out those calculations that works out to 1.2 so the expected value of the random variable X is 1.2 on
• 06:00 - 06:30 average X will equal 1.2 let's see where that expected value falls on the plot of the distribution here's the probability distribution of X the expected value of the random variable X is 1.2 which falls right here on this distribution if 0 1 & 2 were all equally likely then the expected value would be 1 but here 2 is a little more likely than 0 which moves the expectation a little bit higher now
• 06:30 - 07:00 I'm going to carry out a quick simulation to illustrate a point I'm going to simulate 1 million values from this distribution 1 million values where the probability of getting a zero is 0.16 the probability of getting a 1 is 0.4 8 and the probability of getting a 2 is 0.36 here's a relative frequency histogram of 1 million simulated values the observed proportions of zeroes ones and twos are very close to the probabilities as we'd
• 07:00 - 07:30 expect after simulating a million values and if we calculate the average of those million simulated values we get 1 point 1 9 9 8 6 9 which is very close to the expected value of 1.2 the law of large numbers tells us that if we were to sample more and more values from this distribution their average would converge to the expected value as the number of observations increases here we can see that after a million
• 07:30 - 08:00 observations it's getting pretty darn close now let's work out the expectation of a function of X suppose we wish to find the expectation of the square of X to find that we square each x-value multiply that by its probability of occurring and add that up over all X here that's zero squared times 0.16 plus 1 squared times 0.4 8 plus 2 squared
• 08:00 - 08:30 times 0.36 and if we carry out those calculations we'd see that that's equal to 1 point 9 2 on average the square of X will equal one point nine two now let's work out the variance of X the variance of X is defined to be the expectation of X minus mu squared let's take a quick look at a plot to get a feel for what that means here's the probability distribution of the random
• 08:30 - 09:00 variable X and I'm drawing in a line at one point two representing the expected value of x which is just mu the variance is the expectation of the squared distance from the mean visually the variance is a weighted average of the square of the distances represented by these three white arrows let's calculate it here's the probability distribution again and the expectation of X minus mu squared is the sum over all X of X minus
• 09:00 - 09:30 PU squared times P of X and we've previously calculated mu to be one point two so this expectation is zero minus one point 2 squared times its probability of occurring plus one minus one point 2 squared times its probability of occurring plus two minus one point 2 squared times its
• 09:30 - 10:00 probability of occurring and if we carry out those calculations that works out to zero point four eight so Sigma squared the variance of the random variable X is zero point four eight and if we want the standard deviation of the random variable X which will represent by Sigma that would simply be the square root of that the square root of zero point four eight here's the probability
• 10:00 - 10:30 distribution and a recap of the three expectations we just calculated let's verify this relationship the expectation of the square of X is one point nine two and if we subtract the square of the expectation we subtract one point two squared we get zero point four eight which is indeed the expectation of X minus mu squared that we just found so this relationship holds here of
• 10:30 - 11:00 course it does since it's true in general this term is the definition of the variance the variance of a random variable X is defined to be the expectation of X minus its mean squared and this expression is a little more intuitive when we're trying to explain what the variance actually means but it's usually easier to carry out the calculations this way we've often already calculated the expectation of X so we just need to calculate the expectation of the square of X and the
• 11:00 - 11:30 expectation of the square of X is usually a little easier to calculate than the expectation of X minus you squared and that's a brief introduction to expectation and variance of discrete random variables