ANOVA2

Summary

The video delves into the concept of Analysis of Variance (ANOVA), emphasizing its utility in comparing more than two groups. The presenter, Erin Heerey, uses an example data set related to exam anxiety treatments to explain one-way ANOVA, including types of therapy such as exposure therapy, mindfulness meditation, and a control treatment. She discusses key statistical concepts such as factors, treatment levels, and family-wise error rate. The video also addresses the limitations of t-tests for multiple comparisons and highlights the advantage of ANOVA in controlling for type 1 errors.

Highlights

• ANOVA explores differences among multiple groups, surpassing the limitations of simple t-tests ✨.
• Example treatments for exam anxiety include exposure therapy, mindfulness meditation, and control therapy 🌿.
• Understanding factors and levels in ANOVA helps in the correct analysis of data sets with more than two groups 📈.
• Family-wise error rate increases with multiple t-tests, while ANOVA mitigates this risk effectively 🚨.
• Statistically significant Omnibus tests allow for deeper analysis through controlled post-hoc tests 🔍.

Key Takeaways

• ANOVA is used to compare more than two groups to determine differences among means 🎯.
• One-way ANOVA involves one factor with multiple levels, suitable for comparing three or more groups 📊.
• Conducting multiple t-tests increases the risk of type 1 errors, making ANOVA a better choice to control errors 🔍.
• Post-hoc tests can be used after an ANOVA to determine specific group differences, maintaining statistical integrity ✅.
• ANOVA's Omnibus test checks for any differences among group means without indicating which means differ 🎫.

Overview

In this video, Erin Heerey introduces the concept of Analysis of Variance, or ANOVA, a statistical method used when comparing the means of more than two groups. She explains how ANOVA surpasses the capabilities of simple t-tests, which are traditionally used for comparing two groups. Using an example related to treatments for exam anxiety, Heerey illustrates how one-way ANOVA can be applied in real-world situations.

Erin presents three types of treatments as an example – exposure therapy, mindfulness meditation, and a control treatment – to demonstrate how ANOVA determines whether there are any significant differences among these treatment groups. She emphasizes the importance of understanding factors and levels within an ANOVA framework, which are critical for accurate data analysis.

The video further discusses the problem of increased type 1 error when multiple t-tests are conducted, known as the family-wise error rate. By using ANOVA, researchers can avoid this problem, as it combines all effects into a single test with a controlled error rate. If the ANOVA results are significant, post-hoc tests can further explore specific group differences, maintaining the integrity of the statistical analysis.

Chapters

• 00:00 - 00:30: Introduction to ANOVA The chapter 'Introduction to ANOVA' provides an overview of the Analysis of Variance (ANOVA) technique. ANOVA is used to compare the means of more than two groups, expanding beyond the scope of a t-test which compares only two groups. The technique involves determining if differences between means of multiple groups are significant. It answers the question of whether group means are statistically similar or if at least one differs significantly from the others.
• 00:30 - 02:00: Example and Vocabulary The chapter discusses the comparison of three treatments for exam anxiety using an example dataset. The treatments include exposure therapy, where individuals are exposed to anxiety-inducing situations until the anxiety diminishes, and mindfulness meditation, which aims to reduce anxiety. The chapter aims to differentiate between the groups based on their treatment method.
• 02:00 - 03:30: Factors and Levels The chapter titled 'Factors and Levels' discusses methods to control exam anxiety, with a focus on mindfulness and control treatments. Participants are taught a guided mindfulness meditation to manage their anxiety before and during an exam. Additionally, a control treatment involves providing participants with educational texts about exam anxiety, urging them to manage it independently. The chapter sets the context by emphasizing the need to understand key vocabulary, starting with the predictor variable, before comparing the described conditions.
• 03:30 - 05:00: Random Sampling and Representation The chapter introduces the concept of an 'independent variable' in the context of analysis of variance (ANOVA), explaining that it is often referred to as a 'factor.' The chapter uses 'therapy type' as an example of a factor, which includes exposure therapy, mindfulness meditation, or a control group. Additionally, it clarifies that any categorical independent variable can be considered a factor. The chapter further explains that if an experiment involves one factor, it can be analyzed using a statistical method known as 'One-Way ANOVA.' This specific type of ANOVA is discussed further, indicating its use in the analysis presented in the chapter.
• 05:00 - 06:30: Observational Factors The chapter 'Observational Factors' discusses the basics of ANOVA (Analysis of Variance) focusing on a model with a single factor. The chapter explains that an ANOVA factor must have at least two different levels or treatment conditions to be valid. As students progress in their study of statistics, they will encounter more complex designs with multiple factors.
• 06:30 - 08:00: Qualitative vs Quantitative Factors The chapter discusses the different types of factors (qualitative vs quantitative) in the context of analyzing data. It explains that factors can have different levels, and the type of statistical test used depends on the number of levels. For instance, a factor with two levels can be analyzed using a t-test or a one-way ANOVA, while a factor with more than two levels should be analyzed using a one-way ANOVA. An example is given where a t-test is used to compare a drug to a placebo, representing a study design with one factor with two levels.
• 08:00 - 09:30: Balanced Design and Comparing Means The chapter discusses experimental design, specifically comparing outcomes of different groups such as those receiving a drug versus a placebo. It highlights that these comparisons can be analyzed using either a t-test or a one-way ANOVA, implying flexibility in choice of statistical method for such analyses. The focus is on understanding balanced design and comparing means, noting that in many cases, the choice between these statistical methods may not significantly impact the results.
• 09:30 - 12:00: Omnibus Test and Family-Wise Error Rate The chapter discusses statistical methods for analyzing designs with multiple factors and levels. It begins with t-tests used for two-level designs and introduces one-way ANOVA for more than two levels. It further touches on factorial ANOVA, which involves multiple factors with two or more levels, leading to complex interactions between factors. However, this chapter does not delve into modeling these interactions.
• 12:00 - 14:00: Analysis of Variance Solution The chapter discusses the concept of experimental design, particularly focusing on the assignment of participants. It highlights the importance of randomly sampling participants from a population and assigning them randomly to different groups. This method allows researchers to make causal statements based on their findings. However, it also notes that causal statements can still be made even if the sample is not precisely randomly drawn from the population. The emphasis is on understanding that while randomness enhances the validity of causal inferences, it is not an absolute requirement.
• 14:00 - 15:00: Conclusion and Next Steps In the concluding chapter titled 'Conclusion and Next Steps,' the discussion revolves around the complexity of achieving truly random samples. The speaker questions the possibility of obtaining such samples, particularly when involving human subjects, as opposed to inanimate objects like marbles in a jar. The chapter stresses the nuanced understanding required when interpreting the randomness of samples in research, especially in contexts involving human participants and their ability to consent.

ANOVA2 Transcription

• 00:00 - 00:30 so let's move on now to analysis of variance so we use analysis of variants when we can want to compare more than two groups so comparing the means of two groups is really very intuitive we did this in the t-test and what we did is we took the difference between those two means and we asked are they the same or are they not the same is that difference greater than zero or is it approximately zero but what happens if we have more than
• 00:30 - 01:00 two groups so we want to figure out how they differ so we're going to use an example data set in this lecture and the example that we'll use includes a comparison of three treatments for exam anxiety there's exposure therapy where people are just simply thrown in they sit with their anxiety they take an exam even though they have a lot of anxiety and eventually the anxiety sort of goes away people have compared that to mindfulness meditation where people are or participants with
• 01:00 - 01:30 exam anxiety are taught on mindfulness a guided mindfulness meditation that they do before they go into an exam and while they're waiting for an exam to start that can help them control exam anxiety and finally there's control treatment in which participants are given an educational text about exam anxiety and basically told to get a handle on it um so the question is how do we compare these conditions before we do that we need to talk about a little bit of vocabulary so let's start with a predictor variable
• 01:30 - 02:00 or an independent variable an independent variable in analysis of variance is known as a factor in today's example the therapy type whether it's exposure therapy mindfulness meditation or control the theory the therapy type is a factor and any categorical independent variable can be a factor if an experiment has one factor it can be analyzed using one-way Anova and One Way Anova is the kind of Anova that
• 02:00 - 02:30 we're going to cover today as you move on in your statistics Journey you you will cover more different kinds of Anova we can have an over with multiple factors and much more complicated designs this particular Anova that we'll cover today has one and only one factor now to run an anova model the factor must have at least two different levels or treatment conditions so a treatment condition here is a level
• 02:30 - 03:00 so we have factors and nested within factors are levels so one factor with two levels can be analyzed with a t-test or with a one-way Anova one factor with more than two levels needs to be analyzed with a one-way Anova so an example of a t-test of an of a study design where there is one factor with two levels might be if we were comparing um a drug to Placebo where we would have
• 03:00 - 03:30 some people getting a dose of a drug some people getting a dose of placebo and we would measure the outcomes that occurred in both cases that could be analyzed with either a t-test or a one-way Anova it doesn't it frankly it doesn't really matter at all um for that for that comparison however I will also tell you that most of those two factor or two level designs
• 03:30 - 04:00 one factor two level designs are analyzed with t-tests and that's typically how we've thought about the t-test now if we have more than two levels as in the example we'll use today we have to do this one-way Anova we can also have more than one factor Each of which has two or more levels and that's called factorial Anova and factorial Anova is where we get into complicated things like the interactions between factors which we will not model today so experimental factors are ones that
• 04:00 - 04:30 are assigned in the context of an experimental design so participants might be randomly sampled from a particular population they're then randomly assigned to groups and then we can make causal statements by the way we can also make causal statements even if the population or even if our sample is not randomly drawn precisely randomly drawn from a population I know we've said this many many times and it's easy to kind of think about it as a gospel but we need
• 04:30 - 05:00 to be a little bit nuanced in our interpretation of what that means because is there such a thing as a truly random sample and I'm not sure that the answer to that question is yes one can take a truly random sample of things that have no power to consent like for example marbles in a jar um but when we're dealing with people we almost never have truly random samples
• 05:00 - 05:30 so the more important factor is whether the sample is representative of the population now often if we take a convenience we can take convenience samples that are totally representative of the population I gave you an example the very first lecture of this class we talked about James Lind and his scurvy experiment with um with various treatments for scurvy and it turns out so he had a he had a very limited sample they were all men they were all Sailors on the same boat they all got scurvy
• 05:30 - 06:00 um maybe there's a difference between the men who got scurvy and the men who didn't no regardless his sample was totally representative of the population to which he was generalizing which was the population of humans because it turns out that most people's bodies work kind of that same way now there are probably individual differences in how fast your body metabolizes vitamin C how much you can store and so on and so forth but it turns out that if for any single one of us if we don't get enough vitamin C
• 06:00 - 06:30 we're probably getting scurvy and it doesn't matter who you are where you live whether you're male or female it doesn't matter any of those things so his sample was very much a convenient sample it was also representative um if we're looking at more complicated things like attitudes to a particular law or political situation or attitudes toward an out-group those become trickier because it's harder to know
• 06:30 - 07:00 what the population looks like there um and so there we need larger samples are they truly random even when we do random polling the answer is no they're really not and so in those cases we still can make causal statements as long as we have randomly assigned participants to groups so that's an important Nuance that I want you to remember because that will come up on the final exam
• 07:00 - 07:30 observational factors are factors where we select participants for example we could select participants on age we could select participants on the country that they were born in we could select them on the basis of ethnicity or gender we could select them on the basis of whether they have a particular disorder or illness or whether they are they don't and when we're talking about observational factors we can really only detect relationships between those factors we can't make causal statements
• 07:30 - 08:00 factors can be qualitative in which case they cannot be placed in a logical numerical order so if I asked you about your brand preferences for a number of different products we can't really Place brand name in a logical numerical order but we can manipulate it so I can give you items that are from a particular brand and you can you can tell me how much you would be willing to pay for those items um gender ethnicity what color car you
• 08:00 - 08:30 drive all of these things are qualitative variables qualitative factors that are very clearly sort of discrete categories but cannot be placed in a logical numerical order and then we have quantitative factors as well and these can be ordered numerically these might be things like age or what's the dosage of a drug that you've been given so these are the kinds of things that we'll be working with in
• 08:30 - 09:00 the context of analysis of variance treatment is a specific experimental condition we determine whether somebody's in a treatment group based on the factors and levels that we're looking at we often compare treatments to control groups and that can be a really effective way of showing that a treatment is effective we can also compare one particular treatment to a different treatment and that's another way of showing that a treatment is effective
• 09:00 - 09:30 a design is called balanced if each treatment is repeated the same number of times so that means it's a really fancy way of saying the same number of people per condition that gives you a balanced design so when we're thinking about t-test and anovas what we're thinking about is comparing means so Anova allows us to compare means and to consider the relationships between levels of a factor without assuming a
• 09:30 - 10:00 parametric relationship between them so if we have a relationship that we know goes like this so low medium high in terms of you know the number of anxiety symptoms post after following a full treatment of that um then we can order them and use certain kinds of Statistics that that are give us different information than what we get from Anova so if there's a likely or assumed relationship between the
• 10:00 - 10:30 categories they can be ordered and examined using other methods that we'll learn about in um in fact you'll learn about this in the next class in the statistical series if you continue on but sometimes we can't assume that kind of a relationship and in that case we often use a different approach that different approach is called analysis of variance and that doesn't require us to have any particular type of relationship so we talked in when we talked about correlation about there being roughly a linear relationship between the variables and
• 10:30 - 11:00 here with these anxieties and we're not seeing a linear relationship here in this bottom graph these are just pretends data this is a toy example here but the important thing is that analysis of variance allows us to test whether there are differences somewhere in this group of means so what does Anova test well it tests whether there's a difference somewhere within the group of means it doesn't tell us what that difference is just that it's there
• 11:00 - 11:30 that's called an Omnibus test it's an overall test that allows us to ask is what is at least one of these means different from at least one of these other means if I flip back to the previous slide here if you look at the top example this mean might actually be statistically different from this mean and likewise from this mean so all three of these means would be different from each other in this bottom part of the example here um this mean might be different from both of these other means but these two means might not be different from one
• 11:30 - 12:00 another so this is something to consider an anova can help us understand both of these types of things so we learn about this by doing a technique called partitioning the variance and so we're going to compare exposure therapy mindfulness meditation and control treatment now before we do that what's the question why would we do an Omnibus test this is important it's going to come up on an exam
• 12:00 - 12:30 so wouldn't it be more efficient just to conduct all the possible t-tests right so we could compare exposure treatment to mindfulness meditation we can do a difference test between them we get something like a t-test between them we could get a t-test between exposure therapy and control treatment and we could get a t-test between mindfulness meditation and control treatments now we've done three tests and we know exactly which means different from others we would know right away
• 12:30 - 13:00 but it turns out that we have a problem so let's say we do a t-test here looking at this top one so if we adopt a p-value of 0.05 for our comparison we have the chance of making a type 1 error is five percent now we do our second comparison and the chance of getting a type 1 error is also five percent and the same thing happens here with our third comparison our chance of a type 1 error is five percent
• 13:00 - 13:30 so when we add those up we're starting to look a little bit not so good now we have five percent chance plus a five percent chance plus a five percent chance that gives us something approximating a fifteen percent chance now I'll tell you as a spoiler alert it's not exactly 15 but it's pretty darn close we don't do the math exactly this way but regardless the the chances of making a type one error increase the more tests
• 13:30 - 14:00 we do on the same data that's known as family wise error rate fwer femi-wise error rate is the probability of making at least one false conclusion in a set or family of hypothesis tests on the same data it's also you also hear this called in the literature as Alpha inflation or cumulative type 1 error so the more tests you run on the same data the more likely you are to experience false alarms so these are type 1 errors where you say something is
• 14:00 - 14:30 statistically significant when in fact it's really not whoops so the probability of making a type 1 error is one or one hundred percent minus the quantity one minus the alpha value of each individual test raised to the power of the number of tests you're doing so the alpha level for an individual test is Alpha i t so that would be 0.05 the number of comparisons in the example we're using today is three so that formula for the present example
• 14:30 - 15:00 works out is 1 minus the quantity 1 minus 0.05 then cubed because we have three hypothesis we're doing three comparisons we have mindfulness meditation to exposure therapy mindfulness meditation to control therapy and exposure and control therapy that's three different tests so that gives us a number of comparisons of three this guy right here and then the total amount of family-wise error rate for that set of tests will be
• 15:00 - 15:30 4.143 or 14 a little over 14 percent so as I said it doesn't quite add up to 15 but it gets pretty close so and the more tests you do the more inflated this becomes and you can see how it inflates quickly because this is an exponent so it inflates at an exponential rate depending on the number of comparisons you do so that's really problematic because now all of a sudden our conclusions might be based on false alarms and and increasingly likely will be based on
• 15:30 - 16:00 false alarms so in order to solve that problem analysis of variance was invented an analysis of variance controls the family-wise error by combining all the effects in a data set into a single test and that and that single test has a family-wise error rate that is held at five percent now in practice researchers do need to know which groups differ right if I'm Crea if I'm doing a drug study for example and I want to know which drug is better at treating
• 16:00 - 16:30 whatever illness you have I might use different drugs or I might use different doses of the same drug but I need to know which one is the best I need to know what's the optimal treatment so often researchers do need to know which groups differ and what the what the Omnibus test does if the Omnibus test is statistically significant it gives you permission to do additional tests that you then use controls for for family-wise error sometimes these are called post-hoc tests where you do a larger set of you
• 16:30 - 17:00 know un non-hypothetical hypothesis motivated tasks where you're just sort of comparing the groups and looking at what's different and what's the same um and those have different types of error controls we can also do hypothesis tests that are that are very specifically hypothesized prior to the data being collected and researchers often propose those kinds of hypothesis tests as well so what we'll look at in the next video is how Anova works