CorrelatedDesigns2

Summary

In this lecture by Erin Heerey, the focus is on exploring the intricacies of analyzing correlated samples using a one-way repeated measures ANOVA. This analysis is essential when dealing with study designs involving more than two measurement types. Unlike simple t-tests, this ANOVA model accounts for multiple measurements, such as reaction times across different stimuli or pre- and post-treatment evaluations. The lecture emphasizes understanding the ingredients needed for a repeated measures ANOVA, such as one factor with multiple levels and a continuous dependent variable. It delves into details of partitioning variance, examining participant effects, and enhancing signal-to-noise ratios. Through a historic study on fatigue and word cognition, Heerey illustrates the calculation and interpretation of ANOVA tables, highlighting the assumptions of normality and sphericity relevant to these designs.

Highlights

• Discover the one-way repeated measures ANOVA model for correlated samples 🎓.
• Learn how repeated measures ANOVA exceeds basic t-tests in handling multiple data points 📊.
• Understand variance partitioning to improve signal-to-noise ratio 🎺.
• An intriguing old study showcases repeated measures power in real-world settings 🎩.
• Grasp important assumptions like normality and sphericity in statistical tests 📈.

Key Takeaways

• Explore the one-way repeated measures ANOVA for correlated samples 🧠.
• Understand how to manage multiple measurement types beyond basic t-tests 🎯.
• Learn the importance of ingredients required for correlational analysis experiments 🍲.
• Dive into variance partitioning and participant effect in analyses 🌊.
• Study a historical example to see real-world application 📚.

Overview

Erin Heerey guides us through the analytical prowess of the one-way repeated measures ANOVA in dealing with correlated samples. This approach is markedly crucial when research designs involve more than mere pairwise comparisons, making it an essential tool for assessing patterns across multiple time points or conditions.

With the recurrent measure ANOVA, variance is meticulously partitioned not only to observe treatment effects but also to account for participant individuality. Imagine studying the shifts in responses to stimuli under varied conditions or assessing prolonged treatment effects over several months - this ANOVA model empowers such intricate evaluations.

By revisiting a classic study exploring fatigue’s impact on cognition, Heerey hands-on demonstrates the dissection of ANOVA tables, where variance and statistical significance are unraveled, pointing to real-world applications. Simultaneously, the session ensures a rigorous understanding of the assumptions, like normality and sphericity, that underpin these sophisticated analyses.

Chapters

• 00:00 - 01:30: Introduction to Correlated Samples and Repeated Measures ANOVA The chapter introduces the concept of correlated samples and discusses the challenges that arise when a study design includes more than two measurement types. It highlights the issue of taking difference scores and how this can inflate the type 1 error rate. The introduction suggests that there is an analysis of variance model that can effectively address and help understand this issue.
• 01:30 - 03:00: Ingredients for Correlated Samples ANOVA The chapter discusses the concept of a one-way repeated measures ANOVA, which is an extension of the t-test for examining repeated measurements across more than two instances. It is used to analyze how a single group of subjects responds to different conditions or treatments over time. An example given is comparing reaction times to three different stimulus types or evaluating performance pre-treatment and post-treatment.
• 03:00 - 06:00: Conceptual Calculation of Variance Components In this chapter, the author discusses using analysis or variance models to understand changes in individuals over time or across different conditions. Examples provided include looking at changes immediately after a treatment and following up six months later to assess the persistence of treatment gains. The chapter aims to provide insights into the conceptual calculation of variance components within this context.
• 06:00 - 15:30: Example Study: Effect of Fatigue on Word Cognition The chapter titled 'Example Study: Effect of Fatigue on Word Cognition' focuses on guiding students to understand how to analyze study descriptions and determine the appropriate analyses, particularly correlated samples analysis of variance. It prepares students for potential exam problems by emphasizing the importance of identifying components necessary for various statistical analyses.
• 15:30 - 18:00: Assumptions in ANOVA Design The chapter discusses the assumptions involved in ANOVA (Analysis of Variance) design, specifically focusing on scenarios involving factorial repeated measures ANOVA. This design can handle more than one factor, but the focus here is on cases with a single factor or independent variable, typically with three or more levels. These levels can represent various conditions or time changes. The chapter emphasizes understanding the structure when one dependent variable is measured across these levels.

CorrelatedDesigns2 Transcription

• 00:00 - 00:30 all right so we'll continue on with correlated samples and we'll ask the question okay well maybe we have a study design that involves more than two measurement types right we can't just take a difference because again we run into the same problem how do we take the different scores and how does that inflate our type 1 error rate so it turns out that we have an analysis of variance model that allows us to understand that problem
• 00:30 - 01:00 so at this is called a one-way repeated measures Anova repeated measures because the same measure is repeated more than two times so as with the t-test there's an anova model to examine repeated measurements across more than two measurements so we might for example look be looking at reaction times to three different stimulus types we might want to look at how people are doing pre-treatment and
• 01:00 - 01:30 then they do a treatment and then we might want to look at them immediately post-treatment but we might also want to follow them six months later and see if the gains that they made in the treatment are continuing at the six-month follow-up so these are both examples of where we might want to use an analysis or variance model to understand you know how people have changed over time or across conditions so the ingredients here and just as an aside I have a couple of slides over the
• 01:30 - 02:00 course of all of the lectures in this class that look like this they say ingredients um when you are thinking about what it has to do so one of the things I will put on the final exam in this class is I will give you a problem that involves you reading some descriptions of studies and figuring out what analysis they should be so make sure you know what the ingredients for each study are so the ingredients for a correlated samples analysis of variance is one factor
• 02:00 - 02:30 in this version there's there this also scales up to a factorial repeated measures Anova where you can have more than one factor but we're not going to talk about that here but we're we have one factor or independent variable with it could be two but it's usually three or more levels these might be times so you might want to change over time you might want three conditions or four conditions and we have one dependent variable that's
• 02:30 - 03:00 measured within participants so this the data are dependent um where each participant has contributed a score in each one of the conditions or time levels of your Factor the dependent variable needs to be on a continuous scale so typically we talk about interval and ratio scales there although again as with many other measurements that we do in Psychology we sometimes use we sometimes use questionnaire data that we've then
• 03:00 - 03:30 converted in some way to something that looks a bit more like an interval scale so we might take the mean of your extroversion scores or something like that and we would Analyze That so the data in these kinds of samples are one factor with two or more levels and one dependent variable that is a continuous dependent variable where each of the participants has given you multiple scores
• 03:30 - 04:00 so this is the the version of the same graph I showed you in the regular Anova only this time instead of so last time we had three sources of variance we had our total variance which is the blue we had our error variance outlined in red that was a smaller piece of the pie in the last in the last time I showed you a schematic that looked like this and then we have our treatment effect right our some squares treatment so
• 04:00 - 04:30 when we use an analysis of variance model again we're partitioning the variance in a very similar way to the way we did this before and what we're doing here is we're doing um we're using a treatment effect we're using to look at and that's the main thing that we're interested in comparing and we're going to take that treatment effect just like we did before and we're going to compare it to some error variants but we're actually adding a
• 04:30 - 05:00 partition to the variance to account for who our participants are so the particip so that's called the participant effect the sum of squares associated with the participants with individual differences between participants and what we're doing there is we're taking the participant mean and subtracting the grand mean we're squaring that and then summing it again the math on this won't work out this is not how the math works but conceptually at a very conceptual level this is what the calculation essentially is
• 05:00 - 05:30 so what we're doing is we're taking our error variance it doesn't come out of the treatment effect remember what we saw with the little you know sort of toy example on the um in the previous part of the lecture the means of the two groups never changed right so our treatment effect was the same but what we changed was our noise estimate right our estimate of error variance of sampling error so here what we've done is we've taken our a relatively good size estimate of error
• 05:30 - 06:00 variance and we've split it now because some of the scores have been contributed by the same individuals and so we can actually look at the average participant effect and we're not interested in that we're probably not going to do anything with that data because that's not theoretically relevant but what it has done is it's decreased the piece of pie associated with the error variance here so that will
• 06:00 - 06:30 enhance our signal to noise ratio by decreasing the noise element of that estimate so our Anova table is very similar to what we had before there are no differences here in the treatment group here and I've what I've done is I've put a little red box around the bits that change so we add a partition for the variance that's associated with participants so it's the sum of squares participants that's calculated sort of that is essentially the difference
• 06:30 - 07:00 between the participants the participants average and the grand average um the number of degrees of freedom associated with participants is n minus ones the number of participants minus one now that is what we that's what it looked like that's what it looked like our total was in the last type of Anova table I'm going to show you a comparison Anova table later with numbers filled in so you can kind of see what they look like um so here are sum of squares for
• 07:00 - 07:30 participants is the number of participants minus one the sum of squares for our error is the number of participants minus one times the number of levels or conditions they completed minus one so here we're using something that actually looks a lot more like the degrees of freedom in the chi-squared but ins instead of rows minus one times columns minus one it's going to be levels minus one times participants minus one and so what we're doing here
• 07:30 - 08:00 is we're accounting for individual observations now not necessarily total number of participants so we have participants times observations those two sets of degrees of freedom and that gives us our degrees of freedom for the error for the error and then our total degrees of freedom is no longer the number of participants minus one it's the num the total number of observations which is the number of levels you have the number of of levels
• 08:00 - 08:30 of your Factor times the number of participants and that gives you your total number of observations so n minus 1 is participants times levels minus one and that's your total observation or your total number of observations now the this so just like before this plus this plus this will add up to this this plus this plus this will add up to this and our error term here is going to be
• 08:30 - 09:00 the Unexplained error um the variance we can't account for based on individual differences in participants which we now know and can account for this is going to be just sort of standard noise metric um um that's independent of the treatment effect and independent of the individual differences between the participants so that's what our error term is going to be it's still called some squares error and that's mostly how you will see it
• 09:00 - 09:30 written but know that it's the Unexplained error so some squares error divided by degrees of freedom error so then the math here works out very similarly um the mean squared treatment is divided by the sums by the mean squared error and that gives us our F value so that's what an anova table looks like in this kind of a design so let's do an example this is an example from a very very old study from a German psychologist
• 09:30 - 10:00 who's looking at the effect of fatigue on word con cognition and he was looking at people at word associations and at word associations that were puns um you'll have to forgive my I have the necessary coalifications your qualifications are completely irrelevant um so you'll have to excuse the you'll have to excuse the pony Association I didn't think of it but what the psychologist did was he had five participants generate word associations
• 10:00 - 10:30 from a list of a hundred words he had them do this at 9 00 PM so before they got tired at midnight when they were starting to get tired at 3am and then again at 6 a.m when they were you know pretty sleep deprived they didn't they weren't allowed to sleep they stayed up all night and what he was counting was the number of associations that were puns so here's our table it has five participants in it back in the olden days you could do a study with five participants and that was fine um participant one generates eight honey
• 10:30 - 11:00 associations at nine pm at midnight he generates 22 and then 24 and then 34. there's a big jump there um so what you can see is that people increase in their number of puns generally speaking although not perfectly so this person here um goes from 11 to 9. but you can see that over time people's you know puns are generally speaking increasing and so what we want to do is look at that effect
• 11:00 - 11:30 so I'm going to show you two plots of the data here's the data from 9 pm from Midnight from 3 A.M and from 6 a.m and here it is with a bar plot that is what I'm plotting here are the 95 confidence intervals and what you can see from these data are that in general at 9 00 PM when people aren't very tired they're not making a lot of puns then it goes up over the course of the night and then by 6am when you haven't slept at all you're probably
• 11:30 - 12:00 sillier Than You Think You Are so what so I'm going to give you some sums of squares here that are actually calculated from these data um the sum of squares total is 1 352. some squares treatment is 840 um the sum squares for participants is 344 and the sum squares error is 168. so these are the actual numbers calculated off this table using actual formulas and not these like pretend formulas that I've just given you
• 12:00 - 12:30 um and I've also put the means in the table as well so you can see the condition mean and the participant mean and the overall grand mean so our overall error if we were going to analyze this as if the data were independent if we were sort of looking at this independently would be 344 plus 168 so our original our noise estimate if we analyze this as an independent samples design rather than a dependent design would be 344 plus 168 which would be an overall error
• 12:30 - 13:00 of 512. so here's our Anova table we have um based on the previous slide these I'm just plugging in these numbers here into our table so 840 the sum of squares for participants is 344 for error it's 168 and the total is 13.52 for degrees of freedom so we had um we had four different times right they were 9 pm midnight 3 A.M and 6 a.m
• 13:00 - 13:30 so 4 minus one is three we had five participants five minus one is four that gives us the participants degrees of freedom and then in terms of the for the error term five minus 1 times 4 minus one so four times three is twelve so we have 12 degrees of freedom in our for that's associated with our with the sampling error in terms of our of our Anova model here and our total degrees of freedom is the
• 13:30 - 14:00 total number of observations which is 5 times 4. 20 minus 1 is 19. and you can see that these numbers add up so 12 plus 4 plus 3 equals 19. we then do the math to get from these columns to this column is pretty straightforward the mean squared is going to be 840 divided by three gives us 280. um and for the error term it's 168
• 14:00 - 14:30 divided by 12 which gives us 14. so 280 divided by 14 gives us 20. so that's a really big F score a huge F value I can tell you without even looking at a table that's statistically significant um so the important thing that we're looking at and what I would expect you to know about an anova table on an exam is if I gave you these numbers and a design that you could give me these
• 14:30 - 15:00 numbers and then you could give me these ratios I would never expect you to calculate out what happens here but you should kind of know where these numbers come from just like you kind of had to know where the numbers came from when you were looking at probability as well now if we analyze this the other way what would our table look like what would our app value look like are we gaining power here well it turns out that we are so if we did this as an independent table um our treatment effect as we know would not change nor would our degrees of
• 15:00 - 15:30 freedom for the treatment that would just be the same all the way across but what would be different is our error variance would now be 512 divided by 16 as opposed to 168 divided by 12. so that gives us an error variance of 32. now again that's still going to give us a significant F ratio 280 divided by 32 equals 875 which is still significant but you can see that this that these
• 15:30 - 16:00 have dropped dramatically but you should look at what's changed and what has not changed in this table we still have even if we were to pretend that this was an independent design and and it's not but if we were to pretend it was we would still have the same number the same total sum of squares and the same total degrees of freedom as we have up here so bear in mind what is the same and what is different in these Anova tables
• 16:00 - 16:30 and finally we'll talk about the assumptions associated with this type of Anova design so first of all normality so our dependent variable should be normally distributed at each level so we're looking for um that that each of the levels the variable the response variable is normally distributed and we're looking for a new thing we call sphericity and this is called is is a test of the equality of variances for the differences between levels of the
• 16:30 - 17:00 dependent factor so basically what we're asking is is the correlation between each of the pairs of groups the same so is the variance of a minus b equal to the variance of a minus c equal to the variance of B minus C so are all of the correlations the same there is a formal statistical test that we can use to ask this question because you I promise would never want to calculate that out by hand having done that before it's nobody needs to do
• 17:00 - 17:30 that that can be formally tested with a test called Markley's test and there is a statistical package that runs in Python that will allow you to do that test but it's associated with a library that most of you won't have so we won't actually do that when we analyze the data this way in lab this week we'll just we're going to just pretend that we've met our assumption um but in general we we will we will look at it but we won't actually formally do a test so this idea is that the quality
• 17:30 - 18:00 of the variances for each of the for each of the pairs of levels is the same um so that's what you'll that's what that Sphero City assumption is what that test tests and I will leave it there for this lecture thanks for listening