Veritasium: What Everyone Gets Wrong About AI and Learning – Derek Muller Explains

Summary

Highlights

• Derek Muller humorously questions past 'revolutionary' education technologies. 🎥📻
• Derek explains the roles of System 1 and System 2 thinking in learning. 🧠
• He argues that true mastery requires repeated, effortful practice. 🎓
• Muller emphasizes the social importance of in-person education. 👥
• He warns against letting AI make learning too effortless, hindering skill acquisition. ⚠️

Key Takeaways

• AI isn't the education revolution many claim it to be, but it does offer valuable tools. 🤖
• Timely feedback is crucial for effective learning, a role AI can potentially fill. 🕒
• Effortful practice and developing long-term memory are key to mastering subjects. 📚
• There's no substitute for the social dynamics of in-person education. 🏫
• Beware of using AI as a shortcut, which could stifle necessary skill development. 🚧

Overview

Chapters

• 00:00 - 03:00: Introduction and AI in Education The chapter titled 'Introduction and AI in Education' begins with the speaker expressing gratitude to the audience for attending. The speaker is enthusiastic about discussing the impact of AI (Artificial Intelligence) on the field of education. The presentation seems to have a dual focus, as the speaker confesses to an initial intention of integrating AI into the talk, inspired by a strong passion for education itself. The speaker aims to explore the role AI will play in the educational landscape, indicating a keen interest in both technological advancements and educational development.
• 03:00 - 07:30: AI Tutors and Future of Learning The chapter discusses the potential and challenges of AI tutors in the educational landscape. It highlights the need to comprehend the broader educational system before implementing AI solutions. The chapter references a 10-month-old clip demonstrating an AI tutor instructing a student on identifying the sides of a triangle relative to angle alpha. This highlights the evolving role of AI in education.
• 07:30 - 15:00: Historical Expectations vs. Realities in Education This chapter discusses the common misunderstandings and expectations students have about educational concepts, specifically within mathematics. The transcript focuses on a dialogue about identifying the hypotenuse in a right triangle, showcasing how students often confuse terms and concepts such as adjacent sides and the hypotenuse. The discussion highlights the necessity of practical learning and clear explanations to bridge the gap between historical educational expectations and current educational realities.
• 15:00 - 35:00: Cognitive Load and Learning Effectiveness The chapter discusses the advancements in AI tutors and their rapid development over the years. It highlights the impressive capabilities of AI in enhancing learning effectiveness and raises questions about the future of education with AI technology. The speaker reflects on the disbelief they might have had years ago if told about the current state of AI in learning.
• 35:00 - 45:00: Systems of Thought and Learning The chapter discusses the perennial issue of dissatisfaction with educational systems, highlighting that criticism of education has been a consistent theme throughout history.
• 45:00 - 50:00: Chunking and Expertise in Education The chapter titled 'Chunking and Expertise in Education' involves discussions around elements like water (H2O) and transitions into topics concerning climate change and carbon dioxide levels in the atmosphere.
• 50:00 - 60:00: Effortful Practice and Mastery This chapter discusses the correlation between carbon dioxide emissions and climate change, emphasizing the role of various sources including trees, humans, and animals. It presents a reflective commentary on the natural contributions to carbon emissions, specifically highlighting trees as primary emitters. The narrative seems to explore the understanding of environmental impacts concerning effortful practice and mastery, possibly drawing analogies or insights into broader ecological or behavioral patterns.
• 60:00 - 73:00: AI's Role and Challenges in Education The chapter discusses the challenges of education and the questioning of why people aren't learning effectively. It mentions criticism of the education system and implies the potential role of AI in addressing these issues. The transcript begins humorously by mentioning CO2 and trees, but transitions to a more serious examination of educational concerns.
• 73:00 - 75:00: Q&A Session The Q&A Session chapter discusses the potential reasons the current educational system was designed, including suggestions that it was made to create factory workers and the lack of AI tutors at the time. There is a reflection on the outcomes we are getting from the school system and questioning if it is truly succeeding. The chapter also touches upon the idea of revolutionizing education, referencing Salon and his book 'Brave New'.

Veritasium: What Everyone Gets Wrong About AI and Learning – Derek Muller Explains Transcription

• 00:00 - 00:30 thank you thank you everyone for coming tonight it is my absolute pleasure to be here to talk about how AI will change education i must admit maybe first off that I wanted to sort of shoe shoehorn AI into this talk a little bit um because what what I'm really passionate about is education so the my question is really how is AI going to play a role in
• 00:30 - 01:00 that but first we have to understand the bigger picture of how education actually works or doesn't work in some cases you may have seen this clip uh this is an AI tutor from 10 months ago so let's take a look can you first identify which sides of the triangle are the opposite adjacent and hypotenuse relative to angle alpha all right so I'm pretty sure this is the angle alpha right here right perfect that's correct now looking at
• 01:00 - 01:30 the triangle which side do you think is the hypotenuse um remember the hypoten i'm not totally sure i think I think it might be this one but I really am not sure the side AC you're close actually side AC is called the adjacent side to the angle alpha the hypotenuse is along this side of a right triangle and is directly opposite the right angle can you find which one is the hypotenuse oh okay i see so um I think the hypotenuse is this really long
• 01:30 - 02:00 side from A to B will that be correct exactly well done okay so it's pretty impressive and again this was 10 months ago so AI tutors have only gotten better I'm sure in that time i think if you had told me 5 years ago that such a technology would exist I might not have believed you ai as we all know has gone incredibly quickly so the question is is this the future is this how everyone's going to be learning in the future well I wanted to reflect
• 02:00 - 02:30 on two things the first is that you can always complain about education you can do it now people were doing it 10 years 100 years ago you can always complain about the state of education and I can kind of see why i've got I've got some examples of like the failures of the education system for you right here what's water made of water yeah what makes water um water okay what
• 02:30 - 03:00 elements does it take to make water h2o so what does that mean that is water yes yes h2o it is water okay this one uh seems like it's maybe going in a good direction but then it takes a turn see if you can spot uh where things go wrong so what's what's happening to the level of carbon dioxide in the atmosphere right now um well I guess uh you know we've got this whole climate change and you know
• 03:00 - 03:30 um and I guess people believe that obviously carbon dioxide is a uh uh you know impacts on the on climate change you know so you know so carbon dioxide you know everything um emits carbon dioxide you know trees human beings um um I guess like um animals and so trees are contributing to the Oh definitely but it's yeah definitely so yeah so well trees are I guess the you know one of the one of the original primary sources of emitters of uh
• 03:30 - 04:00 CO2 it's the trees guys I don't know if you knew Just cut them all down we'll be okay co2 emitting trees um yeah it's not good you could always complain about education you can always say you know it's it's no good um so we can ask the question why aren't people learning um you might point to certain things maybe the education system is
• 04:00 - 04:30 stale maybe it was made to create factory workers uh maybe it's because we didn't have AI tutors yet um I I'm going to come back to this question but I think it is an important one to think about what is really going on what is the what are the outcomes we're getting from the school system and and why the other thing I wanted to point to the second thing is this word revolutionize when it comes to education um here we have a book by Salon you saw him in the first video um brave new
• 04:30 - 05:00 words how AI will revolutionize education but um I mean he's not the only one saying this uh four ways AI could revolutionize education technology or how AI is revolutionizing the world of education so clearly the expectation of a revolution is here but the truth is people expecting a revolution in education have been around for at least 100 years um we have Thomas Edison this is in 1922 the motion picture is
• 05:00 - 05:30 destined to revolutionize our educational system and in a few years it will supplant largely if not entirely the use of textbooks you remember how we got rid of textbooks right 100 years ago um he he said something like education from a textbook is 2% efficient whereas education from a motion picture is 98% efficient i have no idea where he came up with those numbers i don't think anyone does but you can imagine that in his time the
• 05:30 - 06:00 motion picture would have seemed like magic in the same way that first clip I showed you of an AI agent tutoring a kid and doing it somewhat competently um that also seems like magic so you can see why this argument for revolution is kind of convincing in the 1930s you had people saying radio would revolutionize education why because you could get rid of all the teachers you could just have like babysitters in the classroom and then you'd have the expert radio in and
• 06:00 - 06:30 they could radio in to a thousand classrooms so immediately you have this economy of scale and people see the same thing today elon Musk says AI education will be like having Einstein as a teacher for every child it's the same idea we're going to scale up the best and the brightest you know we're going to have this most intelligent entity teaching everyone instead of I don't know what we have today okay so in the 1950s it was TV academics
• 06:30 - 07:00 actually conducted studies where they would have a lecture going on in one room like this one then they'd have it videoed and closed circuit TV in the lecture room next door and they would test the students to see who learned more or less the answer no significant difference no big surprise because they're getting basically the same experience assuming the TV system uh works reasonably well then in the 1980s I think people at MIT anyway thought that they had it they thought they had
• 07:00 - 07:30 the revolution in education because the thing we had been missing was interactivity and now you could actually interact with the computer you could go back and forth with it you could program so there was this idea that if we taught kids how to program say how to program a turtle then they would get better reasoning skills overall what happened the kids got proficient at programming the turtle but those skills did not transfer to other sorts of reasoning we didn't get better
• 07:30 - 08:00 thinkers and that was the concept that was why computers and interactive computers and doing this whole turtle programming was meant to revolutionize education once more in the 1990s you get this the use of video discs in classroom instruction is increasing every year and promises to revolutionize what will happen in the classroom of tomorrow i don't know many of you here may not know what a video disc is it was a huge CD like thing like a DVD i used to use these uh when I was in high school so
• 08:00 - 08:30 you know they were there did they revolutionize education no there's one more I'll talk about um MOOs massive open online courses about 13 years ago this was all the rage the big idea that can revolutionize higher education it's the MOO will MOOs revolutionize higher education at least they're asking a question uh revolution hits the universities no it's hit uh the phenomenon of MOOs revolutionizing education in the digital age uh the
• 08:30 - 09:00 online education revol revolution drifts off course gets sad the MOO revolution may not be as disruptive as some had imagined why mukes won't revolutionize higher ed what if muks revolutionize education after all a story in 10 10 headlines anyway I hope you take my point people are all too excited all too ready to put the word revolutionize next to education this is old school new school it's the
• 09:00 - 09:30 same school what is going on we shouldn't be using this word revolutionize you keep using that word i do not think it means what you think it means you know it's a good talk if there's a Princess Bride reference okay so why didn't these revolution revolutions materialize or why didn't they eventuate you could say educational institutions they're just they have a lot of inertia they're not willing to change we're all just set in our ways um
• 09:30 - 10:00 you could say it was just technological overhype or you could say we didn't have the capabilities of AI yet this time this time is different or maybe it's something else so I'm going to explore this concept a little bit i think one hint one hint comes from this clip a question that I asked to some people uh in Los Angeles so let's take a look you
• 10:00 - 10:30 go into a toy store and there's a toy bat and a toy ball together they cost $110 and the bat costs a dollar more than the ball how much does the ball cost 10 cents we're all wrong aren't we what's the answer think about what is the answer what is the answer 5 cents yes very good now like I don't think these people are silly i don't think they're stupid i think they could work it out you know if
• 10:30 - 11:00 the ball was 10 cents and the bat was a dollar more than the ball it would be a $110 and so together they would be $120 right so the correct answer obviously you guys know it 5 cents so why do they all say 10 cents it's because that number came to their head as they listened to the question they don't know where that number came from but it was in their head and it sounded right so they blurted it out all at the same time really quite amazing right something I
• 11:00 - 11:30 love about that clip so why do they do that well the the insights are in this book which is one of my favorite books of all time thinking fast and slow by Daniel Conorman so in the book he's talking about our two systems of of thought that we've got two things two kinds of processes going on in our brains at one time the fast processes system one and slow processes system two and at the start of one of his chapters he imagines what if there was a Hollywood movie where these systems are turned into characters and so I decided
• 11:30 - 12:00 to take that inspiration and run with it so I created these uh two characters inside my head uh system one is on the left and system two is on the right now system two is kind of the person who you think you are that's the voice in your head um it's a slow effortful system you know thinking takes a takes some effort you if you ask him to multiply like 13 times 17 he's going to say "I don't want to do it." But you could force him to do
• 12:00 - 12:30 it and he can go through a series of steps so that's the thing about system two is it's slow it's methodical it can catch mistakes it can work through processes it can think about thinking now system one on the other hand is rapid fire this guy is doing things fast in the background you have no idea that he's doing it he's collecting all the information from your senses he's pulling out the relevant information and he is getting rid of everything else because you're just getting bombarded with stimuli so he's he's pulling out uh those key pieces of information that you're going to work with and he does
• 12:30 - 13:00 all of this without you the system two thinking conscious thing really being aware that he's doing it and you noticed also that he was like in a library setting so he's sort of associated with all of your long-term memory so he's got all of that and that's what allows him to work so quickly and so effectively and when when those uh people said 10 cents to me he served up the answer 10 cents and this guy system 2 was just like "Yeah 10 cents." Like I'm not going
• 13:00 - 13:30 to think about it i'm not going to check that answer he could check the answer but he doesn't want to because he's lazy and fair enough the goal for us should not be always to use system two but it should be for us to know when we need system two versus when system one will be fine so the idea is hopefully you want to optimize allow system one to handle everything that they can handle and only hand off to system two uh things that you need to now in my travels I came across these two characters and something that I thought
• 13:30 - 14:00 about later was that they almost embody system one and system two together so the question that I asked them was like um how long does it take for the earth to go around the sun okay so how long does it take for the earth to go around the sun what do you reckon is it 24 hours obviously a day is okay i asked the question to system two
• 14:00 - 14:30 he was like I don't want to like what do you think you know system one over here what do you think he was like obviously a day it sounds good yeah day sun comes up sun goes down yeah and then what did he do he just spat it out he didn't he didn't stop to think about it he was like obviously yeah it's a day what what was amazing about this conversation was this conversation moved on and I asked I had a whole set of questions that I asked them i think at this point I was asking them did dinosaurs live at the same time as humans or something silly
• 14:30 - 15:00 like that anyway anyway that's what I was doing and then I noticed a pause and I and I was like looking at his face and you could see that he was thinking you could see that system two had snapped into action so I just didn't interrupt okay watch what happens hey the Earth doesn't take one day to get around the
• 15:00 - 15:30 sun takes like a year no you see the shock on his face the whole time he was just running system one and then at the at the very end it was like whoa hang on wait a minute all right the thing about system 2 is that it is very limited so this paper came out in 55 it's a classic the magical
• 15:30 - 16:00 number seven plus or minus two the limits on our capacity for processing information so the idea here was really that you can only handle seven new things in your working memory that that system 2 can deal with at one time plus or minus two it turns out over the years that estimate of seven has been revised down to about four and there's a test you can do to kind of assess this and
• 16:00 - 16:30 figure out okay what is my span how how many pieces of information can I just work with at one time so the way this test goes is I show you some numbers that you can look at briefly and then I take them away and what you try to do is say each number on the beat there should be like a beat going i don't know how we're going to do that but uh you say each number but you add one to it so increment each digit by one after the
• 16:30 - 17:00 numbers have disappeared i think I'll do it for you so you guys don't feel weird about this but I just wanted to demonstrate how we can show that this span is is quite limited okay so hopefully I I get this right because I know I know what the numbers are okay so uh here we go there's some numbers 6 5 0 2 right okay that wasn't too hard you can make the task harder by instead of adding one you add three and
• 17:00 - 17:30 instead of having four numbers you can go to five or to six to really push yourself and see what happens what's amazing about this task what's interesting about it is that while you're doing it while you're really activating system 2 there's physiological responses that take place your heart beats a bit faster your skin gets a little sweatier and interestingly your uh pupils dilate so when you're really thinking hard when you're using that system to thinking your pupils are
• 17:30 - 18:00 actually expanding just a little bit so I got a super macro lens and some friends and I showed them uh numbers and then got them to do the the add one task and so we can try to see if you can spot the growing in the pupil of this this eye while he's doing the task here we go 4 3 97 two okay there it goes a little bit 5 4 0 8 3 I feel like in the middle when he's
• 18:00 - 18:30 like in the middle of the numbers and he's still got to remember the remaining numbers and it's like you know like it's just and then at the end it sort of starts collapsing back he's like okay now I can relax a little bit again so it's interesting to see just how effortful uh this is uh this uh measure is known as cognitive load or another way of thinking of how much mental effort you are investing in something and we can break down how the cognitive load is being expended into three
• 18:30 - 19:00 categories so there's intrinsic cognitive load which is just the amount of cognitive load you need for a particular task so in that case you know if I'm giving you six numbers it's it's going to require a large intrinsic cognitive load or sometimes when we're doing something in physics even something simple like F equals MA which to you know physicists and stuff kind of is not that hard but for a new student each of those things is new and sophisticated it's a complex concept and
• 19:00 - 19:30 so in fact even that just teaching that has a large intrinsic cognitive load there's also things that are totally extraneous these could be things like uh I don't know maybe there's someone chewing next to you or your seat's uncomfortable there's just distractions and things around you that are are taking away from your ability to focus so um extraneous cognitive load is not great uh and then finally we have germaine cognitive load which is can you actually use your attention use system 2
• 19:30 - 20:00 to do some maybe thinking about thinking like observing kind of what you what you're going through as you're thinking through a problem or noticing things that might be helpful patterns for you to use later so um germaine load is is great um if you can get it now when it comes to this is another really classic study um because one question you might have is if we have such a limited system too
• 20:00 - 20:30 how do we accomplish anything and if you go back several decades people were wondering what makes a chess master a chess master do they have really high IQ do they have great spatial reasoning do they have a really large uh working memory span which is the kind of thing that we've been looking at you know what makes them unique and these researchers got together and they did a study where they showed some different level chess players from beginner intermediate and
• 20:30 - 21:00 and grandmaster I don't know if they were grandmaster maybe just a master but someone who's very experienced in chess they showed these different people uh a chess board like this that was basically stopped in the middle of a game okay they showed them this chess board for 5 seconds they had it in front of them chessboard's there and then they covered up the chessboard and they had another blank chessboard beside it and they said to these chess players "Okay you've seen it for 5 seconds now put the pieces on the board exactly as they were in that
• 21:00 - 21:30 chess board that you just observed." When you ask noviceses to do this they've only seen it for five seconds when you ask the novice to do this they will get about four pieces correct after looking at the board just once in in further iterations of the study they were allowed to look at it for another five seconds and then put some more down and so they were sort of counting up how many it took but just on the first glance a beginner could do four pieces
• 21:30 - 22:00 the Grandmaster could do 16 pieces after one glance after just five seconds of looking at it why were the Grandmasters so good at remembering where all those pieces were on the board it's a lot it's because of a phenomenon called chunking which is where you can see things that look like disperate bits of information separate pieces of information but you can actually see them as one thing so in the case of the
• 22:00 - 22:30 example I gave you earlier if you reverse the order of these numbers you get 1945 which is no longer four numbers it's one thing the year that the Second World War ended right and so that makes it much easier to remember and this is true for basically everything we come into contact with we're not normally dealing with it in its most uh basic chunk so words you know you don't have to parse out every letter even phrases
• 22:30 - 23:00 you know they have meaning and you can think of them as one thing or equations like this one for a physicist those become one thing even the Schroinger equation yes for for a lot of people in the room I imagine that's just a single entity in working memory and you don't have to you know you could write that down you close your eyes and write write it down again so the point is that the more we get experience with things the more we practice the more we interact the more we use system 2 to uh work through problems the more we develop
• 23:00 - 23:30 this long-term memory and that long-term memory allows us to chunk the things in our world and that allows us to deal with much more complex situations so that's exactly what's happening with a chess master i have heard a lot of people say this that class really taught me how to think or that teacher taught me how to think and I
• 23:30 - 24:00 think that's more of a question than than it might seem at first i mean at first it might seem like yeah that that feels reasonable like I think a lot of us have this experience where we feel like oh yeah after that I I really learned how to think i learned how to reason but if you go back to this uh chessboard study the next thing they did as part of the study was they rearranged the pieces same number of pieces as before but now they positioned them as though they would never occur in a
• 24:00 - 24:30 regular chess match they just put the put the pieces all over the place at random you would never ever see this in uh a regular chess match and they tested again the noviceses and the masters and now the masters did no better than the novices the thing that made them so good at remembering where the pieces should go was that they had seen a lot of chess boards they'd seen a lot of configurations just like that and if the pattern wasn't there then they were no better off than
• 24:30 - 25:00 the noviceses the exact same situation my argument here is that there is no general thinking skill there is no general problem solving skill what there is are these complex webs of long-term memory this is what we build up over our lifetimes so a physicist is not necessarily going to be a good chess player and a chess player is not going to be a good physicist this applies to
• 25:00 - 25:30 all domains I would say i mean you can make arguments for specifics where you could say yes such and such skill will transfer but I think in general the idea is an expert in one field is not an expert in another and it's just because of this it's because that long-term memory that your system one is working with is really specialized to to reflect the experiences you've had and to have recognized all the patterns in those experiences to allow you to to chunk uh
• 25:30 - 26:00 new situations when they come to you it's so powerful when your system one is this really complex web that you don't even feel like you're thinking in the same way that those people said 10 cents to me if you ask Magnus Carlson what is he doing when he plays chess most of the time I know what to do i don't have to figure it out right he's saying I don't need system
• 26:00 - 26:30 two system one is so welldeveloped that it can solve any problem at a glance for him chess is a game of recognition he recognizes the board in the same way that we recognize faces or physicists recognize a physics problem right that's the way we get to be really good at things so really what we want to be doing in education is using system 2's resources very carefully and repeatedly such that we
• 26:30 - 27:00 store information in long-term memory to allow our system one to do things that effectively feel automatic so what are the implications of this for education the first one is that we should eliminate extraneous cognitive load which is fairly obvious so you should have a comfortable seat you should be able to see the board everything should be legible the sound from my microphone
• 27:00 - 27:30 should be pristine and pure i shouldn't I shouldn't have an accent if I have an accent it makes it harder for you that's extraneous just trying to think uh what was that word stuff like that having subtitles sometimes helps my wife loves to watch all TV shows with subtitles so now that's what I do too and uh yeah um so how can we limit intrinsic cognitive load this is point number two
• 27:30 - 28:00 we don't want to overfill what you can handle and my guess for you is that the reason why a lot of physics lectures fail sometimes if they do is because the professor doesn't limit intrinsic cognitive load enough they go through too much novel material in the one lesson and it's just too much for anyone to handle in terms of their working memory once you're overloaded there's basically nothing you can do your system 2 doesn't you know know how to deal with
• 28:00 - 28:30 that so one thing I would say you know when you're teaching is you have to try to obviously start where your students are and you have to keep the work kind of bite-sized because as soon as you start introducing four five six novel concepts in one lesson you're going to lose people um another way to think about limiting intrinsic cognitive load when it comes to music is get people to play songs that they already know this is why we do that I think right learning to
• 28:30 - 29:00 read music takes time and effort so the first things you're going to do with students is get them to play songs they already know then they know what the rhythm should be and so even though like yes in theory they should know how to read these notes and they should know what a quarter note is and understand the beats I can tell you as someone who played French horn for several years uh I didn't really know how to read music for at least a couple years after I started playing like the music told me what fingers to push down because I could have understood the notes but in
• 29:00 - 29:30 terms of rhythms and things like that and really being able to site readad music I think it took a couple years so I think this is one of those things where we limit intrinsic cognitive load by getting students to play songs they already know another way to limit intrinsic cognitive load is to slow things down so anyone who's practicing music for example like this harpist who's a friend of mine uh she describes her practice like this you can practice everything exactly as it is and exactly as it's written um but at just such a speed that you have to think about and
• 29:30 - 30:00 and know exactly where you are and what your fingers are doing and what it feels like so she's really talking about using system two there very slowly deliberately effort thinking through every little thing that she's doing in those moments and then once she does that enough times enough repetitions of this you get something that looks like [Music]
• 30:00 - 30:30 this yeah so good right every time I see you know superhuman performance the thing that I'm thinking about is how they have used system 2 very slowly and effort to build up that structure in long-term memory that allows them to do things that look so amazing and effort effortless it's also at this point that
• 30:30 - 31:00 I I bring up this idea that discovery learning can be dangerous you know when I was going through school I'd say the dominant paradigm in education was something called constructivism and the basic tenant of constructivism is that students are active constructors of their own knowledge something I do not disagree with it's basically what I'm saying up here that that system two is about being active and effortful in how we engage with material how we practice the
• 31:00 - 31:30 problem is I think some people didn't know how to implement teaching for this constructivist paradigm their thinking was if the students need to be active then telling won't work but that's not true i'm telling you stuff right now but I bet you're still being active and effortful in your brains which is the place where you need to be active and
• 31:30 - 32:00 effortful so that was the problem with constructivism and it led to some classrooms where people really pulled the scaffolding away from students i would argue too early so it was we want to teach problem solving here's a problem solve it you don't know how to do that figure it out construct This is just my personal pet peeve uh I have have some hang-ups about constructivism but this is this is where
• 32:00 - 32:30 I say discovery learning can be dangerous i can see flip sides of it for example if you go to a new city and you don't use a GPS it's really hard to figure out where you have to go but if you do it and you use your system two to figure out and follow every street and look at all the signs around and the you read these things and you yeah you look at really carefully at everything at the end of that you'll probably be able to retrace your steps if you use the GPS
• 32:30 - 33:00 it'll be much easier for you you're offloading processing from system 2 you're offloading it to the GPS can you find your way back no you're going to need the GPS to find your way back i think there has to be some balance in education between the kind of GPS guidance and the internal guidance ideally I would love to see like a gradual phasing out of the support to allow you know the person to figure out the directions on their own uh this has been I think well backed up in research
• 33:00 - 33:30 of things like the worked example effect and and this is even a further idea of sort of fading out the assistance as you go but the idea of like first giving someone hey here's a problem and here's how you solve it here's a a problem that is not quite at the end here's a problem partially done and here's one for you to do from start to finish right but it is this idea of like I'm going to reduce your intrinsic cognitive load i'm not just going to give you this problem at the end and say figure it out i'm going to give you this scaffolding because I know that what your working memory is is
• 33:30 - 34:00 very limited that system two has very limited resources and I'm not going to tax them with you trying to think about all sorts of things at the same time i'm going to give you this assistance so to me this is the scaffolding that helps system one move through and learn new problems i think again this is kind of a problem we have um in complex domains like physics where to the physics professor everything's perfectly clear because their system one is so fully developed but to a student it's not so
• 34:00 - 34:30 the the this is the expert novice divide the professor can't see with the student eyes what that problem looks like right only the student can see it that way okay so what else do we need to do in education we need to repeat that effortful practice until we achieve mastery why is mastery so important because when you show a skill with mastery that means it's now a system one
• 34:30 - 35:00 domain for example for all of us here in the room our basic time tables I imagine are mastered they're in system one you don't have to think about it what's so useful about that when you come to do other problems which require that information you're not going to overload system two when you do it because all that stuff is automatic so that's the key that's how we're going to get better and better performance is by building up more and more capabilities which are automatic if
• 35:00 - 35:30 you never get to the level of mastery then moving on is going to always cause you problems because you're going to be having to use extra um areas in your working memory to deal with thinking about that first before you can think about the other thing um next we have increased domain load so how do you get people to engage in really active thinking extra level active thinking one thing that was tried for example with this problem okay so so the bat and ball problem is actually
• 35:30 - 36:00 from uh a test i think it's called the cognitive reflection test and I think there's actually three questions on this test the baton ball problem is only one but this test has been given out to thousands and thousands of people around the world including uh incoming students at very elite colleges and when you give this test you typically find that about 90% of people get at least one wrong at least 90% get at least one of the three questions wrong and they're all kind of like the bat and ball problem in a certain
• 36:00 - 36:30 way so then what some researchers tried was they thought instead of printing out this test in like really clear font on nice white paper we are going to print it out in like a terrible very hardto- read font maybe photocopy it a few extra times so it gets really difficult to read and crumble up the paper or something and then then we'll we'll hand it out and what happens the error rate drops to 35% making the test harder to read made
• 36:30 - 37:00 students more likely to answer it correctly why because they didn't have that easy like okay I can read it and an answer immediately comes to mind they didn't have that experience instead what they had was oh this is challenging to read which kicked their system two into action and so it was activated and was able to find the correct answer so I'm not suggesting this is a good strategy just you know making things more confusing but I will say it can be
• 37:00 - 37:30 effective and I honestly I feel like I'm seeing advertisers apply this sort of principle sometimes in the old days of advertising like the 50s 60s feel like it was all about having a jingle you know showing what your product did why it was better than your competitors and then having this great jingle that would just get stuck in your head and you'd all remember the name of the brand and why it's better and you know and then these days I think we've got so bombarded by advertising that system one
• 37:30 - 38:00 has got great at tuning it out and so now a new strategy I see some advertisers employ is let's confuse people this was uh a billboard that I saw in Sydney near the beach and I was definitely intrigued the power of un unexplained television uncost the earth unpay more unspend more it didn't really tell you anything it's like how is this good advertising
• 38:00 - 38:30 a few days later at a bus stop I saw this unexplained with un there is no stress just unstress no hassle just unhassle with un you can undo what you did you can undrive through the car wash with a window down or unbreak dance in front of your teenage son and his mates or unflood the downstairs laundry that now doubles as the pool room un makes life relaxing and unreal un your life be happy and live for for now don't worry unw worry what is this an ad for insurance
• 38:30 - 39:00 insurance yes this is an insurance ad but how lame are insurance ads and this is pretty great so they really got me um so what role is AI going to play if you believe in this picture of education that I've painted i think the positive role that I see for AI is that it can provide timely feedback and that's essential when you are learning any
• 39:00 - 39:30 skill you're going to go through the repeated rounds of practice and you're going to need immediate feedback in order to learn when you're playing the harp if you miss a note you know immediately that's great immediate feedback if you're practicing tennis and your shots in or out you probably know you know if it hits the net or goes over you know right away so these are really helpful in terms of of training your brain there are some some different industries and some different professions where people seem like experts but they're actually not stock pickers is one
• 39:30 - 40:00 example because you may get some feedback that like maybe this was a good decision or a bad decision but honestly the stock market is at least in the short term incredibly random and so that feedback is never very informative for what you should do the next time this has also been shown with uh political pundits and some economists anyone who's looking to predict these future trends which are uh much less reliable in terms of the validity of the environment so
• 40:00 - 40:30 one thing I see AI doing is providing this timely feedback which will really help as we saw in the in the first case um with Salon and his son but the thing that I'm really worried about is how AI has this opportunity to reduce effortful practice i have four kids who are 8 six four and zero and uh and I worry about them that you know if they're going to be will they write an
• 40:30 - 41:00 essay will they write a hundred essays if there is a generative AI that can write for them what forces them to practice crafting those sentences and if they don't craft those sentences what happens to their brains the argument here is that you get good at your command of the English language you get good at being able to speak in front of people at being able to express your thoughts in writing by doing it again and again and again and
• 41:00 - 41:30 again and you should suck at the beginning and you shouldn't let that stop you and you should keep going and going and making slight tweaks and improving and getting feedback and and getting going if they never do that I really worry what gets into system one you know what is that do they have an an amazing network of connected uh knowledge that they can draw on do they have things that are automated i fear that they won't how do we force people to have to do that painful effortful
• 41:30 - 42:00 work when there's a magic machine that'll do it for you that's a big concern what about drawing you know if you can just ask it to make a picture of whatever you like the bat in the ball was AI by the way um I can't draw so but again like what will happen to people's artistic abilities so this is I think my biggest concern is if it prevents us from going through this painful effortful process which is the core process of learning using your
• 42:00 - 42:30 limited system 2 resources to engage with things and practice again and again and again even when it's hard even when it doesn't feel good even when you're not great at it that is my big concern I want to come back to these two big questions why aren't people learning and why haven't the education revolutions material materialized to the first question i want to be a little bit more generous to the people we saw at the beginning of this talk our
• 42:30 - 43:00 brains are designed to help us be effective in this world which means finding food and shelter finding a mate integrating socially so that we're not ostracized you know being able just to hang out and have fun like all of those things are what we should be doing and maybe it shouldn't be such a surprise that people don't know what elements it takes to make water or even that CO2 could pose an existential threat i know
• 43:00 - 43:30 everyone in this room will will agree like that is that's that's important we should all know that and and we should figure out how to work with that i guess what I'm saying is I think it's understandable that a lot of people don't focus on that don't know that don't think about it it's not part of the world that they exist in just because you know they're so busy with social media and Instagram and whatever because that's about connecting with other people so I want to be a little bit generous there and when it comes to this question of why haven't education
• 43:30 - 44:00 revolutions materialized with film and TV and radio and computers and mukes and now AI part of me wants to say I think we might have already found the best thing being in a room with other people other learners a teacher and some time to talk education is it's a social activity
• 44:00 - 44:30 you know people care about other people i think that the tech hype comes from a place of believing that the problem of education is not being able to get the information to the student that's not the problem it's not the problem now and it wasn't the problem aundred years ago when you have books I mean the information is all there assuming people have access the
• 44:30 - 45:00 students had access to those books and yet they're probably not going to learn very much unless they have a great teacher unless they have a group of like-minded peers to go through that with them unless they have a reason to do it an analogy for you you know the world is full of heavy objects and yet most people are not ripped do you do you see where I'm going
• 45:00 - 45:30 with this the world is full of fields not many people running on them there's plenty of ways to get exercise but obesity is a huge problem i hope the analogy I'll get there i think about teachers because I was one i kind of am one um I think
• 45:30 - 46:00 about teachers a little bit like personal trainers like the gyms are there but unless there's someone who you're going and you're meeting there and you're held accountable to and someone to say "Another one and another give me another rep keep going it's burning." Yeah keep going you know someone to to tell you the homework and someone to hold you accountable and someone to really energize you and maybe a group of other people who are doing it at the same time and we're all like going to this together that's when you see results this doesn't just happen in a
• 46:00 - 46:30 vacuum you know so I guess that's my big thought that you know teachers are are some of the greatest people in the world doing doing an incredible job of connecting with students and creating communities of learners and that's what it's about it's about that social experience it's about getting excited and holding people accountable and and forcing them to put the reps in so for me that's why none of these technologies are ever going to revolutionize
• 46:30 - 47:00 education thank you thought all right thank you Derek that was a fantastic talk um all right so now we have some time for questions so for those in the theater um who would like to ask a question for the speaker we have a microphone set up there on the
• 47:00 - 47:30 stair at the stairs so please make your way there um if you have a have a question um we particularly encourage young members of the audience who have a question to feel free to come up and ask and um just a reminder for anyone and all folks either um online or who are in the theater asking a question um please uh keep your question short and sweet to give us enough time to answer everyone's questions so all right we'll start with uh with you go ahead hi Derek thank you for coming today um you kind of touched
• 47:30 - 48:00 on it a little bit when you talked about your kids but the question that I have is what do you think about um using AI to answer your question when you're working on a on a problem as opposed to looking up for the answers yourself in a maybe more complicated way like in the books or whatever like in a which sorry in the books in the books i think that as long as the AI is
• 48:00 - 48:30 sound and a lot of it is reasonably sound at this point right then I would say I don't think there's a difference really i I think you may as well just talk to the AI assuming assuming the AI is reasonably knowledgeable and is not going to hallucinate things like that but in terms of just the is it better if it takes me longer like no I think I think that's like you it's strictly a question of efficiency and at this point I wouldn't worry about um you know being you know you can be more efficient that I I think it's fine yeah okay thank you
• 48:30 - 49:00 all right we have we have a question from online um uh all right so do you have any suggestions for how to get these insights about education um enacted into policy uh you know is that something that should come from teachers or from the government or from public um what are your thoughts my thoughts are that policy is very challenging um in Australia recently they've implemented what they call a
• 49:00 - 49:30 direct instruction policy which is kind of what I'm talking about which is that you can like telling is not wrong which was kind of taboo for a few decades i think education is a really hard field it's a difficult field to research in because there's so many variables and also because the researchers are not disinterested in the outcomes typically the researchers want to create something good that improves people's schooling but that also encourages them to P hack and do all sorts of other things and probably without even being conscious of what they're doing and it just leads to
• 49:30 - 50:00 a level of research that I feel like is not does not hold up to scrutiny um having said that I feel like there there are some strong uh lead leadings from you know the research like the things I was showing tonight that I think do point us in this direction of there are some things we clearly need to do we need to build up that long-term memory we need to have this effortful practice we need to push people beyond their comfort comfort zone and yeah so I don't know exactly where the movement comes from but there are education researchers
• 50:00 - 50:30 so hopefully it comes from there as I say it's happening in some places it's happening in Australia hopefully it'll it'll spread throughout the world yeah fantastic all right so now we have quite a line for questions so uh just a reminder to keep it to one question that way hopefully we can get through everybody so go ahead hello i'm curious because you mentioned art and I found that many people that choose to go into the arts aren't necessarily doing it for money or for a reward at the end so do you think that and I'm reluctant to use the word fear but do you think that our fear should be
• 50:30 - 51:00 that people start chasing art or the people who observe the art no longer care of the origin of where it was made i'm not entirely sure I understand your question so when you mentioned generative AI and how will people still do things when AI can do it for them people who chase art already are doing it without a necessary promise of a reward yeah yeah yeah do you think that I think that's great and and probably that's really encouraging that that people are just into making art for the
• 51:00 - 51:30 sake of art and so maybe I should be less worried about that i mean like I personally for my kids I'm just worried like are they going to learn how to write if you look at what's happened to our handwriting you know all of us because we don't really handwrite so what do you expect to happen well it's going to get a lot worse we're better typist than we are handwriters and what if we become not that great at um what if we we become not that great at at at writing at all um I think this is one thing that I didn't quite mention in the talk but this idea of it is that
• 51:30 - 52:00 vast structure that I think allows experts to have great insights i was thinking about this yesterday almost something I put in the talk is about how you see a lot of great advancements in physics coming from young physicists and part of the reason for that is I think when they build their networks the teachers who teach them have slightly different networks that followed the historical progress of the science and at some point you know
• 52:00 - 52:30 there were some deadends that didn't work out in physics and those are still in the minds of the teachers but they're not in the minds of the students because they don't get taught to the students and I think that frees up those structures to find new possibilities which is kind of why I think Einstein maybe was open to this idea of well maybe space and time are malleable but for the people that came before that their structure just wouldn't have permitted that kind of like flexing that new idea to come around anyway I think it's really
• 52:30 - 53:00 important that you have like a very well-developed structure of writing in order to be able to express yourself and express ideas and if there's a machine that does it for you maybe we lose all kinds of important insights thank you yeah I don't know that I answered your question but I'm glad that people will still be making art um hi Derek huge fan uh so we you kind of talked a lot about system one and system two and how system one is quick and you know it doesn't make much effort to think and I have observed that with
• 53:00 - 53:30 AI coming over we are using system one quite more often and like keeping system two more on rest and I feel like system 2 is designed to be more curious and um it takes the process slowly and understands the problem to solve it with system two being on rest and curio it not being able to be much curious and I think like curiosity and learning goes hand in hand um how do you feel about
• 53:30 - 54:00 um the learning evolving or let's say my question is like if you had AI at the age of 20 um do you think veritassium would exist these are good questions if AI was around when I was 20 would veritassium exist I don't know that is a crazy counterfactual um I don't know hopefully we're still
• 54:00 - 54:30 all encouraged to follow our our the things that really intrigue us and and and engage our interests um I think as a first gentleman who talked about you know can I use AI as this you know to look up or I need to look it up in a book i mean like it doesn't matter so in some ways I think AI provides a speed up that way um and even these days like I haven't really explored that area very much in terms of the things I create but I'm intrigued to think about what could
• 54:30 - 55:00 I create in future using AI um and maybe it's possible that I can create some things that are really incredible and powerful um I don't know i feel like AI just it's tainted a little bit because there's so much like AI slop out there but I I try to imagine a world in which you have like really high quality AI and maybe it's maybe it's amazing so I don't know is it's a short answer to your question but I will experiment and maybe you'll see some veritas that sort of like has has AI in it and you can tell me whether
• 55:00 - 55:30 you think it's it's better or worse i look forward to that but all right so everyone who's in line for questions stay there but I think like we'll get through this line and then we'll uh we'll move to the atrium so go ahead um hi i was wondering how you'd like solve the problem of scaling because like you mentioned the best way for teaching would be like establishing a personal connection right like that's easy to do when I'm teaching like 10 people but if I scale that up to 100 it's harder for me to build a personal connection with 100 people and like you know having them
• 55:30 - 56:00 be accountable and you know be encouraging to like each one of them individually so like how would you like approach that problem and I think that's somewhere like where AI can be helpful yeah or how do you scale a personal trainer how do you scale a plumber how do you scale an electrician you don't you just have lots of them and I think that's the answer i think that's the solution okay i just think people are always going to complain about education people are always going to be like they don't know anything whatever i
• 56:00 - 56:30 mean it's I I feel like it it might be close to the optimal you know what I mean and that's why it's so hard i am more optimistic about us disrupting say healthcare in the US than I am about us disrupting education it just it's so hard i don't know right i guess that just comes down to like resource management I guess because can't have enough teachers i mean we do have enough teachers in a way and and we and the goal should always be to get more of them and to
• 56:30 - 57:00 make them better and and I think like that's that's a system problem but like we can do that i I think that it's probably the way to go okay thank you yeah uh hi Derek um earlier you touched on the way that people learn like an effective way that people are in the classroom where it's like you have a framework and you slowly take away um like line by line of the equation uh kind of within this context how do you think or where do you think uh the place of AI would be like within this uh this
• 57:00 - 57:30 way of learning if at all yeah it's a great a great question i think you know AI can offer lots of scaffolding you can like say "Hey can I get a hint?" You know so there's there's a remarkable capability for it to sort of fill all kinds of gaps in learning or like you could say to the AI "Hey I need to learn about you know this period of Canadian history ask me 50 questions about this." Like what a great tool to be able to have if you can use it
• 57:30 - 58:00 effectively right um so all of those ways that you can use it as an educational aid I think will be great i think the place where it won't be great is where it allows you to do the work without doing the work and it's that work which is essential for learning so like that's my big concern yeah thank you hello um you know just just going to say you know the the unadvertisement I still think it's one of the most effective ones it's I've never forgotten it yeah but my question was it's like
• 58:00 - 58:30 with the onset of like chat bots and things it's become like so easy to get like assignments like just done without work right but it's like I I've personally never used them because it's like you know I feel like it's against my pride to like you know but I mean other people like you know in postsecary right it's like people just it's like they're like oh yeah no I I use like replet and it's like well I mean but it's like do you think it would be like better to just like outright like remove it from like the early stages of
• 58:30 - 59:00 education and for only to come back for like when you're like older when you've like developed your your your your effectiveness at work and stuff i absolutely think there's going to have to be some of that i think it's a little bit like you know when I was at school we had TI 82 calculators maybe TI83s um and we were allowed to use them on a portion of the exam and then there was a portion of the exam where we weren't and I think it's the same thing i think when it comes to writing there'll probably be some writing where you can have an AI
• 59:00 - 59:30 aid and some where it has to be no AI and I like I I totally think that's essential especially at the lower levels of of schooling is like we're gonna have to have writing sessions for hours in class just to get people that experience of of going through this uh challenging process so yeah yeah all right thank you hi Derek thank you for being here um I just wanted to ask to touch on the previous questions i think that you were going to make veritasium either way to be honest and I
• 59:30 - 60:00 fear that many of the people in this room know how to break their brains and get dopamine out of it so how do you think we can effectively get people excited about breaking their brains on their own because I think that's the key thing if I can share that what do you mean breaking their brains well I mean I feel like everybody here might understand like when you don't understand something it's exciting a lot of people when they don't understand something it's not exciting so how do you think we change that that is hard i feel like that's like
• 60:00 - 60:30 that's the existential question and this comes back to like me asking random people on the street you know what elements does it take to make water what's happening to CO2 in the in the atmosphere right now it is not hard for me to find people who don't know the answers to those questions like most people don't know the answers to like most of these simple questions and so like I say I don't know that you can ever take someone who is just not interested and make them interested it's kind of like people are going to be into what they're into and and this is where
• 60:30 - 61:00 I just have like compassion for like we're kind of not as a species we're not evolved to seek out theoretical physics and yet a small tiny minority of us do but then we shouldn't look around at everyone else and be like "What are you guys doing?" Like "Look how awesome this is." you know like I get it i get it for this room but like I don't get it for the public you know so I don't know that that's my thought like maybe there isn't a way tough question yeah it is it's a tough one thank you
• 61:00 - 61:30 sorry sorry she wanted me to go first so I'll do one first um so you've migrated from like short form video to like longer form and so my question is have like is there a sweet spot where you get content overload you know how you're saying like there's only so much content you can pack into like a lesson for example so I teach so I'm just curious like is there like a minute or like a content like I love how deep you are able to go to stuff i was just curious if there's like do you have to think about content overload when you're doing videos i mean maybe maybe sometimes we
• 61:30 - 62:00 do overload people with stuff honestly like YouTube has algorithmically pushed us i don't know if it's apparent or not but YouTube has algorithmically pushed us toward longer videos so in the early days the videos were 3 minutes 2 three minutes and now they're 30 and why um because if we make a good 30-minute video it gets shown a lot more and that's just the name of the game um so I know that sometimes as we approach sort of 15 minutes into a video 20 minutes into video and we'll start putting integrals and derivatives and stuff into the video and I'll recognize that like
• 62:00 - 62:30 at this point we may be losing some people and maybe this is too much um but then it's there for the people who want to keep watching and I feel like if you've watched 20 minutes why not 25 you know let's keep let's keep going um so I don't know that this is also like kind of a business question for us which is that like what do you put in the first minute that's pretty essential you don't want to you know start with crazy math what what do you put in the first five minutes you know if the writer that I'm working with comes to me and says "Yeah I want to go through this whole derivation in minute 25." I'm like
• 62:30 - 63:00 "Let's do it." You know by that point the only people around are still the true believers for another thing that's great about videos if if someone is feeling like they don't get it they can always go back re-watch they could pause it they could go take out a textbook they could write some things down so hopefully the videos are are still you know can be effective for for a certain population but I do recognize that we may lose people um sometimes and later on thank you what's your favorite video you've ever made in your life
• 63:00 - 63:30 it's such a hard question i do feel like every video is like one of my children like it's very very hard to love some more than others um you know the the one that's done the best for us is on goodles incompleteness theorem so I wonder if I would say that one okay um al also there's a shade balls one on the reservoir have you seen the shade balls one yeah I was going to say the radioactivity one is my go-to which one radioactivity like the most radioactive
• 63:30 - 64:00 places on earth oldie but goody yeah an oldie but a goodie there's one that I really like which is about um it's only like four minutes long but it's about the most common cognitive bias out there and I just feel like it does such a good job of hitting on what science is all about without really talking about it in a sort of boring methodological way like it's a really concrete way to think about it so I was excited by that video yeah thank you thank you thank you
• 64:00 - 64:30 um so if you find that in-person education succeeds because people are forced to engage with the content they're forced to put a conscious effort in why do you find that online edutainment succeeds when people can check out so easily yeah so I kind of think maybe they're doing different things like if I was your teacher or lecturer then there would be stuff to slog through as well as stuff to enjoy i think being an online educator I get to
• 64:30 - 65:00 just pick some highlights and make some exciting stuff that hopefully energizes people shows them what's possible i feel like my job is sort of a combination of educator but also uh magician uh also you know comedian or something not that I'm particularly funny but you know uh I Yeah I think it's all those things um so I I think there's a place for that and then there's a separate place for like now you have to sit down and and practice and that can be you know more
• 65:00 - 65:30 tedious um so yeah hopefully one leads to the other at least in some cases yeah so do you find that people tend to like self- select for edutainment 100% yeah i mean one of the reasons we could do what we do is because we aggregate from across the world if I was ever on a just a broadcast channel somewhere like you know the CBC here or ABC in Australia like I could never go into the depth as as soon as I would put down an a derivative on screen or something they'd be like no no no no no you cannot no
• 65:30 - 66:00 don't do that let's do an analogy to like skiing or something you know you just lose all the rigor so it's the thing I'm so grateful for is the internet came along at a time when I wanted to do that and there was people across the world who were hungry for it um so I just feel so incredibly lucky to be able to to do this and go into depth and yeah reach a lot of people with it thank you thank you hi um my question is if you like start talking to AI or like if AI was in a
• 66:00 - 66:30 human body like talk likes a human acts like a human and you like become friends with it would it count as social interacting i think that is an amazing question amazing question [Music] let me say this that one of the best ways to learn that's been shown has been if you have like a one-on-one tutor there's what's called the two sigma
• 66:30 - 67:00 effect that the performance of those students is two standard deviations higher than everyone else so it's a huge effect and if we could replicate that effect by having something that is so natural and so human seeming uh that it would just feel like you're there with another human that might be really really really powerful i wonder I feel like this is such a big question like how much do we need to know that it's a real human there or how much could a fake human you
• 67:00 - 67:30 know actually make us feel in the exact same way and I think that's what it comes down to that's a such an amazing question thank you thanks all right that's a tough question to follow thanks uh great talk um I'll start here uh so I definitely agree that revolutionized isn't the right word uh but I do think that all of those technologies have
• 67:30 - 68:00 changed education in some way right right so like for me personally I know that the accessibility that was gained by you know so much content being on the internet has completely changed like my educational journey um so my question is if revolutionized isn't the right word what would you say is yeah i feel like a lot of technology just becomes a tool you know I think of them as tools it's I just think the contrast is so weird when you think
• 68:00 - 68:30 about how has the internet changed our lives it's really profoundly changed our lives and like it's touched on education but and and yes like independent learning journeys and so on yes the internet's huge for that but in terms of like what has it done in the classroom what's it done to you know pedagogy curriculum all that sort of stuff taking people up i don't think it's done that much and I just think it's so remarkable when I think about the way all these technologies really have revolutionized things like smartphones right again
• 68:30 - 69:00 totally changed our lives now we're all on social media and we're all depressed but like you know it's it's done that it's had a huge impact um so yeah I just come back to like what what happens in education well it just seems like these things become tools in the hands of educators they're tools to be used and exploited and and so on so that I mean that's the word that comes to mind for me yeah thank you of course stand here stand up stand up here there
• 69:00 - 69:30 we go there we go um if we didn't completely remove AI from education what tools do you think we would add and what tools do you think we would remove if we didn't completely remove if we did or didn't didn't if we don't completely remove AI from education what kind of tools are we going to add and what are we going to take away i don't know if I fully grasp it maybe
• 69:30 - 70:00 what maybe what parts of AI would you keep in the classroom yeah yeah i mean for me the the big role of AI is that kind of like um drill and practice type stuff really getting people to do the reps giving immediate feedback with whatever you know someone's working on i think there can be some powerful use cases yeah the real risky use case is the one where people use it to do their work for them which is I think the one where it's being used most unfortunately so that's that's the one that's concerning yeah that's the one we got to get rid of thank you good question
• 70:00 - 70:30 all right last question from the audience i'm I'm very short so you can keep it like this um thank you for your wonderful talk um my question was hinted at prior and I'd just like to make it a little bit more precise so I'm a grader for this quantum mechanics course and I graded my students midterms recently and the performance was extremely poor it was abysmal and this in no way was
• 70:30 - 71:00 indicated by their performance on assignments and of course I think it's very reasonable to conjecture that this is due to the advent of chat GBT as students can just copy the solutions from chat GPT right whereas on the midterm they don't have that access and see it i think it's very reasonable to argue oh perhaps we should change the evaluation methods for classes to be more exam heavy less assignment heavy but at the same time you can argue that exams aren't the best methods of
• 71:00 - 71:30 evaluation either due to performance anxiety and all these other things so I'm wondering what do you think is a like a preferred rubric or evaluation method for courses nowadays in the with the advent of AI yeah I mean the first thing that came to mind is maybe the way that assignments are done now shouldn't be the way they're done in future like maybe assignments need to be done in the same way that midterms are done to you know remove that that risk because I I
• 71:30 - 72:00 really worry that you know that is how you build that long-term me that huge structure that's what we're after and if we're not building that then yeah like by the time you get to a midterm or the final like it's too late like the the practice has to come in the lesson i can tell you that when I was lecturing first year physics at UTS University of Technology in Sydney I had like stacks of uh cards at the front of my lecture that said in huge letters A and B on one and C and D on the other and I would
• 72:00 - 72:30 force the students to come and get these cards at the beginning of every lecture it was a 400 seat lecture theater and then as I would go through I would ask a question maybe once a minute or once every two minutes and they'd have to hold up these cards um and and that's really helpful for me as a lecturer to make sure that uh they are where I think they are it's also really helpful for them in that it keeps system 2 engaged it keeps them like having to make that decision every few minutes which stops
• 72:30 - 73:00 them for from for example falling asleep which deactivates both system one and two uh but it's also also really easy to tune out and then you know that's also not helping so the whole point of getting this back and forth even in a large lecture theater just with these cards I mean was my attempt at making this interaction and getting people to be thinking all all the time being being really mentally active so I'd say we have to do the same thing with with the
• 73:00 - 73:30 with the this work that they're doing with the assignments like it has to be done um we get we got to find some way to make an AI isolation booth or something i I will can I just say one more thing so I think though when you get into higher education especially in physics the more the higher up you are the longer it takes to complete a question for instance on a three-hour exam there are probably only like three questions because it takes an hour to do each and so reconciling that in a university
• 73:30 - 74:00 setting is much more difficult because you can't just complete a question within the span of a few minutes and then give an answer right so could you comment on that no I mean it's it's a really good point i I just don't see any way of getting away from the idea of there's going to have to be assessments at some point and could be the midterms and the finals and if people are failing those because they're not doing the assignments properly it's just something that people have got to come to terms with and I know that sucks from like a structural and logistic perspective but like there's no other way if you're not learning the
• 74:00 - 74:30 material you're not gradu like just the way it is i you know I I thought that I was an amazing lecturer for my time uh and I also thought that I gave very fair and very reasonable questions and on the midterms my students were scoring 40% so on average I feel that pain yeah it's excruciating so thank you so much all right well that's a good place to leave it thank you so much everyone here and online let's thank Derek again thank you
• 74:30 - 75:00 [Music]