AI Has a Fatal Flaw—And Nobody Can Fix It

Summary

Artificial Intelligence (AI) faces a significant barrier that cannot be overcome. Slidebean's video explains the fundamental limitations of AI models, drawing attention to a mysterious equation. Current AI systems are remarkable at tasks like word prediction, but their intelligence remains limited in certain areas such as math and real-world decision-making. The video dives into the technical aspects of AI, highlighting the challenge of achieving higher intelligence levels due to data and computational constraints. Lastly, it explores possibilities for future advancements and emphasizes the ongoing impact on various industries.

Highlights

• A $100 million equation might limit AI's intelligence forever. 💸
• Despite superiority in various aspects, AI models can't truly think or perform complex tasks. 🤯
• AI still struggles with math and real-world decisions, emphasizing its cognitive boundaries. 🧠
• Recent innovations like reasoning processes are pushing the limits of AI capabilities. 🔍
• Companies are integrating AI with other systems to extend functionalities, like recognizing speech and images. 📸

Key Takeaways

• AI has a mysterious equation that limits its intelligence capabilities. 🧩
• Current AIs excel in tasks like word prediction but fail in math and complex decision-making. 🤖
• The future might bring more efficient methods bypassing the current computational constraints. 🚀
• AI has already replaced some jobs, highlighting the shifting landscape of human skills. 🔄
• Understanding AI is crucial as it continues to influence various industries. 🏭

Overview

The intriguing world of AI faces a peculiar hurdle, a $100 million equation that might permanently cap its intelligence capabilities. The video by Slidebean unravels this concept, exploring how current AI models, though exceptional at tasks like word prediction, fall short in areas such as advanced math and real-world decision-making. Despite advancements, AI can't yet cook an egg or perform tasks requiring true understanding or creativity.

Dive into the technical core of AI, understanding how these models are trained on billions of parameters and yet face the harsh reality of diminishing returns. With limited available data and computational resources, the current trajectory of simply increasing parameters seems unsustainable. This section explores how companies are striving to innovate new ways to enhance AI's efficiency and capabilities, moving beyond mere size and scale.

Finally, Slidebean provides a snapshot of AI's impact on the job market, where it has already started replacing certain jobs. Moreover, it raises thought-provoking questions about the nature of intelligence and what the future holds for AI integration in various sectors. With the line between human and machine skills blurring, understanding these dynamics becomes ever more essential.

Chapters

• 00:00 - 01:00: Introduction to AI Limits The chapter titled 'Introduction to AI Limits' discusses a concept referred to as a $100 million line, described as potentially the most expensive math equation in history. This line represents a limit—a theoretical boundary in physics and mathematics—on how intelligent artificial intelligence can become. The text promises not to delve deeply into the scientific details but aims to explain simply why there is a ceiling to AI's potential intelligence. It notes that this limit has not yet been surpassed by top scientists and suggests that some evidence of this ceiling is already beginning to appear.
• 01:00 - 02:00: Current AI vs. AGI The chapter titled 'Current AI vs. AGI' delves into the discourse surrounding artificial intelligence and artificial general intelligence. It begins by discussing the speculative realm of an AI that surpasses human intelligence, particularly in imaginative terms. However, it emphasizes the current reality where computers outperform humans in specific tasks such as solving mathematical problems and data storage, which isn't equated to true intelligence. The narrative points out that modern AI entities have experienced challenges adjusting expectations against the backdrop of existing technological capabilities.
• 02:00 - 03:00: AI Market and Efficiency This chapter discusses the distinction between current AI technologies and Artificial General Intelligence (AGI), noting that current AI doesn't equate to true intelligence. The discussion includes a hypothetical scenario about giving AI physical abilities, like cooking, to highlight its limitations. It also covers the economic implications of AI's popularity, emphasizing Nvidia's market value driven by three key assumptions: the demand for advanced models requiring extensive GPU resources, the increasing integration of AI into everyday life, and the evolution of AI models.
• 03:00 - 04:00: Understanding GPT Models The chapter discusses recent advancements in AI models, highlighting the rapid improvement in GPT models and the competitive landscape with new players entering the scene.
• 04:00 - 05:00: GPT3's Parameters and Functioning The chapter discusses the challenges and limitations associated with advancing current AI models, particularly focusing on GPT-3. It points out that while many have faith in the transformative possibilities of AI, they often overlook the inherent difficulties in enhancing AI intelligence. The chapter emphasizes the need to understand how these models are trained and operate, hinting at the complexity and potential issues involved. Additionally, it suggests that GPT models are exceptionally good at certain tasks, which may give the appearance of intelligence.
• 05:00 - 06:00: Technicalities of Model Training This chapter discusses the capabilities and limitations of language prediction models like GPT. It is explained how these models can generate coherent text and even excel in standardized tests which measure human intelligence. However, limitations in mathematical problem solving are noted, as GPT only performs at about the 50th percentile compared to students. The discussion hints at the complexity of the metrics involved in model evaluation.
• 06:00 - 07:00: Challenges in Scaling AI The chapter 'Challenges in Scaling AI' discusses the complexity of scaling AI models, specifically focusing on the increase in the number of parameters used in models. It highlights GPT-3 as an example, which employed 175 billion parameters, and compares it to older models. The discussion aims to demystify what these parameters mean and the role of pre-trained transformers in generating text, promising clarity within a brief explanation.
• 07:00 - 08:00: Limits of Training Data The chapter, titled 'Limits of Training Data', discusses how language models process and understand text. It begins by explaining that while computers see text as bits, the model's first task is to break down the text into tokens. The process of tokenization is emphasized, and it's noted that only certain terms, possibly familiar ones, are used. After tokenization, models classify these tokens based on their meaning. Large language models (LLMs) categorize words by associating them with other words of similar meaning. However, the classification is based on tokens rather than whole words.
• 08:00 - 09:00: Achievements and Concerns in AI In this chapter, the author discusses the complexities of classifying words based on their meanings using a 2-dimensional model, which is more suited for numerical data. The concept of word vectors is briefly touched upon, highlighting the difficulty in grouping words by meaning due to the vast number of words and different contexts in which they can be used.
• 09:00 - 10:00: Future Prospects and Reasoning Models The chapter titled 'Future Prospects and Reasoning Models' delves into the complex nature of language processing as applied by GPT-3. It describes the dimensional framework that GPT-3 uses to understand language, consisting of 12,288 different dimensions. The analogy used is that of a grid, which is far beyond human comprehension, similar to a hyper-dimensional tesseract, but expanded to an even greater number of dimensions. Despite the seemingly abstract and incomprehensible nature of this model, the chapter assures that what is crucial to understand is its ability to navigate and process this multi-dimensional space, which is likened to a black hole, and suggests its potential to revolutionize understanding in AI.
• 10:00 - 11:00: AI in Real-World Scenarios This chapter explores the application of AI in real-world scenarios, focusing on how words with similar meanings are grouped in AI models. It highlights a specific example from the GPT-3 paper, where OpenAI utilized about 50,000 tokens, essentially creating a large dictionary of tokens. These tokens were mapped into a 12,000-dimension space, indicating a necessary parameter count of around 600 million for the model. This contrasts with the 175 billion parameters of GPT-3, prompting further investigation into the massive scale of GPT-3.
• 11:00 - 12:00: Conclusion and Future Questions The chapter 'Conclusion and Future Questions' thanks NordPass for its support in sponsoring the content. NordPass is highlighted as a secure password manager developed by the creators of NordVPN, designed for securely storing and sharing passwords and credit card details. It addresses the security issue where one in four people can still access accounts from previous jobs, presenting a risk due to unauthorized access. NordPass allows for quick revocation of shared passwords, giving companies visibility into access rights and making IT management simpler, as demonstrated by a migration to NordPass. The chapter underlines the importance of cybersecurity in managing company accounts and suggests potential areas for future exploration or improvement in digital security solutions.

AI Has a Fatal Flaw—And Nobody Can Fix It Transcription

• 00:00 - 00:30 take a look at this line This is a $100 million line probably the most expensive math equation in history it's a limit an imaginary wall of physics and Mathematics of how intelligent artificial intelligence can ever be so take a good look cuz I promise I'm not going to show it again in this video like I'm not pretending this is a science channel so we're we're not going to go that deep but I will explain in the simplest of words why there's a limit to how smart these models can get a limit that even the top scientists have not been able to overcome and the clues about this have already started started to show up and this is the
• 00:30 - 01:00 equation that was the equation that might put an end to all this uh bubbly Behavior around it so for years we've been imagining an artificial intelligence that outex smarts us but let's get something out of the way computers are already way better than us at plenty of things computers absolutely kick our ass at solving math and they're definitely smarter than us at storing data and reciting it back that's not really intelligence current AI companies have even had to
• 01:00 - 01:30 differentiate our current AI from AGI artificial general intelligence because they know deep down that what we have now is not really intelligence but assuming that we gave hands to a GPT could it actually cook me an egg for breakfast hold that thought the fact that AI is the buzz word of the Year and that Nvidia is worth some trillion dollar number is based on three premises one that the smarter models are going to need all of those gpus two that more people are going to adopt AI into their daily lives and three that our AI models
• 01:30 - 02:00 are going to get exponentially smarter everyone is in panic mode because some Chinese startup allegedly built and trained at chat GPT level model using a fraction of the compute cost and a fraction of the cost one Chinese startup just launched a new AI model to rival open AI deep seek has become the most downloaded free app passing chaty PT which is pretty shocking we're going to get to this it remains to be seen how much the average Joe or the average company adopts AI into their lives but
• 02:00 - 02:30 none of those betting on big Tech AI transformation even questioning the possibility of that third bullet point and therein lies the problem making our current AI models smarter is almost impossible in order to understand why that math equation is so dangerous to these companies we need to understand at least the basics of how our current models are trained and how they think so let's just go to our explainer time explainer time what GPT excels at to the point where it acts like an intelligent
• 02:30 - 03:00 sentient bot enough to fool many of us to write entire essays what it does is is predicting the next word in a sentence but being really good at predicting words or the next word already allows a model like GPT to beat us at most standardized tests which is kind of the way we measure our own human intelligence isn't it and yet GPT doesn't do that well at math it only beat about 50% of the students in these tests why is that well one number that you're going to hear about all the time when people talk about these models is
• 03:00 - 03:30 the number of parameters that a model was trained on how many parameters is a model using gp3 for example which is almost useless dumb compared to the models that we use today used 175 billion parameters what the hell does that mean so let me show you now a model like GPT uses pre-trained Transformers to generate text hence the name now this sounds like nonsense to you right now but I promise it'll make sense in exactly 4 minutes imagine that we feed the model a sentence so that it can try
• 03:30 - 04:00 and predict what the next word is so the first thing that the model needs to do is try to understand what this group of words means like we we're seeing words here but the computer is really just seeing bits out of all this thing and the first thing that the model will do is try to break this into tokens maybe you've heard the term before then what we'll try to do is classify those tokens based on their meaning llms classify words by grouping them together with words that have similar meaning technically this is done not by grouping entire words but tokens
• 04:00 - 04:30 which are fractions of a word but I'm going to stick with the concept of words just for Simplicity sake for example ring may be classified with other words like ear like Jewel maybe around the world Circle so here's a 2d simple two-dimension axis we have a horizontal axis X and a vertical axis y this is great for numbers right because we can just go up or down depending on the number but we're dealing with words here and there are thousands of words out there with thousands of different meanings it would be kind of impossible to group words by meaning into a 2d
• 04:30 - 05:00 space even in a 3D space we'd run out of directions to go to very quickly so gpt3 classifies words or tokens really into 12,288 different dimensions that means a grid with 12,288 axis we can't see it of course we can't even imagine it it's like that Interstellar Tesseract but with 11,284 more Dimensions to go but don't worry what you need to understand is that in this unimaginable Cloud this black hole
• 05:00 - 05:30 of directions words with similar meanings are going to be grouped close to each other so from the gpt3 paper we know that open AI used about 50,000 tokens basically a dictionary of tokens and in map them into this 12,000 Dimension space which already puts the count of parameters that the model is going to need at around 600 million still that's far from the 175 billion parameters that gpt3 had so let's keep digging but before I do that I want to
• 05:30 - 06:00 take a moment to thank nordpass for helping us fun today's explainer nordpass is a secure password manager created by the experts behind nordvpn to help you and your team store and share passwords and credit card details securely one in four people can still log into accounts from their previous jobs granting them access to stuff that they shouldn't have but passwords shared through Nord pass can be revoked in seconds you have full visibility into who has access to which shared Company accounts and the vulnerability of these making life a lot simpler for your it departments we migrated our old password
• 06:00 - 06:30 manager into Nord pass with a simple export import function and everybody hit the ground running in minutes it's easy to use you can sync it across devices and it has this userfriendly interface so I really can't recommend it enough npress also has this really cool feature called data breach scanner which gives you live alerts if any of your corporate data appears on the dark net so it gives you a warning advance to change your passwords before any of your accounts are breached which can of course cause financial and reputation damage we partnered with norpass to bring you 3-month free trial on npass for business
• 06:30 - 07:00 and 20% of their business plans no credit card is required you can just go to nordp pass.com slidebean use the code slidebean at signup or you can just scan this QR code you'll level up your business security you'll save a lot of money and you'll help our channel in the process okay so now let's dig into what happens after the embedding so the mapping of words into this incomprehensible tacct black hole is called embedding this is the embedding step and it's how a model turns words into something that computers can
• 07:00 - 07:30 understand understand and process but just understanding that the word ring lives in a neighborhood of other words we still don't know what it means in this context ring might be a sound might be an earring might be the one ring so how does the model know and so that's where transforming comes in what the model's going to do is well transform the word that essentially move this word in this 12,000 dimensional space this specific sentence it'll move it closer to the meaning that's based on the context around this specific word in this sentence so that that context could
• 07:30 - 08:00 be the word before the word after could be the words mentioned earlier in the conversation for example the model might notice that this R is capitalized even though it's not at the beginning of the sentence must mean something it might also look at adjectives and how they affect nouns so this transformation layer makes tiny adjustments in the region of space where this particular word lives now all of these Transformers are going to run at the same time and that's in part why gpus are so good at doing
• 08:00 - 08:30 this thing because they were built to calculate all the pixels in your screen at the same time now each Transformer in gpt3 has about 1.8 billion parameters around 600 million of those parameters are in this first attention layer which helps Focus the word in space and about 1.2 billion of those parameters are in the feed forward Network layer which is kind of like a like a zoom in on the meaning of the word but that's as far as we're going to zoom in today now gpt3 uses 96 of the Transformers for a total of almost 174 billion parameters we're
• 08:30 - 09:00 almost done now the last few parameters are on the output layer which essentially does the unembedded the inverse of the input layer it brings this word these 12,000 Dimensions into our old 2D bit world and it gives us the result of this massive operation of the model as words not as numbers the result of this massive massive mathematical journey is a list of words along with the probability of which word comes next
• 09:00 - 09:30 now the whole idea of machine learning is that we don't have to go to train each one of those 175 billion parameters to tell it what it needs to do it learns itself AKA machine learning now the first time this runs this thing is going to spit out just gibberish but during training the model adjust these parameters using algorithms to reduce these errors think of them like small knobs that slightly move to generate slightly different mathematical outcomes in the end it's like trial and error on a trillion scale
• 09:30 - 10:00 if a particular set of values helps the model make a correct prediction those values are reinforced if not they're adjusted each of these connections between one value and the other is a neuron which makes a neural network and it works not so differently from Human neurons like it may sound impossible but after billions of operations and training data this thing can actually and pretty accurately predict the next word in a sentence again this thing has consumed billions and billions of texts written by humans and has become so good at predicting words that it can pass our tests and predicting words is the llm
• 10:00 - 10:30 example but you can apply this logic of predicting the next thing at how a pixel should look to generate an image or understanding if this dress is blue or gold same basic principle now you know the reason why it failed the high school math exam a pure GPT model doesn't do math at least not directly in the simplest of terms if you ask it what's 1+ 1 it knows the answer is two because it read a million times that the answer is two and it's incredibly efficient at identifying patterns but not because it
• 10:30 - 11:00 pulled up a calculator and added 1+ one that's not bad per se but it's going to be a problem later the thing is once you have a computer that can understand these relationships within words you can give it instructions in plain English and it'll base its responses on that like this Transformer model with an instruction on top of it is the same concept that grock and Lama are using and they're all limited by the same equation now that 175 billion parameter gp3 model had problems like you could tell it was AI because it didn't write
• 11:00 - 11:30 quite like a human it couldn't count the RS in Strawberry it also had a rather small limit of context how many tokens before the current word are processed and considered for the prediction of the next word so let's just Trin it with more right open AI theorized that by scaling the amount of data and the amount of parameters the model would get a lot smarter and it did a way to measure the effectiveness of the model is with the error rate so the word predictions that are incorrect In in very simple terms it's it's more complicated than that but anyway they they projected the error rate decreasing
• 11:30 - 12:00 the bigger the model was and the more data was used for its training and so they went and did it they spent over a $100 million in training this thing leaked data from open AI says that gp4 uses 1.8 trillion parameters it has more Transformer steps potentially with more dimensions for the tokens and it took about 25,000 gpus running for over 3 months to train GPT 4 but it worked the results were way better than gpt3 so let's just keep doing that right more gpus more data more parameters well
• 12:00 - 12:30 that's when they hit a wall now that wall is this formula I said that I wouldn't show you again cuz you would think that a bigger model in this case you know the bigger the size of the model the better the performance at a at some fantastic astronomical level but open AI has kind of reached this wall of diminishing returns it's kind of like here right there's not a lot that we can do like regardless of the size we just can't get that performance up a lot even if we throw a lot more dat and create neural networks
• 12:30 - 13:00 with quadrillions of parameters the improvements are going to be marginal all the way through 2024 we had lived on this part of the chart right but GPT 5 failures seem to reveal that we've kind of arrived at this Plateau right here and that's not even the worst of it so a recent paper concluded that there is simply not enough data to train them there is a point in this curve where the amount of data needed for training is bigger than the amount of data that
• 13:00 - 13:30 exists we just haven't produced enough data that can be used for training text knowledge images speech to satisfy the needs that the models would have to reach Perfection or or a very very small error rate so in other words we have found the limit of the current machine learning algorithms the models are flawed and Humanity doesn't have the resources to train them let's be let's be real for a second like this series of tubes a series of tubes this Transformer
• 13:30 - 14:00 model is arguably one of the most important scientific breakthroughs of the century and I'm focusing on language models here but we have now built models to predict the shape of proteins which seemed an impossible task for a human if you wanted to produce an image of something that didn't exist you need creative people illustrators Photoshop artists 3D rendering and now an AI can just deduce how something looks from previous training like I don't think enough people talk about what this means for 3D artists when we spent trying to build a world a new world from scratch
• 14:00 - 14:30 and now a computer can reverse engineer that from training data and just give us the same result in a fraction of the time but it's not over though for years we thought there was no other way to reach this level of performance unless we had like two trillion parameters billions of dollars and servers and piles of training data but it looks like there is a better way a way around it based on deep seeks efficiency with apparently a fraction of the parameters and the cost we're yet to see if that's true still I think it's only a matter of time before we find a more efficient way
• 14:30 - 15:00 to do all this process easier and cheaper but even more importantly the answer may not be GPT 5 or 6 or seven nlms have proven that we can make a computer understand natural language and so companies figured The Next Step was connecting other systems to that brain and this is why GPT can now see images or recognize speech giving eyes and ears to the system hello there cutie that eventually will turn into hands but how far is it from cooking an egg or doing my dishes there's some reasoning needed behind
• 15:00 - 15:30 that so this is the whole idea of what models like 01 and more recently 03 try to do this model that we built originally just tries to spit out the next word as quickly as possible but scientists came up with this concept of reasoning Now using the same Transformer the same model at its core it tries to interpret the question and tries to break it down into smaller subtasks or prompts and then it tries to solve each one of those prompts in order kind of giving it like part answers to your original question now once that possible
• 15:30 - 16:00 response is done it analyzes it again to see if it makes sense against the original question and the original context of the conversation so it does a bit like your own brain's thought process you know writing that email response starting over readjusting rereading before you hit send nailed it like this step-by-step process is called Chain of Thought again not too different from your train of thought but let's go back to that table that I showed you earlier this iterative thinking has actually allowed current
• 16:00 - 16:30 models to beat us at General human intelligence tests IQ structure logic decision-making scenarios it's good enough to write basic and even some intermediate code it's got a fair share of Engineers struggling to find jobs which would have thought that they were the first to be replaced by this but anyway in startups at least there's there's an unspoken truth of the number of jobs that AI has already replaced but it's very bad press and nobody really wants to talk about it but when you get out of control environments and into the
• 16:30 - 17:00 real world that's where AI struggles like Common Sense reasoning creativity decision- making in real world scenarios and even advanced mathematics where problem solving and some creativity is required like these models take minutes to process through all of this and it still takes a seconds to make these Advanced decisions that's a processing and capacity problem not an architecture problem also what happens when these models are allowed to escape containment when they can start doing things in the
• 17:00 - 17:30 real world operator is a resarch preview of an agent that uses browser to uh help user to do things I'm not doing anything right now the operator is doing everything by itself it's okay Mom should help I think we we just keep pushing the bar of what intelligence is feelings right creativity true invention we still have some of those and computers don't but the set of human only skills is shrinking it's running out and I think we have to deal with the reality that it's no longer an if but a
• 17:30 - 18:00 when question do computers really think let's just say it all depends on what you mean by thinking now if you enjoyed today's explainer you should watch our video from last week on how money gets created and why 93% of today's money doesn't really exist catch you on the next one [Music]