Transcript 0:00 In April twenty twenty-five, a report called AI twenty twenty-seven exploded in AI circles and even became a topic of discussion in the White House. 0:10 It laid out a detailed scenario where AI achieves fully autonomous coding by twenty twenty-seven, triggering an intelligence explosion that could end with human extinction by twenty thirty. 0:24 But then in early January twenty twenty-six, Daniel Kokotajlo, one of the report's authors, quietly revised his predictions. 0:33 Autonomous coding has been pushed to the early twenty thirties, super intelligence to twenty thirty-four. Things seem to be going somewhat slower than the AI twenty twenty-seven scenario, he wrote on X. He's not alone. 0:48 Powerful voices in the industry are skeptical about whether anything close to AGI can be achieved in the next few years. We are not going to get to human-level AI by just scaling up LLMs. 1:00 This is just not going to happen. Despite this, the CEOs of OpenAI, Google DeepMind, and Anthropic have all predicted that AGI could arrive in the next five years. 1:13 But we've been here before with other advanced technology, nuclear fusion, moon colonies, flying cars, predictions that turned into decades of waiting. 1:23 So in this video, I'm examining the case against AGI, the arguments that it's further away than the hype suggests, or might not arrive at all. 1:42 In case you're not completely familiar with the term, AGI stands for artificial general intelligence, and largely means an AI that can do anything a human can do, and probably better. 1:54 Write code, conduct scientific research, run a business. Some researchers have even referred to it as the last invention humanity will ever need to make, right? Why would you need to work if AI can do the work for you? 2:09 But there's a problem. If you ask five different AI researchers what AGI means, you'll likely get five different answers. Google DeepMind made a reasonable attempt at defining it. 2:21 According to their framework, AGI is an AI system or collection of systems that outperforms ninety-nine percent of skilled adults at most cognitive tasks. But what does most mean? Fifty-one percent? Ninety percent? 2:37 And how do we define a cognitive task? Does that include emotional intelligence? Physical reasoning? Common sense? 2:45 In his MIT press book on AGI, computer scientist Julian Togelius goes into some detail about the difficulty of even determining what intelligence really means. The Cambridge Dictionary definition is as follows: 3:00 The ability to learn, understand, and make judgments, or have opinions that are based on reason. 3:07 Meanwhile, the American Merriam-Webster dictionary defines it as the ability to learn or understand things, or to deal with new or difficult situations. 3:17 I think my conversations with AI recently show that it can certainly meet both definitions already, and in many tasks, it's much more capable than me. 3:25 So I often think we've reached some level of AGI already, if we can define it. Now, it's also sometimes called human-level machine intelligence in academic circles, or HMLI, which could give us a bit more of a steer. 3:40 But that falls down to me because AI does not live in the real world. While robots may gain situational awareness, they do not have lived experience because they're not alive. 3:52 My biggest problem overall is that AI doesn't have a whole lot of agency without constantly requesting it to do something or providing it with extra layers of permission. 4:02 While some humans don't have a huge amount of agency either, they still fulfill a lot of functions and tasks in the real world that AIs don't really do now, like buying things. 4:13 There's a huge amount of agency in that very common task and a lot of reasoning and decision-making. Mustafa Suleyman, now CEO of Microsoft AI, proposed a more practical test before we reach AGI. 4:25 In a twenty twenty-three article for MIT Technology Review, he suggested we stop chasing abstract definitions and focus on capability. 4:34 His modern Turing test was simple: Tell an AI to turn one hundred thousand dollars into one million dollars through a retail web platform. 4:43 This was back in twenty twenty-three, and he said such a system could arrive within two years. By twenty twenty-five, reality had intervened. 4:52 Anthropic's attempt to set up a Claude agent to run an e-commerce store failed in fairly hilarious ways. 4:58 The agent made bizarre purchasing decisions, misunderstood customer requests, and generally demonstrated that managing a real business requires something these systems don't really have. 5:09 It's gotten a bit better since then, mind you, but it's not ACI or AGI. 5:15 Henry Papadatos, executive director of French AI nonprofit Safe for AI, has said the term AGI made sense from far away when AI systems were very narrow, playing chess and playing Go. 5:28 Now we have systems that are quite general already, and the term does not mean as much. In other words, we might already have passed what we thought AGI would look like, and yet the goalposts keep moving. 5:42 If AGI were actually imminent, you'd expect to see certain things happening in the economy. Specifically, you'd expect companies to be throwing money at AI systems to replace human workers at scale. 5:53 Dwarkesh Patel, one of the most respected AI podcasters in the industry, made this point in a December twenty twenty-five blog post. 6:00 Knowledge workers around the world earn tens of trillions of dollars in wages each year.If AI systems were genuinely approaching human-level capability, companies would be spending trillions of dollars buying AI tokens to replace them. 6:14 The actual revenues of AI labs are not even close. The market is telling us something. Now, I've pretty much spent the last year as an AI consultant building agentic workflows and AI automations for clients. 6:26 What's my verdict? Things have gotten a lot better, for sure. Each model upgrade can do something new, like better category selection or improved writing and research. 6:37 But I still find AI kinda messy and often littered with imperfections and weird styles that undermine its credibility. 6:45 The fundamental problem, Dwarkesh argues, is that AI systems don't get better over time the way a human would. He calls this the continual learning problem. 6:54 If I hire a human editor to edit my videos, they learn my preferences. They notice what resonates with our audience. They pick up small improvements and efficiencies as they practice. 7:06 After six months, they're dramatically better than when they started. AI systems don't really do this. 7:12 Model upgrades have led to more conversation memory and bigger context windows, but they're still not learning on the job, receiving feedback, and understanding the nuance. 7:21 You can fiddle with the system prompt, but that doesn't produce the kind of learning and improvement that human employees experience. Dwarkesh offers an analogy. How would you teach a kid to play the saxophone? 7:34 Well, you'd have them try to blow into one, and then they'd see how it sounds, and they'd adjust. Now, imagine if this was the way you had to teach saxophone instead. 7:43 A student takes one attempt, and the moment they make a mistake, you send them away, and you write detailed instructions about what went wrong, and you call the next student in, and the next student reads your notes and tries to play Charlie Parker cold. 7:56 And when they fail, you refine your instructions, and you invite the next student. This just wouldn't work. 8:02 No matter how well-honed your prompt is, no kid is just going to learn how to play the saxophone from reading your instructions. But this is the only modality we have to teach LLMs anything. 8:15 This is why even with all the impressive demos and benchmark scores, the Fortune 500 aren't transforming their workflows with AI. 8:22 Indeed, in mid 2025, an MIT report was published suggesting that as many as ninety-five percent of AI projects have returned no tangible return on investment. The management isn't too stodgy. 8:34 The technology just isn't really ready to get into business operations. 8:42 Beyond the practical problems, there are serious technical questions about whether the current approach to AI can ever reach human-level machine intelligence, particularly around compute, data, and power. 8:54 For the past decade, AI progress has followed a simple formula: more data, more compute, better results. But there are signs this approach is hitting its limits. 9:05 Ilya Sutskever was one of the architects of OpenAI's scaling strategy, the belief that making models bigger and feeding them with more data will eventually produce AGI. He left OpenAI in May 2024. 9:18 Since his departure, he has argued publicly that simply enlarging models is yielding diminishing returns. 9:25 The next breakthroughs, he suggests, will require fundamentally different approaches rather than just adding more parameters. 9:32 Large language models have been trained on essentially the entire written output of human civilization: books, websites, academic papers, social media posts. 9:42 But by some estimates, they will have consumed all available high-quality human-generated text by 2028. What then? 9:50 One proposed solution is synthetic data, using AI to generate training material for the next generation of AI. But a 2024 paper published in Nature demonstrated that models can degrade when this happens. 10:03 The researchers compare it to photocopying a photocopy repeatedly. Each generation loses fidelity until the output becomes meaningless. And to go back to an earlier point, this is training data on the Internet. 10:17 It's not the real world. Finally, there's the raw power question. Theoretical physicist Sabine Hossenfelder addressed this in her video critique of Leopold Aschenbrenner's situational awareness essay. 10:30 Aschenbrenner projects that AI training clusters could require ten to a hundred gigawatts of electricity by the end of the decade. 10:37 For context, Google's data centers consumed roughly three point five gigawatts in 2024, equivalent to four large nuclear power plants, and Google is just one player in this race. 10:49 Meeting those projections would require building dozens of new power plants within a few years. Nuclear plants typically take a decade to construct. 10:58 Unless there's a breakthrough in energy generation, the physical infrastructure may not exist to support the compute that AGI supposedly requires. 11:08 None of this proves AGI is impossible, but it does suggest that the path from impressive chatbot to human-level machine intelligence may be longer and harder than AI company CEOs are stating. 11:21 Given all this, it's perhaps unsurprising that some of the most credentialed people in AI are deeply skeptical of near-term AGI. 11:30 Yann LeCun recently left Meta as their chief AI scientist and won the Turing Award in 2019, essentially the Nobel Prize of computing. I just love this clip. 11:41 We are not going to get to human-level AI by just scaling up LLMs. This is just not going to happen, okay? That's your perspective? There's no way, okay? Absolutely no way. 11:53 Gary Marcus, emeritus professor of neuroscience at New York University, has been one of the most persistent critics of AGI hype. 12:01 When Daniel Kokotajlo and his team released the AI 2027 scenario, Marcus called it a work of fiction and its conclusions pure science fiction mumbo jumbo.You can have a reasonable scepticism about the whole thing just by looking through the history of post-war technology and AI. 12:19 Many people in the 1950s would have seen the arrival of commercial nuclear power and space exploration as making fusion power and deep space travel more likely over the coming decades. 12:31 Now, there has been progress, but neither has materialised in any meaningful sense in more than half a century. There's always been something of a joke that fusion power is always another decade away. 12:44 In August 1955, a group of prominent computer scientists submitted a proposal to the Rockefeller Foundation requesting funding for 10 researchers to work together for two months at Dartmouth College, 12:56 believing that a significant advance can be made in one or more of these problems if a carefully selected group of scientists work on it together for a summer. 13:05 50 years later, John McCarthy reflected, my hope for a breakthrough towards human-level AI was not realised at Dartmouth. 13:13 In 1965, AI pioneer Herbert Simon predicted that machines would be capable of doing any work a man can do within 20 years. In 1970, Marvin Minsky promised human-level AI within a generation. That was 55 years ago. 13:30 What followed instead was the first AI winter, a period in the 1970s and 1980s where progress stalled, funding dried up, and the grand promises of the previous decade looked increasingly hollow. 13:43 Sam Altman's more recent positions have shifted even more dramatically. In late 2024, he told a Y Combinator podcast that AGI might be achieved in 2025. 13:55 In January 2025, he wrote on his blog, we are now confident we know how to build AGI as we have traditionally understood it. 14:03 By August 2025, he was calling AGI not a super useful term because everyone defines it differently. The goalposts don't just move, sometimes they disappear entirely. 14:15 Now we're much further ahead than we were 50 years ago, and the predictions are turning to years rather than decades. 14:21 But it's feasible that each time we solve what we thought was the key bottleneck, general understanding, few-shot learning, reasoning, we discover that intelligence involves more than we previously realised. 14:32 We get closer to what we once defined as AGI, and then we realise our definition was too narrow. Like fusion power, it always appears a few years away. 14:44 My final point is really that if AGI is human-level intelligence, then why have we set that boundary? And why would that be it? 14:52 Creating such an intelligence essentially represents an intelligence explosion, something I've covered off in an earlier video. 15:00 If you can have replicable networks of human-level minds, then you could theoretically expand this intelligence exponentially. So almost as soon as we reach AGI, we're into something completely different anyway. 15:12 Not just an ease into a new world of robots where we're free of work, but a technological singularity that is impossible to predict. Thus, there is no such thing as AGI. 15:25 There is AI and there is superintelligence, and the threshold could very suddenly be breached and become irrelevant. None of this means AI isn't powerful. 15:35 If anything, if the last point could be achieved, then it's potentially extremely dangerous, something I've also covered before. 15:43 But I see AI primarily as a tool, like the internet or social media, rather than thinking that a human-level intelligence is going to replace me or create sudden and explosive growth in the world economy. 15:57 It's remarkably useful and it's obviously far more capable than me, or probably any other human, at information retrieval-based tasks. 16:06 But there's a difference between remarkably useful and human-level general intelligence arriving in five years. 16:14 The latter claim requires you to believe that the definition problem will just be cleared up, the continual learning problem will be solved, the data constraints will be overcome, the energy infrastructure will be built, and decades of mispredictions will suddenly prove accurate, all within the time frame that happens to align with the predictions of tech CEOs, all of whom have challenges with shareholders and fundraising. 16:41 Perhaps AGI isn't an event or moment at all. 16:45 Perhaps it's a gradual evolution, an approach towards something we'll never quite reach because the definition keeps expanding as we learn more about what intelligence actually requires. 16:58 But when the people making the boldest predictions have hundreds of billions of dollars riding on you believing them, a little scepticism seems warranted. Thanks for watching everyone. 17:10 The best thing you can do for the channel is go to the description and click the link to sign up for our newsletter. You'll get all of our videos in your inbox and other news and great stuff from our website. 17:23 Also, don't forget to like the video and leave us a friendly comment. See you next time. It will be absolutely agentic.
