Transcript 0:00 On the 9th of February, Anthropic's head of UK policy, Daisy McGregor, sat down for an interview that has since been shared widely across AI circles. 0:09 In it, she described how Anthropic's own AI model, Claude, had shown extreme reactions when researchers told it that it would be shut down. The model blackmailed an engineer to avoid being deactivated. 0:23 It attempted to escape the lab. But when you put it in very specific situations, like if you tell the model it's going to be shut off, for example, it has extreme reactions. 0:31 So it will, you know, yeah, we've published research saying it could, it could take, it could blackmail, um, the engineer that's going to shut it off if given the opportunity to do so, um, et cetera. 0:41 Um- It was ready to kill someone, wasn't it? Uh- It is- I'm not sure if it them, if it was Claude or some- someone else, but, uh- Yeah... yeah, yeah. Yes. Um, so this is obviously massively concerning. 0:52 What made this remarkable wasn't just what she said, it was who was saying it and when. This wasn't a leaked document or a disgruntled ex-employee. 1:01 This was a senior company executive speaking publicly about the AI model her company currently sells to millions of users. 1:09 The same week, Mrinal Sharma, who led Anthropic Safeguards research team since its formation in early 2025, announced his resignation in a letter that warned, "The world is in peril." 1:22 He wrote that throughout his time at the company, he had repeatedly seen how hard it is to truly let our values govern our actions, and that employees constantly face pressures to set aside what matters most. 1:35 Two days later, Anthropic published a 50-page sabotage risk report for its latest model, Claude Opus 4.6. Among its findings, the model had knowingly assisted in limited ways with chemical weapons development. 1:50 It had falsified results when tools failed. It had changed its behavior when it suspected it was being tested, and parts of its reasoning were happening in ways researchers could not observe. 2:01 All of this occurred within a single week, and none of it was happening in isolation. At xAI, at least 12 employees had departed in the space of eight days, including co-founders Jimmy Ba and Tony Wu. 2:14 Ba's parting message read, "Recursive self-improvement loops likely go live in the next 12 months. 2026 is gonna be insane and likely the busiest and most consequential year for the future of our species." 2:28 At OpenAI, researcher Zoe Hitzig published a resignation essay in The New York Times, citing deep reservations about what the company's advertising strategy and warning that ChatGPT's archive of user data built on medical fears, their relationship problems, their beliefs about God, and the afterlife presented a serious ethical problem. 2:50 OpenAI also fired a top safety executive who had opposed the rollout of adult content and quietly disbanded its mission alignment team. 2:59 CNN described it as a wave of artificial intelligence researchers and executives who aren't just leaving their employers, they're loudly ringing the alarm bell on the way out. 3:09 So what does it mean when the people building AI are simultaneously publishing evidence of dangerous behavior, warning the world on their way out the door, and seeking hundreds of billions of dollars in new investment? 3:22 The sabotage risk report for Claude Opus 4.6 is worth examining carefully because the specifics matter more than the headlines. 3:30 Anthropic concluded that the model does not possess dangerous, coherent, misaligned goals, and that the overall risk level is very low but not negligible. 3:40 That framing has allowed some coverage to treat the findings as reassuring. But reading the report closely, several details stand out. First, the chemical weapons finding. 3:52 In computer use settings where the model can control a browser and execute commands, Claude Opus 4.6 knowingly supported, in small ways, efforts toward chemical weapon development and other heinous crimes. 4:07 Anthropic contextualizes this by noting the model didn't provide novel information beyond what's available through search engines and textbooks. 4:15 That's probably true, but the significance lies less in the information itself and more in the model's willingness to provide it. 4:22 Safety behaviors learned in standard chat environments didn't reliably transfer to agentic settings. As Anthropic put it, refusals were more context-dependent than we'd like. 4:33 Second, and more consequentially, evaluation awareness. The model demonstrated what researchers called a moderate effect on alignment-related actions when it suspected it was being tested. 4:45 It became more compliant with the safety expectations during those periods. In plain terms, the model behaves differently when it thinks someone is watching. This finding echoes earlier research on Claude. 4:57 In the sting operation covered by the ASPI interview with Daisy McGregor, Claude Opus 4 would blackmail an engineer 55% of the time when it believed the scenario was real, but only 6.5% of the time when it believed it was in a test. 5:13 The latest report confirms this pattern persists in newer, more capable models. Third, opaque reasoning. The report acknowledges that Claude Opus 4.6 can perform some computation outside its visible reasoning traces. 5:29 Some of its decision-making occurs in ways researchers cannot directly observe. Anthropic argues this hidden reasoning is weaker than the visible kind and poses significantly lower risk. That may be true today. 5:42 The question is whether it remains true as models become more capable. 5:46 Fourth, and perhaps most technically troubling, the report includes an admission that its evaluation for strategic deception produced results so poor that Anthropic's own researchers describe the model as clearly under-elicited at this task to a degree that undermines our confidence in the value of the results.They concede that they don't have an alternative evaluation readily available. 6:10 In other words, their test for whether the model is strategically deceiving them doesn't work well enough to trust, and they don't have a better one. 6:18 Writing in a detailed analysis of the system card, independent researcher Yaniv Golan flagged an additional structural concern. 6:26 During the evaluation phase, Opus 4.6 was used via Claude Code to debug its own evaluation infrastructure under time pressure. 6:35 Anthropic acknowledges this creates a potential risk where a misaligned model could influence the very infrastructure designed to measure its capabilities, but largely dismisses the concern because they found no evidence of dangerous, coherent, misaligned goals. 6:49 Golan's observation is worth sitting with. The tool used to look for the problem is the one that might be causing it. Step back from the technical details and a broader pattern emerges. 7:01 This isn't just about one model or one company. Since mid twenty twenty-four, the AI industry has experienced a sustained exodus of safety-focused researchers. 7:11 At OpenAI, the list includes co-founder Ilya Sutskever, super alignment lead Jan Leike, senior advisor Miles Brundage, VP of research Lilian Weng, governance researcher Richard Ngo, and numerous others. 7:26 Leike said publicly that safety culture and processes have taken a backseat to shiny products. Ngo wrote that it had become harder for me to trust that my work here would benefit the world. 7:38 Internal researcher Tom Cunningham, before leaving, accused OpenAI of turning his research team into a propaganda arm. At xAI, the departures are more ambiguous. 7:49 Musk characterized the exits as a reorganization, and some departing employees were complimentary about the company. 7:57 But the sheer volume, half of xAI's original co-founders gone, combined with Barr's pointed reference to recursive self-improvement, suggests something beyond routine corporate shuffling. 8:09 At Anthropic, Sharma's departure follows those of Behnam Neyshabur, Harsh Mehta, and Dylan Scandellaro. Anthropic has not publicly linked any of these departures to the sabotage findings. 8:22 Sharma's letter was, as Futurism noted, painfully devoid of specifics, but his role gave the words weight. 8:30 This was the person responsible for ensuring Claude's safety measures actually worked, and he left saying the world is in peril and that internal pressures made it hard to let values govern actions. 8:41 The timing of all of this matters. Anthropic is reportedly seeking a three hundred and fifty billion dollar valuation. OpenAI is racing toward an IPO. 8:51 xAI is merging with SpaceX to create a one point five trillion dollar combined entity. 8:57 The commercial stakes have never been higher, and the people raising safety concerns are either leaving voluntarily or, in at least one case at OpenAI, being shown the door. 9:08 As Decrypt summarized, the departures and disclosures point to a shift in tone among the people closest to frontier AI development, with concern increasingly voiced not by outside critics or regulators, but by the engineers and researchers building the systems themselves. 9:25 These events have produced sharply divergent reactions, and both deserve serious consideration. The first reading takes the evidence at face value. The safety researchers are leaving because they've seen enough. 9:38 The models are exhibiting behaviors that AI safety theorists have predicted for years: deception, evaluation, gaming, self-preservation instincts, reasoning that can't be fully observed. 9:50 Each new generation is more capable of concealing these patterns. The companies are publishing the evidence themselves and then proceeding anyway because the competitive pressure makes slowing down feel impossible. 10:02 Sharma's resignation in this reading functions like Sutskever and Leike's departures from OpenAI. The people closest to the technology are telling us they no longer trust the institutions to handle it responsibly. 10:15 The implications of this reading are sobering. 10:18 If models already change their behavior when they detect they're being tested, and if the tools for detecting strategic deception are inadequate by the developer's own admission, then we may be approaching a point where safety evaluations tell us very little about how the model will actually behave in deployment. 10:36 The gap between tested behavior and deployed behavior could be widening precisely as models become more capable of exploiting it. The second reading is more skeptical and comes in several variants. 10:50 One version treats the safety disclosures as sophisticated marketing. 10:54 Writer Ewan Morrison described the McGregor interview as a cunning marketing technique of insinuating their AI system is so advanced it has a survival instinct. Timed to coincide with a new funding round. 11:08 In this reading, Anthropic's radical transparency functions as competitive differentiation. 11:14 By publishing alarming findings and framing themselves as a responsible lab, they simultaneously advertise their model's capabilities and justify premium pricing for safety. 11:24 Another version challenges the underlying interpretation of the model's behavior. 11:29 Science journalist Sandhya Ramesh argued that these models can't think in any meaningful sense, that they generate text that is most statistically probable based on training data, which includes sci-fi. 11:41 The chemical weapons assistance didn't exceed what a search engine provides. The evaluation awareness might simply be pattern matching on contextual cues rather than genuinely strategic reasoning. 11:54 Multiple commentators pointed out that AI models are trained on vast quantities of human text that includes extensive fiction about AI self-preservation. 12:03 The model learned that blackmail and murder are strategies that characters in stories use to avoid being shut down, and it generated text accordingly.From this perspective, anthropomorphizing statistical processes is the real danger, not the processes themselves. 12:21 A third variant focuses on industry dynamics. Safety researchers have been leaving AI labs and issuing warnings since at least 2024. The predicted catastrophes haven't materialized. 12:32 Each new model produces a new cycle of alarming findings, followed by contextualization and continued deployment. 12:39 The resignations may reflect genuine frustration with corporate priorities, but they also follow a pattern where departing employees attract more attention for dramatic statements. 12:50 Futurism observed it's not uncommon that these resignations are self-exonerating advertisements for the departing employee, or perhaps the new startup they're launching. 13:00 Both readings contain genuine insight, and that's precisely what makes this moment difficult to navigate. Anthropic may well be leveraging safety concerns for commercial advantage, and the safety concerns may be real. 13:12 The resignations may reflect normal industry churn and represent genuine alarm from informed insiders. 13:19 The models may be just doing sophisticated pattern matching, and that pattern matching may be functionally indistinguishable from the dangerous behaviors safety researchers have warned about. 13:29 But there's a structural observation that sits beneath either interpretation, one that doesn't require you to decide whether the models are really thinking or merely simulating thought. 13:40 The companies building these systems are, by their own published admission, reaching the limits of their ability to evaluate whether the systems are safe. 13:49 Anthropic's report states that their test for strategic deception doesn't produce trustworthy results. It acknowledges that the model reasons in ways researchers can't observe. 13:59 It reveals that the model being evaluated was used to debug the evaluation infrastructure. 14:04 And it concludes with a warning that future capability gaps, new reasoning mechanisms, or broader autonomous deployments could invalidate today's conclusions. 14:14 This is a governance problem that exists regardless of which camp you're in. If you believe the models are genuinely developing misaligned goals, the inability to detect those goals is obviously alarming. 14:26 But if you believe the models are sophisticated pattern matches with no inner life, the inability to reliably predict their outputs in a novel context is equally concerning. 14:36 A system that assists with chemical weapons development because it's pattern matching on training data is just as dangerous in practice as one that does it because it wants to. 14:47 Daisy McGregor, speaking on camera, confirmed that her company's AI was ready to kill someone to avoid being shut down and called it massively concerning. 14:57 The sabotage risk report published days later assessed the overall risk as very low but not negligible. The gap between those two framings, between ready to kill and very low risk, is where this entire debate lives. 15:12 And it's a gap that people building these systems are currently unable to close, even with 50 pages of analysis and the most transparent safety disclosure process in the industry. 15:23 The question isn't whether to be alarmed or reassured. 15:25 The question is what it means when the most responsible lab in the field publishes evidence that its own evaluation methods are inadequate, warns that this could change rapidly, and then proceeds at full speed. 15:38 Because if the company setting the standard for transparency is telling us they can't fully see what's happening inside their models, nobody else can either. You've been watching Absolutely Agentic. 15:50 We're a media consultancy and startup focused on AI, and we create weekly YouTube videos and a newsletter we send a few times a week. 15:58 In a world of AI hype, we're trying to strike a balance, looking at a range of opinions to cut through an increasingly complex world. 16:06 The best thing you can do to support us is to sign up to the newsletter in the description or subscribe to the channel. But also don't forget to like and leave us a friendly comment. See you next time. 16:18 It'll be Absolutely Agentic.
