Transcript 0:00 The New York Times recently published an AI writing quiz. Kevin Roose, their technology columnist, and Stuart A. 0:07 Thompson put together five pairs of writing passages, each pair containing one human-written excerpt and one generated by AI. The question they asked wasn't, "Spot the AI." It was, "Which passage do you prefer?" 0:25 Within a day, 86,000 people had taken the quiz. 54% preferred the AI. Roose called it a real moment, and his post on X racked up over three million views in a matter of hours. 0:41 The quiz covered five genres: literary fiction, fantasy, science writing, historical fiction, and poetry. Human authors in the mix included Cormac McCarthy, Ursula K. 0:55 Le Guin, Hilary Mantel, Elizabeth Bishop, and Carl Sagan. These aren't obscure names. They're some of the most widely read in the English language. 1:07 Now, I took the quiz, and I preferred the human writing in three out of five instances. That's pretty close, and my responses weren't exactly certain. 1:17 The percentage of people preferring human writing, like me, may have increased since that first day, but 86,000 people is still a massive sample. 1:27 Just when you take the quiz, you might be thinking that the standard traits of AI writing are sorely missing. It's not X, it's Y. Stupid lists starting with the same word and all the other rubbish. 1:39 So it's received some heat for not being a very fair test, but more on that later. 1:44 It's unsurprising that the results sparked a fierce debate, and it deserves one because the implications reach much further than a viral internet quiz. 1:53 The question of whether AI can write as well as a human and whether you'd even notice if it couldn't touches on something fundamental about what writing is for. 2:04 There's a useful precedent for what happens when you strip the labels off something people think they understand. In May 1976, a British wine merchant named Steven Spurrier organized a blind tasting in Paris. 2:18 He invited nine prestigious French judges to evaluate California wines against top French Bordeaux and Burgundy. France was then, by universal consensus, the home of the world's finest wine. 2:33 Spurrier himself expected the French wines to win easily. He sold French wine for a living. The California wines won in both categories. The immediate reaction from the French wine establishment was hostile. 2:48 Judges questioned the methodology. They demanded retests. One judge reportedly tried to take her scorecard back. The retests were held in 1978 and again in 2006, and the California wines kept winning. 3:04 A statistical analysis later showed that beyond the top two wines, the judges couldn't reliably differentiate quality at all. 3:13 The Judgment of Paris, as it came to be known, didn't prove that Californian wine was better than French wine. 3:19 It proved that when you remove the label, the thing experts thought they were tasting turned out to be, in part, the label itself. 3:29 This is essentially what that NYT quiz did with writing, and the parallel goes further than you might expect because something interesting has been happening in the academic research. 3:41 In October 2025, researchers from three colleges published a pre-registered study. 3:46 They recruited 28 MFA candidates from top American writing programs and asked them to write 450-word excerpts imitating the styles of 50 award-winning authors from Nobel laureates to Pulitzer winners. 4:02 They then had three AI models, ChatGPT, Claude, and Gemini, attempt the same task. 159 readers, both expert and lay, evaluated the results blind. 4:15 When the AI was given standard prompts, experts strongly preferred the human writing. But when the same AI was fine-tuned on each author's complete body of work, the results reversed. 4:28 Experts now preferred the AI output for both stylistic fidelity and writing quality. 4:34 The fine-tuned AI writing was flagged as machine-generated just 3% of the time by state-of-the-art detection tools, compared to 97% for the standard prompted version. 4:45 The cost of fine-tuning and generating a passage was about $81 per author, which the researchers noted represents a 99.7% cost reduction compared to typical professional writer compensation. 5:00 That study on its own is significant, but a second study published around the same time by researchers at Princeton Center for Digital Humanities found something arguably more unsettling. 5:13 This group used Raymond Queneau's Exercises in Style, a 1947 literary work that retells the same mundane story in 99 different styles as an experimental text. 5:26 They created AI-generated versions of 30 of these exercises and showed them to 556 human participants under three conditions: blind, correctly labeled, and with labels deliberately swapped. 5:42 When people didn't know which was which, they chose the AI version 55.3% of the time. But when told a passage was AI-written, they rated it significantly worse. The attribution bias was 13.7 percentage points for humans. 5:59 For AI models asked to do the same evaluation, the anti-AI bias was 2.5 times stronger.The same text with the same words in the same order received opposite evaluations depending entirely on what readers believed about who wrote it. 6:16 So we have a situation where in blind conditions, people consistently lean toward AI writing by a small margin. But the moment they know it's AI, they penalize it heavily. The preference is real, so is the prejudice. 6:31 Before accepting the conclusion that AI writes as well as humans, though, there's something worth examining about the condition under which these tests are run. The NYT quiz used short, decontextualized passages. 6:44 Readers picked whichever they preferred in a few seconds with no knowledge of the broader work either passage came from. 6:51 One of the human written excerpts was from Cormac McCarthy's Blood Meridian, a novel widely regarded as one of the greatest American literary works of the twentieth century. 7:01 The specific passage features the judge's speech on war. "It makes no difference what men think of war," said the judge. "War endures, as well ask men what they think of stone." 7:13 The quiz's feedback to readers who preferred that human passage is revealing. It tells them you're either sharply attuned to the qualities that make for great writing or a lucky guesser. 7:24 It then describes McCarthy's prose as containing clunky phrases caused by the author's aversion to punctuation, and adds that AI used to make mistakes like these. McCarthy's syntax in Blood Meridian is unusual by design. 7:39 The absence of quotation marks, the biblical cadences, and the stripped back punctuation create a voice that reads as ancient and inevitable. 7:49 These are deliberate artistic choices, and they carry the weight of three hundred pages that precede them. Describing them as clunky puts them in the same category as machine errors. 8:00 The quiz, perhaps inadvertently, reveals a bias in its own methodology. It frames smoothness and fluency as the markers of quality and treats unconventional style as a deficiency. 8:13 This connects to something that several of the quiz's critics noticed independently. A commenter on the discussion site Tildes considered the easiest tell to be that AI writing had no rough edges. 8:25 The human writing always had a word or phrase or twist in meaning I wouldn't have expected, whereas the AI writing is stylistically monotone. That momentary pause where you think to yourself, wait, what was that? 8:38 Feels important, as that's when you have to shift your perspective to align with the author's. AI writing never asks that of you. 8:47 Another commenter reflecting on having picked mostly AI passages realized they had been judging based on what felt more cohesive instead of what told the better story. 8:58 On Flowing Data, Nathan Yau observed that the quiz used isolated short passages that don't require sustained coherence and speculated that longer passages or articles would probably favor human writers for now. 9:12 Writer Ben Mork was more pointed, arguing that all attempts to demonstrate AI's literary capability rely on short vibe based excerpts and asking whether AI could produce even a bog standard novel or even a short story that is enjoyable or even on a narrative level legible. 9:33 An LLM generates text by predicting the most likely next token in a sequence. This means its writing gravitates toward the statistical center of its training data. 9:43 Fluent, coherent, frictionless prose that reads well on first encounter. 9:48 An AI could produce unconventional syntax if specifically prompted to do so, but it is structurally unlikely to arrive at that kind of risky, deliberate stylistic decision that makes McCarthy's prose what it is. 10:03 The rough edges that readers identified as absent from AI writing are, in literary fiction, often the whole point. 10:12 The quiz may therefore have been measuring palatability than quality, and the format, short excerpts evaluated on immediate preference, structurally favored the AI's strengths while penalizing exactly the qualities that distinguish literary writing from merely competent writing. 10:30 The finding that fifty-four percent preferred AI after the first day is real. What it signifies is more complicated than the headline suggests. There's a broader question lurking behind all this. 10:42 The NYT quiz tested writing in the format we increasingly encounter it. Short, isolated, and stripped of context. A paragraph here, a snippet there. Kinda like social media. That's just how most of us read now. 10:58 We scroll through headlines, skim articles, and consume threads. We rarely sit with a single piece of writing long enough for its stranger qualities to reveal themselves. 11:09 And if that's the environment in which writing is being evaluated, then of course the AI performs well because AI writing is optimized for exactly that context. It's fluent on first contact. It reads well at speed. 11:23 It never makes you slow down and wonder whether the author made a mistake or a choice. The problem is that slowing down and wondering is a significant part of what literature has always asked of its readers. 11:35 I find myself thinking about whether the quiz results tell us something about AI or whether they tell us something about what we've already lost. 11:43 If we spent a decade training ourselves to prefer writing that goes down easily, writing without friction, then we've been pre-selecting for the kind of prose that machines produce naturally. 11:55 Whatever the quiz's limitations, the broader trajectory is clear. AI generated writing is becoming harder to detect. Readers sometimes prefer it, and the economic incentives to deploy it at scale are significant. 12:08 The question of who gets hurt by this is already being answered.In March 2026, the same week as the NYT quiz, Grammarly launched and then rapidly withdrew a feature called Expert Review. 12:22 The feature allowed subscribers to receive AI-generated writing advice presented as being inspired by named real-world experts, including Stephen King, Carl Sagan, Neil deGrasse Tyson, and dozens of working journalists. 12:36 None of these people consented. Many were not even aware their names were being used. 12:42 Tech journalist Casey Newton discovered that a virtual version of himself was dispensing writing advice to Grammarly's 40 million users. "I've long assumed that before too long, AI might take my job," he writes. 12:55 "I just assumed that someone would tell me when it happened." Kara Swisher's response was less reserved. "You rapacious information and identity thieves better get ready for me to go full McConaughey on you. 13:09 Also, you suck." The feature's absurdities piled up. Author and copy editor Benjamin Dreyer posted paragraphs of lorem ipsum, the meaningless placeholder text used in graphic design, and received tips from Stephen King. 13:24 A Cambridge historian named David Abulafia, who had died in January 2026, continued to offer expertise through the tool posthumously. 13:34 Investigative journalist Julia Angwin filed a class action lawsuit against Grammarly's parent company Superhuman with damages claimed in excess of five million dollars. 13:44 Newton made an observation that complicates any easy outrage about this. What Grammarly did, he argued, was fundamentally the same thing that every chatbot does when a user types, "Edit this the way Casey Newton would." 13:58 Grammarly just had the bad manners to put his name on it. This is where the philosophy catches up with the technology. In 1969, Michel Foucault delivered a lecture called What is an Author? 14:10 In which he proposed that the author is not a person, but a function, something he termed the author function. The author's name, Foucault argued, is functional in that it serves as a means of classification. 14:24 It groups text together, confers authority, and determines how discourse is received. 14:29 Foucault imagined a future in which discourse would circulate without any need for an author and would unfold in a pervasive anonymity. 14:39 He ended the lecture by returning to a line from Samuel Beckett, "What matter who's speaking?" 14:46 Grammarly's expert review was, in a sense, a crude attempt to exploit the author function, to borrow the classification and authority that comes with a name without the person behind it. 14:58 The backlash reveals that even in 2026, and perhaps especially in 2026, the author function carries real commercial and cultural weight. It matters who's speaking, or at least who we think is speaking. 15:12 The Authors Guild has recognized this. In early 2026, they expanded their Human Authored Certification Program, allowing authors to place a trademark seal on their books indicating human authorship. 15:25 Over 3,000 authors have certified more than 5,000 titles. Authors Guild CEO Mary Rasenberger has described it as a response to AI-generated spam titles flooding the various platforms. 15:39 But she has also acknowledged something that sits uncomfortably alongside the program's purpose. No reliable method of detecting AI writing currently exists. So the certification works entirely on the honor system. 15:53 That is, the system of authors declaring in their contracts that their work is original and to disclose any AI use. 16:01 The seal tells readers a human wrote this, and the reader trusts that claim because the author's name is attached. The author function, once again, is doing its work. 16:12 In 1967, two years before Foucault's lecture, Roland Barthes published an essay called The Death of the Author. 16:19 Barthes argued that a text is a tissue of citations resulting from the thousand sources of culture, and that the writer's only power is to combine the different kinds of writing. 16:31 Meaning, he insisted, does not reside in the author's intention. It resides in the reader. The unity of a text is not in its origin, Barthes wrote. It's in its destination. 16:44 Barthes was describing the act of human writing. 16:47 He had no concept of large language models, and yet his description of the writer as someone who recombines existing language into new configurations, whose authority is a constructed function rather than a natural fact, and whose text is a tissue of citations from countless sources, reads now as an almost uncanny description of how AI generates prose. 17:12 The NYT quiz asked 86,000 people to evaluate writing on its own terms, stripped of authorship. In doing so, it performed at enormous scale exactly the experiment Barthes proposed. 17:25 It killed the author and asked the reader to decide. The reader decided, by a slim margin, that they preferred the AI. 17:33 But the studies also show that when the author is resurrected, when readers learn that the passage was AI-generated, their judgment shifts by nearly 14 percentage points. 17:43 The attribution bias found that AI models themselves show an even stronger anti-AI prejudice, devaluing identical texts when told a machine wrote it. We prefer AI writing when we don't know it's AI writing. 17:58 The moment we do know, we punish it. Both reactions are real, and they can't both be right. 18:05 There's a version of the future where this question becomes irrelevant, where AI-generated prose becomes so ubiquitous that the very concept of human writing requires a label and a certification seal to distinguish it.The Authors Guild is already building that future, one $10 certification at a time. 18:25 Nathan Yao at Flowing Data wondered whether a decade from now, in search of human life, we seek craggly imperfections as a signal for a real brain and beating heart. 18:36 The honor system had its first major test this year, and it did not go well for the author in question. 18:42 Hachette Book Group cancelled the US release of a novel called Shy Girl by Mia Ballard and pulled the UK edition from sale. 18:51 After months of reader speculation and a New York Times investigation concluded the book was largely AI-generated. AI detection firm Pangram estimated 78% of the text was machine-written. 19:04 The book had been self-published in early 2025, gone viral, and then been acquired by Hachette on the strength of its sales, then published in the UK with what industry observers described as minimal editing. 19:18 Readers on Reddit and YouTube identified the problem months before anyone at the publishing house acted. A three-hour video essay titled I'm Pretty Sure This Book Is AI Slop gathered 1.2 million views. 19:31 It's believed to be the first time a Big Five publisher has withdrawn a title over AI use, and the detection was effectively crowdsourced by the same readers the certification system asked to simply trust the label. 19:45 Now, I wouldn't be surprised to see that kind of scenario playing out again and again over the next few years. 19:51 If we're being honest, the current state of play is complicated, and there's little to be gained by either denouncing or wholeheartedly embracing AI writing. 20:00 We can be defensive of human creativity, as exemplified by the deliberate syntactical experimentation of Cormac McCarthy, without railing against the practical application of thoughtfully prompted AI text. 20:15 Cards on the table, this script was produced using the kind of hybrid workflow that is becoming more normal. 20:21 AI researching and drafting with a human shaping the argument, making the editorial decisions, refining the style and tone, and catching things that matter. Don't hate me because I used AI. This is a channel about AI. 20:35 It would be most strange if we didn't use any AI at all. 20:39 Foucault ended his 1969 lecture by imagining a world in which the old questions of authorship, who is the real author, have we proof of his authenticity and originality, what has he revealed of his most profound self, would give way to nuance. 20:57 What are the modes of existence of this discourse? Where has it come from? How is it circulated? Who controls it? Behind all these questions, Foucault wrote, we would hear little more than the murmur of indifference. 21:12 What matter? Who's speaking? You've been watching Absolutely Agentic. We're a media consultancy and startup focused on AI, and we create weekly YouTube videos and a newsletter we send a few times a week. 21:26 In a world of AI hype, we're trying to strike a balance, looking at a range of opinions to cut through an increasingly complex world. 21:34 The best thing you can do to support us is to sign up to the newsletter in the description or subscribe to the channel. But also don't forget to like and leave us a friendly comment. See you next time. 21:46 It'll be Absolutely Agentic.
