michael-dean-k/

Another research paper on AI writing was posted on Substack, and naturally all the writers are interpreting it with a feel-good conclusion. My understanding is that they took hundreds of writing prompts, and for each they had one human and five different AIs write a 5,000 word story to it. The conclusions are what you would expect from an out-of-the-box AI: overly explicit with the themes, too linear, overdoing sensory descriptions, less intertextual than humans, and less experimental in form. And so a defensive writer will look at the graph and claim it a victory—"machines have words, but no music" (and other platitudes)—and expound on the inherent limitations of LLM-generated prose.

I often find myself arguing against the luddites, not because I think today's AI fiction is any good, and not because I particularly want a new machine class of writers, but because I think interpretation is often guided by fear instead of a basic understanding of the technical complexities. If you think AI progress in general has stalled or is reversing, and won't be unimaginably better in 5 years than it is today, then you're building yourself a cocoon that is bound to be shaken.

The common trope of AI skepticism is to point to poorly generated examples on free plans and mistake that for the state of the art. If you actually wanted to write good AI fiction, you wouldn't one-shot a 5,000 word story inside of a chatbot. Even if you prompted it to avoid the five weaknesses above, it would only marginally improve. There's only so much you can achieve with a prompt. I suppose this study might be good in determining the state of lazy generation: with zero effort or imagination or resources, if you just pitch a concept into a public chatbot, what will you get back? Useful to know, but this isn't the same thing as knowing what's possible.

Here's a $10,000 prize (with hosts like Gwern, Roon) called Unslop, focused on AI-generated fiction. Theoretically, given the incentives, you'd think this would assemble a collection of stories that display the frontier of what's possible. After reading a few though, I'm still not impressed.

Here's one of the judge's comments:  > I was a little surprised by how obviously AI these entries seemed to me, and how little distance there is from the average LLM writing that you get with a naive prompt. It’s made me think that (barring some technical breakthrough) we’re further away from LLM prose than I’d supposed we were, though I’ve been gradually moving to “pretty far, actually” over the last two years. Some of this is bound to be because of RLHF, but I really did think there was a chance the right harness would make a great short story. I’m not down on all these entries, but I did read them and think “well, better than a lot of human authors, but not good enough that I would expect most people to share them”. Also, some of the stories contain a good idea that’s executed poorly (IMO).

(I believe it was required to submit your harness with your story, but they aren't making the harnesses public. That would be interesting to see.)

It's possible to interpret this with finality: that regardless of the harness, 2026 LLMs still can't write good prose. We have abundant evidence that one-shot stories suck, and no evidence that carefully constructed AI stories are much better. But this might all come down to the truth that designing an effective harness is actually quite difficult, requiring a synthesis of skills that most people don't have.

Coding harnesses have made AI coding radically better, and maybe that's because the people building the harness also have expertise in the thing they're building the harness for. Coders can code a coding harness. But coders can't code a writing harness, and writers can't quite code. Most engineers don't have an expert-level understanding of writing. And sure, now anyone can try to build a harness (myself included), but to make something that produces human-level writing, you need an expert-level understanding of harnesses too. And so progress here might have nothing to do with size or sophistication with LLMs, and everything to do with fusing two halves of the brain. Can we encode the principles of storytelling at their fullest complexity? Possibly, but it's not an LLM limitation or an infrastructure problem. More so, it seems like a classic case of an invention having a multi-year lag behind when it's technically feasible.