michael-dean-k/

If you have an essay with poor conflict, poor cohesion, poor sequence, it’s very possible AI won’t know. AI struggles with essay structure because it thinks through non-linear vectors. A human can easily tell when form is off, because they are slowly reading through mazes of text, from beginning to end, and don’t know how everything connects. Often, only at the end, will they find the key that was necessary to unlock the cryptic prose they just waded through. AI, however, process the whole essay at once. Meaning, it reads the essay insanely quickly, converts it all into math/vectors, and then applies your prompt. It's hard for it to know if your tension is working because you've already spoiled the ending. This is a case for why you need atomic evaluation to either generate/analyze essay form. I needs to think step-by-step (possibly through separate prompts) in order to simulate the linear experience of structure.

Essay Architecture should expand your conception of what an essay can be.

As long as it’s profitable enough to sustain itself, I have the opportunity think differently on the product management front. I can subtly defy “the customer is always right” rule. Each writer has their own goals around writing. Maybe they only want to write personal essays, or academic essays, or memos, or whatever. Part of my vision could be for this app to gradually open your mind to what an essay can be.

The scoring system can and should be confrontational. A flaw of GPT-4o is that it’s sycophantic; it’s so attuned to you that it’s a mirror, reflecting back your strengths but also concealing your blindspots. If I do this right, an objective quality standard will reveal your blindspots instead of hide them.

The purpose of asking “what’s your goal?” on upload isn’t so that all the feedback warps to match your goal; it helps me challenge you (“I know you’re going for X, but why does that preclude you from also achieving Y?”).

Part of my process in building Essay Architecture is writing thousands of words for machines. Basically, it’ll run hundreds of prompts, rollup the results, and point me to areas where the AI scores most diverge from my own. I read them in detail, and as I notice errors in its thinking, I’ll just write paragraph of prose to explain why it’s oversimplifying or over-converging. I then feed these notes back into AI, discuss, and then have it refine its own prompts based on its new understanding (which I then review, edit, and clarify). I’m noticing that instead of writing the prompts to determine how it should think, it’s better than I describe all the nuances and then let it determine how to implement.

I think the "data is all you need" paradigm will be proven wrong in the next 4 years.

My example is, even if you give a 5-year-old the Bible and let them read it cover to cover, they will have read it, but unable to speak about it or understand the full implications. It’s not about more data, it’s enabling deeper understanding of existing data.

And so scaling laws have a limit. Even if you give LLM more time to think, it is still limited by associative connections (it’s effectively just casting and combining more vectors). I’m intuiting that the breakthrough will come from enabling LLMs to inherently make recursive abstractions in chains of thought. After it generates a sentence to itself, it needs the ability to ask: “What is this?” "Is this good?" "Is this (any category within an ontology)?"

What I'm reaching it as it's not about more data, but better labeled data. Early AI was very much about manual labeling and tree construction. It's too vast for humans to do it. We need specialized LLMs to do this abstraction process—to reason over tokens, and then bake back those insights into the token itself, to draw upon during reasoning traces... I think.

This is all pretty fuzzy right now, and limited by my technical understanding, but it could be fun to do some research and properly anchor my "beyond LLMs" ideas. In terms of analyzing essay quality, I could see Essay Architecture evaluations feed directly into model training, but, to what end? I'm all for analysis, but less interested in generation.

**AI Metafiction on Grief:**

• Idea: 1.77 of 5
• Form: 2.00 of 5
• Voice: 3.56 of 5

Overall = 2.44 out of 5

There are mixed reviews on this, so I thought I’d grade it with my framework. It’s great on some dimensions, good on others, but poor on most of them. Overall, it’s barely good. I think this shows that 1) casual readers are often impressed by a single pattern, like if it can do repetition it’s a big deal, 2) skeptics will take the absence of a single pattern to completely discredit it, and 3) even though it’s not “good” yet (an overall 3/5) it’s generally a big deal that we’re close to anyone being able to get good writing for free on-demand. Most people can’t write a 2.44.

Why do LLMs suck at writing? I’d like to do some research and get semi-technical on why it sucks, but then also explain how some technical advances could make it quite good.

Here’s how I’d summarize the 2 core limitations:

• It doesn’t have the ability to edit or delete. It thinks really hard for a second, and then just rams forward without thinking. It takes every word in your prompt, shoots that into the black-box cube of the LLM, and builds something like a snapshot of your problem at hand, and then just spits out words. New “reasoning models,” extend this thinking process. o1 will think up to a minute, and Deep Research for 10s of minutes. This still isn’t how humans write. Ideally, after each sentence, it’s able to re-read what it wrote, and then redefine the whole problem, and perhaps, start over.

• LLMs don’t have an embedded theory on what makes good writing. It uses probability to figure out the most likely next word, making it something like the Internet’s auto-complete. We’re only bottlenecked here by our ability to convert taste into something that is machine readable.

Not sure if IQ is a relevant or legit metric for AIs, but:

• gpt4o = 1 in 6 people (IQ = 115)
• o1 = 1 in 93 people (IQ = 135)
• o3 = 1 in 13,333 people (IQ = 157)

o5 might have an alienness to it, not present in other models. It could be the rarest being you’ve ever encountered, or maybe the rarest on Earth.