michael-dean-k/

When AI leaders (Altman, Amodei) talk about the potentials and dangers, they always bring up curing cancer. Last night I heard Amodei say that curing cancer, Alzheimer’s, schizophrenia, and mental illness would make everything worth it, even 100% job loss. This feels like blatant pathos, an appeal to emotion. I feel like they haven’t been pushed hard on the obvious—what do curing these diseases have to do with public chatbots and agents? Surely, someone isn’t going to use a public chatbot to cure cancer; there would be a specialized internal model with 100x the compute and medical experts. It’s a dishonest use case.

Curing diseases might be a side effect, but they’re obviously more focused on (1) the technical excitement of the problem, and (2) infinite leverage (a datacenter of 24/7 geniuses). There’s no compelling public answer for “why are you doing this?” The average fearful person can’t grasp the answer.

When Amodei was asked, “won’t we all be useless?” his answer was weak (along the lines of, “we’ll have to think about work differently”). How? Here’s my attempt at an explanation: there will be no jobs anymore because any single person will be able to launch and scale a business. Do you have an idea you want to bring into reality? Maybe you’ve had 100s of ideas you’d like to see in the world, but they stay trapped in your head because of the incredible challenge of starting even the most basic idea. What if they could all come to life?

I think the rise of AI will bring Silicon Valley culture as the default mode of work. No more jobs, but everyone becomes a startup. Maybe this sounds exciting to some, but only 1% of startups make it, at least in the current paradigm. I imagine there is a small % of people who want to be in “founder mode,” but maybe AGI will be solid enough that you won’t have to grind and sacrifice your life to spawn a company. You can effectively launch a bad idea from your couch, hope it goes well, forget about it, and earn passive income if it does.

We’re so used to the maxim of “idea are useless, it’s all execution,” but if execution becomes automated, then we’re in for the revenge of the idea guy.

2025 might be the year when AI indisputably crosses human intelligence. It might harness peak human abilities across many fields. It can talk, hear, and see. Bodies aside, it can do anything you can do in the digital plane. It has wallets and permissions. It can work 10-100x our speeds, all day. This means in a single day, it can execute on 6 months of “labor.” So when this thing is agentic, it can run for weeks/months, checking in with you for updates and goal refinement. In 6 months, a single agent can generate 91 years of intellectual power, a synthetic lifetime.

There will be a country of geniuses in a datacenter (says the CEO of Claude). They may be more capable than their masters, but they’ve been designed to stay subservient. Assuming that takeoff, misalignment, and escape aren’t problems (they might be), this paradigm already presents itself with weird, unimagined implications.

At first, there might be technical bottlenecks (who knows how to program these things correctly?). Then, there will be economic bottlenecks (I can only afford one of these things, but a CEO can afford 100). And finally, once it’s cheap, there will only be bottlenecks of vision: what can you imagine?

There has to be a name for this tension: I’m personally excited to have an agent, yet societally I think it will be a net negative. You could just not have an agent (similar to how I just don’t have an Instagram). Or, maybe, they’re not inherently evil. Maybe it’s all about how you would use it, and the #1 rule is don’t be an asshole. If you have an AI-agent posting 50x a day to hack the algo, you’ll get immediately sniffed out.