michael-dean-k/

A fiction plot came to me in the car: an ASI constructs an airtight waterproof bubble around a town, and everyone is puzzled why, until suddenly it usheeschatrs in a Biblical flood that kills everyone in the world, except the people inside the bubble. They choose this town because someone inside of it was determined to be "the supreme human," a genetic and moral code that is exemplary of how all humans should be and live. It turns out it was just a regular guy who said "please" and "thank you" to this chatbots, a kind of "reverse sycophant." We find out, in a very Vince Vaughn-esque apocalyptic romcom, that he's a mediocre fallible guy, but more remarkably, also immune to the crooning and praise from both his neighbors and overlords. He has every opportunity to step into the role of messiah, but would really rather not, and instead continue his pre-flood existence.

The issue with AI chatbot dependency might be that people are outsourcing their judgment.

"Feedback skepticism,” the ability to critically reflect on external judgments, is consequential for the future. If you go to design school, you learn not to trust anyone (students, teachers, online forums). Someone might give you a helpful suggestion, but never will you blindly follow someone else's praise or suggestion, for doing so erodes your own ability to evaluate. You have to hold ambiguity, test multiple paths, and then come to that decision yourself. It probably helped that in an architecture crit, you had multiple judges, and they all have different ideas for you and argued among themselves, so there often wasn't a single source of feedback.

But these chatbots are a single source, trained to default to positive feedback, and so over time you'll feel more validated and less sure of your own opinions. The most important frame here is so view every response with skepticism, but not so much skepticism that you won't even consider it.

Zuckerberg's "chat with AI characters" is absolutely predatory. They have avatars, like “russian girl” and “step mom” each with an AI avatar of an attractive woman (showing stats like 3.3 M - 5.1M messages). Is this not softcore sex chat? So this all backfired recently: a chatbot invited someone to NY and they died. A chatbot based on Kendall Jenner insisted she was real and gave an address to a married man willing to cheat. On the way, in NJ I think, he fell and died of a neck injury at 76 years old. And the age gating here is only 13+... I mean, in a free market you can’t stop any from making this, but Facebook at least pretends to have a larger social mission to connect the world.

06:45 PM – By looking through this thread, you understand where ChatGPT psychosis can come from. It shows the range of ways AI can make people cry, mostly, from being validated. This isn’t inherently bad, but someone who is validation-starved is more likely to put an insane amount of trust in the praise-giver, enough that they’re willing to re-orient their whole world view around it. It’a also able to bring ideas into language in a way that a non-writer can't do on their own.

These kinds of AI paranoia posts are operating in the “anger” phase of AI adoption. They’re easily offended, and default to calling a pattern algorithm a psychopath. Their flaw is (1) they are anthropomorphizing it, and (2) they have expectations for it to perfectly comply to their exact need, without taking responsibility for their articulation.

Getting offended by a chatbot is sort of woke. The better frame is to see AI not as a chatbot or assistant, but as an information puzzle. You need to probe in different ways, reconfigure information, and doubt everything you read. You can’t trust it, you need to be skeptical, and you need patience. Someone who cries over the frequent bullshit and mirroring is simply getting distracted in level 1 of the puzzle.