The Looming AI Clownpocalypse

I appreciate the tone of this article. I am exhausted by the usual existential fear, but doomers are in good company -- during the development of the OG atom bomb, there was fear of the possibility that the fission chain reaction would not stop and all life-as-we-know-it would be destroyed upon detonation. I look at the idea of a rapid AI induced material "robocalypse" (robot-apocalypse) as a similar projected fear, and the result being similarly unlikely, particularly in the near term (coming decades). Even with AI access to sophisticated 3d fabrication facilities, there will be severe supply chain constraints that would impede an overwhelming spawn of robots. If say China or the United States had a ubiquitous deployment of robots already, with manual and mobility capabilities roughly equivalent to humans, the concern would perhaps actually be warranted. We are far from that. Clownpocalyse fits the bill better. Much of the Clownpocalyse are in the ideas themselves, like Nick Bostrom's paperclip improbability.

There are two AI futures I see at the moment that are not so great.

One is the centrally controlled 'large' AI models that become monitoring apparatuses of the state. I don't think there needs to be much discussion on why this is a bad idea.

This said, open (weight) models don't save us from problems either. It's not hard to imagine a small capable model that can boot strap itself into running on consumer hardware and stolen cloud resources being problematic on the net spreading its gremlin like behavior wherever it could. The big AI companies would gladly use AI behaviors like this to dictate why all models/hardware should be controlled and once the general population is annoyed enough, they will gladly let that happen.

Lastly, prompt injections are not a, at least completely, solvable problem. To put it another way, this is not a conventional software problem, it's a social engineering problem. We can make models smarter, but even smart humans fall for stupid things some of the time, and models don't learn as they go along so an attacker pretty much as unlimited retries to trick the model.

“One of the four balloon animals of the AI clownpocalypse” is the best sentence I’ve read all week.

I think this is why the LLM revolution has been so existentially depressing for so many senior engineers - we’ve spent our entire careers fighting for exactly what the author suggests, and we couldn’t make progress against the product and management cabal when code took time and people to write. Now code is “free,” and we’re all being told to just get on the train, don’t worry about the bridge being out, we’ll build a new one when we get there, you see how fast we’re going now?

My generation totally missed the signposts when we RFC'd our way into insecure by default e-mail (and later, web) protocols. In hindsight, it's amazing things held together as long as they did.

It looks like every generation has to learn this for themselves though.

Oof, google api slop !