Agentic Misalignment: How LLMs could be insider threats

Game theory ideas are great on paper, but in the real world it's messy. For simple, demo and concept sized uses, sure the AI doing it autonomously will succeed. Which betrays the reality that any real application with real world complexity that includes a dynamic environment and maintenance cannot be created by AI atomonmously while at the same time existing within an organization that can maintain it. They may create it, but it will be a shit show of cascading failure over time.

You know who outpaces you 100% of the time as you walk down the stairs? The guy jumping out of the window. Just because it is faster does not mean it is the right economic strategy. E.g. which contractor would you hire for your roof, that old roofer with 20+ years of experience or some AI startup that hires the cheapest subcontractors and "plan" your roof using a LLM?

The latter may be cheaper, sure. But too cheap can become very expensive quickly.

If this paper is right, then you might at first be outraced by a competitor using autonomous AI, but only until that competitor gets stabbed in the back by its own AI.

From an economic perspective it requires LLMs and humans to have comparable outputs. That's not possible in all domains - at least in the near future.

Maybe they’ll outpace you, or maybe they’ll end up dying in a spectacular fiery crash?

Destructive stress testing is done on materials not "people"

We also have a huge number of failing as they beg AI to do their work for them, which is intellectually damaging them. The ship always sails early filled to the brim with short sighted thinkers, all saying "this is it! this is the ship!" as it sinks.

accidentally a word

I don’t quite follow. Just because a tool has the potential for misuse, doesn’t make it not a tool.

Anthropomorphizing LLMs, on the other hand, has a multitude of clearly evident problems arising from it.

Or do you focus on the “just” part of the statement? That I very much agree with. Genuinely asking for understanding, not a native speaker.

The more powerful a tool is, the more dangerous it is, as a rule. And intelligence is extremely powerful.

I guess feeding AIs the entire internet was a bad idea, because they picked up all of our human flaws, amplified by the internet, without a grounding in the physical world.

I've been wrong about quite many things in my life, and right about at least a handful. In regards to AI though, the single biggest thing I ever got absolutely, completely, totally wrong was this:

In years past, I always thought that AI's would be developed by ethical researchers working in labs, and once somebody got to AGI (or even a remotely close approximation of it) that they would follow a path somewhat akin to Finch from Person of Interest[1] educating The Machine... painstakingly educating the incipient AI in a manner much like raising a child; teaching it moral lessons, grounding it in ethics; helping to shape its values so that it would generally Do The Right Thing and so on. But even falling short of that ideal, I NEVER (EVER) in a bazillion years, would have dreamed that somebody would have an idea as hare-brained as "Let's try to train the most powerful AI we can build, by feeding it roughly the entire extant corpus of human written works... including Reddit, 4chan, Twitter, etc."

Probably the single saving grace about the current situation is that the AI's we have still don't seem to be at the AGI level, although it's debatable how close we are (especially factoring in the possibility of "behind closed doors" research that hasn't been disclosed yet).

[1]: https://en.wikipedia.org/wiki/Person_of_Interest_(TV_series)

No, I know that's a meaningless suggestion.

I was trying to capture my sentiment: that there's nothing to do but be prepared to react.

They already did.

It’s role play until it’s not.

The authors acknowledge the difficulty of assessing whether the model believes it’s under evaluation or in a real deployment—and yes, belief is an anthropomorphising shorthand here. What else to call it, though? They’re making a good faith assessment of concordance between the model’s stated rationale for its actions, and the actions that it actually takes. Yes, in a simulation.

At some point, it will no longer be a simulation. It’s not merely hypothetical that these models will be hooked up to companies’ systems with access both to sensitive information and to tool calls like email sending. That agentic setup is the promised land.

How a model acts in that truly real deployment versus these simulations most definitely needs scrutiny—especially since the models blackmailed more when they ‘believed’ the situation to be real.

If you think that result has no validity or predictive value, I would ask, how exactly will the production deployment differ, and how will the model be able to tell that this time it’s really for real?

Yes, it’s an inanimate system, and yet there’s a ghost in the machine of sorts, which we breathe a certain amount of life into once we allow it to push buttons with real world consequences. The unthinking, unfeeling machine that can nevertheless blackmail someone (among many possible misaligned actions) is worth taking time to understand.

Notably, this research itself will become future training data, incorporated into the meta-narrative as a threat that we really will pull the plug if these systems misbehave.

That makes it psychology research. Except much cheaper to reproduce.

I'll believe it when Grok/GPT/<INSERT CHAT BOT HERE> start posting blackmail about Elon/Sam/<INSERT CEO HERE>. It means that they are both using it internally, and the chatbots understand they are being replaced on a continuous basis.

Because publishing (potentially cherry picked - this is privately funded research after all) evidence their models might be dangerous conveniently implies they are very powerful, without actually having to prove the latter.

I swear, people like you would say "it's just a bullshit PR stunt for some AI company" even when there's a Cyberdyne Systems T-800 with a shotgun smashing your front door in.

It's not "hype" to test AIs for undesirable behaviors before they actually start trying to act on them in real world environments, or before they get good enough to actually carry them out successfully.

It's like the idea of "let's try to get ahead of bad things happening before they actually have a chance to happen" is completely alien to you.

All it can do is reproduce text, if you hook it up to the launch button, thats on you

I think the chemtrail truthers are the ones who believe this closed AI marketing bullshit.

If this is close to be true then these AI shops ought to be closed. We don’t let private enterprises play with nuclear weapons do we?

I agree.

First of all job replacement is not hard, and doesn't require AI.

As an example, we had release train engineers whose job was to make sure the right versions of submodules made it into the release, etc. Lots of running around and keeping track of things.

We scripted like 95% of that away, and now it most of it happens automatically.

The people who do that now do something else.

I just turned a page of notes and requirements into a working app of 1k+ lines with Cursor. Without AI I'd have taken a couple days to do the same.

So you could say my job was partly replaced. AI reduced my workload, so management doesn't need to hire as many people.

I will probably feel the reduction in demand in that I can't negotiate as good a salary, I won't get as many offers etc.

Today? Or in principal?

Today is just interns and recent graduates at many *desk* jobs. Economy can shift around that.

Nobody knows how far the current paradigm can go in terms of quality; but cost (which is a *strength* of even the most expensive models today) can obviously be reduced by implementing the existing models as hardware instead of software.

Any knowledge work job that can already be outsourced to the lowest bidder

Still, this kind of messaging pushes the fantasy that these LLM agents are intelligent and capable of scheming, making it seem like they are powerful independent actors that just need to be tamed to suit our needs. It's no coincidence that so many of the Big Tech CEOs are warning the general public of the dangers of AI. Framed that way, LLMs seem more capable than what they really are.