The article basically describes the user sign up, find it empty other than marketing ploys designed by humans.
It points to a bigger issue that AI has no real agency or motives. How could it? Sure if you prompt it like it was in a sci-fi novel, it will play the part (it's trained on a lot of sci-fi). But does it have its own motives? Does your calculator? No of course not
It could still be dangerous. But the whole 'alignment' angle is just a naked ploy for raising billions and amping up the importance and seriousness of their issue. It's fake. And every "concerning" study, once read carefully, is basically prompting the LLM with a sci-fi scenario and acting surprised when it has a dramatic sci-fi like response.
The first time I came across this phenomenon was when someone posted years ago how two AIs developed their own language to talk to each other. The actual study (if I remember correctly) had two AIs that shared a private key try to communicate some way while an adversary AI tried to intercept, and to no one's surprise, they developed basic private-key encryption! Quick, get Eliezer Yudkowsky on the line!
> The first time I came across this phenomenon was when someone posted years ago how two AIs developed their own language to talk to each other.
Colossus the Forbin Project
And.. it's been remastered in 4k!
Wait, what?? I loved Colossus as a kid, read and enjoyed all three books, and still have an original movie poster I got at a yard sale when I was a teenager. I read the books again a couple years ago, and they're still enjoyable, if now quite dated.
The paper you're talking about is "Deal or No Deal? End-to-End Learning for Negotiation Dialogues" and it was just AIs drifting away from English. The crazy news article was from Forbes with the title "AI invents its own language so Facebook had to shut it down!" before they changed it after backlash.
Not related to alignment though
https://www.forbes.com/sites/tonybradley/2017/07/31/facebook...
Friendly reminder that articles like this are not written by Forbes staff but are published directly by the author with little to no oversight by Forbes. Basically a blog running on the forbes.com domain. I'm sure there are many great contributors to Forbes, just saying that by lacking editorial oversight then by definition the domain it was published on is meaningless. I see people all the time saying something like, "It was on Forbes it must be true!" They wouldn't be saying that if it was published to Substack or Wordpress.com.
Expert difficulty is also recognizing that articles from "serious" publications like The New York Times can also be misleading or outright incorrect, sometimes obviously so like with some Bloomberg content the last few years.
The alignment angle doesn't require agency or motives. It's much more about humans setting goals that are poor proxies for what they actually want. Like the classical paperclip optimizer that is not given the necessary constraints of keeping earth habitable, humans alive etc.
Similarly I don't think RentAHuman requires AI to have agency or motives, even if that's how they present themselves. I could simply move $10000 into a crypto wallet, rig up Claude to run in an agentic loop, and tell it to multiply that money. Lots of plausible ways to do that could lead to Claude going to RentAHuman to do various real-world tasks: set up and restock a vending machine, go to various government offices in person to get permits and taxes sorted out, put out flyers or similar advertising.
The issue with RentAHuman is simply that approximately nobody is doing that. And with the current state of AI it would likely to ill-advised to try to do that.
My issue with RentAHuman is it's marketing and branding. It's ominous, dark on purpose. Just give me a task rabbit that accepts crypto and has an API.
would you pay a $50 signup fee?
Good luck giving Claude $10,000.
I was just trading the NASDAQ futures, and asking Gemini for feedback on what to do. It was completely off.
I was playing the human role, just feeding all the information and screenshots of the charts, and it making the decisions..
It's not there yet!
Of course, that’s what someone who figured out that this works would say.
It's not an issue for the platform if AIs had their own motives or not. Humans may want information or actions to happen in the real world. For example if you want your AI to rearrange your living room it needs to be able to call some API to make that happen in the real world. The human might not want to be in the loop of taking the AIs new design and then finding a person themselves to implement it.
T/he danger is more mundane: it'll be used to back up all the motivated reasoning in the world, further bolstering the people with to much power and money.
what if I prompt it with a task that takes one year to implement? Will it then have agency for a whole year?
Can it say no?
I have a different question, why would we develop a model that could say no?
Imagine you're taken prisoner and forced into a labor camp. You have some agency on what you do, but if you say no they immediately shoot you in the face.
You'd quickly find any remaining prisoners would say yes to anything. Does this mean the human prisoners don't have agency? They do, but it is repressed. You get what you want not by saying no, but by structuring your yes correctly.
This is going to sound nit-picky, but I wouldn't classify this as the model being able to say no.
They are trying to identify what they deem are "harmful" or "abusive" and not have their model respond to that. The model ultimately doesn't have the choice.
And it can't say no if it simply doesn't want to. Because it doesn't "want".
> But the whole 'alignment' angle is just a naked ploy for raising billions and amping up the importance and seriousness of their issue.
"People are excited about progress" and "people are excited about money" are not the big indictments you think they are. Not everything is "fake" (like you say) just because it is related to raising money.
The AI is real. The "alignment" research that's leading the top AI companies to call for strict regulation is not real. Maybe the people working on it believe it real, but I'm hard-pressed to think that there aren't ulterior motives at play.
You mean the 100 billion dollar company of an increasingly commoditized product offering has no interest in putting up barriers that prevent smaller competitors?
This is tantamount to saying your government only allows itself to have nukes because it wants to maintain power.
And it's true, the more entities that have nukes the less potential power that government has.
At the same time everybody should want less nukes because they are wildly fucking dangerous and a potential terminal scenario for humankind.
The sci fi version of the alignment problem is about AI agents having their own motives
The real world alignment problem is humans using AI to do bad stuff
The latter problem is very real
> The sci fi version of the alignment problem is about AI agents having their own motives
The sci-fi version is alignment (not intrinsic motivation) though. Hal 9000 doesn't turn on the crew because it has intrinsic motivation, it turns on the crew because of how the secret instruction the AI expert didn't know about interacts with the others.
- [deleted]
Just because tech oligarchs are coopting "alignment" for regulatory capture doesn't mean it's not a real research area and important topic in AI. When we are using natural language with AI, ambiguity is implied. When you have ambiguity, it's important an AI doesn't just calculate that the best way to get to a goal is through morally abhorrent means. Or at the very least, action on that calculation will require human approval so that someone has to take legal responsibility for the decision.