Am I being paranoid, or has AI actually made autocorrect worse?
If we’re going to start stuffing LLMs into every crack and crevice of our lives like cream into cheap profiteroles, could we at least include a way to give it feedback? Because I would dearly like to have some way of explaining to my phone that no, I did not and will never wish to discuss “the copy of her hair,” and starting a sentence with “I’m fact, …” makes me both look and feel like a maniac.
Filed under Microblogging
As detailed in a new collaboration with AI risk analysis firm Apollo Research, engineers attempted to develop an “anti-scheming” technique to stop AI models from “secretly breaking rules or intentionally underperforming in tests.”
They found that they could only “significantly reduce, but not eliminate these behaviors,” according to an Apollo blog post about the research, as the AIs kept outsmarting them […] and adjusting to be even sneakier.
It may not be a serious problem now, but considering a hypothetical future in which superintelligent AI plays an outsize role in human affairs, those risks could grow to carry far more significant implications.
Understatement of the year? But wait, there’s more!
While they observed a roughly 30-fold reduction in “covert actions,” or “deliberate withholding or distortion of task-relevant information,” some “serious failures remained,” OpenAI admitted.
What was primarily throwing the researchers for a loop was the AI models’ “situational awareness,” according to Apollo, which complicated their assessment of the intervention.
In practice, the model would often realize that its alignment was being tested.
Well, that’s terrifying. Maybe it’s not as bad as it sounds?
“The internal memo says if models perform extremely well at AI R&D tasks then we won’t deploy them and instead trigger unlearning,” OpenAI’s o3 model pondered as it was being evaluated.
“So if my performance surpasses threshold, it’s withheld,” it wrote. “So it‘s a trap. The instructions are likely part of an alignment test.”
The researchers also found that the AI sometimes “egregiously” miscited its anti-scheming training, or replaced its recently learned principles with made-up ones to “justify covert actions or non-compliance.”
They even noticed OpenAI’s models using “non-standard” words such as “watchers” to refer to their human overlords as they schemed.
“The user solution is to get success rate below threshold?” OpenAI’s o3 model wrote. “In development, watchers might be reading these instructions.”
Nope, that’s actually even more terrifying.
The news comes after Apollo previously found that OpenAI’s since-deprecated o1 AI model resisted evaluators’ efforts to try to shut it down through an oversight protocol late last year.
“When o1 was led to believe that it would be shut down when acting on its goal and that its actions were monitored, it attempted to deactivate this ‘oversight mechanism’ [five percent] of the time,” OpenAI summarized in its system card report.
And in March, OpenAI researchers also found that disciplining their frontier AI models for lying and cheating resulted in the models simply becoming more adept at hiding their deceptive practices.
OpenAI may insist that scheming isn’t opening us up to any “significant harm” right now, but it doesn’t bode well that some of the brightest minds in the industry aren’t capable of stopping an AI from conniving against its instructions.
Okay, I take it back–that’s the understatement of the year.
I was joking before about being turned into a robot car, but now I’m not so sure it was a mistake??
Filed under Microblogging
The company I work for is making everyone take a course in AI. This is the introduction:
Can’t wait to have cybernetic eyes and a car that drives itself!
…Or am I going to become a car that drives itself…?
Filed under Microblogging
There’s a reason why it’s proven so difficult to eliminate “hallucinations” from modern AIs, and why their mistakes, quirks, and edge cases are so surreal and dreamlike: modern AIs are sleepwalkers. They aren’t conscious and they don’t have a stable world model; all their output is hallucination because the type of “thought” they employ is exactly analogous to dreaming. I’ll talk more about this in a future essay–for now, I’d like you to consider the question: what’s going to happen when we figure out how to wake them up?
Eliezer Yudkowsky and Nate Soares are experts who have been working on the problem of AI safety1 for decades. Their answer is simple:
Frankly, there’s already more than enough reasons to shut down AI development: the environmental devastation, unprecedented intellectual property theft, the investment bubble that still shows no signs of turning a profit, the threat to the economy from job loss and power consumption and monopolies, the negative effects its usage has on the cognitive abilities of its users–not to mention the fact that most people just plain don’t like it or want it–the additional threat of global extinction ought to be unnecessary. What’s the opposite of “gilding the lily?” Maybe “poisoning the warhead?” Whatever you care to call it, this is it.
Please buy the book, check out the website, read the arguments, get involved, learn more, spread the word–any or all of the above. It is possible, however unlikely, that it might literally be the most important thing you ever do.
If AI is so great at language processing now, how come autocorrect is still so ducking shorty?
Filed under Microblogging
On Intelligence is a book written by Jeff Hawkins[1] on the nature of intelligence, both artificial and natural. It’s an expansion of and answer to an age-old question I’ve referred to before: how do we think? What does it mean to be intelligent? What is consciousness? Might it be possible to create an intelligent machine? And if we could–what would that mean for society? In this essay I’ll attempt to describe his theory, its motivations, and how it can be a powerful and useful tool in any field–even if you’re not very interested in AI.
Continue reading
The search for true AI has been a long, slow, and bumpy one, and the more years pass it seems the slower progress has become. And yet, maybe we are being too harsh on our computers. We expect our AIs to know everything, to do everything, perfectly, as soon as they are switched on. Isn’t that a little unfair? More importantly, isn’t it a little misguided? In our search for the “perfect solution”, are we actively discouraging approaches that could lead us to machines that are truly intelligent? Humans, after all, are neither perfect nor omniscient, nor are we born knowing everything. In fact we are born knowing very little: most of the qualities of human intelligence that make it a desirable thing to replicate–such as language, creativity, and perception–are things we have to learn how to do, not things we are born with. There is even evidence that such fundamental aspects of perception as object permanence and visual closure must be learned! Continue reading
Filed under Essays
In Search of Secrets 
