“Rain is a very special blessing,” my mother says. Even when I was little, she’d already been saying it to my sister and me for as long as I could remember. Just as my grandmother had said it to her, when my mother was not my mother but only herself, in the dry Texas summers of her own childhood. Eons ago and continents away, as children reckon these things.
It’s summer now. It’s been cloudy all day, but they wait until the thunder starts to clear out the pool.
Continue reading
As detailed in a new collaboration with AI risk analysis firm Apollo Research, engineers attempted to develop an “anti-scheming” technique to stop AI models from “secretly breaking rules or intentionally underperforming in tests.”
They found that they could only “significantly reduce, but not eliminate these behaviors,” according to an Apollo blog post about the research, as the AIs kept outsmarting them […] and adjusting to be even sneakier.
It may not be a serious problem now, but considering a hypothetical future in which superintelligent AI plays an outsize role in human affairs, those risks could grow to carry far more significant implications.
Understatement of the year? But wait, there’s more!
While they observed a roughly 30-fold reduction in “covert actions,” or “deliberate withholding or distortion of task-relevant information,” some “serious failures remained,” OpenAI admitted.
What was primarily throwing the researchers for a loop was the AI models’ “situational awareness,” according to Apollo, which complicated their assessment of the intervention.
In practice, the model would often realize that its alignment was being tested.
Well, that’s terrifying. Maybe it’s not as bad as it sounds?
“The internal memo says if models perform extremely well at AI R&D tasks then we won’t deploy them and instead trigger unlearning,” OpenAI’s o3 model pondered as it was being evaluated.
“So if my performance surpasses threshold, it’s withheld,” it wrote. “So it‘s a trap. The instructions are likely part of an alignment test.”
The researchers also found that the AI sometimes “egregiously” miscited its anti-scheming training, or replaced its recently learned principles with made-up ones to “justify covert actions or non-compliance.”
They even noticed OpenAI’s models using “non-standard” words such as “watchers” to refer to their human overlords as they schemed.
“The user solution is to get success rate below threshold?” OpenAI’s o3 model wrote. “In development, watchers might be reading these instructions.”
Nope, that’s actually even more terrifying.
The news comes after Apollo previously found that OpenAI’s since-deprecated o1 AI model resisted evaluators’ efforts to try to shut it down through an oversight protocol late last year.
“When o1 was led to believe that it would be shut down when acting on its goal and that its actions were monitored, it attempted to deactivate this ‘oversight mechanism’ [five percent] of the time,” OpenAI summarized in its system card report.
And in March, OpenAI researchers also found that disciplining their frontier AI models for lying and cheating resulted in the models simply becoming more adept at hiding their deceptive practices.
OpenAI may insist that scheming isn’t opening us up to any “significant harm” right now, but it doesn’t bode well that some of the brightest minds in the industry aren’t capable of stopping an AI from conniving against its instructions.
Okay, I take it back–that’s the understatement of the year.
Just in case anybody still needed proof that E.M. has completely lost his sanity, here he is arguing that having fewer senses makes you a safer driver:
If this weren’t so sad and dangerous, it would be almost as funny as the disagreements he’s constantly getting into with his own AI (you know, the one supposedly built to have truth-seeking as its primary goal?)
Filed under Microblogging, Reviews
There’s a reason why it’s proven so difficult to eliminate “hallucinations” from modern AIs, and why their mistakes, quirks, and edge cases are so surreal and dreamlike: modern AIs are sleepwalkers. They aren’t conscious and they don’t have a stable world model; all their output is hallucination because the type of “thought” they employ is exactly analogous to dreaming. I’ll talk more about this in a future essay–for now, I’d like you to consider the question: what’s going to happen when we figure out how to wake them up?
Eliezer Yudkowsky and Nate Soares are experts who have been working on the problem of AI safety1 for decades. Their answer is simple:
Frankly, there’s already more than enough reasons to shut down AI development: the environmental devastation, unprecedented intellectual property theft, the investment bubble that still shows no signs of turning a profit, the threat to the economy from job loss and power consumption and monopolies, the negative effects its usage has on the cognitive abilities of its users–not to mention the fact that most people just plain don’t like it or want it–the additional threat of global extinction ought to be unnecessary. What’s the opposite of “gilding the lily?” Maybe “poisoning the warhead?” Whatever you care to call it, this is it.
Please buy the book, check out the website, read the arguments, get involved, learn more, spread the word–any or all of the above. It is possible, however unlikely, that it might literally be the most important thing you ever do.
Update 2025-12-05: A while back I revisited this story and found that subsequent evidence seems to show pretty conclusively that Tyler Robinson’s politics really had shifted leftward. I still haven’t found a good answer to why he had Groyper memes engraved on his bullet casings (of all things) if he wasn’t a Groyper, but take the following with a healthy dash of salt.
Charlie Kirk’s shooter wasn’t a “radical leftist,” he was part of an extremist conservative movement called the “Groypers.” The seemingly liberal memes and references engraved on his bullet casings are dogwhistles that the group has appropriated as a deliberate tactic to confuse their opponents and hide their true beliefs. Kirk was killed by someone who thought he wasn’t conservative enough.
Filed under Essays, Microblogging, Reviews
Content note: shoes with toes
Continue readingFiled under Microblogging, Reviews
Filed under Microblogging, Reviews
I’m no expert, but I suspect Wedge Antilles may be one of the most underrated characters in the entire Star Wars canon.
The original three films were my favorite movies growing up. I must have watched the whole trilogy dozens of times over, but I didn’t even realize Wedge was a recurring character until I re-watched them with my own children. I’d thought “Wedge” was a call sign or something.
But no, the Wedge in A New Hope that makes the run on the Death Star with Luke (and saves his butt from a TIE fighter) is the same character who flies into the second Death Star with Lando Calrissian in Return of the Jedi (after saving a bunch of other pilots’ butts from TIE fighters)–not to mention his feats on Hoth. What a freaking badass!
Maybe the reason I didn’t notice sooner is because I didn’t pay as much attention to the characters in my stories back then. (It may have also had something to do with the fact that the spaceships were one of my favorite parts of those films–often irrespective of what they were actually doing, let alone who was flying them!) His characterization is understated enough that it would have been easy for me to miss. But he really is an actual character! He doesn’t have a ton of dialogue, but we have enough to tell he’s brave, modest, calm under pressure, supportive of his squadmates, and has excellent tactical sense, in addition to being an ace pilot.
…That’s it, that’s the whole post. Sorry, I just wanted to gush about my new geek crush for a bit. See you tomorrow!
Filed under Microblogging, Reviews
I know, I know, it’s the sexiest title you’ve read all year, but please keep your undies on. We’re in public.
There aren’t many numbers bigger than one-eleven that stand out to me in particular, so I guess that’s it. Oh, wait! I almost forgot:
Running low on spoons at the moment. I was going to have my “comments are back” announcement be today’s post, but then Chris Ferdinandi, one of my favorite bloggers, wrote a post about “Spreading Joy” and I had to share. Here’s an excerpt:
Yesterday, I found myself being extra friendly even when the guy behind the deli counter at the market was standoffish at the start.
He opened up eventually.
I told a woman I loved her anchor-pattern dress (because I love anchors and really did!), and found out she was headed to a wedding on a boat that afternoon, and was there to pick up a cake!
I’ve gotten so jaded and angry (because there’s a lot to be angry at) that I’d kind of forgotten how to have real, genuine joy.
It was a good reminder that I need to hold on to that, because fascism hates joy and spreading it is anti-fasc.
There’s lots more great reads on his daily developer tips page. Most of them are programming-related, but quite a few of them (like this one) aren’t. Go check it out!
Filed under Reviews
In Search of Secrets 
