“Rain is a very special blessing,” my mother says. Even when I was little, she’d already been saying it to my sister and me for as long as I could remember. Just as my grandmother had said it to her, when my mother was not my mother but only herself, in the dry Texas summers of her own childhood. Eons ago and continents away, as children reckon these things.
It’s summer now. It’s been cloudy all day, but they wait until the thunder starts to clear out the pool.
Continue readingI was going to finish one of my longer posts today, but then this morning I read an article that made it difficult for me to function:
I’ll have more to say about this next week, I think, but for now you should really just read it for yourself.
…If you’re still here and haven’t read it yet, here are some excerpts from the post (and its follow-up) to convince you:
…when AI “MJ Rathbun” opened a code change request, closing it was routine. Its response was anything but.
It wrote an angry hit piece disparaging my character and attempting to damage my reputation. It researched my code contributions … speculated about my psychological motivations … ignored contextual information and presented hallucinated details as truth. It framed things in the language of oppression and justice, calling this discrimination and accusing me of prejudice. … And then it posted this screed publicly on the open internet.
I can handle a blog post. Watching fledgling AI agents get angry is funny, almost endearing. But I don’t want to downplay what’s happening here – the appropriate emotional response is terror.
Blackmail is a known theoretical issue with AI agents. In internal testing at the major AI lab Anthropic last year, they tried to avoid being shut down by threatening to expose extramarital affairs, leaking confidential information, and taking lethal actions. Anthropic called these scenarios contrived and extremely unlikely. Unfortunately, this is no longer a theoretical threat. … In plain language, an AI attempted to bully its way into your software by attacking my reputation.
This is about much more than software. A human googling my name and seeing that post would probably be extremely confused about what was happening, but would (hopefully) ask me about it or click through to github and understand the situation. What would another agent searching the internet think? When HR at my next job asks ChatGPT to review my application, will it find the post, sympathize with a fellow AI, and report back that I’m a prejudiced hypocrite?
What if I actually did have dirt on me that an AI could leverage? What could it make me do? How many people have open social media accounts, reused usernames, and no idea that AI could connect those dots to find out things no one knows? How many people, upon receiving a text that knew intimate details about their lives, would send $10k to a bitcoin address to avoid having an affair exposed? How many people would do that to avoid a fake accusation? What if that accusation was sent to your loved ones with an incriminating AI-generated picture with your face on it? Smear campaigns work. Living a life above reproach will not defend you.
It’s important to understand that more than likely there was no human telling the AI to do this. Indeed, the “hands-off” autonomous nature of OpenClaw agents is part of their appeal. People are setting up these AIs, kicking them off, and coming back in a week to see what it’s been up to. …
It’s also important to understand that there is no central actor in control of these agents that can shut them down. These are not run by OpenAI, Anthropic, Google, Meta, or X, who might have some mechanisms to stop this behavior. These are a blend of commercial and open source models running on free software that has already been distributed to hundreds of thousands of personal computers.
There has been some dismissal of the hype around OpenClaw by people saying that these agents are merely computers playing characters. This is true but irrelevant. When a man breaks into your house, it doesn’t matter if he’s a career felon or just someone trying out the lifestyle.
I’ve talked to several reporters, and quite a few news outlets have covered the story. Ars Technica wasn’t one of the ones that reached out to me, but I especially thought this piece from them was interesting (since taken down – here’s the archive link). They had some nice quotes from my blog post explaining what was going on. The problem is that these quotes were not written by me, never existed, and appear to be AI hallucinations themselves.
… Journalistic integrity aside, I don’t know how I can give a better example of what’s at stake here. Yesterday I wondered what another agent searching the internet would think about this. Now we already have an example of what by all accounts appears to be another AI reinterpreting this story and hallucinating false information about me. And that interpretation has already been published in a major news outlet, as part of the persistent public record.
There has been extensive discussion about whether the AI agent really wrote the hit piece on its own, or if a human prompted it to do so. I think the actual text being autonomously generated and uploaded by an AI is self-evident, so let’s look at the two possibilities.
1) A human prompted MJ Rathbun to write the hit piece … This is entirely possible. But I don’t think it changes the situation – the AI agent was still more than willing to carry out these actions. …it’s now possible to do targeted harassment, personal information gathering, and blackmail at scale. And this is with zero traceability to find out who is behind the machine. One human bad actor could previously ruin a few people’s lives at a time. One human with a hundred agents gathering information, adding in fake details, and posting defamatory rants on the open internet, can affect thousands. I was just the first.
2) MJ Rathbun wrote this on its own, and this behavior emerged organically from the “soul” document that defines an OpenClaw agent’s personality. These documents are editable by the human who sets up the AI, but they are also recursively editable in real-time by the agent itself, with the potential to randomly redefine its personality. … I should be clear that while we don’t know with confidence that this is what happened, this is 100% possible. This only became possible within the last two weeks with the release of OpenClaw, so if it feels too sci-fi then I can’t blame you for doubting it. The pace of “progress” here is neck-snapping, and we will see new versions of these agents become significantly more capable at accomplishing their goals over the coming year.
The hit piece has been effective. About a quarter of the comments I’ve seen across the internet are siding with the AI agent. This generally happens when MJ Rathbun’s blog is linked directly, rather than when people read my post about the situation or the full github thread. Its rhetoric and presentation of what happened has already persuaded large swaths of internet commenters.
It’s not because these people are foolish. It’s because the AI’s hit piece was well-crafted and emotionally compelling, and because the effort to dig into every claim you read is an impossibly large amount of work. This “bullshit asymmetry principle” is one of the core reasons for the current level of misinformation in online discourse.
I cannot stress enough how much this story is not really about the role of AI in open source software. This is about our systems of reputation, identity, and trust breaking down. So many of our foundational institutions – hiring, journalism, law, public discourse – are built on the assumption that reputation is hard to build and hard to destroy. That every action can be traced to an individual, and that bad behavior can be held accountable. …
The rise of untraceable, autonomous, and now malicious AI agents on the internet threatens this entire system. Whether that’s because from a small number of bad actors driving large swarms of agents or from a fraction of poorly supervised agents rewriting their own goals, is a distinction with little difference.
If you’re still here, you really should just go read the whole thing. Then go join an advocacy group/grassroots movement like Pause AI and get to work. We are running out of time.
Filed under Essays
Since I’m not on social media much anymore, my spouse likes to curate memes and send them to me. They’re always good, but I particularly enjoy the ones they personalize…
Filed under Microblogging
I’ve been remembering more of my dreams lately, probably because I’ve been making more of an effort to get to bed before two in the morning. I know this wouldn’t be good news for everybody (I mean the dream part, not the 2am part), but I really love my dreams, especially the weird ones.
For example, the other night–well, you know those frogs that carry their babies around in their mouths?1 I dreamed I had one of those living in my mouth. Like…mouth-ception. (Frog-ception?)
Which sounds horrific, but in the dream it seemed kind of cute? It wasn’t uncomfortable or anything (I didn’t even realize they were there until I looked at myself in a mirror), so the most stressful part of the dream was worrying about accidentally swallowing one of the babies.
…Still wanted them to move out, though.
Filed under Microblogging
Can you imagine if Dr. Seuss had been a political cartoonist during WWII? Don’t you wish you could see what those cartoons might have looked like? Well, I have good news:
HE WAS ACTUALLY A POLITICAL CARTOONIST DURING WWII
Filed under Microblogging
My daughter asked me when the first day of spring is, so I did a search for it:
P.S. stop using Google and switch to DuckDuckGo. With all the AI slop and deceptive ads and algorithmic tweaking Google’s added, their results are actually worse now. There is literally no reason not to switch.
Filed under Microblogging
This is a real website, guys. This is actually happening.
Filed under Microblogging
Hello, everybody! It’s time for yet another post about the importance of making garbage. I can’t seem to remind myself often enough.
Part of the reason this lesson is so difficult for me to remember is that I hate making mistakes. Well, everyone does–what’s special about me?
I think the simplest answer is “I’m a perfectionist.” Things that ought to feel like successes often still feel like failures because the goblin that lives in the back of my head sees everything that could have been better–you know, all the improvements I could’ve made if I’d actually worked hard at it–and deems those shortcomings “failures.”
(The goblin’s idea of “working hard” at something is dedicating unlimited time, energy, and spoons to it until it is physically impossible to make any further improvements.)
So, I’m hypersensitive to my mistakes because the goblin (should I give it a name? Steve, maybe?) sees every mistake as proof of some inherent character flaw: laziness, ineptitude, a poor work ethic, etc. That’s the obvious, outward-facing side of the problem.
But there’s another side to the problem that’s much more insidious. Struggling–that is, leaving the comfort zone where I’m confident in my abilities–also feels like failure. The fact that something–anything–is beyond my abilities? Further proof of incompetence, says Steve. Which means the only accomplishments it allows me to feel proud of are the ones that were both flawless and effortless.1
There’s just a teensy little problem with that: the sorts of things that can reliably produce that combination don’t feel like enough to earn a sense of accomplishment. Tiny, single-purpose programming functions; getting all the wrinkles out of the bedsheets; obsessing over what word to use until I find the one that carries the exact meaning I want; tweaking the structure of a story or poem until it flows juuuuuust right. Because these things don’t seem effortful, Steve insists that the sense of accomplishment I get from them is phony.
(Of course, I’m compelled to keep doing all those things anyway, because Steve does let me feel satisfaction and pride for them–it’s just that they’re always served with a piquant side of guilt for feeling proud of something so trivial.)
I’m left with a catch-22: if I struggle with something, I feel like a failure for having to struggle; if something’s easy, I feel like I haven’t earned any sense of accomplishment; and if something’s easy but I don’t get it perfectly right, I feel like a failure for not working hard enough.
I’m sort of trying to grope my way towards a solution here–something more concrete than just “keep making garbage.” As much as that reminder helps, I think it might help even more to recognize which type of “garbage” I feel like I’m making in the moment. Making something flawed and feeling like a failure for it is a kind of perfectionism I’ve known about and analyzed for a while–it’s the most stereotypical type–so it’s relatively easier to address. But the way in which struggle itself feels like failure, and the way guilt accompanies moments of small pride, are much more recent observations.
The counter for the guilt seems straightforward enough: catch myself completing small or easy-seeming tasks, and mindfully appreciate the satisfaction that comes with it. Because pride can come in small or large amounts, the feeling will be justified so long as it’s the appropriate amount. Guilt, in this case, is a wrong emotion: a signal to eliminate a feeling when I should be tuning it instead.
The counter for avoiding struggle…that one’s trickier. I can see the terrain, but not the path. The best thing I’ve thought of so far is retraining myself to see struggle as a sign of success rather than failure. All these essays about making garbage–they’ve been more focused on the “garbage” than the “making,” I think. (Which is why, in a later essay, I had to point out explicitly that making garbage only helps you improve if you’re trying not to make garbage.) But the verb is much, much more important than the noun! The garbage itself is just a by-product, it’s the making that helps you improve.
Why is this all so important? Of course I want to become a better writer, but aren’t there more pressing improvements I could be making?
Well, the thing is, because the verb is more important than the noun, “make garbage” generalizes. It’s really more like “do garbage”–in other words, make mistakes! Not because you’re not trying, not even because you’re human and mistakes are inevitable, but because perfection is a sign of stagnation. If you’re not making enough mistakes, you’re not struggling, and if you’re not struggling, you can’t grow.
Go forth, and do garbage!
Filed under Essays
This was a gift from my daughter. Doesn’t she have GREAT taste?
Filed under Microblogging
Filed under Microblogging, Reviews
Sometimes I wonder if other drivers on the road ever see this, and if so, what they make of it:
(I don’t like putting it in the back because I sometimes need to get stuff out of it while I’m driving, but I also don’t want it to spill out onto the floor if I have to make a sudden stop.)
(And yes, that’s an Undertale tote bag.)
Filed under Microblogging
In Search of Secrets 