My planned series of posts on the Drumlin Effect has been overtaken by current events. On September 17th, the internet-popular tech philosopher Venkatesh Rao wrote that “now we have two problems:” AI safety, and the fact that AI safety efforts are dominated by one particular cluster of subcultures. Scott Alexander, a longtime AI safety enthusiast whose audience overlaps a lot with Rao’s, was not amused:
And the problem isn't that we've tried to claim the only seat at the table and prevent other people from contributing. The problem is that we begged everyone to help with the problem for fifteen years, and everyone else just laughed and said "You're a dumb doomsday cult, no smart person has EVER thought AI could be dangerous, I bet you've never read books and worship your own intelligence, hahaha!" So eventually we tried to do some of the work ourselves. Now that reality has dragged (some) other people, kicking and screaming, into admitting there's a problem, the only thing they can think to do is talk about how it's sure suspicious that we haven't immediately stepped aside and handed over all of our projects to the people who spent the last fifteen years bullying us and refusing to lift a finger to solve the problem.
To the extent that Scott is right, this is the Drumlin Effect writ large. Writing about risks from rogue/misaligned AI, until a few weeks ago, automatically made you an Unserious Person. Working on the problem meant accepting grant money from private philanthropists, because no respectable institution would fund you. Researchers made what sometimes turned out to be a devil’s bargain, most notably with OpenAI, because the mainstream position wasn’t just that they were probably wrong, but that they were being ridiculous.
Reputation is sticky. Now that “AI is an existential threat” is the majority position, it doesn’t retroactively make the people who’ve been saying that for fifteen years more respectable. Instead, from the perspective of people who are just now tuning in, it looks like there’s a giant flood coming and the only hope of survival is the local lunatic who heard voices in his head telling him to build a giant boat in his backyard. Obviously we need to confiscate the boat from Noah and turn its management over to our most sensible scientists. They may have failed to predict the flood but at least they’re not nut jobs who think we need to save six of every kosher animal.
I’m torn. On the one hand, it’d be nice to be revered as a prophet for getting into the conversation sixteen years ago. On the other, politics is a large dimension of AI safety at the moment. We need regulations and treaties now now now, and I personally have about as much influence as the guy I saw in Bryant Park recently holding a giant sign reading “Some tattoos attract demons. Google it.”
So, on the whole, I think I’m happy to now have reputable people on my side, even though they earned that reputation by calling me a cultist. Welcome to the resistance. I’m not going to call you out by name like I’d been planning. Just…maybe when drafting regulations, you could still consult us? On the down low, if necessary. We have spent decades thinking about this. You don’t have to start from scratch.