The Drumlin Effect

You go further being loudly and stubbornly wrong than by being quietly and thoughtfully right.

“My rabbi told our congregation that he had a vision and saw that your rabbi had died. I’m so sorry for your loss.”

“Huh? My rabbi’s perfectly fine! I just saw him!”

“Oh, okay, that’s a relief.”

“This sounds like it’s going to be humiliating for your rabbi, though.”

“Nah, so what if he got it wrong? Isn’t it impressive that he was able to see all the way from Krakow to Lemberg?”

— Traditional joke, recounted by Sigmund Freud in his book “Wit and the Relationship to the Unconscious.”

Reputation is sticky. Great people are right, even when they’re wrong. Obscure ones are wrong, even when they’re right.

I’d love it if the scientific community were more of an exception. But I think anybody who read or watched Carl Sagan’s Contact knew, in their bones, that David Drumlin was all too real. Drumlin is a respected scientist with a distinguished career. He’s legitimately pretty good at what he does. He’s just on the wrong side of the key question of the book: is SETI, the search for extraterrestrial intelligence, worth it? Drumlin thinks not, and spends the first part of the story trying to defund it. In spite of him, Dr. Eleanor Arroway is able to discover an unmistakable alien message. At which point, Drumlin quickly takes over her project and grabs the glory.

Drumlin is older and maler than Arroway, and more willing to play politics, but there’s more to it than that. He’s earned a reputation as a level-headed, realistic thinker with his opposition to SETI. Arroway, having spent her life searching for aliens, has the aura of a crank and the career of a nobody. When Arroway is vindicated, it doesn’t undo all that. All Drumlin has to do to win is to refuse to say “oops, I was wrong.”

This is a serious problem, as I will now demonstrate by listing several real-world examples of Drumlin vs. Arroway.¹

Examples of the Drumlin Effect

• Economists Paul R. and Anne H. Ehrlich spent their careers predicting dire consequences from overpopulation, none of which materialized. They enjoyed much more prestige than critics like Julian Simon, whose optimistic predictions were consistently closer to the mark.

• Every year since 2012, AI researcher Gary Marcus has predicted that deep learning is about to hit a wall. Often he triumphantly announces that his old prediction is finally coming true, because a certain model can’t do a certain thing. Then a different model does said thing, and he quietly moves the goalposts and repeats. His commentary continues to be widely read and respected. As recently as last year, his editorial on “Why Chatbots Are A Waste of Time” was printed by the New York Times. Naming an Arroway would be superfluous here—the entire tech industry clearly disagrees. Eventually, Marcus will be right, though. Maybe next year, or maybe when our singleton AGI runs out of mass in the galaxy to convert into computronium, I’m sure Marcus will take a victory lap.

• Biomedical researcher Robert Gallo gained notoriety for stealing the credit for discovering HIV from French researchers and insisting that it was related to the retrovirus he’d already discovered, HTLV. But it’s much worse than that, because he didn’t really discover HTLV either. He claimed he had, recruited researchers to his lab based on that claim, and then fired them after they succeeded where he’d failed. The first key discovery was actually made by researcher Doris Anne Morgan, who deserves a Nobel and whose Wikipedia page I just created.

• Astrophysicist Arthur Stanley Eddington found the idea of a black hole deeply offensive for unclear reasons. When his former student, Subrahmanyan Chandrasekhar, and his colleague, Edward Arthur Milne, began discussing theories that could imply the existence of black holes, Eddington did his best to bully them out of the field. My favorite quote from Eddington: “I have not read professor Milne's paper, but I hardly think it is necessary, for it would be absurd for me to pretend that professor Milne has the remotest chance of being right.”

• Linus Pauling was a Nobel-prize-winning scientist and a Nobel-prize-winning peace activist. He was a generally good guy, but as his prestige grew he become increasingly stubborn about seemingly random scientific beliefs. In one case, he refused to believe in even the possibility of “quasicrystals,” crystals that form in an aperiodic pattern. “There are no quasi-crystals,” he said, “only quasi-scientists.” Discovering a quasicrystal almost destroyed chemist Dan Shechtman’s career; nobody wanted to associate with someone being mocked by Linus Pauling himself.

I’m working on articles about each of these right now. I’ll go back and link them here after I post them.

Automated Drumlin Overlords

A version of this phenomenon is happening today with AI. Hugo Steinhaus predicted it quite precisely in 1963 when he wrote that AI chatbots would be a vessel for conventional wisdom and conventional stupidity; if there’s a popular misconception, the AI will parrot it, even if experts know better. AI is also typically overconfident in how it phrases guesses and intuitions. Which means it is a very bad idea to cite an AI as an expert. It’s a great way for us to bully ourselves out of improving.

Bonus: The Conventional Stupidity of Google Search AI

See https://jdstillwater.blogspot.com/2012/05/i-put-toaster-in-dishwasher.html and later replications chronicled by Experimental History.

My habit of phrasing true facts so that they sound like lies makes it easy to trick Google Search Summary. Unfortunately, it also means that Google Search Summary rarely trusts anything I say in this blog, either.

It’s surprisingly stubborn on this point. It will cheerfully explain that pumpkins turn orange to make themselves more visible to animals so that the animals will eat them, and if asked specifically may grudgingly admit that getting eaten is generally fatal to a pumpkin, but it still won’t concede the syllogism.

I’m not sure you need to cite statistical data to show that high crime rates are correlated with high crime rates.

This is an unfair trick I pulled, but still. It’s answering the question it assumes I meant to ask, and it’s answering it incorrectly. Georgia was the first to be settled by Europeans, not the last. The AI also gave the incorrect (unless you’re taking or grading the SAT) answer to the other question in that article, saying that Keats’s poem was best described as “melodramatic romanticism.” In other contexts, it knows that the poem isn’t melodramatic, but the tautology of referring to a Romantic poet’s tone as “romanticism” was too strong of a lure.

Google Search Summary’s belief that no countries in Africa start with the letter “K” went viral five years ago, which has only made it more confident that it was right. No comment, other than to say that Nairobi has a great superhero-themed sushi restaurant called Hero.

1. This might seem reckless, because a couple of them are still alive, but I’m betting that Robert Gallo won’t have me whacked for putting him on the same list as Linus Pauling.