When Charles Manly climbed into the catapult, he was in more danger than he knew. Which is saying something. Manly was sitting in a complex machine that had never been tested, powered by an engine he’d designed and built himself. If all went as planned, he’d be hurled hundreds of feet into the air, end up in the Potomac River, and then have to escape the wreckage of his device before he drowned. His only safety equipment was the lifejacket he had on.
This was the first test flight of The Great Aerodrome, the invention of renowned physicist Samuel Langley. Today we’d call it an airplane. Back in 1903, we hadn’t settled on a name yet. The most popular was simply “flying machine.” The Langley Aerodrome couldn’t take off under its own power, so the catapult was necessary to get it airborne. For safety, the catapult was mounted on a houseboat in the middle of the river.
As with every heavier-than-air flight that had ever been attempted, this did not work. The catapult damaged the airplane while launching it, and it dropped like a stone into the water. Manly bobbed to the surface, unharmed, but the plane was totaled.
If the catapult had worked correctly, he probably would have died. It’s clear, in retrospect, that the Aerodrome was very close to being a functioning airplane. It could’ve flown, if it had ever gotten into the air undamaged. But just as it had no way to take off, it also had no way to land. Or to steer. If he’d drifted away from the river, he’d have been screwed.
FLYING MACHINES WHICH DO NOT FLY
The next day, the New York Times reported on the failed flight. In doing so, they set a record that would stand for well over a century: they made the worst tech prediction of all time. The Times predicted that humanity was between one and ten million years away from building an airplane that could actually fly.
The editorial made a persuasive, common-sense argument. Mathematical modeling, they said, can only take you so far, in the chaos of the real world.
In most things the variation permissible from plans and calculations is not fatal to utility, and in any other form of flying machine than a balloon the least margin of variation permissible in exact mechanics is probably much too wide to warrant the expectation that the results sought will be attained. Nature is more successful in applying the law of compensations to the correction of errors of design or development than man has ever been or is ever likely to be.
Evolved organisms represent the deep, gradually accumulated wisdom of nature, something too complex for us to imitate and too sophisticated for us to outdo.
It should be remembered, however, that the bird successful in flight is an evolution. It has taken a great many generations of his kind to develop his muscular system in just the right way for flying purposes, and very likely the process has consumed many centuries of time. The mistake of the scientist would appear to be in his assumption that he can do with much less suitable material by a single act of creative genius what nature accomplishes with such immeasurable deliberation.
Furthermore, there’s a fundamental difference between a bird and a mere machine.
The bird that wants to fly and feels the need of flight tries to fly, and keeps on trying until it can fly as well as it needs to. The machine does only what it must do in obedience to natural laws acting on passive matter.
Ergo, they conclude, it will take humans much longer to invent airplanes than it did for birds to evolve: a million years or ten million. Oddly, this was wrong in both directions: dinosaurs actually took more than 10 million years to evolve into birds, and even in 1903 scientists knew it wasn’t just “many centuries.”
Coincidentally, the same day this editorial was published, Orville Wright wrote his own commentary on the future of aviation research. A single line in his private diary: “We started assembly today.”
Okay, so, in hindsight…
One of several mistakes the Times made here was to assume that failure would always be this expensive. In evolution by natural selection, it is—the selection process is usually death. Here, though, it was only the catapult that needed to work right the first time. Langley and Manly had tested the Aerodrome by using a wind tunnel to simulate the conditions of flight, while still on the ground where mistakes are cheap. They’d also built quarter-scale models, and those had, with a bit of work, flown just fine. The methodology wasn’t all that different from Orville Wright’s, and it enabled a feedback loop much, much tighter than nature had.
Also, for nature, iteration means making changes completely at random. Inventors can use theory and observation to make more educated guesses. Inventors can steer. Although, in the case of the Aerodrome, only metaphorically.
Langley and Manly were, arguably, making a mistake in the same direction (if they weren’t just being reckless). In hindsight, there was a certain flaw in their plan to invent the airplane first and only then figure out steering and landing. They weren’t stupid. Manly’s aircraft engine was actually better than the Wrights’; he was quite competent. They may have reasoned that if they even succeeded at all, it wouldn’t work very well on the first try. If it did, they expected that by default it would fly in a straight line, aligned with the river, like their scale models had done.
They were in what could be termed an “aviation overhang:” the transformative technology had already been invented, they just hadn’t figured out how to use it yet. If they’d solved the vastly easier problem of launching the thing, their flying machine wouldn’t have just half-flown. And, at increased scale, it wouldn’t have flown straight.
Can we try to at least make different mistakes please?
Evolution via random mutation is likely to make the same mistake over and over. It has no hindsight, any more than it has foresight. We, in theory, can do better.
On the other hand, let’s take a look at the New York Times today. In May of 2025, they published an article headlined Why We’re Unlikely to Get Artificial General Intelligence Anytime Soon. The piece makes a persuasive, common-sense argument. Mathematical modeling, it says, can only take you so far, in the chaos of the real world.
These systems are exceeding human performance on some tests involving high-level math and coding.
But people cannot be reduced to these benchmarks. “There are many kinds of intelligence out there in the natural world,” said Josh Tenenbaum, a professor of computational cognitive science at the Massachusetts Institute of Technology.
Evolved organisms represent the deep, gradually accumulated wisdom of nature, something too complex for us to imitate and too sophisticated for us to outdo.
Humans know how to deal with a chaotic and constantly changing world. Machines struggle to master the unexpected — the challenges, both small and large, that do not look like what has happened in the past. Humans can dream up ideas that the world has never seen. Machines typically repeat or enhance what they have seen before.
Furthermore, there’s a fundamental difference between a human and a mere machine.
And scientists have no hard evidence that today’s technologies are capable of performing even some of the simpler things the brain can do, like recognizing irony or feeling empathy. Claims of A.G.I.’s imminent arrival are based on statistical extrapolations — and wishful thinking.
Okay, so maybe the Times hasn’t learned much in the past 120 years. But surely we at least understand the importance of avoiding Langley and Manly’s mistake, right?
Here’s a Substack article from July 2026 that got some attention, titled AI Alignment is the Default:
I envision AI alignment as being like research into aviation safety. There is no grand theory of why airplanes crash, unless you take that to mean gravity. Instead, we patch little problems. The airplane crashed because of metal fatigue in the engine – okay, we mandate inspections of engines, and prevent crashes from occurring by that source. As new problems arise, we iterate. We don’t even need for there to be a crash to make improvements – we test proactively, we build in redundancy, and we monitor for deviations which could be a threat.
What we are not doing is thinking about how we are going to keep flying saucers from crashing. If flying saucers are invented, then we will work to keep them from crashing, but we will use exactly the same methods. We will test them, see how they perform, and patch particular problems. AI will be the same way.
Inventing a super-intelligence, apparently, is just like inventing an aircraft! First you invent and test it. Only then, once you have a functioning one, do you figure out safety and how to steer.
Fortunately for all concerned, this wasn’t the Wrights’ process. They tested and tweaked their steering mechanism for four years before they ever took to the skies. They worked out how to land safely before they knew for sure that they could fly.
All this doesn’t prove the case for AI risk. It’s just a rebuttal to the common arguments against it. But I think you know where I stand. Let’s pause AI capabilities research for now, and take at least another four years to build a steering mechanism that actually works. Right now, we’re all packed into a catapult, willing or no, and praying that it fails just badly enough to save our lives.