I follow the AI discussion to some extent (as time permits, which it hasn’t lately) and from initial amusement it’s pivoted to apprehension and doom-kvetching, as if we didn’t get a bellyful of doom-kvetching as COVID passed through. The AIs I’ve played with have had peculiar failure modes, among them expressing that 128-bit registers are larger than 512-bit registers. Numbers aren’t their forte, even down at level of counting on their fingers, since AI image generators don’t have any clear idea how many fingers a hand is supposed to have. (More on that topic here.)
I get the impression that our current generation of AIs have their own way of proposing solutions to problems. What seems obvious to them isn’t always obvious to us. The danger, if there is any, lies in giving them more responsibility than something that can’t count fingers or toes should rightfully have.
Which brings us to an SF flash story that I wrote in 1990 and published in my magazine PC Techniques about that time. Most of my regular readers are familiar with “STORMY Vs. the Tornadoes.” It was designed as a humor piece, and a satire on the concept of AI as it was imagined thirty years ago. A few days ago I realized that I had, in a sense, predicted the future: That AIs will do ridiculous things because those ridiculous things make sense to the AIs. STORMY, a National Weather Service AI, was asked how we might reduce American tornado fatalities.
And STORMY took the question very seriously.
Here’s the whole story, for those who haven’t seen it, or haven’t read it in a long time.
STORMY Vs. the Tornadoes
By Jeff Duntemann
“Mr. Petter, in the last six months, that computer program of yours cut Federal government purchase orders for 18,000 ‘uninhabitable manufactured housing units,’ to a total of 21 million dollars.” Senator Orenby Ruesome (R., Oklahoma) sent the traitor Xerox copies skittering over the Formica tabletop.
U.S. Weather Service Programmer Grade 12 Anthony Petter winced. “Umm…you gave us the money, Senator.”
“But not for rotted-out house trailers!”
Petter sucked in his breath. “You gave us 25 million dollars to create a system capable of cutting annual US tornado fatalities in half. We spent a year teaching STORMY everything we knew about tornadoes. Every statistic, every news item, every paper ever published on the subject we fed him, and we gave him the power to set up his own PERT charts and plan his own project. Umm…I preauthorized him to cut purchase orders for items under $2000.”
“Which he did. 18,000 times. For beat-up, rotted-out, abandoned house trailers. Which he then delivered to an abandoned military base in west Nebraska a zillion miles from nowhere. And why, pray tell?”
Petter keyed in the question on the wireless terminal he had brought from his office. STORMY’s answer was immediate:
TO KEEP TORNADOES FROM KILLING PEOPLE.
Petter turned the portable terminal around so that the Senator could see it. A long pause ensued.
Ruesome puffed out his red cheeks. “Mr. Petter, be at my office at 8:00 sharp tomorrow. We’re going to Nebraska.”
The two men climbed out of the Jeep onto scrubby grass. It was July-muggy, and it smelled like rain. Petter gripped his palmheld cellular remote terminal in one hand, and that hand was shaking.
Before them on the plain lay an enormous squat pyramid nine layers high, built entirely of discolored white and pastel boxes made out of corrugated aluminum and stick pine, some with wheels, most without. The four-paned windows looked disturbingly like crossed-out cartoon eyes. Petter counted trailers around the rim of the pyramid, and a quick mental estimate indicated that they were all in there, all 18,000 of them.
A cold wind was blowing in from the southeast.
“Well, here’s the trailers. Ask your software expert what made him think stacking old trailers in the butt end of nowhere would save lives.”
Lightning flashed in the north. The sky was darkening; a storm was definitely coming in. Petter propped the wireless terminal on the Jeep’s fender and dutifully keyed in the question. The cellular link to STORMY in Washington was marginal, but it held:
TORNADOES ALWAYS SEEM TO STRIKE PLACES WHERE THERE ARE LOTS OF MOBILE HOMES.
Petter read the answer for the Senator. Ruesome groaned and kicked the Jeep hard with his pointed alligator boot. “Goldurn it, son, you call this ‘artificial intelligence?’ That silly damfool program bought up all the cheap trailers it could find and stacked them in Nebraska to get them away from tornadoes in the midwest. Makes sense, right? To a program, right? Save people who don’t live in empty trailers, right?”
The force of the wind abruptly doubled. Lightning flashed all around them, and huge thunderheads were rolling in from all points of the compass. Petter could hear the wind howling through cavities between the trailers.
“I’m sorry, Senator!” Petter shouted over the wind.
But Ruesome wasn’t listening. He was looking to the west, where a steel-grey tentacle had descended from the sky, twisting and twitching until it touched the ground. Petter looked south—and saw two more funnel clouds appear like twins to stab at the earth.
The programmer spun around. On every side, tornadoes were appearing amidst the roiling clouds, first five, then a dozen, and suddenly too many to count, all heading in defiance of the wind right toward them. The noise was deafening—and Petter could now feel through the soles of his feet that unmistakable freight-train rumble of the killer twisters.
Petter had felt all along that he had never quite asked STORMY the right question. Now, suddenly, the question was plain, and he hammered it into the terminal with shaking fingers:
STORMY: FOR WHAT PURPOSE DID YOU BUY ALL THESE TRAILERS?
The answer came back as a single word:
BAIT.
The winds were blowing him to the ground. Petter dropped the terminal and grabbed the Senator by the arm, pulling him toward a nearby culvert where the road crossed a dry creekbed. He shoved the obese man into one three-foot drainpipe, then threw himself into the other.
A moment later, the tornadoes converged on the trailers, all at once. The sound was terrifying. Petter fainted.
Both men lived. Local legend holds that it rained corrugated aluminum in Nebraska for several weeks.
And it was years before another tornado was seen anywhere in the USA.
