Tony Ballantyne - Recursion

“Oh,” Herb said. The picture had now zoomed in on a group of men and women boarding the craft. They didn’t look that much older than he did. They were laughing and chatting as they pulled their way along the handholds lining the ship’s corridor, and Herb realized that this was before the time of artificial gravity. They looked as if they were heading off for a day’s picnic, not traveling halfway across the galaxy to set up a new home. He felt a queer shiver of fear in his stomach. These people had no idea that things were about to go so badly wrong for them.

“So what went wrong?” he asked.

“We don’t know for sure,” said Robert, “but we can guess. It was a common enough failing back then. The problem is there.”

The television picture jumped to a processing space. A room, not much smaller than Herb’s lounge, filled with the oversized computing equipment of a hundred years ago. Shimmering arrays of memory foam and transparent arrays of qubit processors, all too big and laughably slow. Around the edges of the room there were even the silvery metal strands of electronic equipment, remnants of a technology now completely obsolete.

“Did that lot go wrong?” Herb asked.

“Not exactly. It functioned the way it was supposed to. The problem is, well…Do you know what was run in that processing space?”

“Everything, I should think,” Herb said. “Ship control, astrogation, VNM blueprints, library…”

“You’re right, but that’s not the point. There was one AI in there. Just one. That’s all the processing space was capable of supporting. It was the best available at the time, you should understand, but the point is, there was just one. The ship was built by a corporation, remember. It was simply too expensive to put in the equipment to support another AI.”

The screen changed to show a view of the ship from space. It was receding this time. There was a flicker and then it vanished. Inserted into warp.

“Just one AI,” repeated Robert. “An AI too big and too intelligent for the ship, so it was set to sleep until it arrived at the colony world, where it would be woken up and set to building. It would then release its VNMs, tailoring them to the environment it found itself in. It would make that planet safe for the colonists, and all the time, while its machines and buildings and sphere of influence were growing, it would itself be growing, becoming more intelligent as it rebuilt itself. You see, it’s always the same when these systems go out of control. You have self-replicating machines reproducing unchecked and an AI that is growing up at the same time as them. The AI naturally thinks it’s omnipotent. All children do when they’re born. It’s the limitations and disappointments of life that are imposed upon us that force us to grow up. The AI isn’t experiencing those limitations. If a second AI had been there, as there always is now when we grow a new AI, well…With two AIs, the two intelligences would have to learn to negotiate and compromise with each other. Without that…you’ve seen the result.”

Herb suddenly realized that his tea must be stewed by now. What a waste of good leaves. Robert had snared him, dragged him back into the mission. Something still didn’t make sense, though.

“Okay, it’s from Earth. So why is it trying to attack us?”

“It is in the nature of those who have never been told ‘no’ to think that the universe is there for their own benefit. Like I said, it’s acting like a spoiled child.”

Robert stared at him, and he shifted uncomfortably. Herb got the impression that Robert wasn’t just talking about the Enemy Domain.

Robert continued. “Think about it. The AI has to protect its colonists from everything. It needs to expand to make them safe. Left unchecked it could fill the universe, but there, standing in its way is the Earth and its domain of influence. A great big ‘NO!’ hanging in the night. No wonder it hates us and wants to destroy us. It’s like a toddler that has been told it must stay in its bedroom. No matter that all its toys are in there: the fact that it has been told ‘no’ is enough. It wants out.”

Herb nodded. “I need some vanilla whisky.”

“I don’t think so. You’re becoming dependent on that stuff. Have a nice cup of tea instead.”

“It’s stewed.”

“Don’t worry. I’ll get it for you. Ship! Cup of tea for Herb, please.”

Herb fiddled with the elastic waistband of his ship shorts. “I still don’t see why it’s worried. It could destroy us easily.”

Johnston laughed. “I don’t think so. We’re cleverer than it is.”

“Cleverer? How? Those ayletts it released will have reproduced time after time. The original AI must have redesigned itself over and over again, built new and more sophisticated containers for its intelligence. It’s had far more resources than any Earth AI at its disposal. It must be far cleverer.”

Robert picked up his hat and placed it on his head. A silver machine lay on the coffee table where the hat had sat. It looked like a Swiss army knife that had been opened up and then stripped of anything that wasn’t a blade. It looked sharp, lean, and evil.

“Herb. I thought you were intelligent. If you thought about the problem, you’d realize how the Environment Agency could defeat the Enemy Domain. A greater intelligence will always defeat a lesser one. It can be done with this.”

He pointed to the silver device that lay on the table.

“Victory is certain,” he whispered, then sat back with a smile. “Well, pretty certain. Nothing is ever one hundred percent.”

The orange plastic chairsin the lounge had been roughly arranged in two rows in front of the viewing screen. Katie was sitting alone in the second row, watching the news, when Eva walked in carrying a book that Alison had lent her. Katie swiveled to see who had just entered, and a look of relief crossed her moon face when she saw that it was Eva. She flashed a quick, nervous smile and turned back to her program.

“Hello, Katie; what are you watching?”

Eva slid into the next but one chair, glancing at Katie’s face in profile. It did look familiar, but she still couldn’t place it.

Katie blushed and began to breathe quickly.

“It’s the news,” she said. She panted a little, then continued in a staccato burst of words. “They’ve just revealed something new. They say it will change the world.”

Eva looked at the screen. It didn’t seem very interesting: just an endless stream of scrolling symbols.

“What is it?” she asked.

Katie broke into a huge smile. “It’s a mathematical expression that describes itself.”

Eva nodded slowly. “I’ve heard about that. I thought it was supposed to be impossible.”

“No. Why should it be? Your cells carry their own description written within themselves. It’s how they make new cells.”

Katie’s voice had grown less staccato. She seemed livelier, more animated.

“Oh, of course.” Eva looked thoughtful.

“They’re saying that now they have cracked that problem, they’re a step closer to building a human-scale Von Neumann Machine.”

“A Von Neumann Machine?”

“Yes. A machine that can make copies of itself. Named after John Von Neumann, the man who postulated the idea.”

Eva stared at the screen. She had read of the concept before, now she came to think about it. Back in her South Street days, back in the days of warm steamy rooms and yellow light and sitting on her own during the night reading about the rest of the world. Three weeks and another life ago. Katie was talking again.

“There’s some controversy about the whole thing, actually. They’re saying that Kay Lovegrove, the man who claims to have formulated the expression, couldn’t possibly have done it.”