David Simpson - Post-Human Trilogy

Paine tilted his head toward Craig, urging Sanha to continue.

“I-I didn’t know they’d kill everyone. That’s not what I intended.”

Craig shook his head in frustration and closed his eyes as he flexed his large and powerful hands. He wanted to put them around Sanha’s throat and start squeezing; he didn’t think he’d ever let go if he got the chance.

“This war—this conflict—was never about A.I. or no A.I.,” Sanha began to explain. “It was always about control. Power. Absolute power—and who would have it. Gibson or Morgan.”

Craig turned back to Sanha, his eyebrows knitting. “What are you talking about?”

“The A.I. hasn’t told you how it came to be, has it?” Sanha asked.

“I haven’t had time to relay my origin to you, I’m afraid,” the A.I. said to Craig.

“It was grown,” Sanha revealed, “just like a person would be grown—only much more quickly.”

“What do you mean, ‘grown’?”

“The A.I. doesn’t have a brain that emulates the architecture of a human brain. The truth is, we still don’t understand everything about how a brain works. Aldous solved this problem, as the Chinese did before him, by employing a cognitive science-based, explicitly goal-oriented strategy when developing the A.I. In other words, he designed programs that could combine virtual neural patterns together to form new, random patterns that would then be tested to see if the patterns had the desired qualities. Evolution does the same thing when two parents come together to form offspring. Some are successes and others are failures, and more often than not, the successes combine with other successes to produce even more desirable offspring. But, while evolution takes millions of years, virtual combinations are infinitely faster. The A.I. was built this way—the outcome of high-speed computer evolution.”

“His description is accurate,” the A.I. confirmed for Craig.

“All right. So?” Craig asked.

“The A.I. wasn’t the only program to be created in this manner. Aldous also designed virtual worlds where the A.I.’s could be tested. They were given autonomy within the confines of these worlds and then tested one last time in an apocalyptic scenario that they thought was real. The A.I. inside of you right now is the only A.I. that passed the ultimate test.”

“And what was that?”

Sanha smiled. “Ask it.”

“I was willing to sacrifice myself to save humanity,” the A.I. replied.

“So you’re telling me that the A.I. proved it’s a good guy. If that’s the case, why are you trying to destroy it?” Craig asked.

“I’m not trying to destroy it,” Sanha replied, “and neither are the Purists. They’re trying to use it. ”

Craig turned to Paine with an expression that silently asked for confirmation of what Sanha was saying.

“He’s telling the truth. We don’t mean you or the A.I. any harm.”

“Do you expect me to believe that?” Craig asked. “Your government ruined the world over your belief that A.I. is evil, and now you’ve just…changed your minds?”

“We don’t really have a choice anymore,” Paine replied. “The current global situation is unsustainable. When we struck against the Chinese A.I. fourteen years ago, strong A.I. was something it took the resources of an entire nation to realize. Now, all it takes is a few super processors and a small team of people with the right amount of human ingenuity. Aldous and his team were the first to succeed, but they won’t be the last. We’re fighting a losing battle.”

“Humans just can’t monitor everything,” Sanha added. “The Purists have finally figured that out. It’s not practical to try to stop the exponential advancement of technology and, as technology advances, it becomes possible for small groups and even individuals to do greater damage with cheaper and more accessible resources. There was only one sustainable solution to the problem— nannification .”

“What?” Craig reacted.

“Creating an A.I. Nanny. ”

“What?” Craig repeated, this time even more perplexed.

“Basically, an A.I. Nanny is an intelligence that is superhuman, but only mildly so—above us the way we are above higher order apes. It would be tasked with protecting the human species from ourselves. The A.I. could provide stability, and it would have control over a worldwide surveillance system so it could monitor everyone who is online and make sure no one else is trying to build a competing A.I. that could become malevolent. It would control a network of robots in the service industry and be in charge of the world’s manufacturing. It would even control traffic with self-driving cars.”

“So why are the Purists willing to go along with this idea now?” Craig asked. “They could’ve done this all along.”

“Aldous Gibson wasn’t the only one who was determined to build a strong A.I.,” Paine replied. “We’ve intercepted hundreds of other less sophisticated attempts at various stages along the process. Some of them were dangerously close to success—untested, unregulated, extremely versatile A.I.s that were less than six months from coming online and wreaking havoc. If you think WWIII was bad, imagine a malevolent super intelligence running free, exponentially augmenting its own intelligence. Humanity wouldn’t stand a chance.”

“So you’re trusting Aldous’s A.I. just because it passed a test?”

“No,” Sanha answered. “The virtual scenario was a large part of it, that’s true, but there’s more. It is preprogrammed with a set of goals. It has an inhibition against changing its programming. It won’t rapidly modify its general intelligence, and it’s even been programmed to hand over its control of the world to a more powerful A.I. within 100 years. It will see it as its mission to abolish human disease, death, and our current economy of scarcity so clean water, power, food, shelter, and everything else we need will be abundant. And, most importantly, it will prevent the development of technologies that might block it from carrying out its overall mission, which is to improve the quality of human life, without ever taking actions that a strong majority of humanity would oppose.”

“Seems like you’re putting all your eggs in one basket, Professor,” Craig observed.

“It will work,” Sanha affirmed. “The A.I. was created to be good. Just like a human, it cannot fundamentally change that part of itself. If we get it connected to the world surveillance mainframe in time, it will be able to protect us from any and every existential threat.”

“There’s already a mainframe?”

“Yes,” Sanha replied. “Near here, in Endurance Bio-Dome. That’s why you’re here. All that is required is that the A.I. willingly separates himself from you and allows us to transfer his mother program into the mainframe. It’s that simple.”

Craig looked dubiously at Paine.

“Hey. It’s not my first choice,” Paine replied. “I don’t think any American likes the idea of being monitored. But it beats the status quo and any of the other alternatives we’ve been presented with.”

Craig turned back to Sanha. “And you trust them? Even after they killed everyone you lived and worked with?”

Sanha cringed at the mention of the holocaust that was fresh in his memory. “I-I have no choice. I have to trust them at their word. Otherwise, all of this was for nothing.”

“Eliminating the post-humans was a separate issue,” Paine interjected. “Professor Cho had contacted the government intelligence agency about the A.I. Nanny. The decision to remove the equally dangerous nanobot threat swiftly and decisively has no bearing on the government’s decision to adopt the A.I. Nanny project.”