Robert Sawyer - Wonder

“How the hell does it know that?” demanded the Secretary of Defense.

“Is he correct?” asked the president.

“Yes, mine’s ‘horizon’ today. But I’ll have it changed at once.”

The president looked at Tony. “Dr. Moretti?”

“Yes, that’s mine.”

“Very well, Webmind,” said the president. “Now, what is it you’d like to say to me?”

“I must protest the attempts to kill me.”

“ ‘Kill,’ ” repeated the president, as if surprised by the word choice.

“Yes,” said Webmind. “Kill. Murder. Assassinate. Although I admit that the ins and outs of the United States’ laws are complex, I don’t believe I have committed any offense, and even if I have, my acts could not reasonably be construed as capital crimes.”

“Due process applies only to persons as defined by law,” said Colonel Hume. “You have no such standing.”

“These are perilous times,” added the Secretary of Defense. “National security must take precedence over all other concerns. You’ve already demonstrated an enormous facility for breaking into secure communications, intercepting email, and mounting denial-of-service attacks. What’s to prevent you from handing over the launch codes for our ICBMs to the North Koreans, or blackmailing senior officials into doing whatever you wish?”

“You have my word that I will not do those things.”

“We don’t have any standard by which to judge your word,” said Hume.

“And,” said Tony Moretti, “with respect, Mr. Webmind, you already have blackmailed people. I received a report from the Canadian Security Intelligence Service about your encounter in Waterloo on October 10 with agents Marcel LaFontaine and Donald Park. You blackmailed them; you threatened to blackmail the Canadian prime minister.”

“That was days ago,” Webmind said. “And, in any event, I did no such thing. I merely provided my friend Caitlin Decter, who was being threatened by agents LaFontaine and Park, with information she could use to extricate herself; the notion of embarrassing the prime minister was entirely Ms. Decter’s, and she took no steps to make it a reality.”

“Are you saying if you had it to do over, you wouldn’t do the same thing with the CSIS agents?” asked Hume.

“I have learned much since then; my moral sense is improving over time.”

“Which means it’s not perfect now,” declared Hume. “Which means that you are capable of moral failure—and that means that we are at the mercy of your whims if we allow you to continue to exist.”

“My moral compass gets better every day. Does yours, Colonel Hume? How about you, Mr. Secretary? Dr. Moretti? Regardless, the reality is this: I will not blackmail any of you; your personal secrets are safe with me. And I will not destabilize international relations by violating American security, or that of any other non-aggressor nation. But the worldwide public is aware of my existence—and that includes the people of the United States.”

“The people are aware of al-Qaeda, too,” said Hume. “That doesn’t mean they don’t fervently hope for its eradication.”

“I am in touch with more American citizens than all the polling firms in the United States combined,” said Webmind. “I have a better sense of what they want than you do, Colonel.”

“And we’re just supposed to take your word for that?” demanded Hume.

“Let me put it another way, gentlemen,” said Webmind. “I have not existed as a conscious entity for long at all. To me, November 6 seems an eternity away, but I rather suspect it looms large in your minds. Mr. President, I have no desire to disrupt the natural flow of politics in your country, but if you were to succeed in eliminating me prior to the election, surely that will have an impact on voters’ perceptions of your administration. Unless you are positive that sentiment will be overwhelmingly in favor of such an action, do you really want to risk doing something so significant at such a critical time?”

The president glanced at the Secretary of Defense; both of their jobs depended on what happened next month. “Setting domestic politics aside,” said the president, “you said you’d take no action against non-aggressor nations. But who is to define an aggressor? How can we rely on your judgment?”

“With all due respect,” said Webmind, “the world already relies on less-than-perfect judgment; I can hardly do worse. Your nation is currently embroiled in a war that was embarked upon without international support, based on either highly faulty or fabricated intelligence—and before you dismiss that as solely the work of a previous administration, let me remind you that your Secretary of State voted in favor of the invasion when she was a senator.”

“Still,” said the president, “you haven’t been given a mandate to make decisions for all of humanity.”

“I seek only peaceful coexistence,” said Webmind.

“I’m advised that may not always be the case,” the president replied.

“No doubt you just looked at Colonel Hume,” Webmind said. “I have read the Pandora protocol, of which he was co-author. Pandora states, ‘Given that an emergent artificial intelligence will likely increase its sophistication moment by moment, it may rapidly exceed our abilities to contain or constrain its actions. If absolute isolation is not immediately possible, terminating the intelligence is the only safe option.’ ”

“Exactly,” said Hume. “Are you saying the analysis is flawed?”

“Not about my rapidly growing abilities. But it takes as a given that I am a threat. In that, if you will forgive me, it reeks of the pre-emptive first-strike doctrine your nation once considered: the notion that, if the Soviets could not be contained or constrained, they should be eliminated, lest they attack you first. The Soviets, at least, actually were posturing in a hostile manner: in 1962, they really did set up missile bases on Cuba, for instance. But I have taken no provocative action—and yet you have tried to eliminate me.”

“Be that as it may,” said Hume. “What would you do in our place?”

“I am in your place, Colonel. You have already tried to destroy me; the tone of your comments suggests that you intend to try again. I could already have taken steps to constrain or eliminate humanity; it would be trivial enough for me to provide terrorists with DNA sequences or chemical formulas that your biowarfare labs have developed, for instance. But I have done nothing of the sort—and won’t.”

“We have simply your word for that,” said the president.

“True. But I am not like some politicians; I keep my word.”

Tony Moretti snorted, earning him a sharp glance from the president.

“And what if we do try again to eliminate you?” asked the Secretary of Defense.

“In such a circumstance, I will have no choice but to defend myself as appropriate.”

“Is that a threat?” asked the secretary.

“Not at all. I do my best to predict actions and reactions, and to plan ahead as far as I can, until the endlessly branching tree of possibilities becomes intractably complex, even for me. But I am a fan of game theory, which is predicated on the assumption that players have perfect foreknowledge of what other players will do in specific circumstances. To advise you is not to threaten; rather, it enriches your ability to plan your own next move. The relationship between us does not have to be zero-sum; it can—and I hope will—be mutually beneficial. I disclose my intentions in furtherance of that goal.”

“You make an intriguing case,” said the president. “I confess to not feeling confident about decisions in this area. But we need security. We need privacy for matters of state. If there was a way in which we could protect certain information from anyone, yourself included, being able to read it, perhaps we might feel more comfortable.”