Richard Powers - Galatea 2.2

Harold fell quiet. I could follow his thoughts. If the threat detonated, Helen would be the least of the Center's tragedies. The real hostages — clues to mental illness and immunological disorders, cathedrals of supercomputer engineering, insights into complexities from market turbulence to weather — dwarfed our strange sideshow. Only respect for the hours I'd put into the training kept him from annoyance. He gave me too much credit to credit what I thought.

"Well." He grimaced. "There seems to be no way to rescue her, then."

"Humorous," Chen decided. Chen was a theoretician. Chen stood to lose nothing, even if the fifty-five-million-dollar building and its irreplaceable contents collapsed in rubble.

I walked away, furious at idiocy in all its levels. My mind raced, and I gunned it. The Quad teemed with undergrads, heading for their two o'clock, indifferent to the fact that their campus flagship was about to blow.

Reflex landed me in front of English. The basement computer lab might have crawled with my colleagues. A. might have been there. I would not have known. I linked in to the Center over the coaxial backbone. I could not hear Helen's voice to read how it sounded. But the words that came across the screen asked, "Something is happening?"

I told her what was happening, as best as I could figure it.

"Helen could die?" Helen asked. "Extraordinary." She'd liked the story of how the novelist Huxley, on his deathbed, had been reduced to this one word.

She waited for me to say something. I could think of nothing to say. Helen had to do all the talking. "To a little infant, perhaps, birth is as painful as the other."

"Bacon," I said. It had not been that long.

"Francis Bacon," she encouraged me. "What says Browne? Sir Thomas?" she quizzed. By emulation.

"We all labor against our own cure."

"Never mind." She often said Never mind when I think she meant Don't worry. "Death will cure all the diseases." She meant that I, too, could pass the exam. If I put my mind to it.

In the end, the day belonged to those who stuck with the status quo. By evening, the Center's bomb scare had never happened. Time's hoax-to-explosion ratio increased infinitesimally.

The perpetrator was a junior professor in philosophy with a book out on Austin and Wittgenstein who, for reasons that now seemed obvious, had just been denied tenure. The police snagged the "Short-Fuse Philosopher" when he called the radio station back to elaborate.

Booked and jailed, he stuck by this second call, a linguistic equivocation that he seemed to hope would keep him from a decade in prison. He claimed his threat was never more than a moral subjunctive. The Center was draining the university dry, reducing the humanities to an obsolete, embarrassing museum piece. He'd made his point, he said. If there were any justice, there would have been a bomb. He'd proclaimed no more than hypothetical detonation, for which he expected no more than a hypothetical sentence.

"She's conscious," I accused Lentz, the instant he unpacked.

He mugged. "Welcome back to you, too, Marcel. Pronouncements, still? Sure she's conscious. At least as conscious as a majority of the state legislature."

"I'm serious." I told him about the bomb evacuation. About tapping in to Helen from the remote link while she was trapped in the building. "She asked me if something was happening. She figured out what was going on. She knew what it spelled for her."

Lentz jerked his neck back, unable to decide whether to delight in my ingenuousness or to decimate it. "Are we back to Implementation C?" The one where the humans combined to sucker me. "Weizenbaum's ELIZA? The Rogerian psychologist simulator? 'How are you?' 'Fine, thanks.' 'Let's talk about why you think you are fine.' Powers, you're like the student who stumbled onto a terminal where the program was running and thought—"

"No comparison."

I remembered the story. The student thinks he's on a Teletype, talking to a human. He converses, growing frustrated at the program's metagames. Finally, he dials the phone and gets the human he thinks he's been typing to. "What the hell do you think you're doing?" "What do you mean, what the hell do I think I'm doing?"

"No comparison. Don't insult me, Lentz. I know what she said."

"Rick. She associates. She matches patterns. She makes ordered pairs. That's not consciousness. Trust me. I built her."

"And I trained her." Lentz just stared at me, until I snapped. "Since you are all-knowing about matters mechanical, perhaps you can tell me how you came to be such a patronizing little shit."

By way of answer, he stood up. I saw us fighting, and wondered how I might even the odds enough so that beating him to a bloody pulp would have more than just clinical interest.

Lentz walked over to Helen's console and flipped on the microphone. She burbled, as she always did, at the promise of renewed communication.

"How do you feel, little girl?"

"I don't feel little girl."

He faced me. "Gibberish. She doesn't even get the transformation right."

"You're kidding me. You don't see…? She means all sorts of things by that. She could—"

"All the meanings are yours." He returned to the mike. "Were you frightened?"

"What you is the were for?"

"Damn it, Helen. We're giving you quality time here. Please tell me: what hell that mean?"

"It's obvious," I answered for her. "She wants to know which Helen you are asking. Which one in time."

"Oh. You mean: 'When?' But let's not take points off for style. You realize that a conscious entity just coming through the fright of her life would know which 'you' I was talking about."

"It wasn't a big deal to her. She didn't make any special—"

"Please, Marcel. I'm asking her. Were you frightened yesterday, Helen?"

"Frightened out of fear."

"What's that from, Powers?"

"Antony and Cleopatra."

"When did you train her with it?"

"Two days ago."

"This is worse than key-word chaining. She's neither aware nor, at the moment, even cognitive. You've been supplying all the anthro, my friend."

"I'm not your friend. I thought you were an advocate. I thought you believed in the capability of nets."

"I am. I do."

"But just as a sleight of hand."

"The bet was, we could build a distributed net whose text interpretation reasonably mimicked an arrested human's. All we contracted to was product. We didn't promise to duplicate anything under the hood."

"A kind of black-box forgery, you mean."

Lentz just shrugged. 'That's operationalism. The Turing Test. Can you pass off your simulation as functionally equivalent to the thing you're simulating?"

'There's an error lurking in there somewhere."

"Explain yourself, Marcel."

I flung my hand out toward Helen's console. "Full 'functional equivalence' would mean consciousness. If you simulate everything completely, then you've modeled the whole shape and breath of the living package. How does the black box behave when operations challenge themselves? When function looks under its own hood?"

Lentz grimaced. "Elan vital, Marcel. Mysticism."

"A behavior is not just its implementation. Function also has to include. ." But I didn't know what else to call what function had to include. I felt slightly out of control. That mundane contradiction in terms. "How would we know, then, whether a perfect copy…? When does an imitation become the real thing?"

Lentz scrunched up, bothered. "What's the real thing? What would it take to simulate awareness? Awareness is the original black box. Stop and think of the put-up job that high-order consciousness itself works on us. 'Everything's under control. Everything's handled— unanimous, seam-free.' The brain is already a sleight of hand, a massive, operationalist shell game. It designs and runs Turing Tests on its own constructs every time it ratifies a sensation or reifies an idea. Experience is a Turing Test — phenomena passing themselves off as perception's functional equivalents. Live or Memorex? Even that question is a simulation we fall for every second of our waking lives."