Элиезер Юдковски - Three Worlds Collide

empathic faculty to exclude sympathy with pain, and substitute an urge to meliorate pain."

"But -" Akon said.

Dammit, I'm talking again.

"But we chose this; this is what we want."

"That matters less to our values than to yours," replied the Lady 3rd. "But even you, humankind, should see that it is moot. We are still trying to untangle the twisting references of emotion by which humans might prefer pleasure to pain, and yet endorse complex theories that uphold pain over

pleasure. But we have already determined that your children, humankind, do not share the grounding of these philosophies. When they incur pain they do not contemplate its meaning, they only call for it to stop. In their simplicity -"

They're a lot like our own children, really.

"- they somewhat resemble the earlier life stages of our own kind."

There was a electric quality now about that pale woman, a terrible intensity. "And you should understand, humankind, that when a child anywhere suffers pain and calls for it to stop, then we will answer that call if it requires sixty-five thousand five hundred and thirty-six ships."

"We believe, humankind, that you can understand our viewpoint. Have you options to offer us?"

(4/8) Interlude with the Confessor

The two of them were alone now, in the Conference Chair's Privilege, the huge private room of luxury more suited to a planet than to space. The Privilege was tiled wall-to-wall and floor-to-ceiling with a most excellent holo of the space surrounding them: the distant stars, the system's sun, the fleeing nova ashes, and the glowing ember of the dwarf star that had siphoned off hydrogen from the main sun until its surface had briefly ignited in a nova flash. It was like falling through the void.

Akon sat on the edge of the four-poster bed in the center of the room, resting his head in his hands.

Weariness dulled him at the moment when he most needed his wits; it was always like that in crisis, but this was unusually bad. Under the circumstances, he didn't dare snort a hit of caffeine - it might reorder his priorities. Humanity had yet to discover the drug that was pure energy, that would improve your thinking without the slightest touch on your emotions and values.

"I don't know what to think," Akon said.

The Ship's Confessor was standing stately nearby, in full robes and hood of silver. From beneath the hood came the formal response: "What seems to be confusing you, my friend?"

"Did we go wrong?" Akon said. No matter how hard he tried, he couldn't keep the despair out of his voice. "Did humanity go down the wrong path?"

The Confessor was silent a long time.

Akon waited. This was why he couldn't have talked about the question with anyone else. Only a

Confessor would actually think before answering, if asked a question like that.

"I've often wondered that myself," the Confessor finally said, surprising Akon. "There were so many choices, so many branchings in human history - what are the odds we got them all right?"

The hood turned away, angling in the direction of the Superhappy ship - though it was too far away to be visible, everyone on board the Impossible Possible World knew where it was. "There are parts of your question I can't help you with, my lord. Of all people on this ship, I might be most poorly suited to answer... But you do understand, my lord, don't you, that neither the Babyeaters nor the Superhappies are evidence that we went wrong? If you weren't worried before, you shouldn't be any more worried now. The Babyeaters strive to do the baby-eating thing to do, the Superhappies output the Super Happy thing to do. None of that tells us anything about the right thing to do. They are not asking the same question we are - no matter what word of their language the translator links to our

'should'. If you're confused at all about that, my lord, I might be able to clear it up."

"I know the theory," Akon said. Exhaustion in his voice. "They made me study metaethics when I was a little kid, sixteen years old and still in the children's world. Just so that I would never be tempted to think that God or ontologically basic moral facts or whatever had the right to override my own

scruples." Akon slumped a little further. "And somehow - none of that really makes a difference when you're looking at the Lady 3rd, and wondering why, when there's a ten-year-old with a broken finger in front of you, screaming and crying, we humans only partially numb the area."

The Confessor's hood turned back to look at Akon. "You do realize that your brain is literally hardwired to generate error signals when it sees other human-shaped objects stating a different opinion from yourself. You do realize that, my lord?"

"I know," Akon said. "That, too, we are taught. Unfortunately, I am also just now realizing that I've only been going along with society all my life, and that I never thought the matter through for myself, until now."

A sigh came from that hood. "Well... would you prefer a life entirely free of pain and sorrow, having sex all day long?"

"Not... really," Akon said.

The shoulders of the robe shrugged. "You have judged. What else is there?"

Akon stared straight at that anonymizing robe, the hood containing a holo of dark mist, a shadow that always obscured the face inside. The voice was also anonymized - altered slightly, not in any obtrusive way, but you wouldn't know your own Confessor to hear him speak. Akon had no idea who the

Confessor might be, outside that robe. There were rumors of Confessors who had somehow arranged

to be seen in the company of their own secret identity...

Akon drew a breath. "You said that you, of all people, could not say whether humanity had gone down the wrong path. The simple fact of being a Confessor should have no bearing on that; rationalists are also human. And you told the Lady 3rd that you were too old to make decisions for your species. Just how old are you... honorable ancestor?"

There was a silence.

It didn't last long.

As though the decision had already been foreseen, premade and preplanned, the Confessor's hands

moved easily upward and drew back the hood - revealing an unblended face, strangely colored skin and shockingly distinctive features. A face out of forgotten history, which could only have come from a time before the genetic mixing of the 21st century, untouched by DNA insertion or diaspora.

Even though Akon had been half-expecting it, he still gasped out loud. Less than one in a million: That was the percentage of the current human population that had been born on Earth before the

invention of antiagathics or star travel, five hundred years ago.

"Congratulations on your guess," the Confessor said. The unaltered voice was only slightly different; but it was stronger, more masculine.

"Then you were there," Akon said. He felt almost breathless, and tried not to show it. "You were alive

- all the way back in the days of the initial biotech revolution! That would have been when humanity first debated whether to go down the Super Happy path."

The Confessor nodded.

"Which side did you argue?"

The Confessor's face froze for a moment, and then he emitted a brief chuckle, one short laugh. "You have entirely the wrong idea about how things were done, back then. I suppose it's natural."

"I don't understand," Akon said.

"And there are no words that I can speak to make you understand. It is beyond your imagining. But you should not imagine that a violent thief whose closest approach to industry was selling uncertified hard drugs - you should not imagine, my lord, my honorable descendant, that I was ever asked to take sides ."