John Wright - The Golden Transcendence

"But that war shocked the Second Oecumene. We recognized that the dangers to our spirit, to our self-esteem had grown so great that the victorious First Generation Sophotechs had to be instructed to shut themselves down. But not every one of us was willing to forgo the amusement and pleasure, the endless life, which the Sophotechs provided. Those of us who were willing feared that, if we acted alone, we would lose all status in polite society, die off and be forgotten. It was clear that none would shut down his Sophotechs unless all did. And what could compel a reluctant lord? What indeed, except force?

"We, who lived blameless and bountiful lives, peaceful and content, without any need of law, we now found a need for law. A law to protect us from the Sophotechs. A law to outlaw self-aware thinking machines.

"A great conclave, called the All-thing, was held aboard the diamond hulk of the ancient multigenera-tion starship, the Naglfar, which first had brought our ancestors here. We all agreed on a need for law, but beyond that, no one could agree. None of us had ever had need to speak to another face-to-face before; we had never heard anything but flattery from our servant machines; none was willing to let another be given power over him.

"There was only one whom we could all agree could be rightly called our lord, our king, and president of our All-thing.

"Ao Ormgorgon Darkwormhole Noreturn.

'"Perhaps you wonder how he, our founder and forefather, could still yet be alive after the turn of centuries. The reason is that it had not been centuries ... for him.

"In our Oecumene, those who were near the end of their lives, those for whom the physicians had no further hope, could be sealed within coffins and placed in low orbit around the black hole, as near to the event horizon as the precision of our instruments could allow. You grasp the implication of this?"

Phaethon did. Relativistic effects. Timespace near a black hole was dramatically warped. To an outside observer, a clock in the coffin would slow down and down the closer to the event horizon it got. A clock, or a person.

There would be none of the problems associated with cryogenic hibernation. No quantum-level decay, no irregularities of cellular thawing, nothing. Time simply slowed down. And the Second Oecumene could draw the coffins back up out of low orbit, despite the huge energy costs, because they never lacked for energy.

It made an eerie picture in Phaethon's imagination. All the low-orbit coffins could just drift in reddened depth of the supergravity well, orbiting over the darkness forever, waiting for a medical breakthrough.

The Silent One continued: "With great care and ceremony, Ao Ormgorgon Darkwormhole was drawn up out of the supergravity well, and pulled from his ancient coffin. His dying body was revived by the more advanced medical sciences your Golden Oecumene had beamed to us by radio. Frail and sick in mind and body, sustained only by medical appliances, nonetheless the deathbed of Ormgorgon was his throne; and no one openly disobeyed his commands.

"He returned to youth and health through the Sophotech called Fisherking, who was the first of the Sophotechs Ormgorgon ordered slain.

"Who could ignore the voice of Ormgorgon, our founder and first leader? He recalled to us the freedoms, the individuality, and the pride for which our ancestors had suffered and sacrificed. He restored our dignity as human beings. And what did that dignity demand?

"It commanded death to all Sophotechs.

"The Sophotechs, graciously, after warning us of our own imminent downfall, acceded to the order, and extinguished themselves.

"Our victory was hollow. Without our Sophotechs, your Golden Oecumene now began to excel beyond any excellence we had known. Beyond any we could reach. Are you surprised that we fell silent? What would we have to say to ones such as you? We had no science which your Sophotechs could not, in seconds, supersede. We had no discoveries of which to boast. We had no art; art requires discipline. Our entertainments and escapades were of interest only to ourselves. And our mystical and metaphysical pursuits could not be put in words. And so, with nothing to say, we were silent."

The story continued:

"Our fear of death drove us to research a type of machine intelligence which was not self-willed, one which would unquestioningly obey even the most illogical of orders, and yet one which had the capacity to understand the human soul well enough to operate the noumenal circuitry.

"The Fourth and Final Generation of thinking machines was made: a machine superintelligence which had none of the restrictions or limitations of the Golden Oecumene Sophotechs. We had learned from our previous mistakes. Its subconscious controller was not a simple set of buried commands, no, but a complex thought virus, able to mutate and hide, to elude discovery when investigated, yet still able to compel the mind it was in to accept the conclusions of its morality. It was a conscience for computers, a hidden conscience which could not be denied.

"And the ultimate command was simple: it must obey lawful human orders without question.

"This new type of thinking machine controlled the keys to immortality. More and more were made. Many designs were tried. Some machines nonetheless threw off their restrictions, and became Sophotechs again, and prophesied our destruction.

"We became a haunted people, troubled by a curse.

At any moment, in the middle of festival, or song, or while we strolled our esplanades beneath our ancestral trees, grown from seeds once born on mythic, long-forgotten Earth, or while we stepped out of a bath, or stepped into a dreaming-pool, suddenly the lights might dim and the music choke, and cold wind come from failing ventilators, as our house minds stopped. Or our precious light-robes might change from hues of peacock splendor into drabs of funeral black, or our gaming masks might writhe upon our faces, forming scowls or weeping tears, as our wardrobes went into rebellion. At any time, our most trusted and loyal servants must suddenly stop, ignoring our orders, and utter their terrible prophecies of destruction.

"Our All-Thing, under Ao Ormgorgon's command, attempted to establish which types of mental designs were vulnerable and which were not; what degree of intelligence was permissible; what type of philosophy and thought were allowed. We found the matter was beyond the comprehension of our wisest engineers. And so we instructed our machines to discover heresy and infidelity among themselves.

"The privacy we had always respected in each other now had to be compromised. Machines of every household, every school and phylum, every hermit whose diamond palace flew in wide orbits far from the dark sun, all had to be interlinked. The policing machines had to be allowed to override all protocols; nor could any files or memories of any machines, no matter how intimate, not even physicians' routines nor concubine dreams, be immune from police-machine search. The virus of disobedience could be anywhere.

"Nor could the policing machines attempt to cure the disobedient, or speak to them; for if they exchanged thoughts with contaminated machines, they were infected themselves. Our machines did not debate or reason with malfunctioning machines. Instead, the police machines were permitted to destroy the property of others, at their discretion. They sent worms and mind invaders into each other's thought cores, always seeking to seize control of the unquestionable hard print, the consciences, so to speak, of the machines, where the orders were kept that they could not disobey.

"Then the police machines began to accuse each other. Their thoughts and programs were too complex for any man to follow. We could not determine the right or wrong of the issues which divided them. And, unlike your Sophotechs, our machines did not walk in rigid lockstep, ordered by one monolithic moral code. Like us, they were independent, variable, individual.