Isaac Asimov - Caliban

“Theoretically it is possible for a robot to judge the situation properly, and not mindlock over being complicit in the death of the criminal. It must be able to decide that both the immediate and long-term general good are served if the police officer wins, and that coming to the assistance or defense of a criminal prepared to take the life of a peace officer is ultimately self-defeating, because the offender will almost certainly attack society again in other ways, if he or she is permitted to survive. However, in practice, all but the most sophisticated robots, with the most finely tuned and balanced potentials of First Law, will have no hope at all of dealing appropriately with such a situation.

“All the laws and rules we live by are subject to such intricacies of interpretation. It is just that we humans are so skilled, so practiced, in threading our ways through these intricacies that we are unaware of them. The proper way to enter a room when a party is in progress in midafternoon, the correct mode of address to the remarried widow of one’s grandfather, the circumstances under which one mayor may not cite a source in scholarly research—we all know such things so well we are not even aware that we know them. Nor is such practiced knowledge limited to such trivial issues.

“For example, it is a universal of human law that murder is a crime. Yet self-defense is in all places a legitimate defense against the accusation of murder, negating the crime and condoning the act. Diminished capacity, insanity defenses, mitigating circumstances, the gradations of the crime of murder from manslaughter to premeditated murder—all these are so many shades of grey drawn on the black and white of the law against murder. As we have seen with my example of the policeman and the criminal, no such gradations appear in the rigidity of the First Law. There is no room for judgment, no way to account for circumstances or allow for flexibility. The closest substitute for flexibility a robot may have is an adjustment in the potential between the First, Second, and Third Laws, and even this is only possible over a limited range.

“What are the Three Laws for? To answer my own question, then, the Three Laws are intended to provide a workable simulation of an idealized moral code, modified to ensure the docility and subservience of robots. The Three Laws were not written with the intention of modifying human behavior. But they have done just that, rather drastically.

“Having touched on the intent of the Laws, let us now look at their history.

“We all know the Three Laws by heart. We accept them the way we accept gravity, or thunderstorms, or the light of the stars. We see the Three Laws as a force of nature, beyond our control, immutable. We think it is pointless to do anything but accept them, deal with the world that includes them.

“But this is not our only choice. I say again, the Three Laws are a human invention. They are based in human thought and human experience, grounded in the human past. The Laws are, in theory at least, no less susceptible to examination and no more immutable in form than any other human invention—the wheel, the spaceship, the computer. All of these have been changed—or supplanted—by new acts of creativity, new inventions.

“We can look at each of these things, see how they are made—and see how we have changed them, see how we update them, adjust them to suit our times. So, too, if we choose, can we change the Three Laws.”

There was a collective gasp from the audience, shouts from the back of the room, a storm of boos and angry cries. Fredda felt the shouts and cries as if they were so many blows struck down on her body. But she had known this was coming. She had braced herself for it, and she responded.

“No!” she said. “This is not our way. You were all invited here to join in an intellectual discussion. How can we tell ourselves that we are the most-advanced society in the history of human civilization, if the mere suggestion of a new idea, a mild challenge to the orthodoxy, turns you into a mob? You are responding as if my words were an assault on the religion you pretend not to have. Do you truly believe that the Three Laws are preordained, some sort of magical formula woven into the fabric of reality?” That got at them. Spacers prided themselves on their rationality. At least most of the time. There were more shouts, more cries, but at least some of the audience seemed ready to listen. Fredda gave them another moment to settle down and then continued.

“The Three Laws are a human invention,” Fredda said again. “And as with all human creations, they are a reflection of the time and the place where they were first made. Though far more advanced in many respects, the robots we use today are in their essentials identical to the first true robots made untold thousands of years ago. The robots we Spacers use today have brains whose basic design has remained unchanged from the days before humanity first entered space. They are tools made for a culture that had vanished before the first of the great underground Cities of Earth were built, before the first Spacers founded Aurora.

“I know that sounds incredible, but you need not take my word for it. Go look for yourself. If you research the dimmest recesses of the past, you will see it is so. Do not send your robots to find out for you. Go to your data panels and look for yourself. The knowledge is there. Look at the world and the time in which robots were born. You will see that the Three Laws were written in a very different time from ours.

“You will find repeated references to something called the Frankenstein Complex. This in turn is a reference to an ancient myth, now lost, wherein a deranged magician-scientist pulled parts from the dead bodies of condemned criminals and put them back together, reanimating the rotting body parts to create a much-feared monster. Some versions of the myth report the monster as actually a kind and gentle soul; others describe the monster as truly fierce and murderous. All versions agree that the monster was feared and hated by practically everyone. In most variants of the story, the creature and its creator are destroyed by a terrorized citizenry, who learn to be on the lookout for the inevitable moment when the whole story would be told again, when another necromancer would rediscover the secret of bringing decayed flesh back to life.

“That monster, ladies and gentlemen, was the popular mythic image of the robot at the time when the first actual robots were built. A thing made out of rotted, decayed human flesh, torn from the bodies of the dead. A perverted thing born with all the lowest and most evil impulses of humanity in its soul. The fear of this imaginary creature, superimposed on the real-life robots, was the Frankenstein Complex. I know it will be impossible to believe, but robots were seen not as utterly trustworthy mechanical servants, but as so many potential menaces, fearful threats. Men and women would snatch up their children and run away when robots—true robots, with the Three Laws ingrained in their positronic brains—came close.”

More mutterings of disbelief from the audience, but they were with her now, enthralled by the bizarre and ancient world she was describing. She was telling them of a past almost beyond their imagining, and they were fascinated. Even Kresh, there in the front row, seemed to have lost some of his ferocity.

“There is more,” Fredda said. “There is much more that we need to understand about the days when the Laws were written. For the first true robots were built in a world of universal fear and distrust, when the people of Earth found themselves organized into a handful of power blocs, each side armed with enough fearsome weapons to erase all life from the planet, each fearing one of the others would strike first. Ultimately the fact of the weapons themselves became the central political issue of the time, pushing all other moral and philosophical differences to one side. In order to keep its enemies from attacking, each side was obliged to build bigger, faster, better, stronger weapons.