Eliezer Yudkowsky - Harry Potter and the Methods of Rationality

Stared out the glass panes at the big bright burning thing, the fluffy things, the mysterious endless blue in which they were embedded, that strange new unknown place.

It... actually did help, it helped quite a lot, to think that his own troubles were nothing compared to being in Azkaban. That there were people in the world who were really in trouble and Harry Potter was not one of them.

What was he going to do about Azkaban?

What was he going to do about magical Britain?

...which side was he on, now?

In the bright light of day, everything that Albus Dumbledore had said certainly sounded a lot wiser than Professor Quirrell. Better and brighter, more moral, more convenient, wouldn't it be nice if it were true. And the thing to remember was that Dumbledore believed things because they sounded nice, but Professor Quirrell was the one who was sane .

(Again the catch in his breathing, it happened each time he thought of Professor Quirrell.)

But just because something sounded nice, didn't make it wrong, either.

And if the Defense Professor did have a flaw in his sanity, it was that his outlook on life was too negative.

Really? inquired the part of Harry that had read eighteen million experimental results about people being too optimistic and overconfident. Professor Quirrell is too pessimistic? So pessimistic that his expectations routinely undershoot reality? Stuff him and put him in a museum, he's unique. Which one of you two planned the perfect crime, and then put in all the error margin and fallbacks that ended up saving your butt, just in case the perfect crime went wrong? Hint hint, his name wasn't Harry Potter.

But "pessimistic" wasn't the correct word to describe Professor Quirrell's problem - if a problem it truly was, and not the superior wisdom of experience. But to Harry it looked like Professor Quirrell was constantly interpreting everything in the worst possible light. If you handed Professor Quirrell a glass that was 90% full, he'd tell you that the 10% empty part proved that no one really cared about water.

That was a very good analogy, now that Harry thought about it. Not all of magical Britain was like Azkaban, that glass was well over half full...

Harry stared up at the bright blue sky.

...although, following the analogy, if Azkaban existed, then maybe it did prove that the 90% good part was there for other reasons, people trying to make a show of kindness as Professor Quirrell had put it. For if they were truly kind they would not have made Azkaban, they would storm the fortress to tear it down... wouldn't they?

Harry stared up at the bright blue sky. If you wanted to be a rationalist you had to read an awful lot of papers on flaws in human nature, and some of those flaws were innocent logical failures, and some of them looked a lot darker.

Harry stared up at the bright blue sky, and thought of the Milgram experiment.

Stanley Milgram had done it to investigate the causes of World War II, to try to understand why the citizens of Germany had obeyed Hitler.

So he had designed an experiment to investigate obedience, to see if Germans were, for some reason, more liable to obey harmful orders from authority figures.

First he'd run a pilot version of his experiment on American subjects, as a control.

And afterward he hadn't bothered trying it in Germany.

Experimental apparatus: A series of 30 switches set in a horizontal line, with labels starting at '15 volts' and going up to '450 volts', with labels for each group of four switches. The first group of four labeled 'Slight Shock', the sixth group labeled 'Extreme Intensity Shock', the seventh group labeled 'Danger: Severe Shock', and the two last switches left over labeled just 'XXX'.

And an actor, a confederate of the experimenter, who had appeared to the true subjects to be someone just like them: someone who had answered the same ad for participants in an experiment on learning, and who had lost a (rigged) lottery and been strapped into a chair, along with the electrodes. The true experimental subjects had been given a slight shock from the electrodes, just so that they could see that it worked.

The true subject had been told that the experiment was on the effects of punishment on learning and memory, and that part of the test was to see if it made a difference what sort of person administered the punishment; and that the person strapped to the chair would try to memorize sets of word pairs, and that each time the 'learner' got one wrong, the 'teacher' was to administer a successively stronger shock.

At the 300-volt level, the actor would stop trying to call out answers and begin kicking at the wall, after which the experimenter would instruct the subjects to treat non-answers as wrong answers and continue.

At the 315-volt level the pounding on the wall would be repeated.

After that nothing would be heard.

If the subject objected or refused to press a switch, the experimenter, maintaining an impassive demeanor and dressed in a gray lab coat, would say 'Please continue', then 'The experiment requires that you continue', then 'It is absolutely essential that you continue', then 'You have no other choice, you must go on'. If the fourth prod still didn't work, the experiment halted there.

Before running the experiment, Milgram had described the experimental setup, and then asked fourteen psychology seniors what percentage of subjects they thought would go all the way up to the 450-volt level, what percentage of subjects would press the last of the two switches marked XXX, after the victim had stopped responding.

The most pessimistic answer had been 3%.

The actual number had been 26 out of 40.

The subjects had sweated, groaned, stuttered, laughed nervously, bitten their lips, dug their fingernails into their flesh. But at the experimenter's prompting, they had, most of them, gone on administering what they believed to be painful, dangerous, possibly lethal electrical shocks. All the way to the end.

Harry could hear Professor Quirrell laughing, in his mind; the Defense Professor's voice saying something along the lines of: Why, Mr. Potter, even I had not been so cynical; I knew men would betray their most cherished principles for money and power, but I did not realize that a stern look also sufficed.

It was dangerous, to try and guess at evolutionary psychology if you weren't a professional evolutionary psychologist; but when Harry had read about the Milgram experiment, the thought had occurred to him that situations like this had probably arisen many times in the ancestral environment, and that most potential ancestors who'd tried to disobey Authority were dead. Or that they had, at least, done less well for themselves than the obedient. People thought themselves good and moral, but when push came to shove, some switch flipped in their brain, and it was suddenly a lot harder to heroically defy Authority than they thought. Even if you could do it, it wouldn't be easy, it wouldn't be some effortless display of heroism. You would tremble, your voice would break, you would be afraid; would you be able to defy Authority even then?

Harry blinked, then; because his brain had just made the connection between Milgram's experiment and what Hermione had done on her first day of Defense class, she'd refused to shoot a fellow student, even when Authority had told her that she must, she had trembled and been afraid but she had still refused. Harry had seen that happen right in front of his own eyes and he still hadn't made the connection until now...

Harry stared down at the reddening horizon, the Sun was sinking lower, the sky fading, darkening, even if most of it was still blue, soon it would turn to night. The gold and red colors of Sun and sunset reminded him of Fawkes; and Harry wondered, for a moment, if it must be a sad thing to be a phoenix, and call and cry and scream without being heeded.