Eliezer Yudkowsky - Harry Potter and the Methods of Rationality

Again it was a shock. But what would the Hufflepuffs find in me , who never belonged in their House? Acid words, cutting wit, disdain for their inability to keep up with me?

Now it was the Hat's thoughts that were slow, hesitant. "I must Sort for the good of all the students in all the Houses... but I think you could learn to be a good Hufflepuff, and not too out of place there. You will be happier in Hufflepuff than in any other house; that is the truth."

Happiness is not the most important thing in the world to me. I would not become all that I could be, in Hufflepuff. I would sacrifice my potential.

The Hat flinched; Harry could feel it somehow. It was like he had kicked the hat in the balls - in a strongly weighted component of its utility function.

Why are you trying to send me where I do not belong?

The Hat's thought was almost a whisper. "I cannot speak of the others to you - but do you think that you are the first potential Dark Lord to pass under my brim? I cannot know the individual cases, but I can know this: Of those who did not intend evil from the very beginning, some of them listened to my warnings, and went to Houses where they would find happiness. And some of them... some of them did not."

That stopped Harry. But not for long. And of those who did not heed the warning - did they all become Dark Lords? Or did some of them achieve greatness for good, as well? Just what are the exact percentages here?

"I cannot give you exact statistics. I cannot know them so I cannot count them. I just know that your chances don't feel good. They feel very not-good."

But I just wouldn't do that! Ever!

"I know that I have heard that claim before."

I am not Dark Lord material!

"Yes, you are. You really, really are."

Why? Just because I once thought it would be cool to have a legion of brainwashed followers chanting 'Hail the Dark Lord Harry'?

"Amusing, but that was not your first fleeting thought before you substituted something safer, less damaging. No, what you remembered was how you considered lining up all the blood purists and guillotining them. And now you are telling yourself you were not serious, but you were. If you could do it this very moment and no one would ever know, you would. Or what you did this morning to Neville Longbottom, deep inside you knew that was wrong but you did it anyway because it was fun and you had a good excuse and you thought the Boy-Who-Lived could get away with it -"

That's unfair! Now you're just dragging up inner fears that aren't necessarily real! I worried that I might be thinking like that, but in the end I decided it would probably work to help Neville -

"That was, in fact, a rationalisation. I know. I cannot know what the true outcome will be for Neville - but I know what was truly happening inside your head. The decisive pressure was that it was such a clever idea you couldn't stand not to do it, never mind Neville's terror."

It was like a hard punch to Harry's entire self. He fell back, rallied:

Then I won't do that again! I'll be extra careful not to turn evil!

"Heard it."

Frustration was building up inside Harry. He wasn't used to being outgunned in arguments, at all, ever, let alone by a Hat that could borrow all of his own knowledge and intelligence to argue with him and could watch his thoughts as they formed. Just what kind of statistical summary do your 'feelings' come from, anyway? Do they take into account that I come from an Enlightenment culture, or were these other potential Dark Lords the children of spoiled Dark Age nobility, who didn't know squat about the historical lessons of how Lenin and Hitler actually turned out, or about the evolutionary psychology of self-delusion, or the value of self-awareness and rationality, or -

"No, of course they were not in this new reference class which you have just now constructed in such a way as to contain only yourself. And of course others have pleaded their own exceptionalism, just as you are doing now. But why is it necessary? Do you think that you are the last potential wizard of Light in the world? Why must you be the one to try for greatness, when I have advised you that you are riskier than average? Let some other, safer candidate try!"

But the prophecy...

"You don't really know that there's a prophecy. It was originally a wild guess on your part, or to be more precise, a wild joke, and McGonagall could have been reacting only to the part about the Dark Lord still being alive. You have essentially no idea of what the prophecy says or even if there is one. You're just speculating, or to put it more exactly, wishing that you have some ready-made heroic role that is your personal property."

But even if there is no prophecy, I'm the one who defeated him last time.

"That was almost certainly a wild fluke unless you seriously believe that a one-year-old child had an inherent propensity to defeat Dark Lords which has been maintained ten years later. None of this is your real reason and you know it!"

The answer to this was something that Harry would not regularly have said out loud, in conversation he would have danced around it and found some more socially palatable arguments to the same conclusion -

"You think that you are potentially the greatest who has yet lived, the strongest servant of the Light, that no other is likely to take up your wand if you lay it down."

Well... yeah, frankly. I don't usually come out and say it like that, but yeah. No point in softening it, you can read my mind anyway.

"To the extent you really believe that... you must equally believe that you could be the most terrible Dark Lord the world has ever known."

Destruction is always easier than creation. Easier to tear things apart, to disrupt, than to put them back together again. If I have the potential to accomplish good on a massive scale, I must also have the potential to accomplish still greater evil... But I won't do that.

"Already you insist on risking it! Why are you so driven? What is the real reason you must not go to Hufflepuff and be happier there? What is your true fear?"

I must achieve my full potential. If I don't I... fail...

"What happens if you fail?"

Something terrible...

"What happens if you fail?"

I don't know!

"Then it should not be frightening. What happens if you fail?"

I DON'T KNOW! BUT I KNOW THAT IT'S BAD!

There was silence for a moment in the caverns of Harry's mind.

"You know - you aren't letting yourself think it, but in some quiet corner of your mind you know just exactly what you aren't thinking - you know that by far the simplest explanation for this unverbalisable fear of yours is just the fear of losing your fantasy of greatness, of disappointing the people who believe in you, of turning out to be pretty much ordinary, of flashing and fading like so many other child prodigies..."

No, Harry thought desperately, no, it's something more, it comes from somewhere else, I know there's something out there to be afraid of, some disaster I have to stop...

"How could you possibly know about something like that?"

Harry screamed it with the full power of his mind: NO, AND THAT'S FINAL!

Then the voice of the Sorting Hat came slowly:

"So you will risk becoming a Dark Lord, because the alternative, to you, is certain failure, and that failure means the loss of everything. You believe that in your heart of hearts. You know all the reasons for doubting this belief, and they have failed to move you."