Eliezer Yudkowsky - Harry Potter and the Methods of Rationality

"You do?" said Harry. "Gosh, who would have figured."

"Yes indeed," said Dumbledore. "I suppose it is a little predictable, isn't it?" His face turned solemn. "Nonetheless..."

Dumbledore went back to his desk and sat down, pulling out one of the drawers as he did so. He reached in using both arms, and, straining slightly, pulled a rather large and heavy-looking object out of the drawer, which he then deposited on his oaken desk with a huge thunk.

"This," Dumbledore said, "was your father's rock."

Harry stared at it. It was light gray, discolored, irregularly shaped, sharp-edged, and very much a plain old ordinary large rock. Dumbledore had deposited it so that it rested on the widest available cross-section, but it still wobbled unstably on his desk.

Harry looked up. "This is a joke, right?"

"It is not," said Dumbledore, shaking his head and looking very serious. "I took this from the ruins of James and Lily's home in Godric's Hollow, where also I found you; and I have kept it from then until now, against the day when I could give it to you."

In the mixture of hypotheses that served as Harry's model of the world, Dumbledore's insanity was rapidly rising in probability. But there was still a substantial amount of probability allocated to other alternatives... "Um, is it a magical rock?"

"Not so far as I know," said Dumbledore. "But I advise you with the greatest possible stringency to keep it close about your person at all times."

All right. Dumbledore was probably insane but if he wasn't ... well, it would be just too embarrassing to get in trouble from ignoring the advice of the inscrutable old wizard. That had to be like #4 on the list of the Top 100 Obvious Failure Modes.

Harry stepped forward and put his hands on the rock, trying to find some angle from which to lift it without cutting himself. "I'll put it in my pouch, then."

Dumbledore frowned. "That may not be close enough to your person. And what if your mokeskin pouch is lost, or stolen?"

"You think I should just carry a big rock everywhere I go?"

Dumbledore gave Harry a serious look. "That might prove wise."

"Ah..." Harry said. It looked rather heavy. "I'd think the other students would tend to ask me questions about that."

"Tell them I ordered you to do it," said Dumbledore. "No one will question that, since they all think I'm insane." His face was still perfectly serious.

"Er, to be honest if you go around ordering your students to carry large rocks I can kind of see why people would think that."

"Ah, Harry," said Dumbledore. The old wizard gestured, a sweep of one hand that seemed to take in all the mysterious instruments around the room. "When we are young we believe that we know everything, and so we believe that if we see no explanation for something, then no explanation exists. When we are older we realise that the whole universe works by a rhythm and a reason, even if we ourselves do not know it. It is only our own ignorance which appears to us as insanity."

"Reality is always lawful," said Harry, "even if we don't know the law."

"Precisely, Harry," said Dumbledore. "To understand this - and I see that you do understand it - is the essence of wisdom."

"So... why do I have to carry this rock exactly?"

"I can't think of a reason, actually," said Dumbledore.

"...you can't."

Dumbledore nodded. "But just because I can't think of a reason doesn't mean there is no reason."

The instruments ticked on.

"Okay," said Harry, "I'm not even sure if I should be saying this, but that is simply not the correct way to deal with our admitted ignorance of how the universe works."

"It isn't?" said the old wizard, looking surprised and disappointed.

Harry had the feeling this conversation was not going to work out in his favour, but he carried on regardless. "No. I don't even know if that fallacy has an official name, but if I had to make one up myself, it would be 'privileging the hypothesis' or something like that. How can I put this formally... um... suppose you had a million boxes, and only one of the boxes contained a diamond. And you had a box full of diamond-detectors, and each diamond-detector always went off in the presence of a diamond, and went off half the time on boxes that didn't have a diamond. If you ran twenty detectors over all the boxes, you'd have, on average, one false candidate and one true candidate left. And then it would just take one or two more detectors before you were left with the one true candidate. The point being that when there are lots of possible answers, most of the evidence you need goes into just locating the true hypothesis out of millions of possibilities - bringing it to your attention in the first place. The amount of evidence you need to judge between two or three plausible candidates is much smaller by comparison. So if you just jump ahead without evidence and promote one particular possibility to the focus of your attention, you're skipping over most of the work. Like, you live in a city where there are a million people, and there's a murder, and a detective says, well, we've got no evidence at all, so have we considered the possibility that Mortimer Snodgrass did it?"

"Did he?" said Dumbledore.

"No," said Harry. "But later it turns out that the murderer had black hair, and Mortimer has black hair, so everyone's like, ah, looks like Mortimer did it after all. So it's unfair to Mortimer for the police to promote him to their attention without having good reasons already in hand to suspect him. When there are lots of possibilities, most of the work goes into just locating the true answer - starting to pay attention to it. You don't need proof , or the sort of official evidence that scientists or courts demand, but you need some sort of hint , and that hint has to discriminate that particular possibility from the millions of others. Otherwise you can't just pluck the right answer out of thin air. You can't even pluck a possibility worth thinking about out of thin air. And there's got to be a million other things I could do besides carrying around my father's rock. Just because I'm ignorant about the universe doesn't mean that I'm unsure about how I should reason in the presence of my uncertainty. The laws for thinking with probabilities are no less iron than the laws that govern old-fashioned logic, and what you just did is not allowed. " Harry paused. " Unless , of course, you have some hint you're not mentioning."

"Ah," said Dumbledore. He tapped his cheek, looking thoughtful. "An interesting argument, certainly, but doesn't it break down at the point where you make an analogy between a million potential murderers only one of whom committed the murder, and taking one out of many possible courses of action, when many possible courses of action may all be wise? I do not say that carrying your father's rock is the one best possible course of action, only that it is wiser to do than not."

Dumbledore once again reached into the same desk drawer he had accessed earlier, this time seeming to root around inside - at least his arm seemed to be moving. "I will remark," Dumbledore said while Harry was still trying to sort out how to reply to this completely unexpected rejoinder, "that it is a common misconception of Ravenclaws that all the smart children are Sorted there, leaving none for other Houses. This is not so; being Sorted to Ravenclaw indicates that you are driven by your desire to know things, which is not at all the same quality as being intelligent." The wizard was smiling as he bent over the drawer. "Nonetheless, you do seem rather intelligent. Less like an ordinary young hero and more like a young mysterious ancient wizard. I think I may have been taking the wrong approach with you, Harry, and that you may be able to understand things that few others could grasp. So I shall be daring, and offer you a certain other heirloom."