Douglas Hofstadter - The Mind’s I - Fantasies and Reflections on Self and Soul

PAT: Except for your loaded phrase “the machine tries,” this sounds very interesting. But how do you know that this test will get at the essence of thinking? Maybe it’s testing for the wrong things. Maybe, just to take a random illustration, someone would feel that a machine was able to think only if it could dance so well that you couldn’t tell it was a machine. Or someone else could suggest some other characteristic. What’s so sacred about being able to fool people by typing at them?

SANDY: I don’t see how you can say such a thing. I’ve heard that objection before, but frankly it baffles me. So what if the machine can’t tap-dance or drop a rock on your toe? If it can discourse intelligently on any subject you want, then it has shown it can think—to me, at least! As I see it, Turing has drawn, in one clean stroke, a clear division between thinking and other aspects of being human.

PAT: Now you’re the baffling one. If one couldn’t conclude anything from a man’s ability to win at the Imitation Game, how could one conclude anything from a machines ability to win at the Turing game?

CHRIS: Good question.

SANDY: It seems to me that you could conclude something from a man’s win in the Imitation Game. You wouldn’t conclude he was a woman, but you could certainly say he had good insights into the feminine mentality (if there is such a thing). Now, if a computer could fool someone into thinking it was a person, I guess you’d have to say something similar about it—that it had good insights into what it’s like to be human, into the “human condition” (whatever that is).

PAT: maybe, but that isn’t necessarily equivalent to thinking, is it? It seems to me that passing the Turing test would merely prove that some machine or other could do a very good job of simulating thought.

CHRIS: I couldn’t agree more with Pat. We all know that fancy computer programs exist today for simulating all sorts of complex phenomena. In physics, for instance, we simulate the behaviour of particles, atoms, solids, liquids, gases, galaxies, and so on. But nobody confuses any of those simulations with the real thing!

SANDY: In his book Brainstorms, the philosopher Daniel Dennett makes a similar point about simulated hurricanes.

CHRIS: That’s a nice example too. Obviously, what goes on inside a computer when it’s simulating a hurricane is not a hurricane, for the machine’s memory doesn’t get torn to bits by 200-mile-an-hour winds, the floor of the machine room doesn’t get flooded with rainwater, and so on.

SANDY: Oh, come on—that’s not a fair argument! In the first place, the programmers don’t claim the simulation really is a hurricane. It’s merely a simulation of certain aspects of a hurricane. But in the second place, you’re pulling a fast one when you imply that there are no downpours or 200-mile-an-hour winds in a simulated hurricane. To us there aren’t any—but if the program were incredibly detailed, it could include simulated people on the ground who would experience the wind and the rain, just as we do when a hurricane hits. In their minds—or, if you prefer, in their simulated minds—the hurricane would not be a simulation, but a genuine phenomenon complete with drenching and devastation.

CHRIS: Oh, boy—what a science-fiction scenario! Now we’re talking about simulating whole populations, not just a single mind.

SANDY: Well, look—I’m simply trying to show you why your argument that a simulated McCoy isn’t the real McCoy is fallacious. It depends on the tacit assumption that any old observer of the simulated phenomenon is equally able to assess what’s going on. But, in act, it may take an observer with a special vantage point to recognize what is going on. In this case, it takes special “computational glasses” to see the rain, and the winds, and so on.

PAT: “Computational Glasses”? I don’t know what you’re talking about!

SANDY: I mean that to see the winds and the wetness of the hurricane, you have to be able to look at it in the proper way. You —

CHRIS: No, no, no! A simulated hurricane isn’t wet! No matter how much it might seem wet to simulated people, it won’t ever be genuinely wet! And no computer will ever get torn apart in the process of simulating winds!

SANDY: Certainly not, but you’re confusing levels. The laws of physics don’t get torn apart by real hurricanes either. In the case of the simulated hurricane, if you go peering at the computer’s memory expecting to find broken wires and so forth, you’ll be disappointed. But, look at the proper level. Look into the structures that are coded for in the memory. You’ll see some abstract links have been broken, some values of variables radically changed, and so forth. There’s your flood, your devastation—real, only a little concealed, a little hard to detect.

CHRIS: I’m sorry, I just can’t buy that. You’re insisting that I look for a new kind of devastation, a kind never before associated with hurricanes. Using this idea, you could call anything a hurricane, as long as its effects, seen through your special “glasses,” could be called “floods and devastation.”

SANDY: Right—you’ve got it exactly! You recognize a hurricane by its effects. You have no way of going in and finding some ethereal “essence of hurricane,” some “hurricane soul,” located right in the middle of its eye! It’s the existence of a certain kind of pattern —a spiral storm with an eye, and so forth that makes you say it’s a hurricane. Of course there are a lot of things that you’ll insist on before you call something a hurricane.

PAT: Well, wouldn’t you say that being an atmospheric phenomenon is one vital prerequisite? How can anything inside a computer be a storm? To me, a simulation is a simulation is a simulation!

SANDY: Then I suppose you would say that even the calculations that computers do are simulated—that they are fake calculations. Only people can do genuine calculations, right?

PAT: Well, computers get the right answers, so their calculations are not exactly fake—but they’re still just patterns. There’s no understanding going on in there. Take a cash register. Can you honestly say that you feel it is calculating something when its gears turn on each other? And a computer is just a fancy cash register, as I understand it.

SANDY: If you mean that a cash register doesn’t feel like a schoolkid doing arithmetic problems, I’ll agree. But is that what “calculation” means? Is that an integral part of it? If so, the contrary to what everybody has thought till now, we’ll have to write a very complicated program to perform genuine calculations. Of course, this program will sometimes get careless and make mistakes and it will sometimes scrawl its answers illegibly, and it will occasionally doodle on its paper… It won’t be more reliable than the post office clerk who adds up your total by hand. Now, I happen to believe eventually such a program could be written. Then we’d know something about how post office clerks and schookids work.

PAT: I can’t believe you could ever do that.

SANDY: Maybe, maybe not, but that’s not my point. You say a cash register can’t calculate. It reminds me of another favourite passage from Dennett’s Brainstorms —a rather ironic one, which is why I like it. The passage goes something like this: “Cash registers can’t really calculate, they can only spin their gears. But cash registers can’t really spin their gears either; they can only follow the laws of physics.” Dennett said it originally about computers. I modified it to talk about cash registers. And you could use the same line of reasoning in talking about people: “People can’t really calculate; all they can do is manipulate mental symbols. But they aren’t really manipulating symbols; all they are doing is firing various neurons in various patterns. But they can’t really make the neurons fire, they simply have to let the laws of physics make them fire for them.” Et cetera. Don’t you see how this Dennett-inspired reduction ad absurdum would lead you to conclude that calculation doesn’t exist, hurricanes don’t exist, nothing at a higher level than particles and the laws of physics exists? What do you gain by saying a computer only pushes symbols around and doesn’t truly calculate?