David Deutch - The Fabric of Reality

The number of possible tennis games that can be played in a single environment — that is, rendered by a single program — is very large. Consider a rendering of the Centre Court at Wimbledon from the point of view of a player. Suppose, very conservatively, that in each second of the game the player can move in one of two perceptibly different ways (perceptibly, that is, to the player). Then after two seconds there are four possible games, after three seconds, eight possible games, and so on. After about four minutes the number of possible games that are perceptibly different from one another exceeds the number of atoms in the universe, and it continues to rise exponentially. For a program to render that one environment accurately, it must be capable of responding in any one of those myriad, perceptibly different ways, depending on how the player chooses to behave. If two programs respond in the same way to every possible action by the user, then they render the same environment; if they would respond perceptibly differently to even one possible action, they render different environments.

That remains so even if the user never happens to perform the action that shows up the difference. The environment a program renders (for a given type of user, with a given connecting cable) is a logical property of the program, independent of whether the program is ever executed. A rendered environment is accurate in so far as it would respond in the intended way to every possible action of the user. Thus its accuracy depends not only on experiences which users of it actually have, but also on experiences they do not have, but would have had if they had chosen to behave differently during the rendering. This may sound paradoxical, but as I have said, it is a straightforward consequence of the fact that virtual reality is, like reality itself, interactive.

This gives rise to an important difference between image generation and virtual-reality generation. The accuracy of an image generator’s rendering can in principle be experienced, measured and certified by the user, but the accuracy of a virtual-reality rendering never can be. For example, if you are a music-lover and know a particular piece well enough, you can listen to a performance of it and confirm that it is a perfectly accurate rendering, in principle down to the last note, phrasing, dynamics and all. But if you are a tennis fan who knows Wimbledon’s Centre Court perfectly, you can never confirm that a purported rendering of it is accurate. Even if you are free to explore the rendered Centre Court for however long you like, and to ‘kick’ it in whatever way you like, and even if you have equal access to the real Centre Court for comparison, you cannot ever certify that the program does indeed render the real location. For you can never know what would have happened if only you had explored a little more, or looked over your shoulder at the right moment. Perhaps if you had sat on the rendered umpire’s chair and shouted ‘fault!’, a nuclear submarine would have surfaced through the grass and torpedoed the Scoreboard.

On the other hand, if you find even one difference between the rendering and the intended environment, you can immediately certify that the rendering is inaccurate. Unless, that is, the rendered environment has some intentionally unpredictable features. For example, a roulette wheel is designed to be unpredictable. If we make a film of roulette being played in a casino, that film may be laid to be accurate if the numbers that are shown coming up in the film are the same numbers that actually came up when the film was made. The film will show the same numbers every time it is played: it is totally predictable. So an accurate image of an unpredictable environment must be predictable. But what does it mean for a virtual-reality rendering of a roulette wheel to be accurate? As before, it means that a user should not find it perceptibly different from the original. But this implies that the rendering must not behave identically to the original: if it did, either it or the original could be used to predict the other’s behaviour, and then neither would be unpredictable. Nor must it behave in the same way every time it is run. A perfectly rendered roulette wheel must be just as usable for gambling as a real one. Therefore it must be just as unpredictable. Also, it must be just as fair; that is, all the numbers must come up purely randomly, with equal probabilities.

How do we recognize unpredictable environments, and how do we confirm that purportedly random numbers are distributed fairly? We check whether a rendering of a roulette wheel meets its specifications in the same way that we check whether the real thing does: by kicking (spinning) it, and seeing whether it responds as advertised. We make a large number of similar observations and perform statistical tests on the outcomes. Again, however many tests we carry out, we cannot certify that the rendering is accurate, or even that it is probably accurate. For however randomly the numbers seem to come up, they may nevertheless fall into a secret pattern that would allow a user in the know to predict them. Or perhaps if we had asked out loud the date of the battle of Waterloo, the next two numbers that came up would invariably show that date: 18, 15. On the other hand, if the sequence that comes up looks unfair, we cannot know for sure that it is, but we might be able to say that the rendering is probably inaccurate. For example, if zero came up on our rendered roulette wheel on ten consecutive spins, we should conclude that we probably do not have an accurate rendering of a fair roulette wheel.

When discussing image generators, I said that the accuracy of a rendered image depends on the sharpness and other attributes of the user’s senses. With virtual reality that is the least of our problems. Certainly, a virtual-reality generator that renders a given environment perfectly for humans will not do so for dolphins or extraterrestrials. To render a given environment for a user with given types of sense organs, a virtual-reality generator must be physically adapted to such sense organs and its computer must be programmed with their characteristics. However, the modifications that have to be made to accommodate a given species of user are finite, and need only be carried out once. They amount to what I have called constructing a new ‘connecting cable’. As we consider environments of ever greater complexity, the task of rendering environments for a given type of user becomes dominated by writing the programs for calculating what those environments will do; the species-specific part of the task, being of fixed complexity, becomes negligible by comparison. This discussion is about the ultimate limits of virtual reality, so we are considering arbitrarily accurate, long and complex renderings. That is why it makes sense to speak of ‘rendering a given environment’ without specifying who it is being rendered for.

We have seen that there is a well-defined notion of the accuracy of a virtual-reality rendering: accuracy is the closeness, as far as is perceptible, of the rendered environment to the intended one. But it must be close for every possible way in which the user might behave, and that is why, no matter how observant one is when experiencing a rendered environment, one cannot certify that it is accurate (or probably accurate). But experience can sometimes show that a rendering is inaccurate (or probably inaccurate).

This discussion of accuracy in virtual reality mirrors the relationship between theory and experiment in science. There too, it is possible to confirm experimentally that a general theory is false, but never that it is true. And there too, a short-sighted view of science is that it is all about predicting our sense-impressions. The correct view is that, while sense-impressions always play a role, what science is about is understanding the whole of reality, of which only an infinitesimal proportion is ever experienced.