David Deutch - The Fabric of Reality

So it is not just science — reasoning about the physical world — that involves virtual reality. All reasoning, all thinking and all external experience are forms of virtual reality. These things are physical processes which so far have been observed in only one place in the universe, namely the vicinity of the planet Earth. We shall see in Chapter 8that all living processes involve virtual reality too, but human beings in particular have a special relationship with it. Biologically speaking, the virtual-reality rendering of their environment is the characteristic means by which human beings survive. In other words, it is the reason why human beings exist. The ecological niche that human beings occupy depends on virtual reality as directly and as absolutely as the ecological niche that koala bears occupy depends on eucalyptus leaves.

image generatorA device that can generate specifiable sensations for a user.

universal image generatorAn image generator that can be programmed to generate any sensation that the user is capable of experiencing.

external experienceAn experience of something outside one’s own mind.

Internal experienceAn experience of something within one’s own mind.

physically possibleNot forbidden by the laws of physics. An environment is physically possible if and only if it exists somewhere in the multiverse (on the assumption that the initial conditions and all other supplementary data of the multiverse are determined by some as yet unknown laws of physics).

logically possibleSelf-consistent.

virtual realityAny situation in which the user is given the experience of being in a specified environment.

repertoireThe repertoire of a virtual-reality generator is the set of environments that the generator can be programmed to give the user the experience of.

imageSomething that gives rise to sensations.

accuracyAn image is accurate in so far as the sensations it generates are close to the intended sensations. A rendered environment is accurate in so far as it would respond in the intended way to every possible action of the user.

perfect accuracyAccuracy so great that the user cannot distinguish the image or rendered environment from the intended one.

Virtual reality is not just a technology in which computers simulate the behaviour of physical environments. The fact that virtual reality is possible is an important fact about the fabric of reality. It is the basis not only of computation, but of human imagination and external experience, science and mathematics, art and fiction.

What are the ultimate limits — the full scope — of virtual reality (and hence of computation, science, imagination and the rest)? In the next chapter we shall see that in one respect the scope of virtual reality is unlimited, while in another it is drastically circumscribed.

The heart of a virtual-reality generator is its computer, and the question of what environments can be rendered in virtual reality must eventually come down to the question of what computations can be performed. Even today, the repertoire of virtual-reality generators is limited as much by their computers as by their image generators. Whenever a new, faster computer, with more memory and better image-processing hardware, is incorporated into a virtual-reality generator, the repertoire is enlarged. But will this always be so, or will we eventually encounter full universality, as I have argued we should expect to in the case of image generators? In other words, is there a single virtual-reality generator, buildable once and for all, that could be programmed to render any environment that the human mind is capable of experiencing?

Just as with image generators, we do not mean by this that it single virtual-reality generator might contain within itself the specifications of all logically possible environments. We mean only that for every logically possible environment it would be possible lo program the generator to render that environment. We can envisage encoding the programs on, for example, magnetic disks. The more complex the environment, the more disks may be needed in store the corresponding program. So to render complex environments the machine must have a mechanism, just as I have described for the universal image generator, that can read unlimited numbers of disks. Unlike an image generator, a virtual-reality generator may need a growing amount of ‘working memory’ to store the intermediate results of its calculations. We may envisage this as being provided in the form of blank disks. Once again, the fact that a machine needs to be supplied with energy, blank disks and maintenance does not prevent us from regarding it as a ‘single machine’, provided that these operations are not tantamount to changing the machine’s design, and are not forbidden by the laws of physics.

In this sense, then, a computer with an effectively unlimited memory capacity can be envisaged in principle. But a computer with an unlimited speed of computation cannot. A computer of given design will always have a fixed maximum speed, which only design changes can increase. Therefore a given virtual-reality generator will not be able to perform unlimited amounts of computation per unit time. Will this not limit its repertoire? If an environment is so complex that the computation of what the user should be seeing one second from now takes the machine more than one second to compute, how can the machine possibly render that environment accurately? To achieve universality, we need a further technological trick.

To extend its repertoire as far as is physically possible, a virtual-reality generator would have to take control of one further attribute of the user’s sensory system, namely the processing speed of the user’s brain. If the human brain were like an electronic computer, this would simply be a matter of changing the rate at which its ‘clock’ emits synchronizing pulses. No doubt the brain’s ‘clock’ will not be so easily controlled. But again this presents no problem of principle. The brain is a finite physical object, and all its functions are physical processes which in principle can be slowed down or stopped. The ultimate virtual-reality generator would have to be capable of doing that.

To achieve a perfect rendering of environments which call for a lot of computation, a virtual-reality generator would have to operate in something like the following way. Each sensory nerve is physically capable of relaying signals at a certain maximum rate, because a nerve cell which has fired cannot fire again until about one millisecond later. Therefore, immediately after a particular nerve has fired, the computer has at least one millisecond to decide whether, and when, that nerve should fire again. If it has computed that decision within, say, half a millisecond, no tampering with the brain’s speed is necessary, and the computer merely fires the nerve at the appropriate times. Otherwise, the computer causes the brain slow down (or, if necessary, to stop) until the calculation of what should happen next is complete; it then restores the brain’s normal speed. What would this feel like to the user? By definition, like nothing. The user would experience only the environment specified in the program, without any slowing down, stopping or restarting. Fortunately it is never necessary for a virtual-reality generator to make the brain operate faster than normal; that would eventually raise problems of principle because, among other things, no signal can travel faster than the speed of light.