Макс Тегмарк - Life 3.0 - Being Human in the Age of Artificial Intelligence

In most cases, this technology-driven trend has made large entities parts of an even grander structure while retaining much of their autonomy and individuality, although commentators have argued that adaptation of entities to hierarchical life has in some cases reduced their diversity and made them more like indistinguishable replaceable parts. Some technologies, such as surveillance, can give higher levels in the hierarchy more power over their subordinates, while other technologies, such as cryptography and online access to free press and education, can have the opposite effect and empower individuals.

Although our present world remains stuck in a multipolar Nash equilibrium, with competing nations and multinational corporations at the top level, technology is now advanced enough that a unipolar world would probably also be a stable Nash equilibrium. For example, imagine a parallel universe where everyone on Earth shares the same language, culture, values and level of prosperity, and there is a single world government wherein nations function like states in a federation and have no armies, merely police enforcing laws. Our present level of technology would probably suffice to successfully coordinate this world—even though our present population might be unable or unwilling to switch to this alternative equilibrium.

What will happen to the hierarchical structure of our cosmos if we add superintelligent AI technology to this mix? Transportation and communication technology will obviously improve dramatically, so a natural expectation is that the historical trend will continue, with new hierarchical levels coordinating over ever-larger distances—perhaps ultimately encompassing solar systems, galaxies, superclusters and large swaths of our Universe, as we’ll explore in chapter 6. At the same time, the most fundamental driver of decentralization will remain: it’s wasteful to coordinate unnecessarily over large distances. Even Stalin didn’t try to regulate exactly when his citizens went to the bathroom. For superintelligent AI, the laws of physics will place firm upper limits on transportation and communication technology, making it unlikely that the highest levels of the hierarchy would be able to micromanage everything that happens on planetary and local scales. A superintelligent AI in the Andromeda galaxy wouldn’t be able to give you useful orders for your day-to-day decisions given that you’d need to wait over five million years for your instructions (that’s the round-trip time for you to exchange messages traveling at the speed of light). In the same way, the round-trip travel time for a message crossing Earth is about 0.1 second (about the timescale on which we humans think), so an Earth-sized AI brain could have truly global thoughts only about as fast as a human one. For a small AI performing one operation each billionth of a second (which is typical of today’s computers), 0.1 second would feel like four months to you, so for it to be micromanaged by a planet-controlling AI would be as inefficient as if you asked permission for even your most trivial decisions through transatlantic letters delivered by Columbus-era ships.

This physics-imposed speed limit on information transfer therefore poses an obvious challenge for any AI wishing to take over our world, let alone our Universe. Before Prometheus broke out, it put very careful thought into how to avoid mind fragmentation, so that its many AI modules running on different computers around the world had goals and incentives to coordinate and act as a single unified entity. Just as the Omegas faced a control problem when they tried to keep Prometheus in check, Prometheus faced a self-control problem when it tried to ensure that none of its parts would revolt. We clearly don’t yet know how large a system an AI will be able to control directly, or indirectly through some sort of collaborative hierarchy—even if a fast takeoff gave it a decisive strategic advantage.

In summary, the question of how a superintelligent future will be controlled is fascinatingly complex, and we clearly don’t know the answer yet. Some argue that things will get more authoritarian; others claim that it will lead to greater individual empowerment.

Cyborgs and Uploads

A staple of science fiction is that humans will merge with machines, either by technologically enhancing biological bodies into cyborgs (short for “cybernetic organisms”) or by uploading our minds into machines. In his book The Age of Em, economist Robin Hanson gives a fascinating survey of what life might be like in a world teeming with uploads (also known as emulations, nicknamed Ems ). I think of an upload as the extreme end of the cyborg spectrum, where the only remaining part of the human is the software. Hollywood cyborgs range from visibly mechanical, such as the Borg from Star Trek, to androids almost indistinguishable from humans, such as the Terminators. Fictional uploads range in intelligence from human-level as in the Black Mirror episode “White Christmas” to clearly superhuman as in Transcendence .

If superintelligence indeed comes about, the temptation to become cyborgs or uploads will be strong. As Hans Moravec puts it in his 1988 classic Mind Children: “Long life loses much of its point if we are fated to spend it staring stupidly at ultra-intelligent machines as they try to describe their ever more spectacular discoveries in baby-talk that we can understand.” Indeed, the temptation of technological enhancement is already so strong that many humans have eyeglasses, hearing aids, pacemakers and prosthetic limbs, as well as medicinal molecules circulating in their bloodstreams. Some teenagers appear to be permanently attached to their smartphones, and my wife teases me about my attachment to my laptop.

One of today’s most prominent cyborg proponents is Ray Kurzweil. In his book The Singularity Is Near, he argues that the natural continuation of this trend is using nanobots, intelligent biofeedback systems and other technology to replace first our digestive and endocrine systems, our blood and our hearts by the early 2030s, and then move on to upgrading our skeletons, skin, brains and the rest of our bodies during the next two decades. He guesses that we’re likely to keep the aesthetics and emotional import of human bodies, but will redesign them to rapidly change their appearance at will, both physically and in virtual reality (thanks to novel brain-computer interfaces). Moravec agrees with Kurzweil that cyborgization would go far beyond merely improving our DNA: “a genetically engineered superhuman would be just a second-rate kind of robot, designed under the handicap that its construction can only be by DNA-guided protein synthesis.” Further, he argues that we’ll do even better by eliminating the human body entirely and uploading minds, creating a whole-brain emulation in software. Such an upload can live in a virtual reality or be embodied in a robot capable of walking, flying, swimming, space-faring or anything else allowed by the laws of physics, unencumbered by such everyday concerns as death or limited cognitive resources.

Although these ideas may sound like science fiction, they certainly don’t violate any known laws of physics, so the most interesting question isn’t whether they can happen, but whether they will happen and, if so, when. Some leading thinkers guess that the first human-level AGI will be an upload, and that this is how the path toward superintelligence will begin. *

However, I think it’s fair to say that this is currently a minority view among AI researchers and neuroscientists, most of whom guess that the quickest route to superintelligence is to bypass brain emulation and engineer it in some other way—after which we may or may not remain interested in brain emulation. After all, why should our simplest path to a new technology be the one that evolution came up with, constrained by requirements that it be self-assembling, self-repairing and self-reproducing? Evolution optimizes strongly for energy efficiency because of limited food supply, not for ease of construction or understanding by human engineers. My wife, Meia, likes to point out that the aviation industry didn’t start with mechanical birds. Indeed, when we finally figured out how to build mechanical birds in 2011,1 more than a century after the Wright brothers’ first flight, the aviation industry showed no interest in switching to wing-flapping mechanical-bird travel, even though it’s more energy efficient—because our simpler earlier solution is better suited to our travel needs.