Макс Тегмарк - Life 3.0 - Being Human in the Age of Artificial Intelligence

These rules of thumb sometimes fail badly in situations that they weren’t designed to handle, such as when rats eat delicious-tasting rat poison, when moths get lured into glue traps by seductive female fragrances and when bugs fly into candle flames. *1Since today’s human society is very different from the environment evolution optimized our rules of thumb for, we shouldn’t be surprised to find that our behavior often fails to maximize baby making. For example, the subgoal of not starving to death is implemented in part as a desire to consume caloric foods, triggering today’s obesity epidemic and dating difficulties. The subgoal to procreate was implemented as a desire for sex rather than as a desire to become a sperm/egg donor, even though the latter can produce more babies with less effort.

Psychology: The Pursuit of and Rebellion Against Goals

In summary, a living organism is an agent of bounded rationality that doesn’t pursue a single goal, but instead follows rules of thumb for what to pursue and avoid. Our human minds perceive these evolved rules of thumb as feelings, which usually (and often without us being aware of it) guide our decision making toward the ultimate goal of replication. Feelings of hunger and thirst protect us from starvation and dehydration, feelings of pain protect us from damaging our bodies, feelings of lust make us procreate, feelings of love and compassion make us help other carriers of our genes and those who help them and so on. Guided by these feelings, our brains can quickly and efficiently decide what to do without having to subject every choice to a tedious analysis of its ultimate implications for how many descendants we’ll produce. For closely related perspectives on feelings and their physiological roots, I highly recommend the writings of William James and António Damásio.2

It’s important to note that when our feelings occasionally work against baby making, it’s not necessarily by accident or because we get tricked: our brain can rebel against our genes and their replication goal quite deliberately, for example by choosing to use contraceptives! More extreme examples of the brain rebelling against its genes include choosing to commit suicide or spend life in celibacy to become a priest, monk or nun.

Why do we sometimes choose to rebel against our genes and their replication goal? We rebel because by design, as agents of bounded rationality, we’re loyal only to our feelings. Although our brains evolved merely to help copy our genes, our brains couldn’t care less about this goal since we have no feelings related to genes—indeed, during most of human history, our ancestors didn’t even know that they had genes. Moreover, our brains are way smarter than our genes, and now that we understand the goal of our genes (replication), we find it rather banal and easy to ignore. People might realize why their genes make them feel lust, yet have little desire to raise fifteen children, and therefore choose to hack their genetic programming by combining the emotional rewards of intimacy with birth control. They might realize why their genes make them crave sweets yet have little desire to gain weight, and therefore choose to hack their genetic programming by combining the emotional rewards of a sweet beverage with zero-calorie artificial sweeteners.

Although such reward-mechanism hacks sometimes go awry, such as when people get addicted to heroin, our human gene pool has thus far survived just fine despite our crafty and rebellious brains. It’s important to remember, however, that the ultimate authority is now our feelings, not our genes. This means that human behavior isn’t strictly optimized for the survival of our species. In fact, since our feelings implement merely rules of thumb that aren’t appropriate in all situations, human behavior strictly speaking doesn’t have a single well-defined goal at all.

Engineering: Outsourcing Goals

Can machines have goals? This simple question has triggered great controversy, because different people take it to mean different things, often related to thorny topics such as whether machines can be conscious and whether they can have feelings. But if we’re more practical and simply take the question to mean “Can machines exhibit goal-oriented behavior?,” then the answer is obvious: “Of course they can, since we can design them that way!” We design mousetraps to have the goal of catching mice, dishwashers with the goal of cleaning dishes, and clocks with the goal of keeping time. When you confront a machine, the empirical fact that it’s exhibiting goal-oriented behavior is usually all you care about: if you’re chased by a heat-seeking missile, you don’t really care whether it has consciousness or feelings! If you still feel uncomfortable saying that the missile has a goal even if it isn’t conscious, you can for now simply read “purpose” when I write “goal”—we’ll tackle consciousness in the next chapter.

So far, most of what we build exhibits only goal-oriented design, not goal-oriented behavior: a highway doesn’t behave; it merely sits there. However, the most economical explanation for its existence is that it was designed to accomplish a goal, so even such passive technology is making our Universe more goal-oriented. Teleology is the explanation of things in terms of their purposes rather than their causes, so we can summarize the first part of this chapter by saying that our Universe keeps getting more teleological.

Not only can non-living matter have goals, at least in this weak sense, but it increasingly does . If you’d been observing Earth’s atoms since our planet formed, you’d have noticed three stages of goal-oriented behavior:

1. All matter seemed focused on dissipation (entropy increase).

2. Some of the matter came alive and instead focused on replication and subgoals of that.

3. A rapidly growing fraction of matter was rearranged by living organisms to help accomplish their goals.

Table 7.1 shows how dominant humanity has become from the physics perspective: not only do we now contain more matter than all other mammals except cows (which are so numerous because they serve our goals of consuming beef and dairy products), but the matter in our machines, roads, buildings and other engineering projects appears on track to soon overtake all living matter on Earth. In other words, even without an intelligence explosion, most matter on Earth that exhibits goal-oriented properties may soon be designed rather than evolved. Goal-Oriented Entities Billions of Tons5 × 10 30bacteria 400 Plants 400 10 15mesophelagic fish 10 1.3 × 10 9cows 0.5 7 × 10 9humans 0.4 10 14ants 0.3 1.7 × 10 6whales 0.0005 Concrete 100 Steel 20 Asphalt 15 1.2 × 10 9cars 2

Table 7.1: Approximate amounts of matter on Earth in entities that are evolved or designed for a goal. Engineered entities such as buildings, roads and cars appear on track to overtake evolved entities such as plants and animals.

This new third kind of goal-oriented behavior has the potential to be much more diverse than what preceded it: whereas evolved entities all have the same ultimate goal (replication), designed entities can have virtually any ultimate goal, even opposite ones. Stoves try to heat food while refrigerators try to cool food. Generators try to convert motion into electricity while motors try to convert electricity into motion. Standard chess programs try to win at chess, but there are also ones competing in tournaments with the goal of losing at chess.

There’s a historical trend for designed entities to get goals that are not only more diverse, but also more complex: our devices are getting smarter. We engineered our earliest machines and other artifacts to have quite simple goals, for example houses that aimed to keep us warm, dry and safe. We’ve gradually learned to build machines with more complex goals, such as robotic vacuum cleaners, self-flying rockets and self-driving cars. Recent AI progress has given us systems such as Deep Blue, Watson and AlphaGo, whose goals of winning at chess, winning at quiz shows and winning at Go are so elaborate that it takes significant human mastery to properly appreciate how skilled they are.