Макс Тегмарк - Life 3.0 - Being Human in the Age of Artificial Intelligence

Because biologists and chemists once took a stand, their fields are now known mainly for creating beneficial medicines and materials rather than biological and chemical weapons. The AI and robotics communities had now spoken as well: the letter signatories also wanted their fields to be known for creating a better future, not for creating new ways of killing people. But will the main future use of AI be civilian or military? Although we’ve spent more pages in this chapter on the former, we may soon be spending more money on the latter—especially if a military AI arms race takes off. Civilian AI investment commitments exceeded a billion dollars in 2016, but this was dwarfed by the Pentagon’s fiscal 2017 budget request of $12–15 billion for AI-related projects, and China and Russia are likely to take note of what Deputy Defense Secretary Robert Work said when this was announced: “I want our competitors to wonder what’s behind the black curtain.”41

Should There Be an International Treaty?

Although there’s now a major international push toward negotiating some form of killer robot ban, it’s still unclear what will happen, and there’s a vibrant ongoing debate about what, if anything, should happen. Although many leading stakeholders agree that world powers should draft some form of international regulations to guide AWS research and use, there’s less agreement about what precisely should be banned and how a ban would be enforced. For example, should only lethal autonomous weapons be banned, or also ones that seriously injure people, say by blinding them? Would we ban development, production or ownership? Should a ban apply to all autonomous weapons systems or, as our letter said, only offensive ones, allowing defensive systems such as autonomous anti-aircraft guns and missile defenses? In the latter case, should AWS count as defensive even if they’re easy to move into enemy territory? And how would you enforce a treaty given that most components of an autonomous weapon have a dual civilian use as well? For example, there isn’t much difference between a drone that can deliver Amazon packages and one that can deliver bombs.

Some debaters have argued that designing an effective AWS treaty is hopelessly hard and that we therefore shouldn’t even try. On the other hand, John F. Kennedy emphasized when announcing the Moon missions that hard things are worth attempting when success will greatly benefit the future of humanity. Moreover, many experts argue that the bans on biological and chemical weapons were valuable even though enforcement proved hard, with significant cheating, because the bans caused severe stigmatization that limited their use.

I met Henry Kissinger at a dinner event in 2016, and got the opportunity to ask him about his role in the biological weapons ban. He explained how back when he was the U.S. national security adviser, he’d persuaded President Nixon that a ban would be good for U.S. national security. I was impressed by how sharp his mind and memory were for a ninety-two-year-old, and was fascinated to hear his inside perspective. Since the United States already enjoyed superpower status thanks to its conventional and nuclear forces, it had more to lose than to gain from a worldwide bioweapons arms race with uncertain outcome. In other words, if you’re already top dog, then it makes sense to follow the maxim “If it ain’t broke, don’t fix it.” Stuart Russell joined our after-dinner conversation, and we discussed how exactly the same argument can be made about lethal autonomous weapons: those who stand to gain most from an arms race aren’t superpowers but small rogue states and non-state actors such as terrorists, who gain access to the weapons via the black market once they’ve been developed.

Once mass-produced, small AI-powered killer drones are likely to cost little more than a smartphone. Whether it’s a terrorist wanting to assassinate a politician or a jilted lover seeking revenge on his ex-girlfriend, all they need to do is upload their target’s photo and address into the killer drone: it can then fly to the destination, identify and eliminate the person, and self-destruct to ensure that nobody knows who was responsible. Alternatively, for those bent on ethnic cleansing, it can easily be programmed to kill only people with a certain skin color or ethnicity. Stuart envisions that the smarter such weapons get, the less material, firepower and money will be needed per kill. For example, he fears bumblebee-sized drones that kill cheaply using minimal explosive power by shooting people in the eye, which is soft enough to allow even a small projectile to continue into the brain. Or they might latch on to the head with metal claws and then penetrate the skull with a tiny shaped charge. If a million such killer drones can be dispatched from the back of a single truck, then one has a horrifying weapon of mass destruction of a whole new kind: one that can selectively kill only a prescribed category of people, leaving everybody and everything else unscathed.

A common counterargument is that we can eliminate such concerns by making killer robots ethical—for example, so that they’ll only kill enemy soldiers. But if we worry about enforcing a ban, then how would it be easier to enforce a requirement that enemy autonomous weapons be 100% ethical than to enforce that they aren’t produced in the first place? And can one consistently claim that the well-trained soldiers of civilized nations are so bad at following the rules of war that robots can do better, while at the same time claiming that rogue nations, dictators and terrorist groups are so good at following the rules of war that they’ll never choose to deploy robots in ways that violate these rules?

Cyberwar

Another interesting military aspect of AI is that it may let you attack your enemy even without building any weapons of your own, through cyberwarfare. As a small prelude to what the future may bring, the Stuxnet worm, widely attributed to the U.S. and Israeli governments, infected fast-spinning centrifuges in Iran’s nuclear-enrichment program and caused them to tear themselves apart. The more automated society gets and the more powerful the attacking AI becomes, the more devastating cyberwarfare can be. If you can hack and crash your enemy’s self-driving cars, auto-piloted planes, nuclear reactors, industrial robots, communication systems, financial systems and power grids, then you can effectively crash his economy and cripple his defenses. If you can hack some of his weapons systems as well, even better.

We began this chapter by surveying how spectacular the near-term opportunities are for AI to benefit humanity—if we manage to make it robust and unhackable. Although AI itself can be used to make AI systems more robust, thereby aiding the cyberwar defense, AI can clearly aid the offense as well. Ensuring that the defense prevails must be one of the most crucial short-term goals for AI development—otherwise all the awesome technology we build can be turned against us!

Jobs and Wages

So far in this chapter, we’ve mainly focused on how AI will affect us as consumers, by enabling transformative new products and services at affordable prices. But how will it affect us as workers, by transforming the job market? If we can figure out how to grow our prosperity through automation without leaving people lacking income or purpose, then we have the potential to create a fantastic future with leisure and unprecedented opulence for everyone who wants it. Few people have thought longer and harder about this than economist Erik Brynjolfsson, one of my MIT colleagues. Although he’s always well-groomed and impeccably dressed, he has Icelandic heritage, and I sometimes can’t help imagine that he only recently trimmed back a wild red Viking beard and mane to blend in at our business school. He certainly hasn’t trimmed back his wild ideas, and he calls his optimistic job-market vision “Digital Athens.” The reason that the Athenian citizens of antiquity had lives of leisure where they could enjoy democracy, art and games was mainly that they had slaves to do much of the work. But why not replace the slaves with AI-powered robots, creating a digital utopia that everyone can enjoy? Erik’s AI-driven economy would not only eliminate stress and drudgery and produce an abundance of everything we want today, but it would also supply a bounty of wonderful new products and services that today’s consumers haven’t yet realized that they want.