John Brockman - What Should We Be Worried About?

Groupism. The ideal of human rights—that the ultimate moral good is the flourishing of individual people, while groups are social constructions designed to further that good—is surprisingly recent and unnatural. People, at least in public, are apt to argue that the ultimate moral good is the glory of the group—the tribe, religion, nation, class, or race—and that individuals are expendable, like the cells of a body.

Perfect justice. Every group has suffered depredations and humiliations in its past. When groupism combines with the thirst for revenge, a group may feel justified in exacting damage on some other group, inflamed by a moralistic certitude that makes compromise tantamount to treason.

Utopian ideologies. If you have a religious or political vision of a world that will be infinitely good forever, any amount of violence is justified to bring about that world, and anyone standing in its way is infinitely evil and deserving of unlimited punishment.

Warfare as a normal or necessary tactic. Clausewitz characterized war as “the continuation of policy by other means.” Many political and religious ideologies go a step further and consider violent struggle to be the driver of dialectical progress, revolutionary liberation, or the realization of a messianic age.

THE RELATIVE PEACE we have enjoyed since 1945 is a gift of values and institutions that militate against these risks. Democracy selects for responsible stewards rather than charismatic despots. The ideal of human rights protects people from being treated as cannon fodder, collateral damage, or eggs to be broken for a revolutionary omelet. The maximization of peace and prosperity has been elevated over the rectification of historic injustices or the implementation of utopian fantasies. Conquest is stigmatized as “aggression” and becomes a taboo rather than a natural aspiration of nations or an everyday instrument of policy.

None of these protections is natural or permanent, and the possibility of their collapsing is what makes me worry. Perhaps some charismatic politician is working his way up the Chinese nomenklatura and dreams of overturning the intolerable insult of Taiwan once and for all. Perhaps an aging Putin will seek historical immortality and restore Russian greatness by swallowing a former Soviet republic or two. Perhaps a utopian ideology is fermenting in the mind of a cunning fanatic somewhere who will take over a major country and try to impose it elsewhere.

It’s natural to worry about physical stuff like weaponry and resources. What we should really worry about is psychological stuff like ideologies and norms. As the UNESCO slogan puts it, “Since wars begin in the minds of men, it is in the minds of men that the defenses of peace must be constructed.”

VERNOR VINGE

Mathematician; computer scientist; Hugo Award–winning novelist, A Fire upon the Deep, Rainbows End

There are many things we know to worry about. Some are very likely events but by themselves not existential threats to civilization. Others could easily destroy civilization and even life on Earth—but the chances of such disasters occurring in the near historical future seem to be vanishingly small.

There is a known possibility that stands out for being both likely in the next few decades and capable of destroying our civilization. It’s prosaic and banal, something dismissed by many as a danger that the 20th century confronted and definitively rejected: That is war between great nations, especially when fought under a doctrine of Mutually Assured Destruction (MAD).

Arguments against the plausibility of MAD warfare are especially believable these days: MAD war benefits no one. Twentieth-century U.S.A. and U.S.S.R., even in the depths of the MAD years, were sincerely desperate to avoid tipping over into MAD warfare. That sincerity is a big reason why humanity got through the century without general nuclear war.

Unfortunately, the 20th century is our only test case, and the MAD warfare threat has characteristics that made surviving the 20th century more a matter of luck than wisdom.

MAD involves very long time scales and very short ones. At the long end, the threat is driven by social and geopolitical issues in much the same way as with unintended wars of the past. At the other extreme, MAD involves complex automation controlling large systems, operating faster than any real-time human response, much less careful judgment.

Breakers (vandals, griefers) have more leverage than Makers (builders, creators), even though the Makers far outnumber the Breakers. This is the source of some of our greatest fears about technology—that if weapons of mass destruction are cheap enough, then the relatively small percentage of Breakers will be sufficient to destroy civilization. If that possibility is scary, then the MAD threat should be terrifying. For with MAD planning, it is hundreds of thousands of creative and ingenious people in the most powerful societies—many of the best of the Makers, powered by the riches of the planet—who work to create a mutually unsurvivable outcome! In the most extreme case, the resulting weapon systems must function on the shortest of time scales, thus moving the threat into the realm of thermodynamic inevitability.

For the time (decades?) in which we and our interests are undefendable and still confined to a volume smaller than the scope of our weapons, the threat of MAD warfare will be the winner in rankings of likely destructiveness.

There’s a lot we can do to mitigate the threat of MADness:

A resurrection of full-blown MAD planning will probably be visible to the general public. We should resist arguments that MAD doctrine is a safe strategy with regard to weapons of mass destruction.

We should study the dynamics of the beginning of unintended wars of the past—in particular, World War I. There are plenty of similarities between our time and the first few years of the last century. We have much optimism, the feeling that our era is different. And what about entangling alliances? Are there small players with the ability to bring heavyweights into the action? How does the possibility of n -way MADness affect these risks?

With all the things we have to worry about, there is also an overwhelmingly positive counterweight: billions of good, smart people and the databases and networks that now empower them. This is an intellectual force that trumps all institutions of the past. Humanity plus its automation is quite capable of anticipating and countering myriad possible calamities. If we can avoid blowing ourselves up, we will have time to create things so marvelous that their upside is (worrisomely!) beyond imagination.

MARTIN REES

Astronomer Royal; former president, the Royal Society; emeritus professor of cosmology & astrophysics, University of Cambridge; author, From Here to Infinity: A Vision for the Future of Science

Those of us fortunate enough to live in the developed world fret too much about minor hazards of everyday life: improbable air crashes, carcinogens in food, and so forth. But we are less secure than we think. We should worry far more about scenarios that have thankfully not yet happened—but which, if they occurred, could cause such worldwide devastation that even once would be too often.

Much has been written about possible ecological shocks triggered by the collective impact on the biosphere of a growing and more demanding world population, and about the social and political tensions stemming from scarcity of resources or climate change. But even more worrying are the downsides of powerful new technologies: cyber-, bio-, and nano-. We’re entering an era when a few individuals could, via error or terror, trigger a societal breakdown with such extreme suddenness that palliative government actions would be overwhelmed.