Макс Тегмарк - Life 3.0 - Being Human in the Age of Artificial Intelligence

In lockstep with this surge of research came a surge of opinions being expressed, both individually and collectively. The industry Partnership on AI published its founding tenets, and long reports with lists of recommendations were published by the U.S. government, Stanford University and the IEEE (the world’s largest organization of technical professionals), together with dozens of other reports and position papers from elsewhere.9

We were eager to facilitate meaningful discussion among the Asilomar attendees and learn what, if anything, this diverse community agreed on. Lucas Perry therefore took on the heroic task of reading all of those documents we’d found and extracting all their opinions. In a marathon effort initiated by Anthony Aguirre and concluded by a series of long telecons, our FLI team then attempted to group similar opinions together and strip away redundant bureaucratic verbiage to end up with a single list of succinct principles, also including unpublished but influential opinions that had been expressed more informally in talks and elsewhere. But this list still included plenty of ambiguity, contradiction and room for interpretation, so the month before the conference, we shared it with the participants and collected their opinions and suggestions for improved or novel principles. This community input produced a significantly revised principle list for use at the conference.

In Asilomar, the list was further improved in two steps. First, small groups discussed the principles they were most interested in (figure 9.4), producing detailed refinements, feedback, new principles and competing versions of old ones. Finally, we surveyed all attendees to determine the level of support for each version of each principle.

Figure 9.3: Groups of great minds ponder AI principles in Asilomar.

This collective process was both exhaustive and exhausting, with Anthony, Meia and I curtailing sleep and lunch time at the conference in our scramble to compile everything needed in time for the next steps. But it was also exciting. After such detailed, thorny and sometimes contentious discussions and such a wide range of feedback, we were astonished by the high level of consensus that emerged around many of the principles during that final survey, with some getting over 97% support. This consensus allowed us to set a high bar for inclusion in the final list: we kept only principles that at least 90% of the attendees agreed on. Although this meant that some popular principles were dropped at the last minute, including some of my personal favorites,10 it enabled most of the participants to feel comfortable endorsing all of them on the sign-up sheet that we passed around the auditorium. Here’s the result.

The Asilomar AI Principles

Artificial intelligence has already provided beneficial tools that are used every day by people around the world. Its continued development, guided by the following principles, will offer amazing opportunities to help and empower people in the decades and centuries ahead. RESEARCH ISSUES

§1 Research Goal: The goal of AI research should be to create not undirected intelligence, but beneficial intelligence.

§2 Research Funding: Investments in AI should be accompanied by funding for research on ensuring its beneficial use, including thorny questions in computer science, economics, law, ethics, and social studies, such as:

(a) How can we make future AI systems highly robust, so that they do what we want without malfunctioning or getting hacked?

(b) How can we grow our prosperity through automation while maintaining people’s resources and purpose?

(c) How can we update our legal systems to be more fair and efficient, to keep pace with AI, and to manage the risks associated with AI?

(d) What set of values should AI be aligned with, and what legal and ethical status should it have?

§3 Science-Policy Link: There should be constructive and healthy exchange between AI researchers and policy-makers.

§4 Research Culture: A culture of cooperation, trust, and transparency should be fostered among researchers and developers of AI.

§5 Race Avoidance: Teams developing AI systems should actively cooperate to avoid corner-cutting on safety standards. ETHICS AND VALUES

§6 Safety: AI systems should be safe and secure throughout their operational lifetime, and verifiably so where applicable and feasible.

§7 Failure Transparency: If an AI system causes harm, it should be possible to ascertain why.

§8 Judicial Transparency: Any involvement by an autonomous system in judicial decision-making should provide a satisfactory explanation auditable by a competent human authority.

§9 Responsibility: Designers and builders of advanced AI systems are stakeholders in the moral implications of their use, misuse, and actions, with a responsibility and opportunity to shape those implications.

§10 Value Alignment: Highly autonomous AI systems should be designed so that their goals and behaviors can be assured to align with human values throughout their operation.

§11 Human Values: AI systems should be designed and operated so as to be compatible with ideals of human dignity, rights, freedoms, and cultural diversity.

§12 Personal Privacy: People should have the right to access, manage, and control the data they generate, given AI systems’ power to analyze and utilize that data.

§13 Liberty and Privacy: The application of AI to personal data must not unreasonably curtail people’s real or perceived liberty.

§14 Shared Benefit: AI technologies should benefit and empower as many people as possible.

§15 Shared Prosperity: The economic prosperity created by AI should be shared broadly, to benefit all of humanity.

§16 Human Control: Humans should choose how and whether to delegate decisions to AI systems, to accomplish human-chosen objectives.

§17 Non-subversion: The power conferred by control of highly advanced AI systems should respect and improve, rather than subvert, the social and civic processes on which the health of society depends.

§18 AI Arms Race: An arms race in lethal autonomous weapons should be avoided. LONGER-TERM ISSUES

§19 Capability Caution: There being no consensus, we should avoid strong assumptions regarding upper limits on future AI capabilities.

§20 Importance: Advanced AI could represent a profound change in the history of life on Earth, and should be planned for and managed with commensurate care and resources.

§21 Risks: Risks posed by AI systems, especially catastrophic or existential risks, must be subject to planning and mitigation efforts commensurate with their expected impact.

§22 Recursive Self-Improvement: AI systems designed to recursively self-improve or self-replicate in a manner that could lead to rapidly increasing quality or quantity must be subject to strict safety and control measures.

§23 Common Good: Superintelligence should only be developed in the service of widely shared ethical ideals, and for the benefit of all humanity rather than one state or organization.

The signature list grew dramatically after we posted the principles online, and by now it includes an amazing list of more than a thousand AI researchers and many other top thinkers. If you too want to join as a signatory, you can do so here: http://futureoflife.org/ai-principles.