Макс Тегмарк - Life 3.0 - Being Human in the Age of Artificial Intelligence

This thought scared me, but it also strengthened my resolve to keep my New Year’s resolution. I felt that we needed one more person to complete our team of FLI founders, who’d spearhead a team of idealistic young volunteers. The logical choice was Viktoriya Krakovna, a brilliant Harvard grad student who’d not only won a silver medal in the International Mathematics Olympiad, but also founded the Citadel, a house for about a dozen young idealists who wanted reason to play a greater role in their lives and the world. Meia and I invited her over to our place five days later to tell her about our vision, and before we’d finished the sushi, FLI had been born.

The Puerto Rico Adventure

This marked the beginning of an amazing adventure, which still continues. As I mentioned in chapter 1, we held regular brainstorming meetings at our house with dozens of idealistic students, professors and other local thinkers, where the top-rated ideas transformed into projects—the first being that AI op-ed from chapter 1 with Stephen Hawking, Stuart Russell and Frank Wilczek that helped ignite the public debate. In parallel with the baby steps of setting up a new organization (such as incorporating, recruiting an advisory board and launching a website), we held a fun launch event in front of a packed MIT auditorium, at which Alan Alda explored the future of technology with leading experts.

Figure 9.2: Jaan Tallinn, Anthony Aguirre, yours truly, Meia Chita-Tegmark and Viktoriya Krakovna celebrate our incorporation of FLI with sushi on May 23, 2014.

We focused the rest of the year on pulling together the Puerto Rico conference which, as I mentioned in chapter 1, aimed to engage the world’s leading AI researchers in the discussion of how to keep AI beneficial. Our goal was to shift the AI-safety conversation from worrying to working: from bickering about how worried to be, to agreeing on concrete research projects that could be started right away to maximize the chance of a good outcome. To prepare, we collected promising AI-safety research ideas from around the world and sought community feedback on our growing project list. With the help of Stuart Russell and a group of hardworking young volunteers, especially Daniel Dewey, János Krámar and Richard Mallah, we distilled these research priorities into a document to be discussed at the conference.1 Building consensus that there was lots of valuable AI-safety research to be done would, we hoped, encourage people to start doing such research. The ultimate moonshot triumph would be if it could even persuade someone to fund it since, so far, there had been essentially no support for such work from government funding agencies.

Enter Elon Musk. On August 2, he appeared on our radar by famously tweeting “Worth reading Superintelligence by Bostrom. We need to be super careful with AI. Potentially more dangerous than nukes.” I reached out to him about our efforts, and got to speak with him by phone a few weeks later. Although I felt quite nervous and starstruck, the outcome was outstanding: he agreed to join our FLI scientific advisory board, to attend our conference and potentially to fund a first-ever AI-safety research program to be announced in Puerto Rico. This electrified all of us at FLI, and made us redouble our efforts to create an awesome conference, identify promising research topics and build community support for them.

I finally got to meet Elon in person for further planning when he came to MIT two months later for a space symposium. It felt very strange to be alone with him in a small green room just moments after he’d enraptured over a thousand MIT students like a rock star, but after a few minutes, all I could think of was our joint project. I instantly liked him. He radiated sincerity, and I was inspired by how much he cared about the long-term future of humanity—and how he audaciously turned his aspiration into actions. He wanted humanity to explore and settle our Universe, so he started a space company. He wanted sustainable energy, so he started a solar company and an electric-car company. Tall, handsome, eloquent and incredibly knowledgeable, it was easy to understand why people listened to him.

Unfortunately, this MIT event also taught me how fear-driven and divisive media can be. Elon’s stage performance consisted of an hour of fascinating discussion about space exploration, which I think would have made great TV. At the very end, a student asked him an off-topic question about AI. His answer included the phrase “with artificial intelligence, we are summoning the demon,” which became the only thing that most media reported—and generally out of context. It struck me that many journalists were inadvertently doing the exact opposite of what we were trying to accomplish in Puerto Rico. Whereas we wanted to build community consensus by highlighting the common ground, the media had an incentive to highlight the divisions. The more controversy they could report, the greater their Nielsen ratings and ad revenue. Moreover, whereas we wanted to help people from across the spectrum of opinions to come together, get along and understand each other better, media coverage inadvertently made people across the opinion spectrum upset at one another, fueling misunderstandings by publishing only their most provocative-sounding quotes without context. For this reason, we decided to ban journalists from the Puerto Rico meeting and impose the “Chatham House Rule,” which prohibits participants from subsequently revealing who said what. *

Although our Puerto Rico conference ended up being a success, it didn’t come easy. The countdown mostly required diligent prep work, for example me phoning or skyping large numbers of AI researchers to assemble a critical mass of participants to attract the other attendees, and there were also dramatic moments—such as when I got up by 7 a.m. on December 27 to reach Elon on a lousy phone connection to Uruguay, and was told “I don’t think this is gonna work.” He was concerned that an AI-safety research program might provide a false sense of security, enabling reckless researchers to forge ahead while paying lip service to safety. But then, despite the sound incessantly cutting out, we extensively talked through the huge benefits of mainstreaming the topic and getting more AI researchers working on AI safety. After the call dropped, he sent me one of my favorite emails ever: “Lost the call at the end there. Anyway, docs look fine. I’m happy to support the research with $5M over three years. Maybe we should make it $10M?”

Four days later, 2015 got off to a good start for Meia and me as we briefly relaxed before the meeting, dancing in the new year on a Puerto Rico beach illuminated by fireworks. The conference got off to a great start too: there was remarkable consensus that more AI-safety research was needed, and based on further input from the conference participants, that research priorities document we’d worked so hard on was improved and finalized. We passed around that safety-research-endorsing open letter from chapter 1, and were delighted that almost everyone signed it.

Meia and I had a magical meeting with Elon in our hotel room where he blessed the detailed plans for our grants program. She was touched by how down-to-earth and candid he was about his personal life, and how much interest he took in us. He asked us how we met, and liked Meia’s elaborate story. The next day, we filmed an interview with him about AI safety and why he wanted to support it and everything seemed on track.2

The conference climax, Elon’s donation announcement, was scheduled for 7 p.m. on Sunday, January 4, 2015, and I’d been so tense about it that I’d tossed and turned in my sleep the night before. And then, just fifteen minutes before we were supposed to head to the session where it would happen, we hit a snag! Elon’s assistant called and said that it looked like Elon might not be able to go through with the announcement, and Meia said she’d never seen me look more stressed or disappointed. Elon finally came by, and I could hear the seconds counting down to the session start as we sat there and talked. He explained that they were just two days away from a crucial SpaceX rocket launch where they hoped to pull off the first-ever successful landing of the first stage on a drone ship, and that since this was a huge milestone, the SpaceX team didn’t want to distract from it with concurrent media splashes involving him. Anthony Aguirre, cool and levelheaded as always, pointed out that this meant that nobody wanted media attention for this, neither Elon nor the AI community. We arrived a few minutes late to the session I was moderating, but we had a plan: no dollar amount would get mentioned, to ensure that the announcement wasn’t newsworthy, and I’d lord Chatham House over everyone to keep Elon’s announcement secret from the world for nine days if his rocket reached the space station, regardless of whether the landing succeeded; he said he’d need even more time if the rocket exploded on launch.