Bruce Mesquita - The Predictioneer's Game

I’m a political science professor at New York University, where I also run the Alexander Hamilton Center for Political Economy. The Center and all of my courses try to teach students how to solve problems with logic and evidence. The idea is to wean them from knee-jerk conclusions based on gut feel, personal opinions, simple linear reasoning, partisan preferences, or ideology. My colleagues at NYU and I are interested in training students to know how to address problems before they go out into the world. We don’t want them shaking things up without much insight into whether they’re helping to make matters better or worse.

Besides being a professor at NYU, I wear two other hats. I’m a senior fellow at the Hoover Institution at Stanford University. There my job is to think about finding solutions to policy problems. That side of my research is about putting the ideas I teach at NYU to good use by writing op-ed columns, articles, and books, some very technical and some, like this one, designed to spread the word. My third hat is as a partner in a small consuiting company, Mesquita & Roundell, LLC. M&R, as we call it, also uses some of the game theory models I’ve designed to advise people in the national security community and also in the private sector.

I didn’t set out to wear these three hats. The opportunity initially fell into my lap back in 1979 when an official at the State Department called me to ask my opinion about a government crisis in India. He wanted to know who was likely to be the next prime minister. At the time I was a professor of political science at the University of Rochester—where the application of game theory to political questions originated—and I had written my Ph.D. thesis at the University of Michigan about winning and losing strategies among India’s opposition parties. So the State Department official was asking me to be a pundit, to use my “expert” knowledge to speculate about the next Indian government.

It happened that I was on a Guggenheim fellowship at the time, working on a book about war. I had just designed a mathematical model for that project, as well as a little computer program to make calculations that were important for solving that model. The computer program provided a way to simulate decision making under stressful circumstances such as sometimes lead to war. It looked at the choices people could make and calculated the probability that they would get what they wanted if they chose one course of action (say, negotiations) or another (like war), weighting the probabilities by an estimate of how much the decision makers valued winning, losing, or intermediate compromise outcomes. Of course, it also recognized that they had to work out how others might respond to the choices they made.

Like every model, it needed data. The State Department’s phone call about India came in just as I was trying to figure out where to get data to feed into my war and peace model. The timing was perfect. The phone call got me thinking that maybe war and peace decisions really aren’t that different from everyday political confrontations. Sure, the stakes are higher—people get killed in wars—but then any politician seeking high office or about to lose high office sees the personal political stakes as pretty darn high. Probably all of us make similar calculations about how to advance our own well-being in any complex situation involving big risks and potentially big rewards, whether that involves politics, business, or daily life.

The State Department was pressing me for an answer and I wanted to help them. I also wanted to see how well my new model worked. I decided to find out whether the model could really be a useful tool to sort out the political infighting in India. Linking that model to Indian politics was a huge “Aha!” moment for me, one that would change the rest of my life.

I grabbed a yellow pad and picked my own brain, putting together the information the model needed. I wrote down a list of everyone I thought would try to influence the selection of India’s next government. For each of those people (political party leaders, members of India’s parliament, and some members of critical state governments) I also wrote down my estimate of how much clout each had, what their preference was between the various plausible candidates for prime minister, and how much they cared about trying to shape that choice. With just one page of my yellow pad filled with numbers, I had all the information the computer program needed to predict what would happen. I plugged those data into my little program and crunched the numbers overnight. When the computing was done the next morning—computers were slow in those days—I pored over the hundred or so pages of calculated values to see what the model’s predictions looked like.

I thought I had personal insight into what was going to happen in India. My “pundit” knowledge had led me to believe that a man named Jagjivan Ram would be the next prime minister. He was a popular and prominent politician who was better liked than his main rivals for the prime minister’s job. I was confident that he was untouchable—truly unbeatable—in the political arena, and not just in the sense of his caste status. He had paid his political dues and it seemed like his time had come. Many other India watchers thought the same thing. Imagine my surprise, then, when my computer program, written by me and fed only with my data, predicted an entirely different result. It predicted that Charan Singh would become prime minister and that he would include someone named Y. B. Chavan in his cabinet, and that they would gain support—albeit briefly—from Indira Gandhi, then the recently ousted prime minister. The model also predicted that the new Indian government would be incapable of governing and so would soon fall.

I found myself forced to choose between personal opinion and my commitment to logic and evidence as the basis for coming to conclusions about politics. I believed in the logic behind my model and I believed in the correctness of the data I had jotted down. After staring at the output, working out how my own program came to a conclusion so different from my personal judgment, I chose science over punditry. In fact, I told colleagues at Rochester what the model’s prediction was even before I reported back to the State Department. When I spoke with the official at State he was taken aback. He noted that no one else was suggesting this result and that it seemed strange at best. He asked me how I had come to this judgment. When I told him I’d used a computer program based on a model of decision making that I was designing, he just laughed and urged me not to repeat that to anyone.

A few weeks later, Charan Singh became the prime minister with Y. B. Chavan as his deputy prime minister, with support from Indira Gandhi. And a few months after that, Charan Singh’s government unraveled, Indira Gandhi withdrew her support, and a new election was called, just as the computer-generated forecast had indicated. This got me pretty excited. Here was a case where my personal judgment had been wrong, and yet my knowledge was the only source of information the computer model had. The model came up with the right answer and I didn’t. Clearly there were at least two possibilities: I was just lucky, or I was onto something.

Luck is great, but I’m not a great believer in luck alone as an explanation for results. Sure, rare events happen—rarely. I set out to push my model by testing it against lots of cases, hoping to learn whether it really worked. I applied it to prospective leadership changes in the Soviet Union; to questions of economic reform in Mexico and Brazil; and to budgetary decisions in Italy—that is, to wide-ranging questions about politics and economics. The model worked really well on these cases—so well, in fact, that it attracted the attention of people in the government who heard me present some of the analyses at academic conferences. Eventually this led to a grant from the Defense Advanced Research Projects Agency (DARPA), a research arm of the Department of Defense (and the sponsors of research that fostered the development of the Internet long before Al Gore “invented” it). They gave me seventeen issues to examine, and as it happened, the model—by then somewhat more sophisticated—got all seventeen right. Government analysts who provided the data the model needed—we’ll talk more about that later—didn’t do nearly as well. Confident that I was onto something useful, I started a small consulting company with a couple of colleagues who had their own ideas about how to predict big political events. Now, many years later, I operate a small consulting firm with my partner and former client, Harry Roundell. Harry, formerly a managing director at J. P. Morgan, and I apply a much more sophisticated version of my 1979 model to interesting business and government problems. We’ll see lots of examples in the pages to come.