Pedro Domingos - The Master Algorithm - How the Quest for the Ultimate Learning Machine Will Remake Our World

I’m grateful to everyone who read and commented on drafts of the book at various stages, including Mike Belfiore, Thomas Dietterich, Tiago Domingos, Oren Etzioni, Abe Friesen, Rob Gens, Alon Halevy, David Israel, Henry Kautz, Chloé Kiddon, Gary Marcus, Ray Mooney, Kevin Murphy, Franzi Roesner, and Ben Taskar. Thanks also to everyone who gave me pointers, information, or help of various kinds, including Tom Griffiths, David Heckerman, Hannah Hickey, Albert-László Barabási, Yann LeCun, Barbara Mones, Mike Morgan, Peter Norvig, Judea Pearl, Gregory Piatetsky-Shapiro, and Sebastian Seung.

I’m lucky to work in a very special place, the University of Washington’s Department of Computer Science and Engineering. I’m also grateful to Josh Tenenbaum, and to everyone in his group, for hosting the sabbatical at MIT during which I started this book. Thanks to Jim Levine, my indefatigable agent, for drinking the Kool-Aid (as he put it) and spreading the word; and to everyone at Levine Greenberg Rostan. Thanks to TJ Kelleher, my amazing editor, for helping make this a better book, chapter by chapter, line by line; and to everyone at Basic Books.

I’m indebted to the organizations that have funded my research over the years, including ARO, DARPA, FCT, NSF, ONR, Ford, Google, IBM, Kodak, Yahoo, and the Sloan Foundation.

Last and most, I thank my family for their love and support.

If this book whetted your appetite for machine learning and the issues surrounding it, you’ll find many suggestions in this section. Its aim is not to be comprehensive but to provide an entrance to machine learning’s garden of forking paths (as Borges put it). Wherever possible, I chose books and articles appropriate for the general reader. Technical publications, which require at least some computational, statistical, or mathematical background, are marked with an asterisk (*). Even these, however, often have large sections accessible to the general reader. I didn’t list volume, issue, or page numbers, since the web renders them superfluous; likewise for publishers’ locations.

If you’d like to learn more about machine learning in general, one good place to start is online courses. Of these, the closest in content to this book is, not coincidentally, the one I teach (www.coursera.org/course/machlearning). Two other options are Andrew Ng’s course (www.coursera.org/course/ml) and Yaser Abu-Mostafa’s (http://work.caltech.edu/telecourse.html). The next step is to read a textbook. The closest to this book, and one of the most accessible, is Tom Mitchell’s Machine Learning* (McGraw-Hill, 1997). More up-to-date, but also more mathematical, are Kevin Murphy’s Machine Learning: A Probabilistic Perspective* (MIT Press, 2012), Chris Bishop’s Pattern Recognition and Machine Learning* (Springer, 2006), and An Introduction to Statistical Learning with Applications in R ,* by Gareth James, Daniela Witten, Trevor Hastie, and Rob Tibshirani (Springer, 2013). My article “A few useful things to know about machine learning” ( Communications of the ACM , 2012) summarizes some of the “folk knowledge” of machine learning that textbooks often leave implicit and was one of the starting points for this book. If you know how to program and are itching to give machine learning a try, you can start from a number of open-source packages, such as Weka (www.cs.waikato.ac.nz/ml/weka). The two main machine-learning journals are Machine Learning and the Journal of Machine Learning Research . Leading machine-learning conferences, with yearly proceedings, include the International Conference on Machine Learning, the Conference on Neural Information Processing Systems, and the International Conference on Knowledge Discovery and Data Mining. A large number of machine-learning talks are available on http://videolectures.net. The www.KDnuggets.com website is a one-stop shop for machine-learning resources, and you can sign up for its newsletter to keep up-to-date with the latest developments.

An early list of examples of machine learning’s impact on daily life can be found in “Behind-the-scenes data mining,” by George John ( SIGKDD Explorations , 1999), which was also the inspiration for the “day-in-the-life” paragraphs of the prologue. Eric Siegel’s book Predictive Analytics (Wiley, 2013) surveys a large number of machine-learning applications. The term big data was popularized by the McKinsey Global Institute’s 2011 report Big Data: The Next Frontier for Innovation, Competition, and Productivity . Many of the issues raised by big data are discussed in Big Data: A Revolution That Will Change How We Live, Work, and Think , by Viktor Mayer-Schönberger and Kenneth Cukier (Houghton Mifflin Harcourt, 2013). The textbook I learned AI from is Artificial Intelligence ,* by Elaine Rich (McGraw-Hill, 1983). A current one is Artificial Intelligence: A Modern Approach , by Stuart Russell and Peter Norvig (3rd ed., Prentice Hall, 2010). Nils Nilsson’s The Quest for Artificial Intelligence (Cambridge University Press, 2010) tells the story of AI from its earliest days.

Nine Algorithms That Changed the Future , by John MacCormick (Princeton University Press, 2012), describes some of the most important algorithms in computer science, with a chapter on machine learning. Algorithms ,* by Sanjoy Dasgupta, Christos Papadimitriou, and Umesh Vazirani (McGraw-Hill, 2008), is a concise introductory textbook on the subject. The Pattern on the Stone , by Danny Hillis (Basic Books, 1998), explains how computers work. Walter Isaacson recounts the lively history of computer science in The Innovators (Simon & Schuster, 2014).

“Spreadsheet data manipulation using examples,”* by Sumit Gulwani, William Harris, and Rishabh Singh ( Communications of the ACM , 2012), is an example of how computers can program themselves by observing users. Competing on Analytics , by Tom Davenport and Jeanne Harris (HBS Press, 2007), is an introduction to the use of predictive analytics in business. In the Plex , by Steven Levy (Simon & Schuster, 2011), describes at a high level how Google’s technology works. Carl Shapiro and Hal Varian explain the network effect in Information Rules (HBS Press, 1999). Chris Anderson does the same for the long-tail phenomenon in The Long Tail (Hyperion, 2006).

The transformation of science by data-intensive computing is surveyed in The Fourth Paradigm , edited by Tony Hey, Stewart Tansley, and Kristin Tolle (Microsoft Research, 2009). “Machine science,” by James Evans and Andrey Rzhetsky ( Science , 2010), discusses some of the different ways computers can make scientific discoveries. Scientific Discovery: Computational Explorations of the Creative Processes ,* by Pat Langley et al. (MIT Press, 1987), describes a series of approaches to automating the discovery of scientific laws. The SKICAT project is described in “From digitized images to online catalogs,” by Usama Fayyad, George Djorgovski, and Nicholas Weir ( AI Magazine , 1996). “Machine learning in drug discovery and development,”* by Niki Wale ( Drug Development Research , 2001), gives an overview of just that. Adam, the robot scientist, is described in “The automation of science,” by Ross King et al. ( Science , 2009).

Sasha Issenberg’s The Victory Lab (Broadway Books, 2012) dissects the use of data analysis in politics. “How President Obama’s campaign used big data to rally individual votes,” by the same author ( MIT Technology Review , 2013), tells the story of its greatest success to date. Nate Silver’s The Signal and the Noise (Penguin Press, 2012) has a chapter on his poll aggregation method.