Pedro Domingos: The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World

A Brief History of Life , by Richard Hawking (Random Penguin, 1982), summarizes the quantum leaps of evolution in the eons BC. (Before Computers. Just kidding.) The Singularity Is Near , by Ray Kurzweil (Penguin, 2005), is your guide to the transhuman future. Joel Garreau considers three different scenarios for how human-directed evolution will unfold in Radical Evolution (Broadway Books, 2005). In What Technology Wants (Penguin, 2010), Kevin Kelly argues that technology is the continuation of evolution by other means. Darwin Among the Machines , by George Dyson (Basic Books, 1997), chronicles the evolution of technology and speculates on where it will lead. Craig Venter explains how his team synthesized a living cell in Life at the Speed of Light (Viking, 2013).

Abagnale, Frank, Jr., 177, 306

aboutthedata.com, 272

A/B testing, 227, 309

Accuracy, 75-79, 87, 241, 243

Ackley, David, 103

Action potentials, 95-96, 104-105

Acxiom, 272

Adam, the robot scientist, 16, 84, 299

Adaptive systems, 8. See also Machine learning

AdSense system, 160

AI. See Artificial intelligence (AI)

AIDS vaccine, Bayesian networks and, 159-160

Alchemy, 246-259, 309

Markov logic networks and, 246-250

shortcomings, 255-259

tribes of machine learning and, 250-255

alchemy.cs.washington.edu, 250

Algorithms

classifiers, 86-87

complexity monster and, 5-6

defined, 1

designing, 4-5

further readings, 298-299

genetic, 122-128

overview, 1-6

structure mapping, 199-200

See also Machine learning; individual algorithms

AlphaDog, 21

Amazon, 198, 266, 291

A/B testing and, 227

data gathering, 211, 271, 272

machine learning and, 11, 12

Mechanical Turk, 14

recommendations, 12-13, 42, 184, 268, 286

Analogical reasoning, 179, 197

Analogizers, 51, 53, 54, 172-173

Alchemy and, 253-254

case-based reasoning, 197-200

dimensionality, 186-190

Master Algorithm and, 240-241

nearest-neighbor algorithm, 178-186

similiarity and, 179

support vector machines, 53, 190-196

symbolists vs., 200-202

Analogy, 175-179, 197-200

AND gate, 96

AND operation, 2

Anna Karenina (Tolstoy), 66

Apple, 272

Aristotle, 58, 64, 72, 178, 243

Artificial intelligence (AI)

human control of, 282-284

knowledge engineers and, 35-36

machine learning and, 8, 89-90

ASIC (application-specific integrated circuit) design, 49

Asimov, Isaac, 232, 280

Assumptions

ill-posed problem and, 64

of learners, 44

learning from finite data and, 24-25

prior, 174

simplifying to reduce number of probabilities, 150

symbolists and, 61-62

Atlantic (magazine), 273-274

AT &T, 272

Attribute selection, 186-187, 188-189

Attribute weights, 189

Auditory cortex, 26

Autoencoder, 116-118

Automation, machine learning and, 10

Automaton, 123

The Average American (O’Keefe), 206

Average member, 206

Axon, 95

Babbage, Charles, 28

Backpropagation (backprop), 52, 104, 107-111, 115, 302

Alchemy and, 252

genetic algorithms vs., 128

neural networks and, 112-114

reinforcement learning and, 222

Bagging, 238

Baldwin, J. M., 138-139

Baldwin effect, 139, 140, 304

Bandit problems, 129-130

Barto, Andy, 221

Bayes, Thomas, 144-145

Bayesian learning, 166-170, 174-175

Bayesian methods, cell model and, 114

Bayesian model averaging, 166-167

Bayesian models, tweaking probabilities, 170-173

Bayesian networks, 24, 156-161, 305-306

Alchemy and, 250

gene regulation and, 159

inference problem and, 161-166

Master Algorithm and, 240, 245

relational learning and, 231

Bayesians, 51, 52-53, 54, 143-175

Alchemy and, 253

further reading, 304-305

hidden Markov model, 154-155

If … then … rules and, 155-156

inference problem, 161-166

learning and, 166-170

logic and probability and, 173-175

Markov chain, 153-155

Markov networks, 170-173

Master Algorithm and, 240-241, 242

medical diagnosis and, 149-150

models and, 149-153

nature and, 141

probabilistic inference and, 52, 53

See also Bayesian networks

Bayes’ theorem, 31-32, 52-53, 143-149, 253

Beam search, 135

“Beer and diapers” rule, 69-70

Belief, probability and, 149

Belief propagation, 161-164, 242, 253

Bell Labs, 190

Bellman, Richard, 188, 220

Bellman’s equation, 220

Berkeley, George, 58

Berlin, Isaiah, 41

Bias, 78-79

Bias-free learning, futility of, 64

Bias-variance decomposition, 301

The Bible Code (Drosnin), 72

Big data, 21

A/B testing and, 227

algorithms and, 7

clustering and, 206-207

relational learning and, 232-233

science, machine learning, and, 14-16

scientific truth and, 40

Big-data systems, 258

Bing, 12

Biology, learning algorithms and, 15

Black swans, 38-39, 158, 232

The Black Swan (Taleb), 38

Blessing of nonuniformity, 189

Board games, reinforcement learning and, 219

Bohr, Niels, 178, 199

Boltzmann distribution, 103-104

Boltzmann machines, 103-104, 117, 250

Boole, George, 104, 175

Boolean circuits, 123, 136

Boolean variable, 149

Boosting, 238

Borges, Jorge Luis, 71

Box, George, 151

Brahe, Tycho, 14, 131

Brahe phase of science, 39-40

Brain

learning algorithms and, 26-28

mapping, 118

number of connections in, 94-95

reverse engineering the, 52, 302

S curves and, 105

simulating with computer, 95

spin glasses and, 102-103

BRAIN initiative, 118

Breiman, Leo, 238

Brin, Sergey, 55, 227, 274

Bryson, Arthur, 113

Bucket brigade algorithm, 127

Building blocks, 128-129, 134

Buntine, Wray, 80

Burglar alarms, Bayesian networks and, 157-158

Burks, Arthur, 123

Burns, Bob, 206

Business, machine learning and, 10-13

C. elegans , 118

Cajal, Santiago Ramón y, 93-94

Caltech, 170

CancerCommons.org, 261

Cancer cure

algorithm for, 53-54

Bayesian learning and, 174

inverse deduction and, 83-85

Markov logic network and, 249

program for (CanceRx), 259-261, 310

Cancer diagnosis, 141

Cancer drugs

predicting efficacy of, 83-84

relational learning and models for, 233

selection of, 41-42

CanceRx, 259-261, 310

Capital One, 272

Carbonell, Jaime, 69

Carnap, Rudolf, 175

Cars

driverless, 113, 166, 172, 306

learning to drive, 113

Case-based reasoning, 198, 307

Catch Me If You Can (film), 177

Cause and effect, Bayes’ theorem and, 145-149

Cell

model of, 114-115

relational learning and workings of, 233

Cell assembly, 94

Cell phone, hidden Markov models and, 155

Centaurs, 277

Central Dogma, 83

Cerebellum, 27, 118

Chance, Bayes and, 145

Chaos, study of, 30

Checkers-playing program, 219

Cholera outbreak, London’s, 182-183

Chomsky, Noam, 36-38

Chrome, 266

Chunking, 223-227, 254, 309

Circuit design, genetic programming and, 135-136

Classes, 86-87, 209, 257

Classifiers, 86-87, 127

Master Algorithm and, 240

Naïve Bayes, 151-153

nearest-neighbor algorithm and, 183

Clinton, Bill, 18