Steve Brown - The Innovation Ultimatum

Figure 1.1 A simple neural network.

Typically, the more layers there are, and the more nodes in each layer, the more capable the neural network. Neural networks with many layers are known as “deep” neural networks. This is where the term deep learning comes from.

Every node in the hidden layers has both inputs and outputs. Each node is connected to every node in the previous layer and every node in the next layer. The value of each node is influenced by the values of all the nodes it is connected to in the previous layer. Here's the tricky bit: some nodes have a stronger influence on the value of proceeding nodes than others; their influence is weighted. The value of each node is the weighted sum of the values of the previous nodes. These weightings are determined during the training phase and collectively make up what is known as “the model.” The model determines the functionality of the neural network: different weightings, different functionality. Information passes across the network from the input layer to the output layer via this complex web of weighted interconnections.

Neural networks are trained with a process known as backpropagation, or “backprop” as it's known in the business. The details of how backprop works is beyond the scope of this book. At a high level, backprop is a computationally intensive statistical approach that compares the desired output of a neural network with the actual output and then tweaks the weightings in the network to improve the accuracy of results. When the right result is given, the weightings of all the pathways through the neural network that lead to the correct result are strengthened. If the result was incorrect, the pathways that lead to the incorrect result are weakened. Over time, with exposure to more and more data, the model becomes increasingly accurate. The network “learns” the correct complex associations between inputs and outputs.

To train a neural network to read radiology charts and look for tumors, you would expose it to many example charts (the input), each tagged with a radiologist's diagnosis—tumor or no tumor (the desired output). The output of the network is a single number, the probability that an image contains a tumor. Each time the neural net is exposed to a new image, the output of the network is compared with the correct result. If an image of a tumor is presented, the result should be close to 100%. If there's no tumor the result should be close to zero. The backprop process is used to tweak the network's model (the weightings of the connections between the nodes), strengthening the weightings of links that lead to the correct result, and weakening those that don't. Once trained with enough data, the neural network will predict the right diagnosis with impressive accuracy. A more complex network might have several outputs. One could be the percentage chance of a tumor, another the probability of an embolism, another the probability of a broken bone, and so on.

If this all seems too difficult to understand, that's okay. The key thing to understand is that neural nets can infer how to perform tasks from examples, without the need of a domain expert to supply explicit rules on how to perform that task.

Radiologists train for many years to read x-rays, computer tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET) images. After medical school, radiologists do additional training, often involving a four-year residency. Some do additional specialization training after that. Reading images to look for tumors and other ailments uses all of the radiologist's skill, experience, and training. Yet this task is within the reach of a neural network. Given enough training data, an AI can be built with diagnostic abilities similar to those of a human radiologist, a person with about a decade of intense education behind them. As we train the neural network, we are essentially codifying the collective knowledge, and several decades of professional experience, from hundreds of thousands of radiologists. Their experience and diagnostic insight are captured in the model that's generated.

Some radiologists already use AI-based tools to offer a “second opinion” as they read charts. As the accuracy of these tools surpasses that of human radiologists on routine charts, radiologists will be able to focus their attention on more complex, higher-value, and more patient-centered tasks and procedures. The progress made in radiology portends the future for other branches of medicine. Machine learning will be applied to many other fields of medical diagnosis and pathology in the coming decade.

While training an AI requires serious amounts of computing performance to create a model, using that model requires significantly less performance. The process of using a model is known as inference. Often, training occurs on workstations or in the cloud, while inference occurs on devices. Most future computer chips will include inference engines, silicon accelerators optimized to run AI models with relative ease.

Pattern recognition is a core capability of many AI systems, including the radiology example we just discussed. Pattern recognition has many applications and comes in a range of different flavors. It's not important that you remember all these different approaches. They are listed here only to illustrate some of the fundamental capabilities of machine learning. As you read through them, think about how such a capability might be used to solve business problems in your organization.

Classification. AI can classify data into similar types. For example, the radiology AI classifies images as positive or negative. A similar approach might be used to do visual inspection and quality assurance in a manufacturing plant, or to identify spoiled or underripe fruit at a fruit-packing plant.

Clustering. Marketing professionals use clustering algorithms to partition consumers into market segments that share similar characteristics—buying habits, affluence level, and needs or desires. Recommendation engines use clustering, too. Spotify recommends songs that you might enjoy by analyzing historical listening habits. A clustering algorithm finds the complex relationships between songs and listeners. The clustering algorithm might see that I like songs A, B, C, and D, and that you like songs B, C, D, and E. It may conclude that it's probable you will enjoy song A and I might enjoy song E. Clustering is useful to deliver personalized experiences.

Regression analysis finds patterns that describe relationships between pieces of data. For example, regression analysis might observe that if Event A happens, most of the time Event B follows. More complex relationships are found, too, such as “if Datapoint A is below a certain threshold, and Event B and Event C are not happening, then Event D is 46% more likely to occur.” This approach is used to make predictions about the future with predictive analytics tools. Regression analysis is used by Walmart to predict how sales of certain food items are influenced by specific weather conditions.

Sequence labeling is a pattern-recognition approach used in speech recognition, handwriting recognition, and gesture recognition. Sequence labeling is used to break sentences down into constituent words and phrases and to label them in a way that captures their context. For example, sequence labeling identifies which words are nouns, verbs, and proper names. Words are best interpreted in the broader context of a sentence. Sequence-labeling algorithms classify words within a sentence, or cursive letters within a handwritten word, by examining the broader context surrounding them.