Zachary Jarvinen - Enterprise AI For Dummies

Machine learning (ML), a subset of artificial intelligence, enables users to learn from historical data to achieve a desired outcome. It powers targeted ads, personalized content, song recommendations, predictive maintenance activities, and virtual assistants.

ML mimics human learning by absorbing information. Humans learn by reading, watching, listening, and doing. ML learns by processing historical data. For example, a human’s knowledge of elephants is based on historical experience, such as going to the zoo, riding an elephant, watching a documentary, and reading a book. ML gains knowledge of elephants by processing text and images.

The learning phase consists of these steps:

1 Sample historical data (machine activity, customer attributes, and transactions).

2 Apply algorithm to historical data to learn key patterns and trends.

3 Generate a model or set of rules or instructions.

The prediction phase consists of these steps:

1 Load the existing model.

2 Apply the model to new data.

3 Predict the likelihood of an outcome (in other words, customer churn).

The output of the prediction phase feeds back into the input of the learning phase to refine the model.

For the purposes of ML, historical data is called training data. In the case of text mining, the system uses OCR and NLP to process text. For images, the system uses computer vision techniques for detection, recognition, and identification to process the image.

The algorithm processes the data to detect key patterns and trends and correlate them to labels. For example, if you’re doing text mining, the algorithm might notice certain words being associated with elephants, such as large, gray, tusk, and trunk, and associate those with the label “elephant.” Later, in the prediction phase, when the algorithm sees a significant number of these terms, it calculates the probability that the passage is talking about an elephant.

In the learning phase, the system applies statistical techniques or algorithms to the historical data to generate a machine-learning model. You can think of the model as a set of rules or instructions (similar to steps in a recipe) that one must follow to make a business decision.

For example, to approve a loan application, a loan officer considers income, age, net worth, and many other factors. Each attribute of the application is a rule or factor that the officer must evaluate to approve or reject the loan. Machine-learning techniques follow a similar process, comparing various attributes, historical decisions, and the outcome of similar applicants to estimate the credit worthiness of the new applicant. Table 1-3shows how machine learning is like a recipe.

In the prediction stage, the system uses the model to process new data (not historical data), detect patterns and trends, and attempt to match them to patterns from the learning data.

TABLE 1-3Machine Learning as a Recipe

For example, if you process a brochure for the San Diego Zoo using the model, it would recognize the content about elephants and add the tag “elephant” to the document along with a score. The result is a prediction in the form of the percentage probability that the document contains information about elephants. Basically, the model makes a data-driven guess.

In AI and data science, execution is not just implementing a plan. The methodology establishes an iterative process of learning, discovering, and then acting based on new information as opposed to a more traditional IT model of formulating a plan or idea and then rolling it out as planned.

Auto-classification is a machine-learning technique that automates tedious, error-prone tasks such as classifying information for storage and retrieval or answering a question. In a world where the amount of information stored digitally is expected to double every two years well into the next decade, auto-classification makes the difference between using that information and being overwhelmed by it.

Auto-classification uses two machine-learning methods, supervised classification and unsupervised classification, for two different purposes.

Machine learning via supervised classification uses exemplars of known document types to classify new documents in a two-step process:

1 Train the algorithm using known, manually classified content.

2 Classify new content using the trained algorithm.

In a stable content environment, AI teams use supervised classification to set custom classification models specific to a particular application or organization. This method requires human intervention to select the training data and optimize the model, and thus requires substantial involvement and effort in the early phases of the project, but yields predictable, accurate results.

Machine learning via unsupervised classification uses clustering and association algorithms to discover relationships in a heterogeneous dataset:

Clustering algorithms identify commonalities in the data, such as textual content or data format, and extrapolate relationships to create natural groupings and detect anomalous elements, such as security threats or medical issues.

Association algorithms reveal interesting relationships in the data to answer questions to address issues such as reducing customer churn or selecting related products for a promotion.

AI teams use unsupervised classification when attempting to answer these types of questions:

Is there any evidence of fraud in these financial transactions?

Are there any network performance symptoms that indicate a latent issue that would increase the risk of network failure?

Are there any anomalies in customer activity that point to possible buying trends?

Predictive analysis uses data mining, machine learning, and predictive modeling to process transactional and historical data to identify trends that indicate areas of increased risk or reward.

Specifically, predictive modelling software uses known results from existing data to train the model to predict relationships and outcomes that are likely to occur in future data and recommend a course of action. It is a business function, not a math problem or a science exercise.

AI teams use predictive analytics when attempting to answer these types of questions:

Will my customer purchase product X?

Will my customer like a recommended song?

Which of my customers are likely to switch to a competitor or cancel their contract?

Of all recently submitted claims, which ones are likely to require an additional fraud investigation unit review?

Is this applicant likely to default on their car loan in the future?