Vinnie Bansal - Machine Learning with Dynamics 365 and Power Platform

3 Semi‐supervised learning. Semi‐supervised learning is a type of machine learning that uses a combination of both supervised and unsupervised learning techniques. It is used in a scenario where our dataset is a combination of both labeled and unlabeled data.For example, let's assume that we have access to a large number of unlabeled datasets that we like to train a model on. Manually labeling the whole data by ourselves is just not practical. So, instead of labeling the whole dataset, we manually label some parts of the dataset ourselves and use that portion to train our model. But this way, all the unlabeled data will be of no use. As we know, the more data we have to train our model, the better and more robust our model would be. So what can we do to use the unlabeled data of our dataset?This is why semi‐supervised learning was introduced. To prevent our unlabeled data from getting wasted, we can implement a technique of semi‐supervised learning called pseudo labeling.To understand pseudo labeling, let's continue with the example mentioned previously.Our model is trained using labeled data, and it is performing pretty well. Everything to this point is just regular supervised learning. Now we will use unsupervised learning to predict the remaining unlabeled portion of data. We will serve the unlabeled data to our model. The trained model will then process this data, and as a result, it will predict individual outputs for each piece of unlabeled data. Thus, pseudo labeling is a process of labeling the unlabeled data with the output that is predicted by our neural network. With pseudo labeling, we can train on an audaciously larger dataset.

4 Reinforcement learning. There is no predefined data in reinforcement learning. It is the area of machine learning that is concerned with behavioral psychology. In this learning, an agent is put into an environment, and he learns to behave in this environment by performing certain actions and observing the awards that they get from their actions. Reinforcement learning involves software agents that take appropriate actions in a particular situation to earn maximum rewards. There is no expected output in this learning. The reinforcement agent decides what actions to take to perform a task. In the absence of the training dataset, it is bound to learn from its own experience.The following is a list of commonly used algorithms in reinforcement learning:Q‐learningTemporal difference (TD)Deep adversarial networksNow to choose which algorithm is right for your problem, you should categorize your problem according to the following:Categorize by inputLabeled data: supervised learningUnlabeled data: unsupervised learningCombination of labeled and unlabeled data: semi‐supervised learningNo data and want to optimize an objective function by interacting with an environment: reinforcement learningCategorize by outputIf the output of a model is a number: regression problemIf the output of a model is a class: classification problemIf the output of your model is a set of input groups: clustering problemTo detect an anomaly: anomaly detectionUnderstand your constraintsStorage capacity of modelFast predictionFast learningFind the available algorithms: Factors affecting the choice of the model are:Business goalsAmount of preprocessing required in dataAccuracy of the modelScalability of the modelConsider model complexityComplex feature engineeringComputational overheadThese points can help you to choose the right algorithm for developing a solution to a real‐time business problem that requires knowledge of business requirements, rules and regulations, and stakeholders' interests as well as significant expertise. Hence, to solve a machine problem, it is crucial to combine and balance algorithms for valuable results.

Once the machine learning model has been properly trained on a given dataset, then we have to test the model. In this step, we check for the accuracy of the model by rendering a test dataset to it. Testing the model is important to find out the percentage accuracy of the model as per the project requirement or given problem.

The input of this validation stage is the trained model produced by the earlier step in the model learning stage, and the output is a validated model that provides enough information to allow users to check whether the machine learning model is appropriate for its intended purpose. Thus, this validation stage of the machine learning lifecycle deals with whether the model is working properly as desired or not when fed with unseen inputs. Thus, model validation is the process that evaluates a trained model on a test dataset. This step renders the generalization ability of the trained model.

The last step of the machine learning lifecycle is deployment, where we deploy the ML model in the real‐world system. Deployment is a very crucial step in the machine learning lifecycle process. Deployment is a process of making your model available to make predictions in the production environment. The aim of this stage is to check the proper functionality of the model post‐deployment. The models need to be deployed in such a way that they can be used for inference as well as be updated regularly. If the prepared model produces an accurate result as per the specified requirements, with acceptable speed, only then do we deploy the model in the real system. But before deploying the project, you need to check whether it is improving its performance using available data or not and whether you want to go with a Platform as a Service (PaaS) or Infrastructure as a Service (IaaS). A PaaS is excellent for prototyping and businesses with lower traffic. Eventually, when the business grows and traffic increases, you need to switch to IaaS. This is the step to test the ability to predict outcomes in the real world.

Machine learning can help you solve your business issues by predicting future outcomes and make better decisions accordingly. Here are some advantages that can help you understand if machine learning is for you:

Extract meaningful insights from a large volume of data. For example, ML helps in discovering specific patterns from a large volume of data that would not be plausible to humans (e.g., an e‐commerce website like Flipkart has a large volume of data. Using ML, it observes the purchasing and browsing history of the customer, discovers a pattern from it, and then recommends similar products and deals to them).

Algorithms self‐improve the prediction without any manual support.

Machine learning algorithms can learn from experience. So, the accuracy of predictions also improves as the project grows.

Machine learning algorithms can handle a variety of data. For example, multidimensional data.

Machine learning automates time‐intensive and repetitive tasks.

Machine learning helps businesses to make the right decisions by constantly monitoring the change in market demands and facilitating organizations to take proactive measures to maintain their competing edge in real time.

Now that we are aware of all the advantages of machine learning, Let's uncover some disadvantages, as every coin has two sides:

Requires massive datasets to train machine learning algorithms.

Running training data on each algorithm to select the right algorithm for your problem is a time‐consuming and manual task.

Training and testing the huge datasets in all of the algorithms can cause errors that take a lot of time to resolve.

A key point that we would like to cover in this book is really to give you insights about what is important for the decade to come: how to go ahead after an implementation of your “basic” CRM and ERP system and enhance it with Dynamics 365 and Power Platform. As you will see in the next chapters, we will bring those two worlds together.