Alessandro Massaro - Electronics in Advanced Research Industries

The pseudocode highlights that there are two mechanisms in the ANN network: the forward propagation of the estimation of the predicted output y , and the back propagation of the error function as sketched in Figure 1.18b. The output is estimated by considering the summation of the input contributions and is defined as:

(1.10)

where f is the activation function. Some examples of activation functions are plotted in Figure 1.19, where the analytical forms are:

(1.11)

(1.12)

(1.13)

(1.14)

(1.15)

Figure 1.19 Basic mathematical functions defining activation functions.

Other mathematical activation functions are the following [68]:

(1.16)

(1.17)

(1.18)

(1.19)

(1.20)

(1.21)

(1.22)

(1.23)

(1.24)

(1.25)

(1.26)

(1.27)

(1.28)

(1.29)

(1.30)

(1.31)

The activation function represents a basic research element of considerable importance. The correct choice of the activation function defines the best implementation of the logic defining the outputs. The analytical model must therefore be appropriately weighted by the various variables and must be “calibrated” for the specific case study. Another important aspect is the ability of the activation function to self‐adapt [69] to the specific case study providing a certain flexibility [70]. Of particular interest is the possibility to consider a combination of activation functions (activation ensemble [71]). The approach to follow is therefore to define a flexible and modular activation function as is the case for the adaptive spline activation function [72].

Concerning the training models, the full dataset of the neural network is divided into a training set, validation set, and test set ( Figure 1.20). In particular, the function of the training dataset is to fit the model; the validation set is a small partition of the full dataset able to previously estimate prediction error of the selected model; finally, the test set is used for testing the final model. A correct choice of the three parts depends on the SNR of the full dataset.

Figure 1.20 Supervised artificial network model: partitioning of the available dataset into training set, validation set, and test set.

The intelligent algorithms which constitute the core of the Industry 5.0 system, are classified in Figure 1.21. The generic algorithms of data analysis, including statistical data processing and classification roles, are classified as computer science and data mining algorithms; other high‐level algorithms are included in AI algorithms. The main function of the engine processor enables the managing of big data and of data processing. As previously mentioned, a basic concept of algorithm classification is in the learning supervision: in a supervised learning model, the algorithm learns on a pre‐selected dataset with specific labeled attributes filtered by the user; in an unsupervised model all the attributes are unlabeled, and the algorithm tries to extract features and patterns without a guideline. The supervised algorithms mainly support the user to find a solution for a specific problem such as finding a specific defect category or a specific failure system.

Figure 1.21 Algorithm classification and Industry 5.0 facilities.

A simple way to analyze the data trend is regression analysis, where a linear approach is enough to model a relationship between dependent variables and independent ones (see plot example of Figure 1.22a). Typically, line regression provides information about a linear trend prediction. The classification is based on the concept of data categorization: data are classified by considering a generic classification pattern curve defined as shown in Figure 1.22b, where all data above and below the curve appertain to a particular class. Data classification is often used to solve supervised learning models. Finally, data clustering is based on the grouping of datasets forming clusters having similar features. Data clustering is commonly used to solve unsupervised learning models. All analysis can be performed by analyzing a multidimensional domain by taking into account the variable time, which is fundamental for forecasting approaches.