Jane M. Horgan - Probability with R

that best fits the data.

The output is

Call: lm(formula = prog2∼prog1) Coefficients: (Intercept) prog1 -5.455 0.960

Therefore, the line that best fits these data is

To draw this line on the scatter diagram, write

plot(prog2, prog1) abline(lm(prog2∼prog1))

which gives Fig. 3.16.

Figure 3.16The Line of Best Fit

The line of best fit may be used to make predictions. For example, we might be able to predict how students will do in Semester 2 from the results that they obtained in Semester 1. If the mark on Programming 1 for a particular student is 70, that student would be expected to do well also in Programming 2, estimated to obtain . A student doing badly in Programming 1, 30 say, would also be expected to do badly in Programming 2. . These predictions may not be exact but, if the linear trend is strong and past trends continue, they will be reasonably close.

A word of warning is appropriate here. The estimated values are based on the assumption that the past trend continues. This may not always be the case. For example, students who do badly in Semester 1, may get such a shock that they work harder in Semester 2, and change the pattern. Similarly, students getting high marks in Semester 1 may be lulled into a sense of false security and take it easy in Semester 2. Consequently, they may not do as well as expected. Hence, the Semester 1 trends may not continue, and the model may no longer be valid.

Machine learning is the science of getting computer systems to use algorithms and statistical models to study patterns and learn from data. Supervised learning is the machine learning task of using past data to learn a function in order to predict a future output.

The line of best fit is one of the many techniques that machine learning has borrowed from the field of Probability and Statistics to “train” the machine to make predictions. In this case of what is also known as the simple linear regression line in statistics, a set of pairs of data is obtained, is referred to as the independent variable, and is the dependent variable. The objective is to estimate from . The line of best fit, , is obtained by choosing the intercept and slope so that the sum of the squared distances from the observed to the estimated is minimized. The algebraic details of the derivations of and are given in Appendix B.

Often, the data for supervised learning are randomly divided into two parts, one for training and the other for testing. In machine learning, we derive the line of best fit from the training set

The testing set is used to see how well the line actually fits. Usually, an breakdown of the data is made, the 80% is used for “training,” that is, to obtain the line, and the 20% is used to decide if the line really fits the data, and to ascertain if the model is appropriate for future predictions. The model is updated as new data become available.

Suppose there are 50 pairs of observations available for obtaining the line that best fits the data in order to predict from . The data are randomly divided into the training set and testing set, using 40 observations for training ( Table 3.1), and 10 for testing ( Table 3.2).

TABLE 3.1The Training Set

Observation Numbers			Observation Numbers
1	11.8	31.3	21	15.1	80.1
2	10.8	59.9	22	14.7	66.9
3	8.6	27.6	23	10.5	42.0
4	10.3	57.7	24	10.9	72.9
5	8.5	50.2	25	11.6	67.8
6	11.6	52.1	26	9.1	45.3
7	14.4	79.1	27	5.4	30.2
8	8.6	32.3	28	8.8	49.6
9	12.4	58.8	29	11.2	44.3
10	14.9	79.5	30	7.4	46.1
11	8.9	57.0	31	7.9	45.1
12	8.7	35.1	32	12.2	46.5
13	11.7	68.2	33	8.5	42.7
14	11.4	60.1	34	9.3	56.3
15	8.8	44.5	35	10.0	27.4
16	5.9	28.9	36	3.8	20.2
17	13.5	75.8	37	14.9	68.5
18	8.7	48.7	38	12.4	72.6
19	11.0	54.7	39	11.1	54.3
20	8.3	32.8	40	8.9	38.5

TABLE 3.2The Testing Set