Iain Pardoe - Applied Regression Modeling

We can express the model we have been using to estimate the population mean, , as

In other words, each sample ‐value (the index keeps track of the sample observations) can be decomposed into two pieces, a deterministic part that is the same for all values, and a random error part that varies from observation to observation. A convenient choice for the deterministic part is the population mean, , since then the random errors have a (population) mean of zero. Since is the same for all ‐values, the random errors, , have the same standard deviation as the ‐values themselves, that is, . We can use this decomposition to derive the confidence interval and hypothesis test results of Sections 1.5and 1.6(although it would take more mathematics than we really need for our purposes in this book). Moreover, we can also use this decomposition to motivate the precise form of the uncertainty needed for prediction intervals (without having to get into too much mathematical detail).

In particular, write the ‐value to be predicted as , and decompose this into two pieces as above:

Then subtract , which represents potential values of repeated sample means, from both sides of this equation:

(1.1)

Thus, in estimating the population mean, the only error we have to worry about is estimation error, whereas in predicting an individual ‐value, we have to worry about both estimation error and random error.

Recall from Section 1.5that the form of a confidence interval for the population mean is

The term in this formula is an estimate of the standard deviation of the sampling distribution of sample means, , and is called the standard error of estimation . The square of this quantity, , is the estimated variance of the sampling distribution of sample means, . Then, thinking of as some fixed, unknown constant, is also the estimated variance of the estimation error, , in expression ( 1.1).

The estimated variance of the random error, , in expression ( 1.1) is . It can then be shown that the estimated variance of the prediction error, , in expression ( 1.1) is . Then, is called the standard error of prediction .

Thus, in general, we can write a prediction interval for an individual ‐value, as

where is the sample mean, is the sample standard deviation, is the sample size, and the t‐percentile comes from a t‐distribution with degrees of freedom.

For example, for a 95% interval (i.e., with 2.5% in each tail), the 97.5th percentile would be needed, whereas for a 90% interval (i.e., with 5% in each tail), the 95th percentile would be needed. These percentiles can be obtained from Table C.1. For example, the 95% prediction interval for an individual value of picked at random from the population of single‐family home sale prices is calculated as