Yong Chen - Industrial Data Analytics for Diagnosis and Prognosis

One advantage of the Bayesian inference is that the prior knowledge can be included naturally. Suppose, for example, a randomly sampled product turns out to be defective. A MLE of the defective rate based on this single observation would be equal to 1, implying that all products are defective. By contrast, a Bayesian approach with a reasonable prior should give a much less extreme conclusion. In addition, the Bayesian inference can be performed in a sequential manner very naturally. To see this, we can write the posterior distribution of μwith the contribution from the last data point xn separated out as

(3.32)

Equation ( 3.32) can be viewed as the posterior distribution given a single observation x n with the term in the square bracket treated as the prior. Note that the term in the square brackets is just the posterior distribution (up to a normalization constant) after observing n − 1 data points. Equation ( 3.32) says that we can treat the posterior based on the first n − 1 observations as the prior and update the posterior based on the next observation using the Bayes’ theorem. This process can be repeated sequentially for each new observation. The sequential update of posterior under the Bayesian framework is very useful when observations are collected sequentially over time.

Example 3.3:For the side_temp_defectdata set from a hot rolling process, suppose the true covariance matrix of the side temperatures measured at location 2, 40, and 78 of Stand 5 is known and given by

We use the nominal mean temperatures as given in Example 3.2 as the mean of the prior distribution and a diagonal matrix with variance equal to 100 for each temperature variable as its covariance matrix:

Based on ( 3.30) and ( 3.31), the following Rcodes calculate the posterior mean and covariance matrix for μusing the first five ( n = 5) observations in the data set.

Sigma <- matrix(c(2547.4, -111.0, 133.7, -111.0, 533.1, 300.7, 133.7, 300.7, 562.5), nrow = 3, ncol = 3, byrow = T) Precision <- solve(Sigma) Sigma0 <- diag(rep(100, 3)) Precision0 <- solve(Sigma0) mu0 <- c(1926, 1851, 1872) n <- 5 X.n <- side.temp.defect[1:n, c(2, 40, 78)] x.bar <- apply(X.n, 2, mean) mu.n <- solve(Precision0+n*Precision)%*% (Precision0%*%mu0+n*Precision%*%x.bar) Sigma.n <- solve(Precision0 + n*Precision)

The posterior mean and covariance matrix are obtained as

Compared to the sample mean of the first five observations, which is (1943 1850 1838) T, the posterior mean has some deviations from both the sample mean and the prior mean μ 0. Now we use the first 100 ( n = 100) observations to find the posterior mean by changing nin the Rcodes from 5 to 100. The posterior mean and covariance matrix are

Compared to the sample mean vector of the first 100 observations, which is (1944 1849 1865) T, the posterior mean with n = 100 observations is very close to the sample mean, while the influence of the prior mean is very small. In addition, the posterior variance for the mean temperature at each of the three locations is much smaller for n = 100 than for n = 5.

Multivariate normal distribution and its inference are thoroughly discussed in multivariate statistics books, for example, Johnson et al. [2002], Rencher [2003], and Anderson [2003]. Particularly, proofs of many theoretical results and properties can be found in Anderson [2003].

1 Consider two discrete random variables X and Y with joint probability mass function p(x, y) given in the following table:

1 Find the mean vector, the covariance matrix, and the correlation matrix of the random vector (X Y )T.

1 A random vector X = (X1 X2 X3 X4)T has mean vector and covariance matrix given as

1 Let and . Please findE(X1)E(AX1)cov(X1)var(AX1)E(X2)E(BX2)cov(X2)cov(BX2)

1 Repeat Exercise 2, but with A and B replaced by

1 Let X = (X1 X2 X3)T be a random vector with X ∼ N(μ, Σ) with

1 Which of the following random variables are independent? Please explain.X1 and X2X2 and X3X1 and X3(X1, X2) and X3 and X3

1 Consider the random vector X in Exercise 2.Find the distribution of X2 + X3 + X4.Find the distribution of 3X2 − 2X3 + X4.Find the joint distribution of X2 + X3 + X4 and 3X2 − 2X3 + X4.Find the distribution of X1 − X2 + 2X3 + X4Find a 2×1 vector c such that X2 and are independent.Find a 2×1 vector c such that X2 and are independent.

2 Consider the random vector in Exercise 4.Find the conditional distribution of X1, given that X3 = x3.Find the conditional distribution of X1, given that X2 = x2.

3 Consider the random vector X in Exercise 2.Find the conditional distribution of X1, given that X2 = x2 and X3 = x3.Find the conditional distribution of X2, given that X3 = x3 and X4 = x4.Find the conditional distribution of X3, given that X2 = x2 and X4 = x4.Find the conditional distribution of (X2 X3)T, given that X4 = x4.

4 Calculate by hand the maximum likelihood estimates of the mean vector μ and the covariance matrix Σ of (X2 X3)T based on the first five observations of the last two variables in Table 2.1, assuming the observations are from a bivariate normal population.

5 Consider a random sample of size n = 3 from a bivariate normal population as shown in the following table.

1 Evaluate the T2-statistic used to test H0 : μ = μ0 based on this data set, where μ0 (8 4)T. What is the distribution of the T2-statistic in this case?

1 Consider the data from a bivariate normal population in the following table: