Douglas C. Montgomery - Introduction to Linear Regression Analysis

1 COVER

2 TITLE PAGE

3 COPYRIGHT PAGE

4 PREFACE

5 ABOUT THE COMPANION WEBSITE

6 CHAPTER 1: INTRODUCTION1.1 REGRESSION AND MODEL BUILDING 1.2 DATA COLLECTION 1.3 USES OF REGRESSION 1.4 ROLE OF THE COMPUTER

7 CHAPTER 2: SIMPLE LINEAR REGRESSION2.1 SIMPLE LINEAR REGRESSION MODEL 2.2 LEAST-SQUARES ESTIMATION OF THE PARAMETERS 2.3 HYPOTHESIS TESTING ON THE SLOPE AND INTERCEPT 2.4 INTERVAL ESTIMATION IN SIMPLE LINEAR REGRESSION 2.5 PREDICTION OF NEW OBSERVATIONS 2.6 COEFFICIENT OF DETERMINATION 2.7 A SERVICE INDUSTRY APPLICATION OF REGRESSION 2.8 DOES PITCHING WIN BASEBALL GAMES? 2.9 USING SAS® AND R FOR SIMPLE LINEAR REGRESSION 2.10 SOME CONSIDERATIONS IN THE USE OF REGRESSION 2.11 REGRESSION THROUGH THE ORIGIN 2.12 ESTIMATION BY MAXIMUM LIKELIHOOD 2.13 CASE WHERE THE REGRESSOR x IS RANDOM PROBLEMS

8 CHAPTER 3: MULTIPLE LINEAR REGRESSION 3.1 MULTIPLE REGRESSION MODELS 3.2 ESTIMATION OF THE MODEL PARAMETERS 3.3 HYPOTHESIS TESTING IN MULTIPLE LINEAR REGRESSION 3.4 CONFIDENCE INTERVALS IN MULTIPLE REGRESSION 3.5 PREDICTION OF NEW OBSERVATIONS 3.6 A MULTIPLE REGRESSION MODEL FOR THE PATIENT SATISFACTION DATA 3.7 DOES PITCHING AND DEFENSE WIN BASEBALL GAMES? 3.8 USING SAS AND R FOR BASIC MULTIPLE LINEAR REGRESSION 3.9 HIDDEN EXTRAPOLATION IN MULTIPLE REGRESSION 3.10 STANDARDIZED REGRESSION COEFFICIENTS 3.11 MULTICOLLINEARITY 3.12 WHY DO REGRESSION COEFFICIENTS HAVE THE WRONG SIGN? PROBLEMS

9 CHAPTER 4: MODEL ADEQUACY CHECKING4.1 INTRODUCTION 4.2 RESIDUAL ANALYSIS 4.3 PRESS STATISTIC 4.4 DETECTION AND TREATMENT OF OUTLIERS 4.5 LACK OF FIT OF THE REGRESSION MODEL PROBLEMS

10 CHAPTER 5: TRANSFORMATIONS AND WEIGHTING TO CORRECT MODEL INADEQUACIES5.1 INTRODUCTION 5.2 VARIANCE-STABILIZING TRANSFORMATIONS 5.3 TRANSFORMATIONS TO LINEARIZE THE MODEL 5.4 ANALYTICAL METHODS FOR SELECTING A TRANSFORMATION 5.5 GENERALIZED AND WEIGHTED LEAST SQUARES 5.6 REGRESSION MODELS WITH RANDOM EFFECTS PROBLEMS

11 CHAPTER 6: DIAGNOSTICS FOR LEVERAGE AND INFLUENCE6.1 IMPORTANCE OF DETECTING INFLUENTIAL OBSERVATIONS 6.2 LEVERAGE 6.3 MEASURES OF INFLUENCE: COOK’S D 6.4 MEASURES OF INFLUENCE: DFFITS AND DFBETAS 6.5 A MEASURE OF MODEL PERFORMANCE 6.6 DETECTING GROUPS OF INFLUENTIAL OBSERVATIONS 6.7 TREATMENT OF INFLUENTIAL OBSERVATIONS PROBLEMS

12 CHAPTER 7: POLYNOMIAL REGRESSION MODELS7.1 INTRODUCTION 7.2 POLYNOMIAL MODELS IN ONE VARIABLE 7.3 NONPARAMETRIC REGRESSION 7.4 POLYNOMIAL MODELS IN TWO OR MORE VARIABLES 7.5 ORTHOGONAL POLYNOMIALS PROBLEMS

13 CHAPTER 8: INDICATOR VARIABLES8.1 GENERAL CONCEPT OF INDICATOR VARIABLES 8.2 COMMENTS ON THE USE OF INDICATOR VARIABLES 8.3 REGRESSION APPROACH TO ANALYSIS OF VARIANCE PROBLEMS

14 CHAPTER 9: MULTICOLLINEARITY9.1 INTRODUCTION 9.2 SOURCES OF MULTICOLLINEARITY 9.3 EFFECTS OF MULTICOLLINEARITY 9.4 MULTICOLLINEARITY DIAGNOSTICS 9.5 METHODS FOR DEALING WITH MULTICOLLINEARITY 9.6 USING SAS TO PERFORM RIDGE AND PRINCIPAL-COMPONENT REGRESSION PROBLEMS

15 CHAPTER 10: VARIABLE SELECTION AND MODEL BUILDING10.1 INTRODUCTION 10.2 COMPUTATIONAL TECHNIQUES FOR VARIABLE SELECTION 10.3 STRATEGY FOR VARIABLE SELECTION AND MODEL BUILDING 10.4 CASE STUDY: GORMAN AND TOMAN ASPHALT DATA USING SAS PROBLEMS

16 CHAPTER 11: VALIDATION OF REGRESSION MODELS11.1 INTRODUCTION 11.2 VALIDATION TECHNIQUES 11.3 DATA FROM PLANNED EXPERIMENTS PROBLEMS

17 CHAPTER 12: INTRODUCTION TO NONLINEAR REGRESSION 12.1 LINEAR AND NONLINEAR REGRESSION MODELS 12.2 ORIGINS OF NONLINEAR MODELS 12.3 NONLINEAR LEAST SQUARES 12.4 TRANFORMATION TO A LINEAR MODEL 12.5 PARAMETER ESTIMATION IN A NONLINEAR SYSTEM 12.6 STATISTICAL INFERENCE IN NONLINEAR REGRESSION 12.7 EXAMPLES OF NONLINEAR REGRESSION MODELS 12.8 USING SAS AND R PROBLEMS

18 CHAPTER 13: GENERALIZED LINEAR MODELS13.1 INTRODUCTION 13.2 LOGISTIC REGRESSION MODELS 13.3 POISSON REGRESSION 13.4 THE GENERALIZED LINEAR MODEL PROBLEMS

19 CHAPTER 14: REGRESSION ANALYSIS OF TIME SERIES DATA14.1 INTRODUCTION TO REGRESSION MODELS FOR TIME SERIES DATA 14.2 DETECTING AUTOCORRELATION: THE DURBIN–WATSON TEST 14.3 ESTIMATING THE PARAMETERS IN TIME SERIES REGRESSION MODELS PROBLEMS

20 CHAPTER 15: OTHER TOPICS IN THE USE OF REGRESSION ANALYSIS 15.1 ROBUST REGRESSION 15.2 EFFECT OF MEASUREMENT ERRORS IN THE REGRESSORS 15.3 INVERSE ESTIMATION—THE CALIBRATION PROBLEM 15.4 BOOTSTRAPPING IN REGRESSION 15.5 CLASSIFICATION AND REGRESSION TREES (CART) 15.6 NEURAL NETWORKS 15.7 DESIGNED EXPERIMENTS FOR REGRESSION PROBLEMS

21 APPENDIX A: STATISTICAL TABLES

22 APPENDIX B: DATA SETS FOR EXERCISES

23 APPENDIX C: SUPPLEMENTAL TECHNICAL MATERIAL C.1 BACKGROUND ON BASIC TEST STATISTICS C.2 BACKGROUND FROM THE THEORY OF LINEAR MODELS C.3 IMPORTANT RESULTS ON SS RAND SS RES C.4 GAUSS–MARKOV THEOREM, VAR( ε ) = σ 2I C.5 COMPUTATIONAL ASPECTS OF MULTIPLE REGRESSION C.6 RESULT ON THE INVERSE OF A MATRIX C.7 DEVELOPMENT OF THE PRESS STATISTIC C.8 DEVELOPMENT OF C.9 OUTLIER TEST BASED ON R -STUDENT C.10 INDEPENDENCE OF RESIDUALS AND FITTED VALUES C.11 GAUSS-MARKOV THEOREM, VAR( ε ) = V C.12 BIAS IN MS RESWHEN THE MODEL IS UNDERSPECIFIED C.13 COMPUTATION OF INFLUENCE DIAGNOSTICS C.14 GENERALIZED LINEAR MODELS

24 APPENDIX D: INTRODUCTION TO SAS D.1 BASIC DATA ENTRY D.2 CREATING PERMANENT SAS DATA SETS D.3 IMPORTING DATA FROM AN EXCEL FILE D.4 OUTPUT COMMAND D.5 LOG FILE D.6 ADDING VARIABLES TO AN EXISTING SAS DATA SET

25 APPENDIX E: INTRODUCTION TO R TO PERFORM LINEAR REGRESSION ANALYSIS E.1 BASIC BACKGROUND ON R E.2 BASIC DATA ENTRY E.3 BRIEF COMMENTS ON OTHER FUNCTIONALITY IN R E.4 R COMMANDER

26 REFERENCES

27 INDEX

28 WILEY SERIES IN PROBABILITY AND STATISTICS

29 END USER LICENSE AGREEMENT

1 Chapter 1 Figure 1.1 ( a ) Scatter diagram for delivery volume. ( b ) Straight-line relation... Figure 1.2 How observations are generated in linear regression. Figure 1.3 Linear regression approximation of a complex relationship. Figure 1.4 Piecewise linear approximation of a complex relationship. Figure 1.5 The danger of extrapolation in regression. Figure 1.6 Acetone–butyl alcohol distillation column. Figure 1.7 The designed experiment for the distillation column. Figure 1.8 Regression model-building process.

2 Chapter 2 Figure 2.1 Scatter diagram of shear strength versus propellant age, Example 2.... Figure 2.2 Situations where the hypothesis H 0: β 1= 0 is not rejected. Figure 2.3 Situations where the hypothesis H 0: β 1= 0 is rejected. Figure 2.4 The upper and lower 95% confidence limits for the propellant data.... Figure 2.5 The 95% confidence and prediction intervals for the propellant data... Figure 2.6 Scatter diagram of satisfaction versus severity. Figure 2.7 JMP output for the simple linear regression model for the patient s... Figure 2.8 JMP output for the model relating team wins to team ERA for the 201... Figure 2.9 Two influential observations. Figure 2.10 A point remote in x space. Figure 2.11 An outlier. Figure 2.12 Scatter diagrams and regression lines for chemical process yield a... Figure 2.13 True relationship between yield and temperature. Figure 2.14 Scatter diagram of shelf-stocking data. Figure 2.15 The confidence and prediction bands for the shelf-stocing data.

3 Chapter 3 Figure 3.1 ( a ) The regression plane for the model E ( y ) = 50 + 10 x 1+ 7 x 2. ( b ) ... Figure 3.2 ( a ) Three-dimensional plot of regression model E ( y ) = 50 + 10 x 1+ 7 Figure 3.3 ( a ) Three-dimensional plot of the regression model