Daniel J. Denis - Applied Univariate, Bivariate, and Multivariate Statistics

1 COVER

2 TITLE PAGE

3 COPYRIGHT PAGE

4 DEDICATION PAGE

5 PREFACE ACKNOWLEDGMENTS

6 ABOUT THE COMPANION WEBSITE

7 1 PRELIMINARY CONSIDERATIONS 1.1 THE PHILOSOPHICAL BASES OF KNOWLEDGE: RATIONALISTIC VERSUS EMPIRICIST PURSUITS 1.2 WHAT IS A “MODEL”? 1.3 SOCIAL SCIENCES VERSUS HARD SCIENCES 1.4 IS COMPLEXITY A GOOD DEPICTION OF REALITY? ARE MULTIVARIATE METHODS USEFUL? 1.5 CAUSALITY 1.6 THE NATURE OF MATHEMATICS: MATHEMATICS AS A REPRESENTATION OF CONCEPTS 1.7 AS A SCIENTIST, HOW MUCH MATHEMATICS DO YOU NEED TO KNOW? 1.8 STATISTICS AND RELATIVITY 1.9 EXPERIMENTAL VERSUS STATISTICAL CONTROL 1.10 STATISTICAL VERSUS PHYSICAL EFFECTS 1.11 UNDERSTANDING WHAT “APPLIED STATISTICS” MEANS Review Exercises Further Discussion and Activities

8 2 INTRODUCTORY STATISTICS 2.1 DENSITIES AND DISTRIBUTIONS 2.2 CHI‐SQUARE DISTRIBUTIONS AND GOODNESS‐OF‐FIT TEST 2.3 SENSITIVITY AND SPECIFICITY 2.4 SCALES OF MEASUREMENT: NOMINAL, ORDINAL, INTERVAL, RATIO 2.5 MATHEMATICAL VARIABLES VERSUS RANDOM VARIABLES 2.6 MOMENTS AND EXPECTATIONS 2.7 ESTIMATION AND ESTIMATORS 2.8 VARIANCE 2.9 DEGREES OF FREEDOM 2.10 SKEWNESS AND KURTOSIS 2.11 SAMPLING DISTRIBUTIONS 2.12 CENTRAL LIMIT THEOREM 2.13 CONFIDENCE INTERVALS 2.14 MAXIMUM LIKELIHOOD 2.15 AKAIKE'S INFORMATION CRITERIA 2.16 COVARIANCE AND CORRELATION 2.17 PSYCHOMETRIC VALIDITY, RELIABILITY: A COMMON USE OF CORRELATION COEFFICIENTS 2.18 COVARIANCE AND CORRELATION MATRICES 2.19 OTHER CORRELATION COEFFICIENTS 2.20 STUDENT'S t DISTRIBUTION 2.21 STATISTICAL POWER 2.22 POWER ESTIMATION USING R AND G *POWER 2.23 PAIRED‐SAMPLES t ‐ TEST: STATISTICAL TEST FOR MATCHED‐PAIRS (ELEMENTARY BLOCKING) DESIGNS 2.24 BLOCKING WITH SEVERAL CONDITIONS 2.25 COMPOSITE VARIABLES: LINEAR COMBINATIONS 2.26 MODELS IN MATRIX FORM 2.27 GRAPHICAL APPROACHES 2.28 WHAT MAKES A p ‐VALUE SMALL? A CRITICAL OVERVIEW AND PRACTICAL DEMONSTRATION OF NULL HYPOTHESIS SIGNIFICANCE TESTING 2.29 CHAPTER SUMMARY AND HIGHLIGHTS Review Exercises Further Discussion and Activities

9 3 ANALYSIS OF VARIANCE: FIXED EFFECTS MODELS 3.1 WHAT IS ANALYSIS OF VARIANCE? FIXED VERSUS RANDOM EFFECTS 3.2 HOW ANALYSIS OF VARIANCE WORKS: A BIG PICTURE OVERVIEW 3.3 LOGIC AND THEORY OF ANOVA: A DEEPER LOOK 3.4 FROM SUMS OF SQUARES TO UNBIASED VARIANCE ESTIMATORS: DIVIDING BY DEGREES OF FREEDOM 3.5 EXPECTED MEAN SQUARES FOR ONE‐WAY FIXED EFFECTS MODEL: DERIVING THE F ‐RATIO 3.6 THE NULL HYPOTHESIS IN ANOVA 3.7 FIXED EFFECTS ANOVA: MODEL ASSUMPTIONS 3.8 A WORD ON EXPERIMENTAL DESIGN AND RANDOMIZATION 3.9 A PREVIEW OF THE CONCEPT OF NESTING 3.10 BALANCED VERSUS UNBALANCED DATA IN ANOVA MODELS 3.11 MEASURES OF ASSOCIATION AND EFFECT SIZE IN ANOVA: MEASURES OF VARIANCE EXPLAINED 3.12 THE F ‐TEST AND THE INDEPENDENT SAMPLES t ‐TEST 3.13 CONTRASTS AND POST‐HOCS 3.14 POST‐HOC TESTS 3.15 SAMPLE SIZE AND POWER FOR ANOVA: ESTIMATION WITH R AND G *POWER 3.16 FIXED EFFECTS ONE‐WAY ANALYSIS OF VARIANCE IN R: MATHEMATICS ACHIEVEMENT AS A FUNCTION OF TEACHER 3.17 ANALYSIS OF VARIANCE VIA R’s lm 3.18 KRUSKAL–WALLIS TEST IN R AND THE MOTIVATION BEHIND NONPARAMETRIC TESTS 3.19 ANOVA IN SPSS: ACHIEVEMENT AS A FUNCTION OF TEACHER 3.20 CHAPTER SUMMARY AND HIGHLIGHTS REVIEW EXERCISES Further Discussion and Activities

10 4 FACTORIAL ANALYSIS OF VARIANCE 4.1 WHAT IS FACTORIAL ANALYSIS OF VARIANCE? 4.2 THEORY OF FACTORIAL ANOVA: A DEEPER LOOK 4.3 COMPARING ONE‐WAY ANOVA TO TWO‐WAY ANOVA: CELL EFFECTS IN FACTORIAL ANOVA VERSUS SAMPLE EFFECTS IN ONE‐WAY ANOVA 4.4 PARTITIONING THE SUMS OF SQUARES FOR FACTORIAL ANOVA: THE CASE OF TWO FACTORS 4.5 INTERPRETING MAIN EFFECTS IN THE PRESENCE OF INTERACTIONS 4.6 EFFECT SIZE MEASURES 4.7 THREE‐WAY, FOUR‐WAY, AND HIGHER MODELS 4.8 SIMPLE MAIN EFFECTS 4.9 NESTED DESIGNS 4.10 ACHIEVEMENT AS A FUNCTION OF TEACHER AND TEXTBOOK: EXAMPLE OF FACTORIAL ANOVA IN R 4.11 INTERACTION CONTRASTS 4.12 CHAPTER SUMMARY AND HIGHLIGHTS REVIEW EXERCISES

11 5 INTRODUCTION TO RANDOM EFFECTS AND MIXED MODELS 5.1 WHAT IS RANDOM EFFECTS ANALYSIS OF VARIANCE? 5.2 THEORY OF RANDOM EFFECTS MODELS 5.3 ESTIMATION IN RANDOM EFFECTS MODELS 5.4 DEFINING NULL HYPOTHESES IN RANDOM EFFECTS MODELS 5.5 COMPARING NULL HYPOTHESES IN FIXED VERSUS RANDOM EFFECTS MODELS: THE IMPORTANCE OF ASSUMPTIONS 5.6 ESTIMATING VARIANCE COMPONENTS IN RANDOM EFFECTS MODELS: ANOVA, ML, REML ESTIMATORS 5.7 IS ACHIEVEMENT A FUNCTION OF TEACHER? ONE‐WAY RANDOM EFFECTS MODEL IN R 5.8 R ANALYSIS USING REML 5.9 ANALYSIS IN SPSS: OBTAINING VARIANCE COMPONENTS 5.10 Factorial Random Effects: A Two‐Way Model 5.11 FIXED EFFECTS VERSUS RANDOM EFFECTS: A WAY OF CONCEPTUALIZING THEIR DIFFERENCES 5.12 CONCEPTUALIZING THE TWO‐WAY RANDOM EFFECTS MODEL: THE MAKE‐UP OF A RANDOMLY CHOSEN OBSERVATION 5.13 SUMS OF SQUARES AND EXPECTED MEAN SQUARES FOR RANDOM EFFECTS: THE CONTAMINATING INFLUENCE OF INTERACTION EFFECTS 5.14 YOU GET WHAT YOU GO IN WITH: THE IMPORTANCE OF MODEL ASSUMPTIONS AND MODEL SELECTION 5.15 MIXED MODEL ANALYSIS OF VARIANCE: INCORPORATING FIXED AND RANDOM EFFECTS 5.16 MIXED MODELS IN MATRICES 5.17 MULTILEVEL MODELING AS A SPECIAL CASE OF THE MIXED MODEL: INCORPORATING NESTING AND CLUSTERING 5.18 CHAPTER SUMMARY AND HIGHLIGHTS Review Exercises

12 6 RANDOMIZED BLOCKS AND REPEATED MEASURES 6.1 WHAT IS A RANDOMIZED BLOCK DESIGN? 6.2 RANDOMIZED BLOCK DESIGNS: SUBJECTS NESTED WITHIN BLOCKS 6.3 THEORY OF RANDOMIZED BLOCK DESIGNS 6.4 TUKEY TEST FOR NONADDITIVITY 6.5 ASSUMPTIONS FOR THE COVARIANCE MATRIX 6.6 INTRACLASS CORRELATION 6.7 REPEATED MEASURES MODELS: A SPECIAL CASE OF RANDOMIZED BLOCK DESIGNS 6.8 INDEPENDENT VERSUS PAIRED‐SAMPLES t ‐TEST 6.9 THE SUBJECT FACTOR: FIXED OR RANDOM EFFECT? 6.10 MODEL FOR ONE‐WAY REPEATED MEASURES DESIGN 6.11 ANALYSIS USING R: ONE‐WAY REPEATED MEASURES: LEARNING AS A FUNCTION OF TRIAL 6.12 ANALYSIS USING SPSS: ONE‐WAY REPEATED MEASURES: LEARNING AS A FUNCTION OF TRIAL 6.13 SPSS TWO‐WAY REPEATED MEASURES ANALYSIS OF VARIANCE MIXED DESIGN: ONE BETWEEN FACTOR, ONE WITHIN FACTOR 6.14 Chapter Summary and Highlights Review Exercises

13 7 LINEAR REGRESSION 7.1 BRIEF HISTORY OF REGRESSION 7.2 REGRESSION ANALYSIS AND SCIENCE: EXPERIMENTAL VERSUS CORRELATIONAL DISTINCTIONS 7.3 A MOTIVATING EXAMPLE: CAN OFFSPRING HEIGHT BE PREDICTED? 7.4 THEORY OF REGRESSION ANALYSIS: A DEEPER LOOK 7.5 MULTILEVEL YEARNINGS 7.6 THE LEAST‐SQUARES LINE 7.7 MAKING PREDICTIONS WITHOUT REGRESSION 7.8 MORE ABOUT ε i 7.9 MODEL ASSUMPTIONS FOR LINEAR REGRESSION 7.10 ESTIMATION OF MODEL PARAMETERS IN REGRESSION 7.11 NULL HYPOTHESES FOR REGRESSION 7.12 SIGNIFICANCE TESTS AND CONFIDENCE INTERVALS FOR MODEL PARAMETERS 7.13 OTHER FORMULATIONS OF THE REGRESSION MODEL 7.14 THE REGRESSION MODEL IN MATRICES: ALLOWING FOR MORE COMPLEX MULTIVARIABLE MODELS 7.15 ORDINARY LEAST‐SQUARES IN MATRICES 7.16 ANALYSIS OF VARIANCE FOR REGRESSION 7.17 MEASURES OF MODEL FIT FOR REGRESSION: HOW WELL DOES THE LINEAR EQUATION FIT? 7.18 ADJUSTED R 2 7.19 WHAT “EXPLAINED VARIANCE” MEANS AND MORE IMPORTANTLY, WHAT IT DOES NOT MEAN 7.20 VALUES FIT BY REGRESSION 7.21 LEAST‐SQUARES REGRESSION IN R: USING MATRIX OPERATIONS 7.22 LINEAR REGRESSION USING R 7.23 REGRESSION DIAGNOSTICS: A CHECK ON MODEL ASSUMPTIONS 7.24 REGRESSION IN SPSS: PREDICTING QUANTITATIVE FROM VERBAL 7.25 POWER ANALYSIS FOR LINEAR REGRESSION IN R 7.26 CHAPTER SUMMARY AND HIGHLIGHTS REVIEW EXERCISES Further Discussion and Activities

14 8 MULTIPLE LINEAR REGRESSION 8.1 THEORY OF PARTIAL CORRELATION 8.2 SEMIPARTIAL CORRELATIONS 8.3 MULTIPLE REGRESSION 8.4 SOME PERSPECTIVE ON REGRESSION COEFFICIENTS: “EXPERIMENTAL COEFFICIENTS”? 8.5 MULTIPLE REGRESSION MODEL IN MATRICES 8.6 ESTIMATION OF PARAMETERS 8.7 CONCEPTUALIZING MULTIPLE R 8.8 INTERPRETING REGRESSION COEFFICIENTS: CORRELATED VERSUS UNCORRELATED PREDICTORS 8.9 ANDERSON’S IRIS DATA: PREDICTING SEPAL LENGTH FROM PETAL LENGTH AND PETAL WIDTH 8.10 FITTING OTHER FUNCTIONAL FORMS: A BRIEF LOOK AT POLYNOMIAL REGRESSION 8.11 MEASURES OF COLLINEARITY IN REGRESSION: VARIANCE INFLATION FACTOR AND TOLERANCE 8.12 R‐SQUARED AS A FUNCTION OF PARTIAL AND SEMIPARTIAL CORRELATIONS: THE STEPPING STONES TO FORWARD AND STEPWISE REGRESSION 8.13 MODEL‐BUILDING STRATEGIES: SIMULTANEOUS, HIERARCHICAL, FORWARD, STEPWISE 8.14 POWER ANALYSIS FOR MULTIPLE REGRESSION 8.15 INTRODUCTION TO STATISTICAL MEDIATION: CONCEPTS AND CONTROVERSY 8.16 BRIEF SURVEY OF RIDGE AND LASSO REGRESSION: PENALIZED REGRESSION MODELS AND THE CONCEPT OF SHRINKAGE 8.17 CHAPTER SUMMARY AND HIGHLIGHTS Review Exercises Further Discussion and Activities