Daniel J. Denis - Applied Univariate, Bivariate, and Multivariate Statistics

15 9 INTERACTIONS IN MULTIPLE LINEAR REGRESSION 9.1 THE ADDITIVE REGRESSION MODEL WITH TWO PREDICTORS 9.2 WHY THE INTERACTION IS THE PRODUCT TERM x i z i: DRAWING AN ANALOGY TO FACTORIAL ANOVA 9.3 A MOTIVATING EXAMPLE OF INTERACTION IN REGRESSION: CROSSING A CONTINUOUS PREDICTOR WITH A DICHOTOMOUS PREDICTOR 9.4 ANALYSIS OF COVARIANCE 9.5 CONTINUOUS MODERATORS 9.6 SUMMING UP THE IDEA OF INTERACTIONS IN REGRESSION 9.7 DO MODERATORS REALLY “MODERATE” ANYTHING? 9.8 INTERPRETING MODEL COEFFICIENTS IN THE CONTEXT OF MODERATORS 9.9 MEAN‐CENTERING PREDICTORS: IMPROVING THE INTERPRETABILITY OF SIMPLE SLOPES 9.10 MULTILEVEL REGRESSION: ANOTHER SPECIAL CASE OF THE MIXED MODEL 9.11 CHAPTER SUMMARY AND HIGHLIGHTS REVIEW EXERCISES

16 10 LOGISTIC REGRESSION AND THE GENERALIZED LINEAR MODEL 10.1 NONLINEAR MODELS 10.2 GENERALIZED LINEAR MODELS 10.3 CANONICAL LINKS 10.4 DISTRIBUTIONS AND GENERALIZED LINEAR MODELS 10.5 DISPERSION PARAMETERS AND DEVIANCE 10.6 LOGISTIC REGRESSION 10.7 EXPONENTIAL AND LOGARITHMIC FUNCTIONS 10.8 ODDS AND THE LOGIT 10.9 PUTTING IT ALL TOGETHER: LOGISTIC REGRESSION 10.10 LOGISTIC REGRESSION IN R 10.11 CHALLENGER ANALYSIS IN SPSS 10.12 SAMPLE SIZE, EFFECT SIZE, AND POWER 10.13 FURTHER DIRECTIONS 10.14 CHAPTER SUMMARY AND HIGHLIGHTS REVIEW EXERCISES

17 11 MULTIVARIATE ANALYSIS OF VARIANCE 11.1 A MOTIVATING EXAMPLE: QUANTITATIVE AND VERBAL ABILITY AS A VARIATE 11.2 CONSTRUCTING THE COMPOSITE 11.3 THEORY OF MANOVA 11.4 IS THE LINEAR COMBINATION MEANINGFUL? 11.5 MULTIVARIATE HYPOTHESES 11.6 ASSUMPTIONS OF MANOVA 11.7 HOTELLING’S T 2: THE CASE OF GENERALIZING FROM UNIVARIATE TO MULTIVARIATE 11.8 THE COVARIANCE MATRIX S 11.9 FROM SUMS OF SQUARES AND CROSS‐PRODUCTS TO VARIANCES AND COVARIANCES 11.10 HYPOTHESIS AND ERROR MATRICES OF MANOVA 11.11 MULTIVARIATE TEST STATISTICS 11.12 EQUALITY OF COVARIANCE MATRICES 11.13 MULTIVARIATE CONTRASTS 11.14 MANOVA IN R AND SPSS 11.15 MANOVA OF FISHER’S IRIS DATA 11.16 POWER ANALYSIS AND SAMPLE SIZE FOR MANOVA 11.17 MULTIVARIATE ANALYSIS OF COVARIANCE AND MULTIVARIATE MODELS: A BIRD’S EYE VIEW OF LINEAR MODELS 11.18 CHAPTER SUMMARY AND HIGHLIGHTS REVIEW EXERCISES Further Discussion and Activities

18 12 DISCRIMINANT ANALYSIS 12.1 WHAT IS DISCRIMINANT ANALYSIS? THE BIG PICTURE ON THE IRIS DATA 12.2 THEORY OF DISCRIMINANT ANALYSIS 12.3 LDA IN R AND SPSS 12.4 DISCRIMINANT ANALYSIS FOR SEVERAL POPULATIONS 12.5 DISCRIMINATING SPECIES OF IRIS: DISCRIMINANT ANALYSES FOR THREE POPULATIONS 12.6 A NOTE ON CLASSIFICATION AND ERROR RATES 12.7 DISCRIMINANT ANALYSIS AND BEYOND 12.8 CANONICAL CORRELATION 12.9 MOTIVATING EXAMPLE FOR CANONICAL CORRELATION: HOTELLING’S 1936 DATA 12.10 CANONICAL CORRELATION AS A GENERAL LINEAR MODEL 12.11 THEORY OF CANONICAL CORRELATION 12.12 CANONICAL CORRELATION OF HOTELLING’S DATA 12.13 CANONICAL CORRELATION ON THE IRIS DATA: EXTRACTING CANONICAL CORRELATION FROM REGRESSION, MANOVA, LDA 12.14 CHAPTER SUMMARY AND HIGHLIGHTS REVIEW EXERCISES Further Discussion and Activities

19 13 PRINCIPAL COMPONENTS ANALYSIS 13.1 HISTORY OF PRINCIPAL COMPONENTS ANALYSIS 13.2 HOTELLING 1933 13.3 THEORY OF PRINCIPAL COMPONENTS ANALYSIS 13.4 EIGENVALUES AS VARIANCE 13.5 PRINCIPAL COMPONENTS AS LINEAR COMBINATIONS 13.6 EXTRACTING THE FIRST COMPONENT 13.7 EXTRACTING THE SECOND COMPONENT 13.8 EXTRACTING THIRD AND REMAINING COMPONENTS 13.9 THE EIGENVALUE AS THE VARIANCE OF A LINEAR COMBINATION RELATIVE TO ITS LENGTH 13.10 DEMONSTRATING PRINCIPAL COMPONENTS ANALYSIS: PEARSON’S 1901 ILLUSTRATION 13.11 SCREE PLOTS 13.12 PRINCIPAL COMPONENTS VERSUS LEAST‐SQUARES REGRESSION LINES 13.13 COVARIANCE VERSUS CORRELATION MATRICES: PRINCIPAL COMPONENTS AND SCALING 13.14 PRINCIPAL COMPONENTS ANALYSIS USING SPSS 13.15 CHAPTER SUMMARY AND HIGHLIGHTS REVIEW EXERCISES Further Discussion and Activities

20 14 FACTOR ANALYSIS 14.1 HISTORY OF FACTOR ANALYSIS 14.2 FACTOR ANALYSIS AT A GLANCE 14.3 EXPLORATORY VERSUS CONFIRMATORY FACTOR ANALYSIS 14.4 THEORY OF FACTOR ANALYSIS: THE EXPLORATORY FACTOR‐ANALYTIC MODEL 14.5 THE COMMON FACTOR‐ANALYTIC MODEL 14.6 ASSUMPTIONS OF THE FACTOR‐ANALYTIC MODEL 14.7 WHY MODEL ASSUMPTIONS ARE IMPORTANT 14.8 THE FACTOR MODEL AS AN IMPLICATION FOR THE COVARIANCE MATRIX ∑ 14.9 AGAIN, WHY IS ∑ = ΛΛ′ + ψ SO IMPORTANT A RESULT? 14.10 THE MAJOR CRITIQUE AGAINST FACTOR ANALYSIS: INDETERMINACY AND THE NONUNIQUENESS OF SOLUTIONS 14.11 HAS YOUR FACTOR ANALYSIS BEEN SUCCESSFUL? 14.12 ESTIMATION OF PARAMETERS IN EXPLORATORY FACTOR ANALYSIS 14.13 PRINCIPAL FACTOR 14.14 MAXIMUM LIKELIHOOD 14.15 THE CONCEPTS (AND CRITICISMS) OF FACTOR ROTATION 14.16 VARIMAX AND QUARTIMAX ROTATION 14.17 SHOULD FACTORS BE ROTATED? IS THAT NOT CHEATING? 14.18 SAMPLE SIZE FOR FACTOR ANALYSIS 14.19 PRINCIPAL COMPONENTS ANALYSIS VERSUS FACTOR ANALYSIS: TWO KEY DIFFERENCES 14.20 PRINCIPAL FACTOR IN SPSS: PRINCIPAL AXIS FACTORING 14.21 BARTLETT TEST OF SPHERICITY AND KAISER–MEYER–OLKIN MEASURE OF SAMPLING ADEQUACY (MSA) 14.22 FACTOR ANALYSIS IN R: HOLZINGER AND SWINEFORD (1939) 14.23 CLUSTER ANALYSIS 14.24 WHAT IS CLUSTER ANALYSIS? THE BIG PICTURE 14.25 MEASURING PROXIMITY 14.26 HIERARCHICAL CLUSTERING APPROACHES 14.27 NONHIERARCHICAL CLUSTERING APPROACHES 14.28 K‐ MEANS CLUSTER ANALYSIS IN R 14.29 GUIDELINES AND WARNINGS ABOUT CLUSTER ANALYSIS 14.30 A BRIEF LOOK AT MULTIDIMENSIONAL SCALING 14.31 CHAPTER SUMMARY AND HIGHLIGHTS REVIEW EXERCISES Further Discussion and Activities

21 15 PATH ANALYSIS AND STRUCTURAL EQUATION MODELING 15.1 PATH ANALYSIS: A MOTIVATING EXAMPLE—PREDICTING IQ ACROSS GENERATIONS 15.2 PATH ANALYSIS AND “CAUSAL MODELING” 15.3 EARLY POST‐WRIGHT PATH ANALYSIS: PREDICTING CHILD'S IQ (Burks, 1928) 15.4 DECOMPOSING PATH COEFFICIENTS 15.5 PATH COEFFICIENTS AND WRIGHT'S CONTRIBUTION 15.6 PATH ANALYSIS IN R—A QUICK OVERVIEW: MODELING GALTON'S DATA 15.7 CONFIRMATORY FACTOR ANALYSIS: THE MEASUREMENT MODEL 15.8 STRUCTURAL EQUATION MODELS 15.9 DIRECT, INDIRECT, AND TOTAL EFFECTS 15.10 THEORY OF STATISTICAL MODELING: A DEEPER LOOK INTO COVARIANCE STRUCTURES AND GENERAL MODELING 15.11 THE DISCREPANCY FUNCTION AND CHI‐SQUARE 15.12 IDENTIFICATION 15.13 DISTURBANCE VARIABLES 15.14 MEASURES AND INDICATORS OF MODEL FIT 15.15 OVERALL MEASURES OF MODEL FIT 15.16 MODEL COMPARISON MEASURES: INCREMENTAL FIT INDICES 15.17 WHICH INDICATOR OF MODEL FIT IS BEST? 15.18 STRUCTURAL EQUATION MODEL IN R 15.19 HOW ALL VARIABLES ARE LATENT: A SUGGESTION FOR RESOLVING THE MANIFEST‐LATENT DISTINCTION 15.20 THE STRUCTURAL EQUATION MODEL AS A GENERAL MODEL: SOME CONCLUDING THOUGHTS ON STATISTICS AND SCIENCE 15.21 CHAPTER SUMMARY AND HIGHLIGHTS REVIEW EXERCISES Further Discussion and Activities

22 REFERENCES

23 INDEX

24 End User License Agreement

1 Chapter 2 Table 2.1 Contingency Table for 2 × 2 Design Table 2.2 Contingency Table for 2 × 2 × 2 Design... Table 2.3 Contingency Table for 2 × 2 Diagnostic Design Table 2.4 Mathematical versus Discrete Random Variable Table 2.5 Favorability of Movies for Two Individuals in Terms of Ranks Table 2.6 Power Estimates as a Function of Sample Size and Estimated Magnitud... Table 2.7 Conversions for r → r 2 → d. 11 Table 2.8 Matched-Pairs Design Table 2.9 Randomized Block Design

2 Chapter 3 Table 3.1 Achievement as a Function of Teacher Table 3.2 Hypothetical Achievement DataTable 3.3 Summary Table for One‐Way Fixed Effects Analysis of VarianceTable 3.4 Hypothetical Data on Two Independent SamplesTable 3.5 Hypothetical Data on Dependent Variable Y and Independent Variable XTable 3.6 Pairwise Differences Between Achievement Means for Respective Teacher A...Table 3.7 R 2→ f 2→ f Conversions 4Table 3.8 Number of Bacteria Colonies by Plate and Sample (Fisher, 1925/1934)

3 Chapter 4TABLE 4.1 Achievement as a Function of Teacher and TextbookTABLE 4.2 Cell Means of Sleep Onset as a Function of Melatonin Dose and Noise Lev...TABLE 4.3 Cell Means Layout for 2 × 3 Factorial Analysis of Variance...TABLE 4.4 Deviations Featured in One‐way and Two‐way Analysis of Variance...TABLE 4.5 ANOVA Summary Table for Two‐Way Factorial DesignTABLE 4.6 Nested Design: Teacher is Nested Within TextbookTABLE 4.7 Learning as a Function of Trial (Hypothetical Data)TABLE 4.8 Achievement Cell Means Teacher*Textbook