Daniel J. Denis - Applied Univariate, Bivariate, and Multivariate Statistics

There is often stated a distinction between the so‐called “soft” sciences and the “hard” sciences (Meehl, 1967). The distinction, as is true in many cases of so many things, is fuzzyand blurryand requires deeper analysis to fully understand the issue. The difference between what is “soft” and what is “hard” science has usually only to do with the objectof study, and not with the method of analytical inquiry.

For example, consider what distinguishes the scientist who studies temperature of a human organism compared to a scientist who studies the self‐esteem of adolescents. Their analytical approaches, at their core, will be remarkably similar. They will both measure, collect data, and subject that data to curve‐fitting or probabilistic analysis (i.e., statistical modeling). Their objects, however, are quite different. Indeed, some may even doubt the measurabilityof something called “self‐esteem” in the first place. Is self‐esteem real? Does it actually exist? At the heart of the distinction, really, is that of measurement. Once measurement of an object is agreed upon, the debate between the hard and soft sciences usually vanishes. Both scientists, natural and social, are generally aiming to do the same thing, and that is to understand, document phenomena, and to identify relationsamong these phenomena. As Hays (1994) put it so well, the overreaching goal of science, at its core, is to determine what goes with what. Virtually every scientific investigation you read about has this underlying goal but may operationalize and express it in a variety of different ways.

Social science is a courageous attempt.Hard sciences are, in many respects, much easier than the softer social sciences, not necessarily in their subject matter (organic chemistry isdifficult), but rather in what they attempt to accomplish. Studying beats‐per‐minute in an organism is relatively easy. It is not that difficult to measure. Studying something called intelligenceis much, much harder. Why? Because even arriving at a suitable and agreeable operational definition of what constitutes intelligence is difficult. Most more or less agree on what “heart rate” means. Fewer people agree on what intelligence really means, even if everyone can agree that some people have more of the mysterious quality than do others. But the study of an object of science should imply that we can actually measure it. Intelligence, unlike heart rate, is not easily measured largely because it is a constructopen to much scientific criticism and debate. Even if we acknowledge its existence, it is a difficult thing to “tap into.”

Given the difficulty in measuring social constructs, should this then mean the social scientist give up and not study the objects of his or her craft? Of course not. But what it does mean is that she must be extremely cautious, conservative, and tentativeregarding conclusions drawn from empirical observations. The social scientist must be up front about the weaknesses of her research and must be very careful not to overstateconclusions. For instance, we can measure the extent to which melatonin, a popular sleep aid, reduces the time to sleep onset (i.e., the time it takes to fall asleep). We can perform experimental trials where we give some subjects melatonin and others none and record who falls asleep faster. If we keep getting the same results time and time again across a variety of experimental settings, we begin to draw the conclusion that melatonin has a role in decreasing sleep onset. We may not know whythis is occurring (maybe we do, but I am pretending for the moment we do not), but we can be reasonably sure the phenomenon exists, that “something” is happening.

Now, contrast the melatonin example to the following question— Do people of greater intelligence, on average, earn more money than those of lesser intelligence? We could correlate a measure of intelligence to income, and in this way, we are proceeding in a similar empirical(even if not experimental, in this case) fashion as would the natural scientist. However, there is a problem. There is a bigproblem. Since few consistently agree on what intelligence isor how to actually measure it, or even whether it “exists” in the first place, we are unsure of where to even begin. Once we agree on what IQ is, how it is measured, and how we will identify it and name it, the correlation between IQ and income is as reputable and respectable as the correlation between such variables as height and weight. It is getting to the very measurementof IQ that is the initial hard, and skeptics would argue, impossible part. But we know this already from experience. Convincing a parent that her son has an elevated heart rate is much easier than convincing her that her son has a deficit in IQ points. One phenomenon is measurable. The other, perhaps so, but not nearly as easily, or at minimum, agreeably.

Our point is that once we agree on the existence, meaning, and measurement of objects, soft science is just as “hard” as the hard sciences. If measurement is not on solid ground, no analytical method of its data will save it. All students of the social (and natural, to some extent) sciences should be exposed to in‐depth coursework on the theory, philosophy, and importance of measurementto their field before advancing to statistical applications on these objects, since it is in the realm of measurement where the true controversies of scientific “reputability” usually lay. For general readable introductions to measurement in psychology and the social sciences, the reader is encouraged to consult Cohen, Swerdlik, and Sturman (2013), Furr and Bacharach (2013), and Raykov and Marcoulides (2011). For a deeper and philosophical treatment, which includes measurement in the physical sciences as well, consult Kyburg (2009). McDonald (1999) also provides a relatively technical treatment.

One of the most prominent advances in social statistics is that of structural equation modeling. With SEM, as we will survey in Chapter 15, one can model complex networks of variables, both measurableand unmeasurable. Structural equation modeling is indeed one of the most complex of statistical methods in the toolkit of the social scientist. However, it is a perfectly fair and reasonable question to ask whether structural equation modeling has helped advance the causeof social science. Has it increased our knowledge of social phenomena? Advanced as the tool may be statistically, has the tool helped social science build a bigger and better house for itself?

Such a question is open to debate, one that we will not have here. What needs to be acknowledged from the outset, however, is that statistical complexity has little, if anything, to do with scientific complexityor the guarantee of scientific advance. Indeed, the two may even rarely correlate. A classic scenario is that of the graduate student running an independent‐samples t ‐test on well operationally defined experimental variables, yet feeling somewhat “embarrassed” that he used such a “simple” statistical technique. In the lab next door, another graduate student is using a complex structural equation model, struggling to make the model identifiable through fixing and freeing parameters at will, yet feeling as though she is more “sophisticated” scientifically as a result of her use of a complex statistical methodology. Not the case. True, the SEM user may be more sophisticated statistically(i.e., SEM isharder to understand and implement than t ‐tests), but whether her empirical project is advancing our state of knowledge more than the experimental design of the student using a t ‐test cannot even begin to be evaluated based on the statistical methodology used. It must instead be based on scientific meritand the overall strength of the scientific claim. Which scientific contribution is more noteworthy? That is the essential question, not the statistical technique used. The statistics used rarely have anything to do with whether good science versus bad science was performed. Good science is good science, which at times may require statistical analysis as a tool for communicating its findings.