Keith Stanovich - How to Think Straight About Psychology

Knowledge in most subareas of psychology, such as cognitive psychology, developmental psychology, organizational psychology, personality psychology, and social psychology, is stated in terms of actuarial predictions. In contrast, some subgroups of clinical psychological practitioners claim to be able to go beyond group predictions and to make accurate predictions of the outcomes of particular individuals. This is called clinical, or case, prediction . When engaged in clinical prediction, as opposed to actuarial prediction, professional psychologists claim to be able to make predictions about individuals that transcend predictions about “people in general” or about various categories of people. . . . Where professional psychologists differ is in their claim to understand the single individual as unique rather than as part of a group about which statistical generalizations are possible. They claim to be able to analyze “what caused what” in an individual’s life rather than to state what is “in general” true. (Dawes, 1994, pp. 79–80)

Clinical prediction would seem to be a very useful addition to actuarial prediction. There is just one problem, however. Clinical prediction doesn’t work.

For clinical prediction to be useful, the clinician’s experience with the client and her or his use of information about the client would have to result in better predictions than we can get from simply coding information about the client and submitting it to statistical procedures that optimize the process of combining quantitative data in order to derive predictions. In short, the claim is that the experience of psychological practitioners allows them to go beyond the aggregate relationships that have been uncovered by research. The claim that clinical prediction is efficacious is, thus, easily testable. Unfortunately, the claim has been tested, and it has been falsified.

Research on the issue of clinical versus actuarial prediction has been consistent. Since the publication in 1954 of Paul Meehl’s classic book Clinical Versus Statistical Prediction , six decades of research consisting of over a hundred research studies have shown that, in just about every clinical prediction domain that has ever been examined (psychotherapy outcome, parole behavior, college graduation rates, response to electroshock therapy, criminal recidivism, length of psychiatric hospitalization, and many more), actuarial prediction has been found to be superior to clinical prediction (Kahneman, 2011; Morera & Dawes, 2006; Swets et al., 2000; Tetlock, 2005).

In a variety of clinical domains, when a clinician is given information about a client and asked to predict the client’s behavior, and when the same information is quantified and processed by a statistical equation that has been developed based on actuarial relationships that research has uncovered, invariably the equation wins. That is, the actuarial prediction is more accurate than the clinician’s prediction. In fact, even when the clinician has more information available than is used in the actuarial method, the latter is superior. That is, when the clinician has information from personal contact and interviews with the client, in addition to the same information that goes into the actuarial equation, the clinical predictions still do not achieve an accuracy as great as the actuarial method. The reason is of course that the equation integrates information accurately and consistently. This factor— consistency—can overcome any informational advantage the clinician has from information gleaned informally.

Regarding the research showing the superiority of actuarial prediction over clinical prediction, Paul Meehl (1986) said, “There is no controversy in social science which shows such a large body of qualitatively diverse studies coming out so uniformly in the same direction as this one” (pp. 373–374). Yet, embarrassingly, the field of psychology does not act on this knowledge. For example, the field continues to use personal interviews in the graduate admissions process and in the mental-health-training admissions process, even though voluminous evidence suggests that these interviews have virtually no validity. Instead, practitioners continue to use specious arguments to justify their reliance on “clinical intuition” rather than aggregate predictions that would work better. For example, Dawes, Faust, and Meehl (1989) noted,

A common anti-actuarial argument, or misconception, is that group statistics do not apply to single individuals or events. The argument abuses basic principles of probability. . . . An advocate of this anti-actuarial position would have to maintain, for the sake of logical consistency, that if one is forced to play Russian roulette a single time and is allowed to select a gun with one or five bullets in the chamber, the uniqueness of the event makes the choice arbitrary. (p. 1672)

An analogy to the last point would be to ask yourself how you react to the scientific findings that the probability of a successful kind of surgery is higher for surgeons that perform many of that particular type of surgical operation (Grady, 2009; Groopman, 2007). Would you rather have your operation done by a surgeon A, who is practiced in that type of surgery and has a low failure probability, or by surgeon B, who is unpracticed in that type of surgery and has a high failure probability? If you believe that “probabilities don’t apply to the single case,” you shouldn’t mind having your surgery done by the unpracticed surgeon.

The field of psychology has little to lose in prestige by admitting the superiority of actuarial to clinical judgment in a domain such as predicting psychotherapeutic outcome because the same is true of professionals in numerous other domains as varied as medicine, business, criminology, accounting, and livestock judging. Although the field as a whole would have little to lose, individual practitioners who engage in activities in the role of “experts” (i.e., in courtroom testimony) and imply that they have unique clinical knowledge of individual cases would, of course, lose prestige and perhaps income.

In fact, the field, and society, would benefit if we developed the habit of “accepting error in order to reduce error.” In attempting to find unique explanations of every single unusual case (unique explanations that simply may not be possible given the present state of our knowledge), we often lose predictive accuracy in the more mundane cases. Recall the red–blue light experiment again. The “100 percent red strategy” makes incorrect predictions of all of the minority or unusual events (when the blue lights flash). What if we focused more on those minority events by adopting the “70-percentred-30-percent-blue strategy”? We would now be able to predict 9 of those 30 unusual events (30 times 0.30). But the cost is that we lose our ability to predict 21 of the majority events. Instead of 70 correct predictions of red, we now have only 49 correct predictions (70 times 0.70). Predictions of behavior in the clinical domain have the same logic. In concocting complicated explanations for every case, we may indeed catch a few more unusual cases—but at the cost of losing predictive accuracy in the majority of cases, where simple actuarial prediction would work better.

Wagenaar and Keren (1986) illustrated how overconfidence in personal knowledge and the discounting of statistical information can undermine safety campaigns advocating seat belt use because people think, “I am different, I drive safely.” The problem is that the vast majority of the population thinks that they are “better than the average driver” (De Craen, Twisk, Hagenzieker, Elffers, & Brookhuis, 2011)—obviously a patent absurdity.

The same fallacy of believing that “statistics don’t apply to the single case” is an important factor in the thinking of individuals with chronic gambling problems. In his study of gambling behavior, Wagenaar (1988) concluded,