Allen Rubin - Practitioner's Guide to Using Research for Evidence-Informed Practice

Some searches will fail to find any rigorous studies that clearly supply strong evidence supporting the effectiveness of a particular intervention approach. Perhaps, instead, you might find many seriously flawed studies, each of which supports the effectiveness of a different intervention approach. Some searches might just find what interventions are ineffective (at least those searches might help you in deciding what not to do).

Some searches might find that the best scientific evidence supports an intervention approach that doesn't fit some aspect of your practice situation. For example, although exposure therapy and EMDR both have strong evidence for their effectiveness in treating PTSD, some clients refuse to participate in those therapies because they fear that the treatment process will be too painful in requiring them to recall and discuss the details of the trauma or perhaps visit places in vivo that resemble the site of the trauma (clinicians often succeed in helping clients surmount their fears of these therapies, but that is not always the case). Also, as noted earlier, these interventions can be harmful to clients with substance abuse disorders or who are suicidal if they are provided to such clients before those comorbid disorders are alleviated.

Likewise, some interventions with the best evidence might never have been evaluated with a population of clients like yours, and your clients might have attributes that in some important ways are not like the attributes of those clients who participated in the evaluations. Suppose, for example, you reside in Alaska and want to start a program to treat Native Alaskan girls who have been victims of physical or sexual abuse and who suffer from PTSD. If you search the literature for effective treatments for PTSD, you are likely to find that the best evidence supports the effectiveness of interventions such as exposure therapy, EMDR, or cognitive restructuring. We say the “best” evidence because those interventions are likely to have been supported by the most scientifically rigorous outcome evaluations. However, in a search completed in preparing for a talk on EBP that Rubin presented in Anchorage, Alaska, in 2006, he found no rigorous evaluations of the foregoing evaluations in which Native Alaskans participated.

He did, however, find numerous articles discussing the high prevalence of comorbidity with substance abuse among physically or sexually abused Native Alaskan girls. That illustrates another difficulty. Most of the evaluations offering the best evidence regarding the effectiveness of these treatments have excluded participants whose PTSD was comorbid with substance abuse. Thus, you would face a double whammy in trying to develop your treatment program based on the best evaluations. You would have serious doubts as to whether the findings of those studies can be generalized to Native Alaskan girls or girls with comorbidity. Even if the ethnicity issue didn't matter, the comorbidity issue might matter a great deal.

Even if you can't find the best sorts of evidence supporting the effectiveness of an intervention with clients just like yours, you still can operate from an EIP framework. One option would be to look for less rigorous evaluations that have involved clients like yours and which – while not offering the best evidence from a scientific standpoint – are not fatally flawed and thus offer some credible evidence supporting a particular intervention. If that option doesn't pan out, an alternative would be to use your practice judgment in deciding whether an intervention supported by the best evidence with clients unlike yours seems to be worth proposing to your client. If you monitor client progress (or lack thereof) during your client's treatment, you can change course if the intervention is not achieving the desired result. When you do discover a lack of evidence specific to your particular client population or target problem or problems, you may even be inspired to partner with researchers to test interventions and contribute to the research evidence. Novel practices can come from practitioners who are frustrated with the limitations of the interventions or the currently available research evidence.

As we mentioned earlier, in your real world of everyday practice, you may encounter some practical obstacles limiting your ability to implement the EIP process in an ideal fashion. Some strategies for overcoming those obstacles are presented in Box 2.2.

The time and cost it takes to learn and provide some interventions, however, might be out of your reach. For some of the most effective interventions, the evidence supporting them is based on evaluations involving clinicians who – as part of the research study and before they delivered the interventions to the research participants – received extremely extensive and costly training and repeated practice under supervision and monitoring. Franklin and Hopson (2007) pointed out that this training process “is too slow, cumbersome, inflexible, and time intensive for many community-based organizations” (p. 8). As an example, they cite Brief Strategic Family Therapy, which “has considerable research support for Hispanic families who need help with an adolescent with a drug abuse problem … but costs $4,000 per therapist to participate in the training” (p. 10). Moreover, agencies may hesitate to invest in costly training for their practitioners if they fear that practitioners will leave the agency and take their training with them.

Cut some corners. If you lack the time and access to bibliographic databases needed to conduct a thorough search for and appraisal of evidence, you might need to cut some corners. Instead of searching for and critically appraising individual studies, for example, you might choose to rely on books, reviews, and practice guidelines developed by EIP experts who have reviewed the literature, identified interventions supported by the best evidence, and described the nature of that evidence. Table 2.3lists some Web sites that can be useful in accessing such materials. As we mentioned earlier, however, authors of some sources might have a vested interest in promoting or debunking a particular treatment modality, and you should be cautious when relying exclusively on those sources. You should rely mainly on resources known for their objectivity, some of which we identified when discussing Step 2 of the EIP process. Chapter 8of this book discusses what to look for in appraising whether a particular review has sufficient credibility to merit guiding practice.

Read titles and abstracts. As we mentioned earlier, when conducting your search, you don't have to read every study that you find. You can examine their titles and abstracts to ascertain which ones are worth reading. You can decide which studies to read based on the relevance of the study to your practice question as well as any mention in the abstract of attributes that might tip you off about the quality of the study.

Work with a team. Another strategy to improve the feasibility of the EIP process is to work with a team of practitioners to move through the EIP process. Taking on each of the steps can feel daunting if you go it alone, but working with colleagues to split up the effort can be more efficient and provide you with a forum where you can discuss what you find and problem solve the challenges along the way. Your team could include colleagues at your agency or a group of independent practitioners serving similar clients.

Use manuals or checklists. Before you can provide the interventions that have the best supportive evidence, you might have to learn more about them. You can start by obtaining readings on how to implement the intervention or consulting with colleagues who have used the intervention. Some interventions have treatment manuals that provide very specific step-by-step guidance. Increasingly, many interventions also provide fidelity checklists and other tools designed to measure whether or not you are delivering the intervention as intended.