There are several steps in doing EBP, but the number varies a bit by author. First Step.

Social worker Len Gibbs points out that the first step is to become motivated to do EBP. One's introduction might be based on a carrot or a stick or a hammer! If shown the utility of EBP to real world practice, professionals and students would begin with a positive orientation and motives. On the other hand being forced to do EBP by managed care could create considerable resentment. Most authors don't mention this step as part of EBP. Core Steps: Step 1. Develop a clear and answerable question derived from the client's problem or need. Such questions may be about diagnosis, treatment, side effects, prognosis as well as costs and benefits or efficiency of care (though now the concern shifts to overall benefit, not just to the client.) Step 2. Search the literature for relevant research that could help answer this question. The EBP model places greatest credibility in results of randomized controlled trails [RCTs] or meta-analyses of experimental studies. (See Rating the Evidence.) Step 3. Conduct a critical appraisal of this information and rank the evidence for its validity and applicability to the client's need and situation. The client's wishes and needs (stated and implicit) must also be considered, along with the professional's competencies. Step 4. Formulate and apply an intervention based on the most relevant and applicable findings which we can call the "best available evidence." The assumption is that the evidence will clearly point to a best intervention. In many situations the model may point to a "best intervention." However, in many situations the evidence will be lacking, of variable quality or irrelevant, making the yield of this step a bit more ambiguous than the model suggests. Additional Steps: Step 5. A few authors (Gibbs, for one) appear to make practice evaluation an aspect of EBP. That is, the professional should audit the intervention (to verify it was done appropriately) and evaluate its yield. This makes some sense, but note that the practice evaluation of the single case would be done using methods quite different from those used in EBP. Single case or single system designs can help identify progress, but are based on replication logic rather than the sampling logic underlying experimental research. That is, the case studies one would use in practice evaluation are not highly valued in EBP research summaries. Step 6. A few authors also add sharing your results with others and work toward improving the quality of available evidence (Gibbs). This would be useful but again does not necessarily draw on the same core logic of experimental research EBP emphasizes. In fact, case studies are often viewed as the least useful source of evidence in many EBP "evidence hierarchies". Note, however, that such work may be very helpful in identifying to whom and in what circumstances the best research evidence does not work or is not appropriate. Ironically, very small

scale research may be very useful in shaping how and when and where to use large scale experimental evidence to best advantage. The University of Michigan's Social Work Library offers a fine page on the steps of EBP (but also a disclaimer that the pages are not intended as a course for beginners.) Note that all steps are meant to be transparent and replicable by others. That is, the steps should be so clear you could re-do them yourself with enough time and access. It also means many things are accepted at face value (or as face valid) such as definitions of mental and social disorders (usually defined via DSM or ICD) though these categories do change over time. Measures of treatments are assumed to be adequate, valid, reliable and complete. Treatments though often only broadly described, as assumed to be replicable by others in different settings, with different training and with different backgrounds. Note, too, that EBP focuses on the outcome of treatment, not the processes by which change occurs. Understanding both outcome and change process is the cornerstone of science. to Social Work Resources Home Page
text copyright J. Drisko page begun 3/17/04; last update 06/8/08

There are several steps in doing EBP, but the number varies a bit by author. First Step. Social worker Len Gibbs points out that the first step is to become motivated to do EBP. One's introduction might be based on a carrot or a stick or a hammer! If shown the utility of EBP to real world practice, professionals and students would begin with a positive orientation and motives. On the other hand being forced to do EBP by managed care could create considerable resentment. Most authors don't mention this step as part of EBP. Core Steps: Step 1. Develop a clear and answerable question derived from the client's problem or need. Such questions may be about diagnosis, treatment, side effects, prognosis as well as costs and benefits or efficiency of care (though now the concern shifts to overall benefit, not just to the client.) Step 2. Search the literature for relevant research that could help answer this question. The EBP model places greatest credibility in results of randomized controlled trails [RCTs] or meta-analyses of experimental studies. (See Rating the Evidence.) Step 3. Conduct a critical appraisal of this information and rank the evidence for its validity and applicability to the client's need and situation. The client's wishes and needs (stated and implicit) must also be considered, along with the professional's competencies. Step 4. Formulate and apply an intervention based on the most relevant and applicable findings which we can call the "best available evidence." The

assumption is that the evidence will clearly point to a best intervention. In many situations the model may point to a "best intervention." However, in many situations the evidence will be lacking, of variable quality or irrelevant, making the yield of this step a bit more ambiguous than the model suggests. Additional Steps: Step 5. A few authors (Gibbs, for one) appear to make practice evaluation an aspect of EBP. That is, the professional should audit the intervention (to verify it was done appropriately) and evaluate its yield. This makes some sense, but note that the practice evaluation of the single case would be done using methods quite different from those used in EBP. Single case or single system designs can help identify progress, but are based on replication logic rather than the sampling logic underlying experimental research. That is, the case studies one would use in practice evaluation are not highly valued in EBP research summaries. Step 6. A few authors also add sharing your results with others and work toward improving the quality of available evidence (Gibbs). This would be useful but again does not necessarily draw on the same core logic of experimental research EBP emphasizes. In fact, case studies are often viewed as the least useful source of evidence in many EBP "evidence hierarchies". Note, however, that such work may be very helpful in identifying to whom and in what circumstances the best research evidence does not work or is not appropriate. Ironically, very small scale research may be very useful in shaping how and when and where to use large scale experimental evidence to best advantage. The University of Michigan's Social Work Library offers a fine page on the steps of EBP (but also a disclaimer that the pages are not intended as a course for beginners.) Note that all steps are meant to be transparent and replicable by others. That is, the steps should be so clear you could re-do them yourself with enough time and access. It also means many things are accepted at face value (or as face valid) such as definitions of mental and social disorders (usually defined via DSM or ICD) though these categories do change over time. Measures of treatments are assumed to be adequate, valid, reliable and complete. Treatments though often only broadly described, as assumed to be replicable by others in different settings, with different training and with different backgrounds. Note, too, that EBP focuses on the outcome of treatment, not the processes by which change occurs. Understanding both outcome and change process is the cornerstone of science. to Social Work Resources Home Page
text copyright J. Drisko page begun 3/17/04; last update 06/8/08

Steps in the Process of Evidence-Based Practice

Step 1: Framing the Clinical Question

The first step in applying evidence to a clinical decision is framing the specific question about which evidence will be sought. One widely-used approach to framing these questions is known as PICO, for Population, Intervention, Comparison, Outcome. Ensuring that the clinical question addresses all four of these areas will help to ensure that the evidence will be relevant to the particular circumstance faced by the clinician. P opulation Stroke patients I ntervention Early initiation of aphasia treatment C omparison Aphasia treatment after initial/spontaneous recovery time O utcome Functional communication abilities Ability to consistently produce /s/ Return to work/school

Kindergarteners with articulation disorders 17-year-old male with a severe head injury

Individual pull-out Group pull-out treatment treatment Cognitive rehab No cognitive rehab

The first question, then, could be written as, "Are patients with aphasia who received SLP services shortly after their stroke more or less likely to achieve functional communication abilities than stroke patients who received such treatments later?". The most difficult aspect of framing appropriate clinical questions is determining the level of specificity of what goes into each of these categories. To some, it is difficult because it is an art, rather than a science, and usually requires a fair amount of trial and error before the question is in its final form.

Population
The second example was Kindergarteners with articulation disorders. Here, the clinician will need to decide whether he or she wants evidence drawn only from Kindergarteners, or would studies including children one or two years younger or older also be of interest. What about a study of sixth graders? Let's say the specific circumstance that prompted this clinician's search for evidence was the arrival on his/her caseload of a girl who just arrived in the U.S., from China, and has limited English proficiency. Should the evidence that is sought be limited to Englishlanguage learners, or should children whose first language is English also be included?

Intervention and Comparison

The intervention of interest is specified as individual pull-out treatment, and the comparison group pull-out treatment. Here the decision has to be made whether that is specific enough, i.e., whether more information is needed about what actually transpires during those sessions, or perhaps whether it is too specific, and the real issue is whether the child should be treated using a pull-out model (whether individual or group) as

opposed to a classroom-based or collaborative consultation approach.

Outcome
Finally, the outcome of interest as written here is the child's ability to pronounce /s/. Another approach would be to look at more functional outcomes such as the extent to which the teacher felt that the child was better able to participate in classroom activities requiring speech. The two different types of outcomes would likely involve very different types of evidence (for more on this topic, see Robey R (2004). A five-phase model for clinical-outcome research. Journal of Communication Disorders, 37, 401-411). How to make these choices? There are two factors that will influence the final question. The first is the theoretical model that informs the clinician's thinking about this disorder. It is the clinician's expertise that will dictate whether he or she thinks it is important to differentiate between left- and right-hemisphere strokes for a particular question, or to what extent the age of a patient should be considered a factor. This is one important reason that true EBP must involve the expertise and experience of the clinician. Once these initial decisions are made, the second factor comes into play, and that is the availability of evidence. This serves as a reality check on the feasibility of garnering sufficient evidence to address the question as posed. Perhaps, the question was originally posed as elementary-school students with articulation disorders, but it turns out that there is a large body of evidence specific to Kindergartners and/or specific to English-language learners, so that the question can be more focused, and thus more directly relevant to the particular clinical circumstance that prompted it. More common, however, is a dearth of evidence, and the clinician is faced with using his or her own expertise to decide to what extent the criteria can be relaxed (e.g., going from evidence on "left-hemisphere stroke patients" to just stroke patients) while still maintaining relevance. For more on framing clinical questions, see The University of Washington's web-based tutorial, Constructing Well-built Clinical Questions using PICO.

Step 2: Finding the Evidence

Ideally, evidence-based clinical practice guidelines relevant to your clinical question will already exist (see Making the Decision). When that is not the case, however, the clinician needs to seek out scientific evidence to help inform the treatment decision. Two major types of evidence may be useful: Systematic Reviews Individual Studies

Systematic Reviews
Systematic reviews form the basis for evidence-based clinical practice guidelines. They are formal assessments of the body of scientific evidence related to a clinical question, and describe the extent to which various diagnostic or treatment approaches are supported by the evidence, but stop short of making specific recommendations for clinical practice.

They are useful in helping clinicians make treatment decisions in that, when done properly, they have pulled together and in a systematic way characterized the available evidence on a clinical question.

Where to Find Systematic Reviews

ASHA's Evidence Maps ASHA/N-CEP's Compendium of Guidelines and Systematic Reviews Cochrane Collaboration Campbell Collaboration What Works Clearinghouse (U.S. Department of Education) Psychological Database for Brain Impairment Treatment Efficacy speechBITE: Speech Pathology Database for Best Interventions and Treatment Efficacy Evidence-based Communication Assessment and Intervention (EBCAI) Journal

Individual Studies
When clinical practice guidelines or systematic reviews are not available, not current, not trustworthy, and/or not relevant, one can turn to individual studies to seek evidence to help make treatment decisions. The first place to find individual studies would be an online bibliographic database. For health care studies, the best place to start would be MEDLINE, the world's largest online bibliographic database of health related studies. MEDLINE cites over 12 million articles from 4,000 peer-reviewed journals. Unfortunately, publication in a peer-reviewed journal is not a guarantee of scientific quality. It is also important to keep in mind that studies published in English langauge, and particularly American journals, are less likely to include studies with negative findings than are European and other non-English journals. In order to get a comprehensive view of the evidence, both positive and negative, searching a European database such as CINAHL, the Cumulative Index to Nursing and Allied Health Literature, is valuable. Although CINAHL is more likely than MEDLINE to contain studies with negative findings, it is still more likely to publish a study with a positive than a negative finding. To find studies with negative findings, and indeed some with positive findings, one can look to the "gray" literature. The gray literature is the term given to the body of research that is not published in peer-reviewed literature. This can take the form of technical reports, conference proceedings, testimony and other unpublished evidence. Finding the gray literature is a difficult task and is typically done through conversations with content experts, relevant professional groups/organizations, and internet search engines.

Step 3: Assessing the Evidence

Systematic Reviews
There are at least two important factors to keep in mind when assessing a systematic

review. The first is relevance of the review to your specific clinical question (see framing the clinical question). If the brain-injured patient whose care prompted your question is a member of a cultural or linguistic minority, for example, how useful is a brain-injury review that excludes or makes no specific mention of culturally or linguistically diverse populations? If you are treating an autistic teenager, of what relevance are reviews based primarily on studies of younger children? Once again, the expertise and experience of the individual clinician is an absolutely essential part of evidence-based practice. The second factor to consider is who wrote and published the review. While many reviews are produced by academic institutions and interdisciplinary collaborations, others are produced by advocacy groups or payors. It is important to consider who produced the reviews and to what extent they would likely be affected by positive or negative findings. However, reviews eminating from a "trusted" source are no more guaranteed to be of high quality than are reviews coming from a less objective source guaranteed to be flawed.

Individual Studies
As noted elsewhere, publication of a study in a peer-review scientific journal is not a guarantee of quality. Individual studies are generally assessed along two dimensions: level of evidence and study quality. Level of evidence refers to the establishment of a hierarchy of study designs based on the ability of the design to protect against bias. While there is no one universally accepted hierarchy, randomized controlled trials (RCTs) are considered to be the design least susceptible to bias, and various hierarchies follow from there through observational studies and non-experimental designs. The table below is one example of a hierarchy of levels of evidence. Leve Description l Ia Ib IIa IIb III IV Well-designed meta-analysis of >1 randomized controlled trial Well-designed randomized controlled study Well-designed controlled study without randomization Well-designed quasi-experimental study Well-designed non-experimental studies, i.e., correlational and case studies Expert committee report, consensus conference, clinical experience of respected authorities

Adapted from the Scottish Intercollegiate Guidelines Network Study quality is an assessment of the extent to which a study, of whatever design, was designed and implemented appropriately. Again, there is no single universally accepted set of criteria for what constitutes a high quality study. For examples of study quality criteria, see the Scottish Intercollegiate Guidelines Network.

Step 4: Making the Decision

Finally, the time will come to combine clinical expertise, the patient's perspective, and the available scientific evidence in making a specific clinical decision with a specific patient. In some instances, evidence-based clinical guidelines will already have been developed on topics related to your particular question. There are at least three important factors to keep in mind when considering whether and to what extent to follow the guidance contained in such documents. The first is relevance of the guideline to your specific clinical question (see framing the clinical question and EBP Compendium). If the brain-injured patient whose care prompted your question is a member of a cultural or linguistic minority, for example, how useful are brain-injury guidelines that exclude or make no specific mention of culturally or linguistically diverse populations? If you are treating an autistic teenager, of what relevance are guidelines developed primarily on the basis of evidence from younger children? Once again, the expertise and experience of the individual clinician is an absolutely essential part of evidence-based practice. The second consideration is the extent to which clinical practice guidelines are truly evidenced-based. Many guidelines are produced via an expert consensus process or other non-systematic approaches. While expert consensus can certainly be a valuable source of information, the conclusions are particularly vulnerable to the biases held by the "experts", and history is full of examples of such conclusions being simply wrong. If a guideline is truly evidence-based, the methodology by which evidence was identified and evaluated should be transparent. Unfortunately, transparency in itself is not a guarantee of quality. It can be a challenge for clinicians to determine what is and is not a high quality evidence-based practice guideline. Systems for evaluating practice guidelines have been developed and can be useful tools to help determine whether a guideline should be applied. The most prominent of these tools is the Appraisal of Guidelines Research and Evaluation (AGREE) framework. AGREE was developed by the European Union and has subsequently been endorsed by the US Agency for Health Research and Quality (AHRQ). The third factor to consider is who wrote and published the guideline. While many guidelines are produced by academic institutions and interdisciplinary collaborations, others are produced by advocacy groups or payors. It is important to consider who produced the guidelines and to what extent they would likely be affected by positive or negative recommendations. However, guidleines eminating from a "trusted" source are no more guaranteed to be of high quality than are guidelines coming from a less objective source guaranteed to be flawed. Application of the AGREE or other objective criteria should be the final determinant of the guideline's quality.

Where to Find Evidence-Based Practice Guidelines

N-CEP's Compendium of Guidelines and Systematic Reviews (ASHA) Scottish Intercollegiate Guidelines Network (SIGN) The National Guideline Clearinghouse

American Academy of Pediatrics U.S. Department of Veterans Affairs Academy of Neurologic Communication Disorders and Sciences (ANCDS) National Electronic Library for Health (National Health Service of the UK) Royal College of Speech-Language Therapists

When guidelines are either inapplicable or non-existent, then the clinician must rely on the evidence identified and assessed in the previous steps (Finding the Evidence and Assessing the Evidence). There typically is no magic formula for determing how much evidence is "enough." Factors such as patient preference, cost effectiveness, potential for harm and availability of alternative treatments all come into play in ultimately making the treatment decision