OverviewofEvidence

BasedPractice
Charles Wilson, MSSW, Executive Director of Chadwick Center
TheSamandRoseSteinChaironChildProtection
Rady Childrens Hospital-San Diego

www.cachildwelfareclearinghouse.org

HowThingsChange
AProblemis
Recognized
ActionAnyAction

ActionCreationofOrphanTrains

Between1854and1929100,000200,000
childrenwereplacedinnewfamiliesviathe
OrphanTrains.

http://www.orphantraindepot.com

Children were taken in small groups of 10 to 40,

under the supervision of at least one adult, and
traveled on trains to selected stops along the way,
where they were taken by families in that area.
http://www.pbs.org/wgbh/amex/orphan/teachers.html

HowThingsChange
AProblemis
Recognized
ActionAnyAction

InformedAction

SeriesofTrail
andErrors
Adjustments
SomeBetter
SomeWorse

Family
Foster
Care
TrialandError

Orphanagesand
Boardingschools

TennesseePreparatorySchoolforDependentChildren

HowThingsChange
AProblemis
Recognized

InformedAction
InformedActionBasedonScience

Sohowdoweknowwhatworks
vs.
meremarketingmarketing
hyperbole?

LettheBuyer
Beware

ThoughtFieldTherapy

Thought field therapy with Callahan techniques is a powerful therapy

exerted through nature's healing system to balance the body's energy
system. This therapy promotes stress management and stress relief as
well as the reduction or elimination of anxiety and anxiety related
problems.
This
Roger
J. Callahan,
PhDincludes help for weight control and weight loss, trauma
or sleep difficulties, depression, addictions and the disorders
associated with past trauma including nightmares and post traumatic
stress disorder.
(underlines added)
Retrieved from http://www.tftrx.com/, November 17, 2006

MoreClaimsforTFT
Q. How Can TFT Benefit You? What Kind of Problems Can Be Helped?

Anxiety and Stress

Personal fears or your childrens fears
Anger and Frustration
Eating or smoking or drinking problems
Loss of loved ones
Social or public speaking fears
Sexual or intimacy problems
Travel anxiety including fear of flying or driving on the freeways
Nail biting
Cravings
Low moods and mood swings

Retrieved from http://www.tftrx.com/profaq.php?PHPSESSID=

f4cf66c40b9678b742b82989fee7b377# on November 17, 2006

NPRAllThingsConsidered,
March29,2006
AccordingtopsychologistRogerCallahan,the
creatorofthoughtfieldtherapy,majorproblemslike
depressioncanbecuredquicklywiththismethod.He
saysposttraumaticstressdisorderiseasilydispatched
in15minutes,andeventhemostseriouscasesof
anxiety,addictionandphobiasarelikewisesubjectto
quarterhourcures.

ResearchonTFT?
Has any research been carried out on TFT?
There have been no control (sic) studies on the
success of TFT
From the Thought Field Therapy Training Center of La Jolla

Retrieved from http://thoughtfield.com/faqs.htm on November 17, 2006

Distinguishinggroundless
marketingclaimsfromreality

The Problem:
All sorts of
intervention
s are
available out
there.

WaitingRoomSign

BenSaunders
MUSC

EvidenceBasedSocialWork
Professionaljudgmentsandbehaviorsshouldbeguided

bytwointerdependentprincipals:
1. Wheneverpossible,practiceshouldbegroundedon
priorfindingsthatdemonstrateempiricallythatthey
arelikelytoproducepredictable,beneficial,and
effectiveresults.
2. Everyclientssystem,overtimeshouldbeevaluated
Evidence Based Practice Manual
Oxford University Press
2004
Albert Roberts, PhD
Kenneth Yeager, PhD, LISW

GlobalDefinitionofEBP
Theconscientious,explicitandjudicioususeofcurrentbest
evidenceinmakingdecisionsaboutthecareofindividualpatients.
IncludingBoth

Individual clinical expertise

Thebestavailableclinical
evidencefromsystematic
research

DavidSackett

HugePolicyImplications
ShouldpolicymakerssupportadoptionofEBP?

Ifso,whichonesWhenaretheyReadyforPrimetime
Whatisthestandardofevidence?

Ifso,howbestcantheysupportadoption?

Whatarethepitfallsofastateornationalpolicy
leveladoptionofEBP?
ImpactonInnovation
Misapplicationofgoodmodels?Onesizedoesnotfitall
Wateringdownofempiricallybasedpracticedangerof
implementinginnameonly
Ideologyvs.Sciencewhoisthejudgeofthescience?
ShouldwelimitwhatwedotoEBP?

Parachuteusetopreventdeathand
majortraumarelatedtogravitational
challenge:systematicreviewof
randomizedcontrolledtrials

(GordonCSmith,JillPPell,2005)

Theperceptionthatparachutesareasuccessfulinterventionisbased.
largelyonanecdotalevidence

Observationaldatahaveshownthattheiruseisassociatedwith,morbidityand
mortalityduetobothfailureoftheinterventionandmechanicalcomplications.In
addition,naturalhistory"studiesoffreefallindicatethatfailuretotakeor
deployaparachutedoesnotinevitablyresultinanadverseoutcome...

Theeffectivenessofaninterventionhastobejudgedrelativetonon
intervention.

Understandingthenaturalhistoryoffreefallisthereforeimperative.
Iffailuretouseaparachutewereassociatedwith100%mortalitythenany
survivalassociatedwithitsusemightbeconsideredevidenceofeffectiveness.

Therefore,studiesarerequiredtocalculatethebalanceofrisksand
benefitsofparachuteuse.

Why Evidence-Based Practice Now?

A growing body of scientific knowledge
Increased interest in consistent application of
quality services
Increased interest in outcomes and
accountability by funders
Past missteps in spreading untested best
practices that turned out not to be as effective as
advertised
Because they work !!

ProblemsintheChildAbuse
FieldintheU.S.
Empiricalevidenceofefficacyhasnotbeenacommoncriteriafortreatment
selectioninthechildmaltreatmentfield.
Lackofoutcomeresearchformanycommonlyusedinterventions.
Readywillingnessamongsometouse,embrace,promote,andstaunchly
defendpracticesthathavenoevidencefortheirefficacyandquestionable
theoreticalbases.
Poordisseminationofthesignificantclinicaloutcomeresearchthathasbeen
done.
Ineffectiveapproachestocontinuingeducation.
Pooradoptionofempiricallysupportedtreatmentsinrealworldclinical
settings.
Disconnectionbetweencurrentscientificknowledgeandpracticeinthefield.

ScaredStraight

TFCBT

ReactiveAttachmentDisorderand
AttachmentTherapy

pioneered by psychoanalyst Aaron Lederer, the RAD

Consultancys creator and director. His methods yield
remarkable results within weeks.

Retrieved from http://www.radconsultancy.com/, November 17, 2006

Why should we worry

about using Evidence
Supported Treatments?

InstituteofMedicine:

ApplythePrinciplesandMethodsof
EvidenceBasedPractice
Integration of:

Best Research Evidence

Best Clinical Experience
Consistent with Client Values
http://www.shef.ac.uk/scharr/ir/netting/
http://ebmh.bmj.com/
http://cebmh.com/
http://www.cebm.utoronto.ca/

Understand Adoption of Innovation

MTFC
1991
Innovators
Early
Late
Majority Majority
Early
Adopters

Traditionalists

CommonErrorsWhenDeciding
aboutInterventionEffectiveness
Reliancesolelyonindividualanecdotesandrememberedcases.
Thatchildmadesuchamazingchangesduringtreatment.
Confusingclientsatisfactionwithclinicalimprovement.
Thefamilyjustlovedcomingtotherapy.Nevermissedasession
duringtheir3yearsoftherapy.Amazing.Toobadtheyhadtomove
away.
Misattributionofthecauseofchange.
Failuretoappreciateresilienceandnaturalrecovery.
Thefamilygotmultipleservicesandwraparoundcare.
WithtreatmentherPTSDresolvedinabout3monthsaftertherape.
Gurueffectintrainingandtreatmentadoption.
IheardDr.McDreamyisdoingalevelIItraining.And,itsinSan
DiegoinJanuary!
Thosevideoswerejustsoamazing!Ihavegottotrythat.
BenSaunders
MUSC

WhattolookforinaPractice?
Treatmentorinterventionprotocolthathasatleastsomescientific,empirical
researchevidenceforitsefficacywithitsintendedtargetproblemsand
populations.
Evidencemaybebasedonavarietyofresearchdesigns.
RandomizedClinicalTrial(RCT)
Controlledstudieswithoutrandomization
Opentrials,prepost,oruncontrolledstudies
Multiplebaseline,singlecasedesigns
Thedegreetowhichwearepersuadedthatthetreatmentiseffectivewillvaryby
thequalityofempiricalsupport.
NumberofRCTs
Replicationbyresearchersotherthanthetreatmentdevelopers
Sampling,samplesizeused,comparisontreatment,effectsize
Variousmethodshavebeendevelopedforclassifyingthelevelofempirical
supportenjoyedbytreatmentapproaches.
Shouldbeusefulforfrontlinepractitioners

CEBCWebsite:www.cachildwelfareclearinghouse.org

CurrentDataonVisitorstotheWebsite
TotalNumberofVisitstotheWebsite
46,635

Percentage of Total Visitors from over

131 International Countries
Percentage of Total
Visitors from U.S.

14%

86%
Percentage of Total Visitors
from California

33%

Data based on numbers as of

September 1, 2007

CEBCs Definition of Evidence-Based

Practice for Child Welfare
Best Research Evidence
Best Clinical Experience
Consistent with Family/ Client Values

(modified from The Institute of Medicine)

http://www.iom.edu/

The California Evidence-Based

Clearinghouse
for Child Welfare (CEBC)
In 2004, the California Department of Social Services,
Office of Child Abuse Prevention contracted with the
Chadwick Center
for Children and Families, Rady
Childrens Hospital-San Diego in cooperation with the
Child and Adolescent Services Research Center to create
the CEBC.
The CEBC was launched on 6/15/06.

Advisory Committee
TheAdvisoryCommitteeiscomposedof15membersdrawnfromabroadcross
representationofcommunitiesandorganizations.
Therearerepresentativesfrom:
CaliforniaDepartmentofSocialServices
ChildWelfareDepartmentsfromCaliforniaCounties
ChildWelfareDirectorsAssociation(CWDA)
CaliforniaChildWelfareTrainingLeaders
PublicandPrivateCommunityPartnersWithintheState
TheroleoftheAdvisoryCommitteeisto:
DeterminethetopicalareasfortheCEBC
EnsuretheCEBCremainsuptodatewithemergingevidence.
AssistindisseminatingtheproductsoftheCEBC.
ProvidefeedbackontheutilityoftheCEBCproducts.

National
Scientific Panel
The National Scientific Panel is composed of five core
members and up to 10 selected Topical Experts.
The Panel is nationally recognized as leaders in child
welfare research and practice, and who are
knowledgeable about what constitutes best
practice/evidence-based practice.
The Panel assists in identifying relevant practices and
research and provide guidance on the scientific
integrity of the CEBC products.

Scientific Rating Scale

and
Relevance to Child Welfare Scale

Rating Scale Development

Goals:
Multiple categories
High standard for top ratings Randomized
Controlled Trials
Clearly defined criteria
Focus on peer-reviewed research and ability to
replicate program

Gold Standard for Evidence

Randomized controlled trial (RCT)
Participants are randomly assigned to either an
intervention or control group. This allows the
effect of the intervention to be studied in
groups of people who are the same, except for
the intervention being studied.
Any differences seen in the groups at the end can
be attributed to the difference in treatment alone,
and not to bias or chance.

Peer-Reviewed Research
PeerreviewAprocessusedtocheckthe
qualityandimportanceofresearchstudies.It
aimstoprovideawidercheckonthequality
andinterpretationofastudybyhavingother
expertsinthefieldreviewtheresearchand
conclusions.

Efficacy vs. Effectiveness

Efficacyfocusesonwhetheranintervention
worksunderidealcircumstancesandlooksat
whethertheinterventionhasanyimpactatall.
Effectivenessfocusesonwhetheratreatment
workswhenusedintherealworld.
Aneffectivenesstrialisdoneaftertheintervention
hasbeenshowntohaveapositiveeffectinan
efficacytrial.

ScientificRatingScale

6.ConcerningPractice

If multiple outcome studies have been conducted, the overall

weight of evidence suggests the intervention has a negative
effect upon clients served.

and/or

There is a reasonable theoretical, clinical, empirical, or

legal basis suggesting that, compared to its likely benefits,
the practice constitutes a risk of harm to those receiving it.

5.EvidenceFailstoDemonstrateEffect

Two or more randomized, controlled outcome studies (RCT's)

have found that the practice has not resulted in improved
outcomes, when compared to usual care.

If multiple outcome studies have been conducted, the overall

weight of evidence does not support the efficacy of the
practice.

4.Acceptable/EmergingPractice
EffectivenessisUnknown

There is no clinical or empirical evidence or theoretical basis

indicating that the practice constitutes a substantial risk of
harm to those receiving it, compared to its likely benefits.

The practice has a book, manual, and/or other available

writings that specifies the components of the practice
protocol and describes how to administer it.

The practice is generally accepted in clinical practice as

appropriate for use with children receiving services from
child welfare or related systems and their parents/caregivers.

The practice lacks adequate research to empirically

determine efficacy.

3.PromisingPractice
Same basic requirements as Level 4 plus:

At least one study utilizing some form of control (e.g.,

untreated group, placebo group, matched wait list) has
established the practices efficacy over the placebo, or found it
to be comparable to or better than an appropriate comparison
practice. The study has been reported in published, peerreviewed literature.

Outcome measures must be reliable and valid, and

administered consistently and accurately across all subjects.

If multiple outcome studies have been conducted, the overall

weight of evidence supports the efficacy of the practice.

2.WellSupportedEfficaciousPractice
Same basic requirements as Level 3 plus:

Randomized controlled trials (RCTs): At least 2 rigorous

RCTs in highly controlled settings (e.g. University laboratory)
have found the practice to be superior to an appropriate
comparison practice.
-The RCTs have been reported in published, peer-reviewed
literature.

The practice has been shown to have a sustained effect at

least one year beyond the end of treatment, with no evidence
that the effect is lost after this time.

1.WellsupportedEffectivePractice

Same basic requirements as a Level 2 plus:

Multiple Site Replication: At least 2 rigorous randomized

controlled trials (RCTs) in different usual care or practice
settings have found the practice to be superior to an
appropriate comparison practice.
- The RCTs have been reported in published, peerreviewed literature.
The practice has been shown to have a sustained effect
at least one year beyond the end of treatment, with no
evidence that the effect is lost after this time.

Child Welfare Ratings

Not every program that is evidence-based will
work in a Child Welfare setting
We also examined each programs experience
and fit with Child Welfare systems and
families

Relevance to Child Welfare Scale

1.

High:
The program was designed or is commonly used to meet the needs of
children, youth, young adults, and/or families receiving child welfare
services.

2.

Medium:
The program was designed or is commonly used to serve children,
youth, young adults, and/or families who are similar to child welfare
populations (i.e. in history, demographics, or presenting problems) and
likely included current and former child welfare services recipients.
Low:
The program was designed to serve children, youth, young adults,
and/or families with little apparent similarity to the child welfare
services population.

3.

Child Welfare Outcomes

We also examined whether programs had included outcomes
from the Child and Family Services Reviews in their peerreviewed evaluations:

Safety
Permanency
Well-being

CommonContinuingEducation
DisseminationModel
One day
workshop
Book

Therapist

Use Tx with
appropriate
clients

XLayingtheGroundworkfor
Implementing
EvidenceBasedPractice

LevelsofImplementation
Fixenetal

PaperImplementation
ProcessImplementation
PerformanceImplementation
Fixsen,D.,Naoosm,S.,Blas,
K.,Friedman,R.,Wallace,F.(2005)

InstituteforHealthcare

ImprovementModel
Environmental Context

Organizational Context

Microsystem

Community, Government,
Funders

Organizations
Departments
and Programs
Within
Organizations

Patient and
Community
Social Workers, Therapists,
Medical Professionals and
Families

Transtheoretical Model of Change

5 Stages of Change
Precontemplation
Compliant Status Quo

Contemplation
Changes in orientation

Preparation
Planning for change
Organizational and environmental
readiness

Action
Training

Maintenance
Monitoring/Institutionalization

Driven at each stage by:

Self Efficacy
&
Decisional Balance

ComponentsofImplementation
Select a Solution that Fits a Problem
Prepare the internal and external environment
Supervision and Leadership Buy-in
Acquire knowledge and skills
Use practice with support, supervision and
consultation
Adapt practice to environment
Monitor fidelity
Teach others
Institutionalize Practice

PracticeSelection

Attributesthatcanfacilitateadoption

RelativeAdvantageclear,unambiguousadvantageineither
effectivenessorcosteffectiveness
Coststraining/materials/ongoingconsultationlossproductivity
duringstartupcostsofdelivery
CompatibilityHowcompatibleisthepracticewiththeorganizational
andworkforcesvalues,norms,andclinicaltraditionsandorientation
Complexityperceivedasmoresimpletouseandtoimplement
Trialabilityabletoexperimentwithinalimitedbasis
ObservabilityofBenefitsoutcomesorinterimresults/measures
Reinventionifcanadapt,refineorotherwisemodifyittomeetown
needs
Riskifthereishighercertaintyofoutcomes
TaskIssuesIfrelevanttoperformanceofintendedusersworkand
improvedtaskperformance
Knowledgeifknowledgecanbecodifiedandtransferredfromone
contexttoanother
Augmentation/Supportifprovidedwithtraining/consultation
FromGreenhalghetal

OrganizationalReadiness

OrganizationalCulture/Traditions/History
Leadership
Supervision
CapacitytoevaluatechangeKnowifitisworking
SupportofOpinionLeaders
Connectionswithothersupportiveorganizations/individuals
Doesorganizationhavethetechnologytosupportthechange
Staffreadiness

StaffReadiness

StaffDirectlyandIndirectlyinvolved

UnderstandWhatBenefitsWilltheAdoptionoftheEBPBring

MeaningWhatdoesthechangemeantothestaff?

Whatconcernswillstaffhaveaboutadoption

Howcongruentarethetrainersinorientationandvalueswiththe
staff

PresenceofChampions

ReadinessofExternalEnvironment
CongruencewithCommunity/Cultural/FamilyValues
ReferralSourceUnderstandingandSupport
FundingSourceSupport
PoliticalSupport
RoleofSocialInfluence/DemandforServices
RoleSocialMovementTheory

SupportiveImplementationModel
Administrative Leadership and Support for EBT
Obtain
client
feedback

Technical Assistance

Supervision
Expert
Consultation
Therapist

Training

Use EST with

appropriate
clients

Materials
Community/Consumer Support for EBT

FindingEvidenceSupported
TreatmentsontheWeb
www.nctsn.org
www.cachildwelfareclearinghouse.org/
http://modelprograms.samhsa.gov/template.cfm?
CFID=119292&CFTOKEN=55491051
www.strengtheningfamilies.org/
www.ncptsd.va.gov/topics/treatment.html
www.childtrends.org
www.wsipp.wa.gov
http://ebmh.bmjjournals.com/
www.cochrane.org
www.campbellcollaboration.org
www.colorado.edu/cspv/blueprints/model/overview.html

ContactInformation
Download reports from:

www.chadwickcenter.org

E-mail:
cwilson@rchsd.org

www.cachildwelfareclearinghouse.org