IE

berta Heritage Foundation for Medical Research

Al

Acupuncture: Evidence from Systematic Reviews and Meta-analyses

Patricia Leggett Tait, Laurie Brooks, Christa Harstall

March 2OO2 27: Series A Health Technology Assessment

HT A

Cop).right Alberta Herltage Foundation for Medical Research, 2002

This Health Technolosr Report has been prepared on the basis of available information of whtch the Foundatlon ls aware from public literatue and expert opinion and attempts to be curent to the date of publication. It has been externally reviewed. Additional information and comments relatlve to the report are welcome and should be sent to:

Director, Health Technology Assessment Alberta Heritage Foundation for Medical Research 1500, 10104 - 103 Avenue Edmonton Alberta TsJ 4A7

CANADA
TeI:. 780-423-5?27,
F

ax

780-429-3509

ISBN l-896956-56-4

Alberta's health technolory assessment program has been established under the Health Research Collaboration Agreement between tie Alberta Heritage Foundation for Medical Research and the Alberta Health Ministry.

AcKNowLEDGMENTS
The Alberta Heritage Foundation for Medical Research is most grateful to the following peFons for provision of information and comments on the draft report. The views expressed in the final report are those ofthe Foundation.

Dr. Brian Berman, MD, Complementary Medicine Prog.am, Baltimore, MD

D!. Stephen Birch, Stichting (Foundation) for the Study for Traditional East Asian Medicine (STEAM), Amsterdam Netherlands
Dr. Andrew Vickers, Memorial Sloan-Kettering Cancer Center, New York, New York

Dr. Harald Walach, Universitatsklinikum Freiburg, Institute fur Umweltmedizln und Krankenhaushygiene, Fdeburg, I.BR Dr. Adrian R. White, Department of Complementary Medicine, Unlversity of Exeter, Exeter, UK

Alberta He.itage Foundation for Medical Research Heahh Technology Assessment

SUMMARY
This report's objective is to provide an overview ofevidence from systemaric reviews and meta-analyses on the effectiveness ofacupunctule for any condition that has been assessed in this manner.

Two-thirds ofCanadians used some form ofalternative or complementary medicine in 2000, and l-27o reported seeing an acupuncture practitioner in 1998/99.
Acupuncture, in the stdctest sense refers to "insertion of dry needles, at specially chosen sites, for the treatment or prcvention of symptoms and conditions." It is a relatively safe procedure, but it can lead to both minor (fainting, exacerbation of symptoms) and se ous (hepatitis, traumatic injury of body tissue) adverse events. Twenty-three reviews were included: two on dental and temperomaldibular dysfunction (TMD) pain; one on headaches; one on tinnitus; three on asthma; one on stroke rehabilitation; two on antiemesis; ftve on neck/back pain; two on chronic pain; one on fibromyalgia: one on induction oflabour; one on addictions; two on smoking cessation; and one on weight reduction. Unanimously these reviews call for higher quality research with greater sample size. Among the studies included in the review there are wide variatlon oftreatments such as manual or electrical stimuladon, number of needles per treatment, technique ofneedle insertion, and fuequency oftreatment. All ofthese factors may influence the outcome. Acupuncture was found to be effective for the treatment of dental and TMD pain and antiemesis (nausea/vomiting) in comparison to other chosen interventions. The results for idiopathic headaches and fibromyalgia were reported as encouraging. The evidence was inclusive for the treatment ofback pain, chronic pain, smoking cessation. and asthma. Effectiveness was not supported by the evidence for tinnitus, stroke rehabilitation, neck pain, addictions, and weight reduction. Dental and TMD pain and antiemesis appear to be two areas in which acupuncture is reproducibly effective. Fo. all other indications the methodology design and quality is either too weak to dmw conclusions, the studies have not been done, or, in studies of better quality, acupuncture does not appear to be more effective than standard ofcare or control chosen. Clearly, more research of higher mefiodological quality is called for. Issues of blinding, the use ofa credible control, varying diagnosis amongst differing philosophical approaches, and the diversity oftreatment points chosen and techniques used challenge this particular area olcomplementary medicine.

Alberta Herila8e Foundation for lledical Research Heahh Technology Assessment

Alberta Heritage Foundation for Medical Research Health Technology Assessment

Acupuncture Evidence from Systematic Reviews and Meta,analyses

Appendix A:

Methodology.......................

....................-.......................24

Appendix B: Data Extraction and Quality Assessment oflncluded Studies....................29 Appendix C: Quality Assessment .......,,,,,,,,.....................,,,..51

Tools....

Appendix D: Crite Appendix

for Evaluating Systematic Reviews.,.....,...........................................56

.................. 56 ................,,,........................ 57

Treatment......,,,........ References.................
Tables:

E:

Checklist of Data Required in a Complete Report of Acupuncture

Table 1: Conclusions and quality rating ofthe systematic reviews.................................... 13 Table 2: Data extraction and qualitv assessment ofincluded studies ,,,..................,,,,,,,....2g

Alberta Heritage Foundation for Medical Research Health Technology Assessment

lNTRoDUcTtoN
Two-thirds ofCanadians used some form ofcomplementary or alternative medicine in 2000, mostly vitamins 1. Sixteen percent of Canadians saw an altemative health care provider in 1998/99, and of those people , 160/o (or l-Za/. ol t}Le total population) saw an acupuncturist. In Canada, the incorporation of complementary or alternative medicine into one's health care regime is more common in the western provinces, in women aged between 25 and 44 years, and in those living with chronic conditions 2. Medical schools are incorporating complementary and alternative medicine training in their cufficula, with ten out ofsixteen Canadian medical schools providing some exposure to acupuncture 3. A large body of research exists in acupuncture, covedng viftually every symptom a. In the 1950's the Central Committee in China was interested in this question and thousands of studies were performed. The results led the Committee to bestow equal recognition to Traditional Chinese Medicine (TCM) and Western Medicine 5.
This project is Part II ofa request from several Regional Health Authorities and Alberta Health and Wellness, regarding two key objectives: to present information on the regulation ofacupuncture in Alberta, including the scope ofpractice and coverage of services: and to conduct a systematic review of currcnt scientiffc evidence to determine the effectiveness ofacupuncture, and the conditions lor which it has been found to be effective. Objective two is addressed in this report. Part I, titled "Overview of the Regulation of Acupuncture in,4lrerra ", addressed the firct objective and was completed in May 2001 6. In Part I, three categories ofcertification or governance were found to be in place: those who are members ofa professional college or association (eg. physical therapists) prior to their study ofacupuncture; those who have completed training in an acupuncture program and have passed a provincial exam; and those who have knowledge ofacupuncture (eg. from another country), but are practicing without a governing body. Acupuncture is usually performed in a clinical setting, and is not covered by the Alberta Health Care Insurance plan. As the insertion ofneedles will be considered a 'restricted activity' under the new Health Professionals Act, these activities will only be allowed to be practiced by those governed by a regulating body.

Growing demands on the health care system for provision ofcomplementary medicine, the evolution oflegislation regarding the regulation ofhealth professionals, in conjunction with requests fiom the community for funding coverage for acupuncture treatment illustrate the importance and timeliness ofthis review.

Alberta Heritage Foundation for Medical Research Health Technology Assessment

Acupuncture: Evjdence ftom Systematic Revlews and Meta-analyses

ScoPE oF THE REPoRT

Due to tlle breadth ofthis topic and the challenge ofreviewing the extensive body of research on acupuncture, the approach of systematically assessing the available reviews was chosen to evaluate the current evidence for the efficacy ofacupuncture. In choosing this approach, it is acknowledged that there are areas in which acupuncture is used that have not been assessed and reported as a systematic review.

Though the scope for this report did not limit the search ofscientific evidence to speciffc medical conditions, systematic reviews meeting methodological inclusion crite a a.re reviewed on the following topics; dentistry, headaches, tinnitus, asthma, stroke rehabilitation, nausea/vomiting, neck,/back pain, chronic pain, fibromyalgia, labour induction, addiction, smoking cessation, and weight/appetite reduction.
This assessment will not be comprehensive including all ofthe primary acupuncture research, and all the conditions currently heated by acupuncture, but for the systematic reviews and meta-analyses published in English between 1990 and 2001 that met the inclusion criteria (see Appendix A). Primary studies published since the reviews have not been included; the results from these primary studies may change some ofthe conclusions derived at in this systematic review. The findings from this review are summarized by medical condition and the overall results compared with those from the United States'National Institute of Health Consensus statements and other systematic reviews of rcviews. There are many issues in acupuncture research that need to be explored and addressed in future studies which a.e pointed out briefly in this report. These issues range from the quality and assessment ofmethodology to the appropdate acupuncture treatment regimen.

DEFINITION
Acupuncture. in the stdctest sense, refers to "inseftion of dry needles, at specially chosen sites for the treatment or prevention of symptoms and conditions" i. The Chinese term "zhenjiu" references both acupuncture and moxibustion. The latter is the use of a herb, fienrbJa yulgan3 (mugwort) which is burned over the acupuncture site for purposes of warming L9. Acupuncture is performed with solid needles ranging in length from I cm to l0 cm r0. They are made ofgold, silver, copper, stainless steel, or a combination of metals 10 and may be inserted under the skin no more than 8 cm deep. As technology evolved it was applied to traditional forms ofhealing. Electroacupuncture refers to the technique of introducing an electrical curent to the inserted needles at various frequencies. A laser beam, directed at the acupuncture site and stimulated, has developed into a technique

Alberta Heritage Foundation for Medical Research Heahh Technology Assessment

Acupuncture: Evidence from Syslematic Reviews and Meta-anaryses

called laser acupuncture, Techniques of fire needles involve inserting red-hot needles at an acupuncture point.
Staple puncture is the application ofa metal staple to an acupuncture point where it remains lor a prolonged period of time. Cupping is a technique by which a vacuum force is applied to acupuncture sites, Bloodletting refers to tie pricking ofthe skin for the purpose of releasing blood. This may be aided with the application of a cup over the site. A less lnvasive procedure is acupressure which refers to the stimulation ofa point manually with pressure with the intention ofstimulating Qi flow. Intramuscular stimulation is a technique of applying needles to areas of tenderness. There is debate in the acupuncture community over which ofthese techniques fall under the classification

ofacupuncture.
'De qi' is the sensation ofnumbness, tingling, electrical sensation, fullness, distension. soreness, warmth, and itching which may be felt subjectively around the acupuncture point 8. The practitioner may have a sensation of tenseness or dragging to the needle 11. This sensation may be sought by some practitioners through twirling, plucking, or thrusting ofthe needles. In Western terms these are signs that A-delta fibers are activated r0. In TCM this indicates that the Qi has affived. There is controversy amongst practitioners ofacupuncture as to whether it is necessary to elicit this sensation ro render the rreatment effective 8.

CoMPLICATIoNS oF AcUPUNCTURE
Acupuncture is a relatively safe procedure, but it can also lead to both minor and se.ious adverse events. There is an increasing amount ofliterature on adverse effects of acupuncture; however, there is still a concem about under reporting in the studies of the more minor adverse effects ofacupuncture. Norheim's study ofthe literature from 1981 to 1994 examined 78 case reports (N= 193 treatments) of adverse effects with needle acupuncture 12. He classified the complications as mechanical organ injuries such as pneumothorax (n=23) and medulla spinalis injury (n=13); infections such as hepatitis (n=100) and auricular chondritis (n=16); and other effects such as argyria (n=5) and problems with implanted needles (n=5). Many ofthe effects seem to be linked to practitioner competence, or patients' ongoing health conditions. White et al. and MacPherson et al. both examined the adverse events occurring with acupuncturists in the U.K., in over 32,000 and 34,000 consultations respectively 13,1r. In MacPherson et al, no serious adverse effects were reported, as defined as requiring hospitalization, or leading to permanent disability or death. and there was a rate of 1.3/1000 consultation of minor adveEe events such as severe nausea and fainting. White et al. also found no serious adverse events in theb study. Significant minor events such as fainting, lost needles, and exacerbation of symptoms were reported at a rate of l4l10,000 consultations. In addition, both studies reported on'minor events'or
Albena Heritage Foundation for Medical Research
Health Technology Assessment

Acupuncture: Evidence from Systematic Revieu,s and Meta,analyses

a much larger percentage ofthe beatments, though these were not considered significant.

'mild transient reactions' occurdng in

Using combined data from published literature, Ernst rs found that the most commonly reported adveme events were drowsiness and fainting, increased pain, nausea/ vomiting, and infections. Infections, including Hepatitis C, were usually due to using,/reusing unstedle needles, or practitioner error. In a 2001 systematic review by Emst and White on tie safety of acupuncture they reviewed nine studies, which included almost 250,000 treatments 16. Thirty-eight percent of all patients experienced bleeding and 45% experienced an aggmvation of pain symptoms. Overall 28% of patients expe enced some adverse effect, serious effects were rare, as supported by their systematic review of life threatening adveme effects in 1997 r?.
Across the studies common mild or 'non-significant' adverse effects included pain at the needling site, nausea, bleeding, aggravation ofsome symptoms and faintness or fatigue 1315 17 r8. Infections, such as hepatitis. were often due to poor hygiene technique such as reusing unsterilized needles. Some cases oftraumatic injury ofbody tissue including pneumothorax, cardiac tamponade, spinal trauma, and hematoma re have also occurred usually due to practitioner eror.

identified in consistent reporting ofadverse effects are summarized by Emst and White 10. The method of defining adverse events and of collecting/reporting ofthese events diflers across the studies. As well, the style of acupuncture differs across cultures and practitioner groups, which have differing risk levels for adverse events. Finally, the rate ofadverse events may vary according to the condition being treated, and the body part being stimulated during that treatment. Yamashita et al. suggest that a distinction should be made between the reporting of results due to negligence and those due to adverse events ofthe acupuncture technique itself r8.
The studies' conclusions are consistent in that they found that the mte or incidence of serious adverse events due to acupuncture treatment is low but that they do occur 12 13 15 18. Acupuncture is not free of risk. it is important to track and report minor side eflects as they may progress into a more serious event. Ifthere is not a consistent and thorough process for reporting these events, the evidence on adverse effects remains limited and incomplete. MacPherson and colleagues ra stated that the adverse event mte, when compared with primary care drugs, suggests that acupuncture is a relatively safe treatment. and many researchers concur that it is a relatively safe technique 12.15.16.

Some key challenges

FtNDtNGS
Of the thirty three studies selected, twenty-three systematic reviews met the inclusion criteria, including five Cochrane Reviews (see Appendix A). A table of data extraction and quality assessment ofincluded systematic reviews can be found in Appendix B. Though there is growing debate as to whether the Cochrane Reviews should continue
Albefta Heritage Foundation for Medical Research
Health Technolo&y Assessment

Acupuncturet Evidence from Systematic Reviews and Mera-analyses

to be a 'gold standard' for systematic review methodology, they curently have the most rigorous methodology, and therefore, a quality assessment ofthese reviews was not undertaken 20. The other reviews were assessed using criteria based on those set out in Greenhalgh 21 (see Appendix D). Though this quality assessment may not be as rigorous as initially intended by its authors, it has been consistent across the reviews. Once agreed upon criteria have been developed for the assessment ofthe methodological quality ofprimary studies in acupuncture, the same approach should be taken for the assessment ofsystematic reviews.
The methodology of a review was considered to be satisfactory if it contained a concise research question(s) and inclusion criteria, an adequate search strate$/, and included a quality assessment evaluation 22 28. If a review also attempted to integrate andlor statistically analyze the data, it was considered to be of good quality 2e-36. The rest of the reviews were considered to be ofpoor methodological quality 37 3e.

Dental and temperomandibular pain

Two reviews focused on dental pain, temperomandibular dysfunction (TMD) and facial pain. Eight ofthe l6 studies reviewed by Ernst and Pittler 24 regarding dental pain were also included in Rosted's review 22. In Emst and Pittler, eleven studies examined pain due to dental surgery and five studies evaluated pain induced in an experiment. Methodological quality was assessed using the Jadad Scale 40 (Appendix C). Though the authors state that the heterogeneity oftreatments and low quality ofmany ofthe studies reviewed limit the conclusions that can be drawn, the data suggest that acupuncture can be effective in the treatment ofdental pain. The method and regimen ofacupuncture that most eflectively rclieves dental pain. however, remains unclea..
Rosted's review 22 examined seven studies on pain associated with tooth extraction and eight studies on TMD/facial pain. Overall the review's methodology was fairly dgorous. Descriptions ofthe treatment and practitioner's qualifications were assessed as part ofthe quality assessment, but these details were not presented. Eleven studies indicated that acupuncture was effective and seven ofthese studies were rated at a high level due to their methodological quality. Rosted concluded that most of the studies suggest that acupuncture is effective in controlling dental and TMD,/facial pain. He stated concerns, however, about the clinical relevancy ofthis treatment as a surgical analgesic, as the time needed for acupuncture to take effect was much longer than other analgesics.

Headache
Only one review met the inclusion criteria.al. This Cochrane Review addressed the question whether acupuncture was more effective than no treatment. sham or other treatments used for headache ol three types: mig.aine, tension. and mixed. The Jadad Scale 40 was used to assess study quality (Appendix C) ofthe 26 RCTs (16 RCTs for migraine headaches, six RCTS for tension type headaches, and four RCTs for various
Alberta Heritage Foundation for Medical Research Health Technology Assessment

Acupuncture: Evidence from Systematic Reviews and Meta-analyses

headache types). Evaluation ofthe appropriateness ofacupuncture point selection was undertaken by someone trained in both Western and TCM acupuncture.

Sham'controlled studies for migraine and tension type headaches, reported favourable results for acupuncture, though the methodological quality ofmany ofthe studies was rated as weak. There were a small number of studies for all types of headaches indicating mixed results for acupuncture compared with each of physiotherapy, drugs, and massage/relaxation. Four out of 16 studies evaluating the effectiveness of acupuncture for the treatment of migraines were rated at a high level in relation to their methodological quality. Of these four studies, two studies showed a benefit from acupuncture, one study showed no difference in comparison to sham acupuncture, and one study noted that acupuncture was less effective when compared to standard care.
For tension-type headaches, two out ofthe six RCTS were rated as high for methodological quality. These two studies indicated that acupuncture was effective for the treatment of tension type headaches. The authors stated that acupuncture appears relatively safe when administercd by qualified providers- Although the type of acupuncture used and whether it should be widely recommended could not be answered by the evaluators, and patients wishing to try it should not be discouraged.

Tinnitus
One systematic review focused on acupuncture as a tleatment for tinnitus. Park et al. 23 identified six RCTS which compared electro or manual acupuncture in the treatment of tinnitus to sham, physiotherapy, biofeedback, or medication. Park and colleagues evaluated methodological quality using the Jadad Scale {0 (Appendix C). The type of acupuncture stimulation and number ofsessions were described but not the appropriateness ofthe acupuncture treatment. Four studies found no effect of acupuncture on tinnitus; three ofthese studies achieved passable methodological quality scoring by the Jadad criteria. Due to the low methodological quality overall, the heterogeneity of the samples. and the ffndings of'no effect', the conclusion of the rcviewerc was that evidence did not support the use ofacupuncture for the trcatment of

tinnitus.

Asthma
Two systematic reviews assessed the effectiveness of acupuncture in the treatment of asthma, in addition to a Cochrane Review. Kleijnen and colleagues 28 reviewed 13 RCTs that were based on needle acupuncture. They reported on the style ofacupuncture (all but one were based on formula acupuncture) but did not evaluate the appropriateness ofpoints chosen. The methodological quality of the 13 studies was rated on the Kleijnen scoring system (see Appendix C). No studies of high enough quality were found to conclude ifacupuncture was effective in the treatment ofasthma.
Alberta Heritage Foundation for Medical Rsearch Health Technology Assessment

Acupuncturer Evidence from Systematic Reviews and Meta analyses

In 1996 Linde and his colleagues published a review 2e of 15 tials including all but two ofthe studies included in the Kleijnen et al. review. These excluded studies were not .andomized trials. They were concemed that there had been no evaluation as to the
appropriateness of point selection; therefore four physicians who also taught and practiced acupuncture evaluated the appropriateness ofthe acupuncture points chosen. There was vadability in the assessment of adequacy oftreatment choice, but none ofthe studies were evaluated as being totally inadequate. Jadad's scale a0 was used as well as Linde's own scale to assess methodological quality ratings and were compared to the ratings obtained in the Kleijnen scoring system (Appendix C). More similarity was found between the Linde and Kleijnen ratings than those of Jadad. They concluded that there was insufficient research ofhigh enough quality to recommend to acupuncturists to stop treating asthma patients, nor to recommend to non-acupuncturists to start its
use,

A Cochrane Review by Linde, Jobst and Patton {2 using similar inclusion criteria was published in 2000. Seven studies matched the criteria and were evaluated using the Jadad Scale. One ofthe authors was experienced in acupuncture and evaluated the adequacy ofthe sham-acupuncture, but not the appropriateness ofthe acupuncture treatment. Objective measurements for lung function were included as well as drug
use. Subjective results were also accounted for in these studies. They concluded that the efficacy of acupuncture for asthma can not yet be determined. variety ofcriteria used to evaluate the methodological quality of the studies, the results obtained were consistent in stating that the evidence did not support or refute the use ofacupuncture in the treatment of asthma.

Although there were

Stroke rehabilitation
A review by Park and colleagues 2s included nine
RCTS comparing needle acupuncture

to standard medical and rehabilitative treatments or sham electro-acupuncture, The search included more than one complementary database, and study quality was assessed usingJadad's Scale {0 (Appendix C). They identified variability in treatment schedules, types ofstimulation, time ofinitiation ofacupuncture, acupuncture sites, and use of quality of life measures. Numerous different stroke assessment scales were used in the primary studies to measure the outcome, challenging the ability to make comparisons between the studies. No mention was made of the evaluation of the appropriateness ofteatment, although the duration ofheatment and whether it was manual or electroacupuncture was documented. The authors found the quality of studies to be poor (only two studies obtained a Jadad score of 3 or more) and stated that the evidence does not support the use ofacupuncture for stroke rehabilitation, though the findings show some promise.

Nausea and emesis

Peiicardium 6 (P6) is used in the treatment of post-operative nausea and vomiting (PONV). Two systematic reviews evaluated the effectiveness ofP6. TheLeeandDone
Alberla Herilage Foundation for Vedical Research Health Technology Assessmnt

Acupuncture: Evidence from Systematic Reviews and Meta'analyses

meta analysis 30 included 19 studies that evaluated post-operative nausea and vomiting by comparing acupuncture andlor TENS to a control group receiving eitier sham acupressure or treatment or a pharmacological intervention. No specific complementary database rvas searched to locate primary studies nor was there a stated search ofthe grey literature. Eleven ofthe 19 studies scored three or better onJadad's Scale a0 (Appendix C). though there were many issues noted that hindered comparison, including diversity of techniques used for stimulation of the point. Acupuncture was not deemed effective in the control ofnausea and vomiting in the pediatric population. P6 stimulations fo. early or late PONV compared to pharmaceutical treatments was reported to have an equal effect, and when compared to sham or no treatment was supedor in 20% to 25% ofadults within 6 hours oflaparoscopic and gynecological procedures. There was inadequate data to determine effects oftreatment versus sham for late PONV.

Vickers' review 31, which included a complementary database in the search strategy, identified 33 studies and evaluated the methodological quality using the Vickerc Scale a0 (Appendix C). They divided the studies into nausea and vomiting post-operatively, following cancer chemothe.apy, and morning sickness. A diveNity of treatments were assessed in the primary studies such as acupressure, electroacupuncture. needle acupuncture, TENS, and acupoint injections. Using acupuncture while under anaesthesia was found to be ineffective at controlling emesis, in four studies. All but two ofthe remaining 29 studies reported a positive effect for acupuncture in P6 anti-emesis. The author concluded that PO stimulation seems to be effective except when it was administered under anaesthesia.

Back and neck pain

Five systematic reviews, including a Cochrane Review, examined the efncacy of acupuncture for the treatment ofneck or back pain. White and Ernst 33 reviewed the methodological quality of 14 RCTS on the treatment of acupuncture in neck pain usjng a modified Jadad Scale. The initial diagnosis varied, including ankylosing spondylitis, myofascial pain, osteoarthritis, and pain ofundefined etiology. The method of acupuncture treatment varied, although most studies used formula acupuncture. The control groups varied. including sham needling, TENS or laser, physiotherapy, waiting Iists, and medication. These reviewers were ofthe opinion that there was no evidence f.om sound clinical trials to support the use ofacupuncture for neck pain. The Smith et al. review 32 examined the use ofacupuncture on traditional and non-traditional points for chronic neck and back pain in l3 RCTs. Eleven RCTs evaluated the effectiveness ofacupuncture in chronic neck or back pain and two studies assessed acupuncture lor acute low back pain. They used theJadad Scale a0 to assess methodological quality and also tested their own tool, the Oxford Pain Validity Scale (OPVS) in the review (see Appendix C). There was no assessment as to appropriateness ofacupuncture site selection. Nine out of II studies used multiple treatments for chronic neck and back pain. Acupuncture, electro acupuncture or laser acupuncture
Aiberta Heritage Foundation for Medical Research Heal!h Technology Assessmenr

Acupuncture: Evidence from Systematic Reviews and Mera-analyses

was compared to sham, needling. laser, or TENS, waiting lists, or standard clinical therapy. Most of the studies on chronic neck and back pain found either no difference between acupuncture and control treatments, or found an initial positive effect for acupuncture, but often after 24 hours there was no significant difference between the treatments. The two studies examining acute low back pain after a single treatment were evaluated to have no benefit over sham or acupressure with anaesthetic spray. The conclusion ofthe reviewerc was that there was no evidence to support the treatment ofback or neck pain by acupuncture.

Using methodological cdteria adapted from Koes (see Appendix C), Strauss 26 reviewed results from four controlled clinical trials for chronic low back pain (LBp). There was no evaluation olthe appropriateness of the acupuncture treatment, although the discussion did address many ofthe problems associated with the assessment of acupuncture including the skill of the acupuncture provider. Three of the studies reported positive results for acupuncture, however these were ofpoor methodological quality. The heterogeneity of patients and treatment methods, and practitioner qualifications, made drawing any conclusion regarding the effectiveness ofacupuncture in chronic LBP difficult. Though the author believed that acupuncture was a safe and popular treatment for LBP, he recommended that rigorous research was needed to determine the most appropriate treatment methods for specific conditions ofLBp.
Ernst and White 36 included 12 RCTs (9 into the meta-analysis) on back pain. Methodologically this review was rigorous, with good data integration and assessment of trcatment adequacy. Acupuncture was shown to be superior to waiting list and physiotherapy but was not found to be superior to placebo except in one study on severe pain. Odds ratios for unblinded studies suggested a strong placebo effect. They recommended that further studies explore the specific and non-specific effects of acupuncture, as well as the adveme elfects and cost-effectiveness ofthe various treatments for back pain. to assist in determining the usefulness ofthese therapies. The Cochrane Review by van Tulder 8 examined l1 RCTS on the effect of acupuncture in chronic and acute lower back pain. This review followed the Cochrane Back Review Croup's rules for assessing methodological quality (see Appendix C). No assessment as to appropriateness of trcatment was made. There was conflicting evidence from low quality trials comparing acupuncture to no treatment, moderate evidence that acupuncture was not more effective than TENS or trigger point injections. and limited evidence that acupuncture was not more effective than sham for the treatment of chronic LBP. Overall this review reports that the ellectiveness ofacupuncture in t1-re treatment ofLBP was unclear and, since there are effective altematives, the authors do not.ecommend acupuncture as a regular treatment for LBp.

Chronic pain
Two reviews focused on the effectiveness ofacupuncture for the treatment ofchronic pain. The appropriateness of treatment was not evaluated in either review.
Alberta Heritage Foundation for Medical Research
Health Technology Assessment

Acupuncture: Evidence from Systematic Reviews and Meta analyses

The review by Ezzo et al. 3a used the Jadad Scale (see Appendix C) to evaluate 51 RCTS in which patients with pain longer than 3 months were treated with needle acupuncture. The review assessed the number offieatments, number ofpoints needled, eliciting of'de qi', and type of acupuncture (whether formula or individualized). On$ 'number oftreatments' seemed to be co.related with a positive outcome. The authors found that the control group participants in studies using sham acupuncture (needles were inserted) as the control had a proportionally higher improvement rate compared to the control group participants in studies using inert controls such as, TENS, sugar pills, and mock acupuncture (in which needles were not inserted), This led the authors to propose, amongst other possibilities, that sham acupuncture was not physiologically inert. They stated that they found limited evidence that acupuncture was more effective than waiting lists and the evidence was inconclusive on whether acupuncture was more effective than physiologically inert controls, sham acupuncture, or standard The ter Riet. Kleijnen and Knipschild's meta-analysis 3e also evaluated 51 studies using needle acupuncture (excluded surlace electrodes or laser acupuncture). but these studies included patients with chronic pain of at least 6 months duration. They assessed methodological quality based on criteria developed by ter Riet and colleagues (see Appendix C) and found that further research needed to be conducted with more homogeneous study groups, and better methodological design. The reviewers stated that there are no published studies ofhigh enough quality and that the efficacy of acupuncture lor this condition remains inconclusive.

Fibromyalgia
One review addressed the use of acupuncture for the trcatment of fibromyalgia 27. The reviewers used the Jadad Scale to rate the methodological quality ofthe studies (see Appendix C). They did not, however, identify the style (eg. classical TCM or formula acupuncture), appropriateness of treatment, or the qualifications of the acupuncture practitioner. The authors based their conclusions on one high quality RCT, which found signiffcant improvement in both subjective and objective pain measures compared to sham acupuncture but the duration of benefit was unknown. A lew patients had woNening of symptoms during the treatment with acupuncture. They state that their review may provide some practical information for practitioners on possible benefits and risks ofacupuncture. Based on limited evidence, acupuncture is more effective than sham acupuncture for improving symptoms (pain relief. reducing morning stiffness, increasing pain threshold, and improving global ratings) in patients

with fibromyalgia syndrome.

Obstetrics
A Cochrane Review was conducted by Smith and Crowther a3 to determine the effects of acupuncture lor the induction of labour. The authors noted that there were limited
observation studies published that suggested acupuncture appeared safe and effective.
Alberta Heritage Foundaljon for Medical Research
Halth Technology Assessmni

10

Acupuncture: Evidence from Systemalic Reviews and Meta-analvses

None ofthe published trials, however, met the inclusion suggest the need for a well-designed RCT.

teria and the authors

Addictions
Ter Riet and colleagues 38 conducted a meta-analysis that included 2Z studies. euality of studies was assessed using criteria developed by the authors (see Appendix C). The outcomes assessed, however, were not clearly defined in terms ofaddiction treatments, as they were only stated as the cessation of smoking. use of heroin, or use of alcohol. In addition, no biochemical vedfication of cessation,/abstinence was included. Fifteen ofthe 22 studies examined the use ofacupuncture (excluded surface electrodes or laser acupuncture) in smoking cessation and reported acupuncture as not effective in comparison to placebo; however placebo treatment was not identified. Five studies reviewed the use of acupuncture in hercin addiction. The methodological quality of all five studies was rated as low and therefore it was difficult to draw any conclusions. Two studies using acupuncture for the treatment ofalcohol addiction reported a positive effect for acupuncture reatment but these studies suffered from high drop-out rates. No mention was made ofthe appropriateness ofthe acupuncture points used in any ofthese studies. though the practitioner and treatment description were assessed as part of the quality assessment. The conclusion lrom this review was that the evidence does not support the use ofacupuncture in the treatment ofaddictions.

Smoking cessation
Two reviews examined acupuncture in the treatment ofsmoking addiction; the Cochrane Review by White and colleagues aa and a meta analyses by White et al. 35. The Cochrane Review included 18 RCTs in which smoking cessation was the outcome. Acupuncture was compared to sham acupuncture or an alternative form ofcessation intervention or to no inteNention. There was no assessment of appropdateness of acupuncture sites chosen, but the treatment regimen was descdbed in the primary studies. Only four studies reported any form of biochemical validation of smoking cessation. Three studies indicated strong positive results for acupuncture in the treatment olsmoking addiction. In two ofthese studies prolonged auricular acupuncture was applied. The authors proposed that perhaps more rigourous study into the effects ofintensive and continuous treatment was warranted. As \a,ell. they suggested the importance olstudying acupuncture effects during acute nicotine withdmwal. The review concludes that acupuncture was not superior to sham acupuncture. Compared with other anti-smoking interventions there was no difference but early results indicated it was superior to no intervention.

of 14 RCTs (12 RCTs sham-controlled) was thorough and methods. as well as limitations, were clearly stated. The authors came to the same conclusions as the Cochrane Review that there was no evidence that acupuncture was more or less effective than sham acupuncture or other smoking cessation interventions.

The meta analysis

35

Alberta Heritage Foundation for Medical Research Health Technology Assessmenr

11

Acupuncture: Evidence from Systematic Revie$,s and Meta analyses

Weight loss
One systematic review on the eflectiveness of acupuncture and acupressure in weight loss and hunger suppression was identified 37, Four sham-controlled clinical trials were assessed by an "accepted instrument" used by Kleijnen, Knipschild, ter Riet (see Appendix C), One study used an acupressure device. while all other studies used varying auricular points. The conclusion of the reviewers was that fufther, well designed research needs to be conducted to provide sufficient evidence regarding the effectiveness ofacupuncture in appetite or weight reduction, but that there curently was no convincing evidence to support the effectiveness of acupuncture for weight loss or hunger suppression.

Summary
For the various conditions listed in Table 1, the respective reviews found that the evidence supports acupuncture as an ellective treatment for dental pain, and nausea./vomiting. Though the evidence for the other conditions such as idiopathic headaches, back pain, chronic pain, and fibromyalgia was often inconclusive due to methodological weaknesses, andlor conflicting results reported by the primary studies included in the reviews, the results look promising. These reviews. the majority with a good quality rating, found acupuncture to be as effective in the short term as the conventional interventions or no t.eatment for these conditions. Many of the authon noted that better quality studies provided negative results while poorer quality studies tended to report positive results. Furthermore. they agreed that there appeared to be insufficient evidence and that better quality research was needed.

Alberta Heritage Foundation for Medical Research Halth Technology Assessment

t2

Acupuncturc: compilalnxt ofsystonralic ft:vicws ancl mcra analyscs

Table 1: Conclusions and quality rating of the systematic reviews

Condition
Dental and TMD

Review
Ernst E, Piltler MH'?a 1998 Rostad P

Conclusions
Acupuncture can be effective in alleviating dentalpain Acupuncture was more effeclive than sham and had a similar effect as conventional lreatment Evidence supported the value of acupuncture for the treatmenl of idiopalhjc headaches bul1he quality and amount of evidence is
not convirlcing

Quality Rating
Salisfactory
SaLisfactory

"

1998
a1

Headache

Lindc K, et al.
2001

Cochrane Review

Tinnitus

Park J, et al.'z3

2000
Kleilnen J, et al.
1991

Evidence of efficacy does not suppod acupuncture for the trealmenl of chronic tinnilus Efficacy not supported by the resulls ofwellperlormed cljnical
lnsufficaent evidence to draw reliable conclusions

Satisfaclory Satisfactory

Linde K, et al.'zs
1996

Linde K, et al.
2000

a'z

Not enough evidence to make recommendations aboutlhe value of acupuncture Evidence does not support acupuncture in stroke rehabilitation

Slroke
Nausea and Emesis

Park J, et aI. 2001 1999

1C96

'z5

Satisfactory
Good Good Good Good Good

Evidence indicates signjficanl reduction in adults ve6us no -te3!!9!!q9rp{?!lqr99q!!!lqgqsi!!t9 qss Acupuncture seems to be effective exceptwhen it is administered
Evidence from clinicaltrials does not suppot the {realment of .neck pain

Back and Neck Pain

White AR, Ernst E 3l 1999

Ernst E, While AR
1C98

36

Combined results indicaled lhat acupuncture for back pain was

glp-e,ILoIlg SqlllgLlllgrvenlions but not to sham Evidence from valid trials indicates no analgesic efficacy for neck and back pain

Smith LA, et a|.32 2000 Strauss AJ

1S9S
26

Efficacyfor chronic low back pain has not been demonstrated by good clinicalstudies Evidence indicates that acupunclure is not proven effective for the treatmenl ot low back Dain

Satisfactory

Van Tulder I\,4W, et al. u 2oo1

Albcrla Ilcrilagc Irou'rda(ion for M{{iical ltcsearch Hcalth Tcchnology Asscssmcnt

l3

Acupu cturci Ividdrcc trom Syslomatic Rcvi{}ws and Mcta anatysos

Table

l:

Conclusions and quality rating of the systemaiic reviews (cont,d)

Review
Ezzo J, et at. 2000
3a

Condition
Chronic Pain

Conclusions
Limited evidence indicates that acupuncture is more effective than no lreatmenl; inconclusive that it is more effective than sham, standard care or inerl conlrols
3'g

Quality Rating
Good

ter Riel G, el al.

1990

Efficacy of acupuncture in chronic pain (at leasl6 monlhs) remains doubtlul

27

Fibromyalqia

Berman BM, et al. 1999

wtl4!9!!

Based on one good quality triatthe evidence jndicated significant symptom improvement compared to sham but duration of benefit

Satisfactory

Obstetrics

Smith CA. Crowlher

cA

43

2oo1
33

Observalional studies provided promising findings but no randomired controlled studtes were lo.ated
Efficacy for smoking, heroin and alcohotaddiclions are not suppotled bv evidence lrom qood chnicdl studies Evidence indicates that acupunclure does not appear to be effeclive for smokinq cessalion

Cochrane Review

Addictions

ter Riet c, et al1990 2000

Cochrane Review Good

lreatino smokino a.l.liclions c6mnarp.i

Evidence indicates that acupuncture appears to be better in 1^ thnaa riclc Based on two rigorous sludies there was no elfcct on body

Weight Reduction

Ernst E 1997

37

^n 'lrifinn

Albcrta Ilcritage Foundation for Modical Rcsoarch Hcallh l cchnology Asscssrncnt

14

Acupuncture: compilation of systematic reviews and meta-analyses

OTHER REVIEwS
Systematic reviews are a synthesis and critical appraisal of p mary studies and therefore play an important role in evidence-based decision making. Many of the primary studies included in a systematic review may not be easily accessible to a number ofpmctitione$ or busy practitioners may not have time to read all the published research, hence the value of systematic reviews. A main limitation of this systematic review ofsystematic reviews is that it did not take into account the evidence from new research that may add to or change the conclusions. For example, since the publication ofthe systematic review by Park and colleagues 25 on the eflectiveness of acupuncture for stroke, a sham controlled study considered ofgood quality, indicated negative results. The addition of this study to the systematic review would strengthen the evidence to recommend against the use ofacupuncture for this indication.

National lnstitute of Health held a2 d,ay conference on acupuncture ^nd,1/2 specifically to evaluate the scientific data on the conditions, sks, and benefits. They stated that there seemed to be potential usefulness based on the studies but, due to flaws in design, sample size, and other factors, the results ofthe research were often equivocal. The role of acupuncture in nausea and vomiting resulting f.om chemotherapy as well as post-operative surgical and dental pain appeared to have some of the best evidence. They also acknowledged that there are many other conditions for which acupuncture may be useful as an alternative or adjunct treatment. Their concluding comments focused on the issues oftraining and licensure, summadzing that there was sufficient evidence to support further research and integration into conventional medicine a5. Based on these results Medicare does not cover acupuncture services. The coverage and analysis group, however, a.e open to receiving furthe. evidence on the efficacy ofacupuncture (informed placement ofneedles with or lvithout twirling, but not with electrical stimulation or moxibustion) for post operative chemotherapy pain and nausea in adults and post-operative dental pain for dental conditions covered by Medicare 46.

In

1997 the

Ernst and White { reviewed seven systematic reviews on the effectiveness of acupuncture for dental pain, low back pain, neck pain, osteoarthritis, stroke, smoking cessation and weight loss. They concluded that there was strong evidence on the efficacy of acupunctu.e for dental pain, low back pain, and nausea/vomiting. In addition, they stated the need for .igorous research by experts in the field, and lunding support to allow for the expansion of acupuncture .esearch.

Linde et al. a7 published a bibliography ofsystematic reviews in acupuncture. The reviews they included were on the lollowing topicsr chronic pain, headaches, dental/TMD pain, rheumatic diseases, addiction, nausea, asthma, tinnitus, weight,/appetite reduction. and stroke rehabilitation. They only found convincing evidence in support of acupuncture for postoperative nausea and against acupuncture for smoking cessation. They also concluded that there were key issues around
Alberta Heritage Foundation for Medical Research Health Technology Assessment
15

Acupuncture: Evidence from Systematic Reviews and Meta analyses

methodological problems, lack ofresearch infrastructure and funding for research and pointed to the complexities ofacupuncture as a group oftreatments for many and vadous medical conditions.

A final review by Vickers as published in the fall of200I looked ar effectiveness in the treatment ofacute pain. chronic pain, addiction, astima, nausea/vomiting, obesity, stroke rehabilitation, tinnitus, and various other conditions. They found acupuncture to be effective for postoperative and chemotherapy nausea/vomiting. and postoperative dental pain. They also found that the evidence for acupuncture in obesity, sm;king cessation and tinnitus suggested it is 'unlikely to be of benefit'. For the other conditions, the evidence was insufficient to support any conclusions.
Comparison ofthese reviews with this report finds consistent support for the effectiveness of acupuncture in the treatment ofpostoperative nausea,/vomiting, and dental pain.

lssuEs

tN

AcupuNcruRE TREATMENT AND REsEARGH

a

Many issues have arisen with regards to developing research. Three key areas will be discussed:

study model for acupuncture

. . .

selection ofcontrol groups;

complexities of acupuncture; and

study design and assessment of methodological quality.

Selection of control groups

The use ofa control group is a key part ofclinical trials. It is the comparator group for the experimental group receiving the treatment being investigated. The selection of 'credible' controls a.26, ae,50 poses a challenge for acupuncture research as controls can range from placebo or 'sham' controls, to standard care, to no treatment at all. The use ofstandard care or no treatment versus placebo or sham, and the effects of some .sham' techniques on outcomes. are issues ofdebate in the literature 51.52. Vickers and de Craen reviewed methodological literature and provided a summary of arguments for and against the use ofplacebo controls in acupuncture 52. placebo use enables blinding and potentially decreases drop out rates ofpa.ticipants in ,known control groups'. Non-placebo control groups can have a higher drop-out rate, because participants know they are not receiving treatment. Vincent and Lewith suggest routine assessment of control g.oup members' perceptions oftheir treatments through a 'credibility scale' 53. The aim is to reflect patient perceptions olefficacy oftheir treatment, and therefore the credibility ofthe placebo control. Streitberger and Kleinhenz have developed a 'placebo needle', which mimics the visual and tactile sensations ofacupuncture with a needle that does not break the skin, but disappea$ into the handle 5a. Irnich et al. used'sham' laser acupuncture with visual
Alberta Heritage Foundation for Medical Rsearch Health Technology Assessment
16

Acupuncture: Evidence from Syslematic Reviews and Meta-analyses

and acoustic signals similar to those found during active laser acupuncture ss. These placebo or sham controls increase the patient's perception ofactually receiving acupuncture treatment, and also enables double-blinding.
Placebo or 'sham' were defined in the studies included in the systematic reviews as using non-traditional acupuncture points, superficial puncturing ofthe skin without stimulation, introduction ofa sensation without puncturing (eg. acupresssure), or, in the case of elechoacupuncture, the use ofelectro stimulators without connecting the cables s. 'Sham' acupuncture, the most commonly used control in acupuncture studies, is where needling is done at theoretically irelevant sites t3.50. It was initially believed that acupuncture at these sites would have no effect, but many people now believe that inserting a needle anywhere in the body or applying pressure to any site evokes a response s2. s3, 57. This evocation of response can also be found with other placebo controls mentioned. Others believe that there is a strict process to ensuring that'sham' is truly placebo. based on where the needling is done in relation to the treatment aCuPuncture 56.

The specific and nonspecific effects ofsham techniques are unclear. For example, ifthe sham control group also shows benefits, the acupuncture featment may be deemed ineffective in comparison to the 'control' group; however, this may be misleading ifthe 'sham' featment was actually evoking a physiological response similar to the acupunctu.e treatment group. Though this does not clarify the issue around placebo controls, it does illustrate the complexity, and the impact ofindividual trcatment styles. Though the effects, both specific and non-specific, of acupuncture at various sites need to be determined, the value ofsham acupuncture as a control is clear: the patients can then be blinded to treatment, 'improving' the quality ofthe research study 58. The choice ofcontrol group in acupuncture research. Iike in conventional medicine research, needs to be guided by the research question. and the objectives ofthe research 52.

Complexities of acupuncture
Acupuncture is a complex 'umbrella' oftreatment apprcaches. Acupuncture includes such a diverse constellation ofphilosophies and treatment styles. This means the most accurate determination ofeffectiveness of acupuncture should include the evaluation of each single, well-defined approach, versus evaluating the 'umbrella' oftreatments as a single approach 58. However, the many types and methods of acupuncture are often combined and compared in the systematic reviews. For example, manual stimulation and electrostimulation have seldom been compared to each other as to their effectiveness, but are considered the same in many systematic reviews.
As well, many microsystems are used in treating varying conditions. Ear acupuncture is perhaps the most widely used, although other systems such as scalp, hand, foot, nose, and abdominal acupuncture are also considered specialties. Formula and TCM acupuncture are two diiferent styles. which are also often grouped together in reviews. TCM focuses on a balanced system. It uses point selection based on symptoms, pulse, and tongue diagnoses, and the choice of points used may vary from day to day as the
Alberla Heritage Foundation for Medicai Research
Health Technology Assessment
17

Acupuncturet Evidence from Sysrematic Reviews and Meta analyses

balance shifts. The "formula" or standardized approach in which the same prescription ofpoints are used for each patient repeatedly is better suited for research, but perhaps not reflective ofactual experience 5.7.

The individualization of diagnosis and treatment may be more similar to psychotherapy or physiotherapy where the skill ofthe therapist and the bond with the patient are as important in producing an elfect as the treatment strategies 7.ae. Thempy is adjusted according to the subtle shifLs as they occur rather than continuing with a standard pattern. There have not been studies to elucidate the effectiveness ofany one ofthese acupuncture approaches over the other or whether they are equal in their effect. yet they are compared against placebo or sham in studies.
There is also variability in the technique ofneedle insertion and manipulation that may influence the efficacy but are often not reported in studies. Electrical or manual stimulation may alter the outcome. Diameter. length, depth of insertion, duration of retention, the number of needles pe. treatment, tempemture ofthe needles, the number oftreatments, and materials ofthe needles may all be factors which influence the
OUtCOme 7,49.

Linde et al. 2e included four expert opinions in acupuncture to evaluate the adequacy of the acupuncture treatments from a clinician's percpective. They were given a questionnaire to evaluate the choice ofacupuncture points used in the studies. Linde et al. found a low level olagreement bet$,een the four experts and posed questions of clinical relevance.
Few researchers have investigated what adequate acupuncture treatment is, due to the complexity described above, and little agreement has been reached for the various conditions treated with acupuncture. Birch broke this challenge down into the administration ofadequate treatments, and the adequacy ofthe repo.ting ofthe treatments 5r. The difnculties in determining adequate treatment can be captured by the following: Which sources,/evidence does one use? Can the treatments from a study be standardized to a broader population, or is it specific to those individuals? How many treatment points and sessions are the correct number for certain conditions? Is the condition used alone or in conjunction with any other modes oftreatment? sr. The issue ofinadequate reporting makes assessment ofthe research difftcult and makes the gene.alizability impossible. The inclusion ofkey information is necessary to be able to determine the adequacy ofthe treatment used.

Study design and assessment of quality

The issue ofstudy design is a challenge for acupuncture research. Some ofthe systematic reviews. and the primary studies reviewed within. either lacked a st.ong research design, and,/or an adequate description ofthe design on which a reader could base an opinion 5s. For example, Linde points to the problem of small sample size in many studies, leading to underpowering of the results 47. In addition, some of the reviews found a positive co elation between low methodological study quality and
Alberta Heritage Foundation for Medical Research Health Technology Assessment

l8

Acupuncture: Evidence from Systematic Reviews and Meta-analyses

positive outcomes 28 32 34 37. 38. This makes the determination of efficacy very difficult, as it is hard to differentiate between true positive effects, and false positive effects due to poor study quality, leading to inconclusive results.
There is also the issue ofassessing methodological quality ofstudies, not only for acupuncture specifically, but for complementa.y medicine overall. Acupuncture is based on differing philosophical models 4. 4e than Western Medicine. Using methodological c te.ia validated in conventional clinical tdals, to evaluate acupuncture trials may not be appropriate 4e. As in the determination of adequate treatment, one needs to attempt to separate the quality ofthe research from the quality ofthe

reporting

60.

Many systematic reviews examined in this report used the Jadad Scale 40.61 as their quality assessment tool, as it is 'the'validated tool among the assessment scales available 61. This scale includes five criteria (see Appendix C), four ofwhich look at randomization and blinding. Therefore, if a study does l1ot describe the randomization process or blinding methodology, the quality is deemed to be poor, without considering other criteria 5e. It also does not evaluate specifics important in acupuncture elficacy research, such as the appropriateness oftreatment, the skill ofthe the.apist, and the type and duration of treatment. Difficulties in blinding both practitioner and patient a ae are intrinsic to acupuncture, and some criticism ofthe Jadad Scale has been based on this 5e. Double blinding can, however, also be ofthe patient and the assessor ofthe results, which means that acupuncture research could meet this c terion after all 6t. 62. This latter inclusion for double blinding is not known or understood by some researchers. so studies may meet that criterion and be underscored in the quality
assessment.

Any quality scale should explore the clinical relevance ofthe question, the intemal and extemal validity, the appropdateness ofthe methodologies, and the ethical implications. There are many scales presently being used. though the key criteria to be assessed have not been agreed upon. Experts debate whether five criteria, such as the Jadad Scale, are enough to effectively determine quality of any research 5e and. on the other hand, whether longer lists of criteda may be too unwieldy. There is also some discussion whethe. scoring studies using set criteria is a useful tool for determining the quality olthe research 60 61. The criteria are often used solely to present the study data (10 trials were randomized. 12 were not) in a standard format, rather than to use it as a tool to analyze the study. These issues also extend to the systematic reviews, as the quality ofreviews varied f.om poor to good, and the review details provided were minimal in some cases 28. 37, 3e. Associated with this is the lack of agreement on an appropriate tool to assess the quality ofstudies in complementary medicine.
The continuing goal is the development ofstandardized and accepted criteria that are effective in evaluating the quality ofstudies in complementary and alternative mediajne 26,39,59.

Alberta Heritage Foundation for Medical Research Health Technology Assessment

19

Acupuncture: Evidence from Systematic Reviews and Meta analyses

DrscussroN
This project was undertaken to provide a critical appraisal ofthe scientific literature and determine the status ofacupuncture as a treatment option for various conditions, to assist health care decision-makers in Alberta, both rcgionally and provincially regarding acupuncture services. Acupuncture has become increasingly popular, especially for conditions of a chronic or recurring nature. Though the determination of effectiveness of trcatment for each condition is the primary objective ofthe report, much ofthe discussion has focused on the quality ofthe evidence and the issues in acupuncturc treatment and research.
Just as there are methodological limitations ofthe prima.y research studies, there are methodological limitations ofsystematic reviews, The quality ofthe systematic review is impacted by the quality of the reporting of the studies included in the review. This is even more ofan issue when critically appraising systematic reviews, which is further removed lrom the primary rcsearch.

Overall. the systematic reviews examined (10 out of 18, excluding Cochrane Reviews, had ratings ofpoor to satisfactor, were oflow quality methodologically, and reported mixed findings with inconclusive results. Dental pain, and nausea,/vomiting are the two conditions for which evidence supported the efficacy ofacupuncture as a treatment.
For dental and TMD pain. two reviews both found that acupuncture can be effective as a treatment, though there was no discussion as to the specific type and method of acupuncture that would be the most appropriate 2?. 24. Rosted, finding most studies in favour ofacupuncture, had concerns with the clinical relevancy ofsuch findings, as there are other analgesics available, with simpler procedures 22.

A Cochrane Review oa headaches found that though the procedures seemed

safe, there

were mixed results, and therefore the authoN made no statements regarding the efficacy ofacupuncture for migraine or tension headaches a1. The one review on tinnitus found that there was no difference between acupuncture and sham, and that the evidence did not support ofthe use ofacupuncture 23. Three reviews including a Cochrane Review on asthma reported inconclusive results, and that claims of efficacy \ /ere not supported. One difference with the study by Kleijnen et al. 28. however, is that they used only relative effectiveness as their outcomes, meaning that acupuncture would have to be more, not equally effective to the controls, to show results. Though the evidence was not strong enough to support claims of efficacy. Linde et al. 2e concluded that the evidence was also not shong enough to recommend to those using it, to discontinue. Overall, however, the use of acupuncture was not supported for the treatment ofasthma. The one review on st.oke rehabilitation found that though the evidence did not support acupuncture effectiveness, the findings were promising enough to warrant
Alberta Heritage Foundation for Medical Research Health Technology Assessment 20

Acupuncture: Evidence from Sysremaric Revie$,s and Meta analyses

further and better research to determine the actual effectiveness ofacupuncture as a treatment option 2.. However, two recently published blinded, placebo conbolled trials showed no effect of acupuncture (personal communication, Vickers, White).
In the two reviews on nausea and vomiting 30.3r, acupuncture was found to be effective, except for children and when acupuncture was administered under anaesthesia. The positive effects were better than sham acupuncture or no treatment, and equivalent to antiemetics. Four ofthe five reviews focused on back or neck pain, found that the effectiveness of acupuncturc for these types ofconditions was not supported by strong evidence 8 26 32 33. A rigorous meta-analysis36 by Ernst and White found that acupuncture was an effective treatment relative to other types oftreatment, and that thei. rclative usefulness needs to be investigated. Though Strauss 26 believes that acupuncture is safe and should be evaluated in more rigorous studies, the Cochrane Review Group concluded that since there were effective altematives, that acupuncture not be recommended as a regular treatment 8. These conflicting conclusions add to the uncertainty in the interpretation ofthe research to date.
The two reviews on chronic pain 3l rs found that the evidence was inconclusive overall. Ezzo et al. 3r also concluded that acupuncture for patients with chronic pain was more effective than the waiting list control group. Ezzo and colleagues also questioned whether sham acupuncture is inert, and what impact that may have. The review on fibromyalgia discussed adverse reactions to the acupuncture treatment, and even questioned whether this was an appropriate treatment for fibromyalgia 27, Though their findings were based on one quality RCT with positive results. they still felt that their review provided information on the benefits and risks, and raised questions on efficacy that required further research.

Two reviews on addictions to alcohol or heroin; smoking addiction; and weight/appetite reduction 38 3i indicated that the evidence was ofpoor quality and found little support for the effectiveness ofacupuncture. Acupuncture appea-red to be better than doing nothing for smoking addiction according to the results ofthe Cochrane Review aa and another systematic review 3t with a good quality rating. The review on weight and appetite reduction provided little information on which to evaluate the quality, and the treatments being reviewed together were very heterogeneous, both of which made the evaluation extremely difficult 37. Currently the evidence does not support the use ofacupuncture in addiction treatment or weight/appetite reduction.

Alberta Heritage Foundation for Medical Research Health Technology Assessment

2t

Acupunctur: Evidence from Systematic Reviews and Meta analyses

CoNcLUSIoNS
Growing demands on the health care system for public funding ofcomplementary health seryices, the changes in legislation regarding the regulation ofhealth care professionals, in conjunction with demand from the community for funding coverage for acupuncture treatment underline the importance and timeliness of this review.

A large body of primary research exists in acupuncture, covering virtually every symptom. Due to the breadth of this topic and the challenge of reviewing the extensive body ofresearch on acupuncture, the approach of systematically assessing the available reviews was chosen to evaluate the current evidence for the efficacy ofacupuncture. In choosing this approach, it is acknowledged that there are limitations.
There are many issues in acupuncture research which are highlighted briefly in this report that need to be explored and addressed in future studies. These issues range from the assessment ofstudy methodology to the appropriateness ofan acupunctu.e treatment regimen. The studies included in the reviews had many limitations and variations. Variability among the studies included the technique of needle insertion and manipulation, grouping of range ofacupuncture techniques, the number ofneedles per treatment, temperature ofthe needles, material composition ofthe needles, and selection ofconhol comparatoF. All ofthese factors may influence the study's outcomes and the overall conclusions ofthe systematic reviews,

Many researchers concur that acupuncture is a relatively safe procedure howevet. it is not without risk. Acupuncture can lead to both minor (drowsiness. nausea and fainting) and serious (traumatic injury ofbody tissue) adverse events. There is an increasing amount of literature published on adverse effects ofacupuncture, but there is still a concern about under reporting. Twenty-three systematic reviews on conditions such as dental pain/TMD, headaches, tinnitus, asthma, stroke, nausea/vomiting. neck/back pain, chronic pain, ftbromyalgia, labour, addictions, and obesity, were included in this appraisal ofsystematic reviews. This systematic review confirms the findings from other reviews which indicate consistent support for the effectiveness of acupuncture in the treatment ofpostoperative nausea/vomiting. and dental pain. For other indicators the robustness ofthe effect of acupuncture is debatable and its clinical value questionable for conditions such as idiopathic headaches, chronic pain, smoking and fibromyalgia, ho'"'i,ever some reviews indicated promising results. The results from these reviews 27 3136 ar, the majority of which had a good quality rating, found acupuncture to be as effective as the alternative interventions or no treatment in the short term.

Alberta Heriiag Foundalion for \,ledical Research Heahh Technology Assessment

22

Acupuncture: Evidence hom Systenatic Reviews and Meta analj.ses

Overall, in terms of the volume ofresearch that has been created in studying acupuncture there is a paucity of good quality research with large sampie sizes, randomization, and control for placebo effects. There was a lack of study detail provided in the reviews in regards to descriptions ofthe practitioners inirolved, Hence it was not possible to relate treatment effect or no effect to service provider.

Alberta Heritage Foundation for Medical Research Health Tehnology Assessment

23

Acupuncture: Evidence from Systematic Revie$6 andMeta analyses

APPENDx

A: METHoDoloGy

Two searches were performed inJanuary andJuly 2001. The following outlines the search strategy and the databases used. Effort was made to find criteri; accepted by the acupuncture community as well as the scientific community for use in the critical appraisal ofthe quality ofsystematic reviews for acupuncture. No quality assessment tool specific to acupuncture was found. Two ofthe co-authors (LB and CH) selected the arlicles based on the inclusion and exclusion criteria while two co-authors (LB and pLT) extracted data lrom the re\,iews and evaluated their methodological quality using criteria by Greenhalgh 2r as outlined in Appendix B, The authors of the reviews were not contacted for misiing information.
Search Strategy
Databases Searched
IVEDLiNE (Ovid) 1990-May2001

Subject headings (Bolded) and Textwords combinations

Acupuncture (exploded) OR acupuncture

acupressure OR Electroacupuncture OR electro-acupuncture OR staple acupuncture OR staple-acupuncture OR stapleacupuncture OR staple puncture OR staple-puncture OR slaplepuncture OR moxibusiion

and PTeMEDLINE
to Ju v 21. 2001 HeahhSTAR (Ovrd) 1991- Jan 2000 - database

disconlinued

Best evidence (Ovid) Jan/Feb 2001 CINAHL (ovid) 1990-March 2001

EMBASE (Ovid)
199
l\4av 2001 2AA1

AMED (Ovid)
Cochaane Database of

Syslematic Reviews
1st Ouarter 2001

CMA praclice guide lnesCPG lniobase

June 22,2aA1

Acupunctur' OR acupressure OR eleciroacupuncture OR e ectro-acupuncture OR staple acupunct!re OR staple-acupuncture OR stapleacupuncture OR staple puncture OR slap e-punciure OR stapleouncture OR nroxibusiion acupuncture OR moxibustion

National guide ine

acupunclure OR moxibLtstiof

cearinghouse
June

22

2AA1 AcLrp OR moxibustion

DARE HTA, EED June.2001

Aiberta Heritage Foundation for Medicat Research Health Technology Assessment

24

Acupuncture: Evidence from Systematic Reviews and Meta-analyses

Subiect headings (Bolded) and Tgxtwords combinations

WWW| ECRI, Bandolier, and
other HTA agencies

websites

acupuncture OR acupressLrre OR electroacupuncture OR electro-acupuncture OR stap e acupuncture OR staple-acupuncture OR stapleacupuncture OR staple punciure OR cture OR staDleounciure OR moxibustion

Two other databases, ISTAHC. Psyclnfo (February 2001), were searched but there were no relevant studies found. Articles were submitted by various people interested in acupuncture. and access was gmnted to a private collection ofjournals ofacupuncture. This 'grey literature'was hand searched for articles that complied with rhe inclusion cdte.ia. Reference lists of retrieved reviews were search for systematic reviews and
meta analyses.

Publlcation type limirs (where available): meta-analysis, systematic review "A systematic review is an oveNiew olprimary studies that use explicit and reproducible methods" 21. "A meta analysis is a mathematical synthesis ofthe results of two or more primary studies that addressed the same hypothesis in the same way" 2r.
These publication types were searched as textwords and where publication type Iimiting was not available by using this search stringr (Subject headings OR Textwords) AND (systematic review OR meta analysis OR critical appraisal OR metaanaly$ OR meta-analy$ OR metanalys OR critical$ apprais$ OR systematic$ review$)

Inclusion diteria: Articles were selected if they were systematic leviews, which includes but is not limited to meta-analyses. The study must have human participants, but with no restriction ofage group or nationality. Reviews were requi.ed to have an intervention of acupuncture as being the primary treatment intervention in the study. Studies addressing any medical indication were included if they were published within the past ll years (1990 - 2001). Only reviervs available in English were evaluated.
Exclusion criteria: Reviews were excluded if the use ofa tool to evaluate the methodological quality of the primary studies rvas not apparent. If reviews used the same methodological c.iteda and had the majority of primary studies in common, the older publications were excluded.

lncluded studies:

. . .

Ernst E, Pitder
Rosted

MH.
21

The elfectiveness ofacupuncture

in treating acute dentalpain:

systematic review

P.

The use of acupuncturc

in dentistry:

review of the scientilic validity

as a

of

published papers22

ParkJ, White AR,EtnstE. Ellicacy ofacupuncture

trcatment fot Tinnitus23

Atberta Heritage Foundation for Medical Resarch Health Technology Assessment

25

Acupuncturei Evidence from Systematic Reviews and Meta-analyses

. . . . . . . . . . r . . . . . r . .

KleijnenJ, ter Riet G, Knipschild P. Acupuncture andasthma:a review ofcontrclled

trials2s

Linde K, Worku F, Stor W, Wiesner-Zechmeister M, Pothmann R, Weinschutz T, et al. Randomized clinical trials of acupuncture lor asthma - a systematic rcview 2s ParkJ, Hopwood V, White AR, Ernst E. Ellectiveness ofacupuncture fot stmke: a
systematic review2s

Lee A, Done ML. The use of nonphamacologic techniques to prevent postoperative nausea and vomiting: a mek-analysis 30

Vickers
ac upu nc

A.

Can acupuncture have specific etects on health?

A systematic review of

ture antiemes is

tri als 3l

White AR, Ernst E. A systematic reuiew ofrandomized controlled trials ofacupuncturc for
neck Pain33

Smith LA, Oldman AD, McQuay Hj, Moorc RA. Teasing apart quality and validity in systematic feviews: an example from acupuncture trials in chnnic neck and back pain 32
Strauss AJ. Acupuncture and the treatment of chronic low-back pain: a review of the Iiterature26

Berman BM, EzzoJ, Hadhazy V, SwyerslP. Is acupuncture ellective in the treatmert of fibromyalgia? 27
Ezzo J, Berman B, Hadhazy VA, Jadad AR, Lao L, Singh BB . Is acupuncture ellective f1r the treatment ofchronic pain? A systematic review3a

ter Riet G, Kleijnen J, Knipschild P. Acupuncture and chronic pain: a criteia-based meta
analysis
3s

White AR, Resch KL, Ernst E. A meta-analysis

cessation 3i

of acupuncture techniques for smoking

ter Riet G, KleijnenJ, Knipschild P. A meta-analysis ofstudies into the ellect of ac up u ncture on addic tio n 38 Ernst E. Acupuncturc/acupressure for weight rcduction? A systematic review3r

Linde K, Melchart D, Fischer P, Berman B, White A, Vickers A, et al. Acupuncturc fot

idiopathic headache (Cochnne review)
!1

Linde K,JobstK, Panton J. Acupuncture for chronic asthmaa2

van Tulder MW. Cherkin DC, Berman B, Lao L. Koes BW. Acupuncture for low back pains

White AR, Rampes H, Ernst E. Acupuncture for smoking cessation44 Smith CA. Croq,ther CA. Acupuncture for induction of laboura3

Albena Heritage Foundation for Vedical Research

Heaith Technology Assessment

26

Acupuncture: Evidence from Sysrcmatic Reviews and Meta analyses

. . . . . . . . . . . .

Ernst E, White AR, Acupuncturc for back pain: trials 36

a meta-analysis of

nndomized contt,lled

Excluded Studies:
Ernst E, White AR. Acupuncturc as a treatment for temporomandibularjoint dydunction: a systematic review of randomind trials63 - methodological quality was not discussed Ernst E. Acupuncture as a symptomatic tftatment of osteoarth tis. A systematic revlew ia did not use a 'tool' to evaluate methodological quality Ernst E. Acupuncturc as an adjuvant thetapy in stroke rchabilitation? discuss methodological quality
6a

'

does

nol

Hopwood V. Acupuncture in quality was not discussed

stroke recovery: a litetature tevlew

65

- methodological

Rosted P. Survey of recent clnical studies on the treatment of skin diseases with acupuncture 66 Melenger A, Borg steir\J. Acupuncturc and sports medicine. A review ofpublished
studies6T

South

NA.
68

processei

Acupuncture for the treatment of trithdrawa] sruptoms in detoxilication - did not review methodological quality olthe studies,

Ashenden R, Silagy CA, Lodge M, Fowler G. A meta-analysis of the ellectiveness of acupuncturc in smoking cessalrbr 6e - included in the review by white and Rampes
Green CJ; Kazanjian A; Rothon DA. Acupuncturc in the management of alcohol and dMg dependence 7a report in dmft stage

Baillie AJ, Mattick RP, Hall W, Webster P. Meta-analytic rcview of the elficacy of smoking cessation interventionsTt - no clear systematic review of the literature is outlined. Databases used are not identified.

Melcha D, Linde K, Fischer

P,

White A, Allais G, Vickers A, Berman B.

72

Acupuncture fot recurent headaches: a systematic review of tundomized controlled trials - included in Cochrane reviews

Three 'reviews' were identified by the search that are currently in the form ofprotocols, not reviews, with the Cochrane group. They are as follows:

. . r

HeL,ZhouD,WuB,LiN.

Acupuncture fot Bell's palsy (Protoco, n

Green S, Buchbinder R, Hall S, Barnsley L, Forbes A, SmidrN,elal. Acupuncture for laterul elbow pain in adults (Protocol for a Cochrane Review)73

Richardson MA, Allen C, ExxoJ, Lao L, Ramirez G, RamirezT,etal. Acupuncture for chemothenpy-induced nausea or vomiting among cancet patients (Protocol for Cochrane
Review)
7a

Alberta Heritage Foundation for Medical Research Health Technology Assessmnt

27

AcupuncLurc: coDrpila&rn ofsysrcmatic rcviows and mola anatyscs

AppENDrx

B:

DATA

ExrRAcIoN

AND

euaLtry AssEssMENT oF

INCLuDED STUD|ES

Table 2: Data extraction and quality assessment of included studies

Study
(Ouality Ratins)
Design lype

Endpoini

Oata inregEtion

Reviewers assessment

Dentalan d TMD Pain

Jff*""""-,

svsr"."t"
l"=,"

lo *rtu

Dala reviewed suggest

eiJeclive in allevjating

Clear question, search extraction was delailed, and quality assessment Results interprcted in a
using variouslechniques and duration. and for

L,
Emsr E.

*,'n,".

Ihe conclusions lhat can

be drawn are limiled,
I

P,(er

I
I

lhe olher s werc

ccrs

tsarisracroryt

lu"ot,n..
I

!ii,ff'-.""
sludies.lrials

setting.

methodological qualily of Comparisons of sludies methods of lhe sludy acupuncture techniques. Difficuli lo glean the praclical implications. lnadequale description acupuncture treaiment

l."uo". lctscotr rr'.t

I

r'o,",v.

I I

o'" rr*.

o,scurs.n
]

Sham acupunclure had posilive eflects as well. No conclusive evidence

l",pu,r"in
I oL,orrsrreo

tn"
ano

I I I

acupunclure that is best for denlal analgesia, and

I I

wdl

_l

AIbc{a Ilcritagc loundalion for Mcdical Rcscarch

Hcalth
'l

cchnology Asscssmcnr

28

Ampuncturri: Ilvidcn{l) liom Systc )alic Rcvicws and Mrta-anatyscs

Table 2: Data extraction and quality assessment of included studies (cont,d)

Design type Sample size

Endpoint

ReYiewers assessnent

Search sir.teqy
Dala inlegralion
N= 15-all RCTs lisr ol 24

exlraction studaes, three

Objeclive is clear, search stElegy was adequale and inclusion/erclusion Outcomes descibed in a simplislic way of positive or negalive. Makes il diffacull to determine lhe

resulls. This linding

musl be intefpreled with ol 92 poinls,

Rosled P

1998"

Biological

lSatisractoryl Science

ctscoM

tanguages English,

questionable, as the time relaled to lhe effcacy of needed for acupuncture before a procedure is much longer than other assessmenl of method methods ot aneslhetic. qualily was more lhan
For facialpai. and TMO,

Gelman, Danish.

appropriale compadsons and conclusions drawn on resulls, and could be 60%. This is nol a lreatment effecliveness study, as much as a melhodoloqy Dala was grouped to illuslrate if lhere wre pailems between lho methodolog ical score and

TENS or laser

Eleven of 1 5 RCTS lvre in favour of acupunctue and shown ir ro be befler lhan sham, or similar lo

convenlional treatmenls.
highest melhodologically

Albcrla Ileritagc lroundalion for Mcdi0al Research

Hcallh l cchnology Asscssmcnt

29

A(upunclurc: Ilvidorcc

fro

Systcmalic Itcvicws and Mota nnalyscs

Table 2: Dala extraction and quality assessment of included studies (cont'd)

Study (Quality Ratins) Endpoinl
Data integration

Reviewers assessment

N=26
[,IEDLINE,

trealmenl of rccurrenl headaches, but most Oualily of werc melhodoloqically

(15) (6) - mixed (1)

(1)

Albcda Ilcritagc loundalio lbr Mcdicat Itcscarch

I

Icalth Tcchnology Asscssmcnl

30

Acupuncturc: Iividcncc liom Systcmalic Iicvicws an(l Mcla analyscs

Table 2: Data extraction and quality assessment of included studies (cont,d)

Study (Oualily Ratinq)
Design type

Endpoint

Data inlegration

Reviewers assessmerr

Tinnilus
20oo
23

N=

all RCTS

positive eflecl and four sludies thai we.e sham differcnce belreen lhe suggesting non-specifl c etrects oI acupuncture.

The objeclive was clear, and the search stralegy

adequale. Theinclusion Ouality assessmenl was adequale according lo the Jadad scoring, lhough the sample was quile small. adequate, excepi for the

lSalisfacloryl

crscoi/
AMED and Brilish Library

[,lelhodological quallty was Poor, with only lhree

acuPunclurc may have

Studies reviewed provide little information, as lhe oulcome measures varied across sludies, and there was no inlegralion of dala

Diflicull to inlerprel rcsult as lhe grcups were very result lhatwould be useful as a practical applicalion.

supported by evidence.

Albcrta Flcritagc Foundation for Medical ltcscarch

Hcalth Tcchnology AsscssDlcDt

3l

Acupundurc: Ilviclcncc liom Syslcmalic Rcvicws and Mcra rnatysos

Table 2: Data extraction and quality assessment of included studies (cont,d)

(auality Rating)
Study
Oesiqn type

Endpoinl

Data intgration

Reviewers assessment

Qualily
N

Data integmtion:

13

regading

arcund whelher sham acupuncture is an aclive placebo or not, and lhe lradiiional and Weslern

The objective is faidy clear, bul the search was not comprehensive. The inclusion/exclusion crile.ia
Quality was assessed
usang

Kleijnen J,
19C1

elal
thal [ledline CDRON4s,

lsalisfactoryl

of 100.
Bolh lhe

standad criteria

Concerns wilh the use of mela-analysjs and the

Study oulcomes we only listed as positive or

Though eight of lhe 13

results, only lhree had quality scores above 50 and lhe five negative result studies had scores

prcsenlation in lhe inilial

in English

English studies.

Claims ofthe eflicacy of lrealment of aslhma are not suppoded by well

peformed clinical lials.

Albc(a Ilcritagc loundation for Mcdical Rcscarch

Hcalth Tcchnology Asscssnrcrl

Acupuncturc: Iividcnc{} liom Sysl{}malic Itcvicws

aDCl

Mc1:r analvscs

Table 2: Data extraction and quality assessment of included studies (cont,d)

Study
(Ouality Ratins) Design lype

En.lpoint

Data inteqration

RevieweB arsessment

From a clinical standpoint comparabilily of the tr ats

N = 15 RCTS

The obleclive is clear. The

inlerprelation and clinical applicalion is very difticutt.

Difficull io interp.el the low

Qualily assessment was comprehensive manner using two scaies separalety Quanlitalive measures of lhe studies results were incllded fo. readers lo see compared on key variables. Their conclusions were delailed and supported by the.evlew of the trials in

1996,e IGoodl
l',ledline CD' ROMS (1983

rateFagreemeni f indings assessing ac!puncture quality. May bedue toa

lack of agreement otwhat good qualily acupuncture is, a poorly designed tool, or an inabilily lo interpret for clinical applications,

AMED,

clscor\4).
(1991).

moslly on the beliefs and allitudes of lhe.eviewers.

Due lo lhe hete.ogeneity

Dulch)

insufficient data, the highly conlradictory resulls are

Can neilher recommend to acupuncturists to cease lreating aslhma Dalienls nor recommend io start its

Albcrla Ucritagc loundalion fbr Mcdicat ltcsca.ch Hcallh Tcchnology Asscssmcnl

Acupunc(urci

lividunr liot)

Systcma(ic llcvicws and Mcla analyscs

Table 2: Data extraction and quality assessment ot included studies (cont'd)

Study (Quality Ratins)
Design type Sample size Search strategy

Endpoinl

Reviewe.e assessment

Did not find acupuncture

N=7RCTS

needed. Poinls ofsome

global
of lhe sham lreatmenls would be considered for lrcatment of asthma by

Albcrla Hcritagc F'oundation for Modi(al Rcscarch

34

Hcalth l echnology Asscssrncnt

Acupuncturc: Evidencc fi.om Systonaric R{}vicws and Mcra analyscs

Table 2: Data extraction and quality assessment of included studies (cont,d)

Design type Endpoint

Stroke Rehabilitation
Lack of homogeneity of iniervenlions and outcohe
N= 9 RCTS
P5ark

J, el al.2001

syslematic review very difiicult improvement when using Chinese scales, and no signiticant improvement when using a Weslem

The objellive or the siudy was clear. Search slrategy was suflicient and inctusion
Qualrty assessmenl was done using a validated toot, and dal8 en.acliob was

ctscoM
(including AMED), olher

Daily Living

sufiicienlly promising to

integralion, so the overatt findings, though rcbust. were based on summary

The authors state that lhe

Overall evidence does not supporl ihe effeciiveness of acupuncture tor shoke

poor melhodologicat q!atity impacls on lhe cerialnly of the inlerpretalions of ihe resulls This indicales the

Albcrla Hcritagc F-oundation Ibr Mcdicat Rcsoarch

I

lcalr

l1

Tcchnology Asscssmcnt

35

Acupuncturc: Iavidcncc tr$D Sysrcmaric ltcvicws and Mcta anatyscs

Table 2: Data extraction and quality assessment of included studies (cont,d)

Design type

Endpoint

Nausea and Emesis

Well designed pedialric siudies tailed to show a signilicant benelit using
rnclusion criteria were

clear. The seamh strategy complementary dalabases

lale (G48
1999
30

There was a siqnficant reducl on in early onset

The qualily assessmenl supported by reliability Clear oulcome measures and strong quanlitative
Data was integraied well,

PONV

PONV in adults versus placebo or drugs, and in late onset PONV versus

IGoodl

variaiion in lhe length of trealmenl trom 5 minules Lale oulcomes results were based on studics wilh small samples, so

descriplion or assessmenl Findings are rcbust and should be generalizable lo the larger population due io lhe strength of melhods The aulhors .ecognized and staled limilations and the potenliat impact on lhe

PONV

ihose who have adverse reactions lo anliemelic drogs or wish to reduce

Albcrta IIcrilagc Foundation for Mcdical Rcscarch I Icallh Tcchnology Asscssncnt

36

Acupuncluro: tividorcc fr.orD Sysrcnraljc ltcvicws and Mcra anatyscs

Table 2: Data extraction and qualily assessment of included studies (cont,d)

Design type

Endpoinl

Sludy quality
N = 33 CCTS, RCTs

Pe6 slimulalion is an

lechnique,lhough not

objeclive was clear, as were the inclusion criteia. Search was inadequale,
oppodunily lo find srey

clscoM,
vomiling IGoodl
Ieading

found lillle difference between groups, which

weighling ilself, in lrealmeni and

Assessment of sludy quality was adequate. bur lhe scorinq was quile subjeclive, and there was not slandard weighting of Outcome measures were rcporled wilh staiislical data,lhough there was liltle inlegralion of lhe data, and no mention of

conclusions up to lhe
27 of29 studies lbal did

lhough lhe

showed acupuncture as Summary of lhe quality and the resulls. minimal dala inlegration of the Based on lhe number of posilive versus negalive findings, the authols feasible; however. wilh

(includins

TENS, acupoint

Albcrta Hcritagc lounrlatton

tor.

vca icai

llcalth Tcchnology Asscssmcnt

ncscarch

37

Acupuncturc: Ividcncc fr.oln Sysrcmarj(j Rcvicws and Mcta anatyscs

Table 2: Data extraction and quality assessment of included studies (cont'd)

Study (Quality Ratino)
Design type

Endpoint

Data integration

Reviewers assessment

search 6tralegy

Data integralion
N

Sludies werc balanced belween posilive and belweeo a better quality

Search adequale, the inclusion andexclusion

14

of5.
E 19SS
33

[4otion (RO[4)

Qualily was assessed using a standard tool. clinical lrials needs to be The endpoints werc well described with p,values included. but lillle data Though the adequacy ol
Some studies suggest that precise techniques

language
ClSCO14, lheir

IGoodl

felt that it

lisling no

assessed. the rcview's authors rccognized this limitalion and lhe need in future studies for lhis vaiable ro be included.

equipoise in the resulls of lhe .eview ro justify

Their conclusions alrd recommendations make assuming acupuncture is sense and arc supported by the lindings in lheir

treaiment of neck pain is not suppoded by curent

Albcrta Hcrilagc Foundalion for Mcdical Rcscarch I lcalth lcchnology Asscssmcnt

38

Acupun{1urci Itvidorrcc lionr Svsrcnlatic R(:vn}ws and Mcr:l anatvscs

Table 2: Data extraction and quality assessment ot included studies (cont'd)

Design type

Endpoint

N = 12 RCTS

inleglaled using

The high odds mtio of unblinded studies may suggesf an association between acupuncture Acupuncture has been shown lo be superior to
interuentions. but it was not shown lo be superior

Objeclive is clear, search sitategy was adequale, and inclusion/exclusion Qualily assessment used Endpoinis were slaled,

ralng
1998
36

tGoodl

CISCO[,'1, gray

Spanish,llalian

Dala was analyzed well. and integ.aled thoroughly.

Further studies rcquired

effects as well as nonlrial

lreatment adequacy was rigorous,lhough only I of the 12 included sludies were incorpomred jnlo the The limitaiions of lhe

descibed, and next sleps for research suggesled.

The findings were clear

Albc[a llcritago loundation Ibr Mcdical Rcscarch

Heallh'f cchnolouy Assossmcnt

39

Acupunclurc: l-vidcnco fmm Systoomtic licvicws all.l Mota-analvscs

Table 2: Dala extraction and quality assessment of included studies (cont'd)

Sludy
(Quality Ratinq)
Design type

Endpoint

Data inlegmtion

lnt .ptul"lion,.d

RevieweIs assessment

validily in

N= 13 RCTS

global

(11 in chrcnic

Though the lrial conclusions lead to no rclationship belween sludy validity and lial findings,lhis review s aulhols found that ther

objeclives and inclusions

criteria werc clear. and the search was faifly thorough, no search on a complementary medicine

dalabase. Dataexlracted
Large discrepancy in melhods of lrealmenl

24 hours) or negalive findings. 2000

which.educed the degree

of analysis lhat could be Qualily/validily assessment Iool was added a lurther degrce of slrength lo their findings

'
lhe OPVS, (oPVS score)

There was insufficienl data io answer many

tcoodl

OPVS may be a useful acco.ding to validlly in qualilalive rcviews. One benelil oflhis melhod inclusion citeria can be used melhodologically, wilhout decreasing llle
No convincing evidence

findings groups, <10

liallindings
validity

Dala was inlegraled jn the analysis of validity and study outcomes, but was minjmalregarding lhe

aclualoutcomes and
acupunclule in lreating Their ovemll conclusions were robust. and rculdn l be significantly allercd if inclusion c teria were adjusted lo include or exclude cerlain sludies

Alberta Horitage loundation for Mcdical Rcscarch

Ilcalth Tcch olo&y Asscssmcnr

40

Acupuncturc: tsvidcncc from Syslcl alic Rcvicws and Mclr .rnalyscs

Table 2: Data extraction and quality assessment of included studies (cont'd)

Study (Otlality Ratins)
Design lype

Endpoint

Reviewers assssment

Three studies showed This rcview shows some

Slrauss AJ 1999

Search slrategy was not

complemenlary medicine

N=4CCIS
weighling, [,EDLINE,
CINAHL,
t!1ANTtS,

literalure. The objeclives and inclusion crileia were

=100
English

ISalislacloryl

future research design. lmponant thal singteblind sham conlrolled

RCTS

AIIIED, AIVII,

Aulhors {lsed a sel of craleia lhat assessed lhe ne@ssary items lo be

a completed to

show lhe validily of this

lreatmenf s efticacy. No menlion of who did the Lillle dala inlegration

:onclusions regarding the rcupunclurc, from lhese )oorly designed kials.

acupunclure trcatmenl and the practitioner were Very few details, outside of the lable of quality

sludies. Difticull lo check

or duplicate the review's melhods and findings.

Albcrta llcritage loundalion for Mcdiml Rcscarch Hcallh Tcchnok)sy Asscssmcnt

4l

Acupundurc: Ividcncc fr.om Systcnr.rtic ltovjcws and Mola,analyscs

Table 2: Data extraction and quality assessment of included studies (cont,d)

Study (Qualily Ratinq)
Design type

Endpoint

Data inlegration

N.11

RCTS

proven effective fo. lhe lrcatment of back pain accordlng ro rhe siudies idenlitred in lhis review

Disability Scale,

Albofla rlcritagc IroLndalion ibr Mcdicat Roscarctr

Hcallh
't

cchnology Asscssmcnt

42

Acupuncturc: Lvi(lcnce fron) Systonatir Itcvicws artd Mela analyscs

Table 2: Data extmction and quality assessment of included studies (cont'd)

Sludy
(Ouality Raling)
Design type Search slralegy

Endpoint

RevieweB assessment

Chn
Obje.tives and inclusion

con t rol qroup lype makes

lhe

resulls more comparable lo

outcomes assessmenl deiine signficance but

l.
we

not pEsented in lhe Ev

eu

Dala Ms well Jnlegraled across studies 5nd rhe use oi stalislical melhodologies makes lhe lindinqs more

tEatment examined. lhe

Auihors reporled lhe sulls unknowns (eg. Possible

Review lindings are robusl and lhe removal of one or

Alberta Heritagc Foundation for Mcdical ltesearch Hcalth Tcchnology Asscssmcnt

43

Acupuncluro: Evidcncc from Syslcmalic Rcviows and Mcla analyscs

Table 2: Data extraction and quality assessment of included studies (cont,d)

Study (Ouality Rating)

Q!estion/

Desiqn type

Endpoint

Dala integration

Reviewe6 assessment

No def initive conclusions on lhe efficacy can be

N=51

comprehensive, but their objeclives and inclusion

Used a veified lool to

alleviating

melhodolog ical quality of

1990

3e

lvledline cDROI\,1S,

100

Some rcviewer bias may be involved as lhey were nol blinded to lhe sludy More research needs lo be conducted wilh more

Data was integraled well

methodological scores. Study oulcomes were only lisled as positive or slatislical data presented. The aulhors examined lhe rclalionships betwen
varaables well, but lhe

English abslract

(ifan existing
English was

melhodological qualily
Results arc conlmdictory

between positive and

chronic pain condilions

included studies are not

Albcrta Hcritage Foundalion for Mcdical Rosearch Hcalth Tcchnology Asscssmcnt

44

Acupunclurc: Ividcncc from Systcnratic Revicws and Mora-analyscs

Table 2: Data extraction and quality assessment of included studies (cont,d)

Srudy (Quality Ralins)
Oesign type

Endpoint

Samplesiz
Fibromyalgia

Dala integ.ation

ReviweB assessm6nt

Objectives and inciusion cnleda we clear, and the acupuncture, which may be claled to lhe etiology ol ihe disease. Physicians should MANTIS, number ofFMS palienls

Only three studies had any follow 0p per ods, and iew used any endpoint measues lhal wen t self-repo.led. Qua ily assessment uses lwo aulhors dont report the

c{scoM, NtH

This revrew may provide praclilioners some practical infomation on lhe beneiits and risk s of acuponctu re,
raisinq questions for lulure

summary of data extraciion (even lhough a sludy s'high s@p is mentioned in the Evidence is based on only 1 slody. and lhe.efore would be considered sensiliee Then .onclusron lhat this review en be used lor some

Eseatuh, nol addressing

acu

P u

nclue experienced

sign icant improvehenls in subjective and objelive

.aisi.g reseaah quesiions,

Albcrla Hcritage loundattonforMcdical Rcsarch Hcalth Tcchnology Asscssnrnt

45

Acupuncturc: Ividcnce from Systcmalic llcvicws and Mcla anatysos

Table 2: Data extraction and quality assessment of included studies (cont,d)

Design type Search slrategy

Endpoinl

Data integralion

Reviewers assessmert

lnduction of Labour
Need for a welldesigned RCT lo evaluate the mle

N=0RCTS
Smalh CA,

Review 2001

a3

childbinh
Group lrials

bibliogmphies

Albcrla Hcriragc [oundarion for Medical Rcsoarch

Hcalth Tcchnology Asscssmcnt

46

Acuplrn(xurci lividoncc li.om Systcnratic Rcvicws and Mirla analyscs

Table 2: Data extraction and quality assessment of included studies (cont'd)

Study
lQuality Rating)
Design type

Endpoint

Reviewers assessment

A lisr oI 18
N = 22 CCTS

100. Points

Trcnd between quality of studies and resull, in

ihal higherqualiiy

broadt it could be mo.e specif ically slaled. Their literalure search was inclusion craleia were
No iesults other

RCTs 1990
36

There is no evidence
Medline CD RO[,{, Excerpta

tPoorl

th.n

+/-

lreatment of addiction The ulility of acuPunclurc in heroin

alcohol addiclion needs drop out rales seen in

robust in that they saw a common Pattern across sludies. which wouldn t have changed dmstically ifa study was removed.

English was

when the sludies are of

involving significantly Acupunclure has not prcven lo be efficacious as a trealmenl for these

Albcfia Hcrilagcloundatio'1 lbrMcdicalRoscarch Hcalth l cchnology Asscssmcnt

47

Acupundulc: lividcmc lionl Syslcrrlalic Rovbws and Mota analysos Table 2: Data extraction and quality assessment of included studies (cont'd)
Sludy (O!al'ty Rating)
Desisn type Endpoint
Data integration

Reviewers assessmnl

Tool
Search strategy

Majority of studies did

N = 18 RCTS

randomizalion prccess

Rview 2000

"
Group lrials

ofskill.
acupuncturc incrcases

smoking compared Io sham, bul appeared lo

(less than 6

rclhing
12 monlhs. lf Social

poinls (including Smoking&

Heallh.

Biological
DRUG.

bibliographies.

Alhcrta Hcrilagc Found:rlion for Mcdical Rcscarch rlcall h 'l cchnology Asscssmcnt

4lr

Acupunclurc: Ividcnco from Systcmalic llcvicws and Mda anatyscs

Table 2: Data extraciion and quality assessment of included studies (cont,d)

Design lype

Reviewers assessment

Subgroups

English, Frcnch
1999
15

smoking at3 righl afier

in smoking

E[,IBASE, Brilish Libmry,

Acupuncl{r.e did seem lo Though it was stated that be more efleclive than lhey contacted oiginal authols for additional study details, lhey then effecl, lhougtt short{erm stated lhal lack ofdelall in Acupunclurisl should be
quality oi lhe lechnique This review reflecls thal lack oi clear hypoiheses re:role of acupuncture an
defl

ctscot\,1,
lhe

IGoodl

ciency across sludies.

up lo 100

smoking cessalion bolh

This meta-analysis was lhorouqh. ll clearly staled its melhods, limilations of findings.

Future research needs

Fairly robust

removing

melhodological aspecls, biochemical validation of cessaiion, longer follow delails, efiective lhan sham. or lhal one acupuncture efieclive lhan another

scenarios ifa study was rcmoved, or some sludies werc grouped rogether and analyzed, rcsulls

igour.

Alberta Hcritagc loundation for Medical Rcsoarch I Ioallh Tcchnology Asscssmcnt

49

Acupuncturc: lvidcnco trom Syslomalic Rcvicws and Mcla analysos

Table 2: Data extraction and quality assessment of included studies (cont'd)

Study
(Ouality Ratins)
Design type

Endpoint

Data intesration

Reviewers assessment

Objectives and inclusion crite a are not specific. and the search stralegy trealment peiod, sl'rdy Ernsl lPoorl Qualily assessmenl was

1997

37

ctscoM,

= 100

design.

methodological qualily
results. whlle the 2 morc igoro'rs sl'rdies did not. Future studies should be

Due lo lhe heterogeneity of lhe slsdy melhods, and findings, in conjunclion wilh lhe small sample of

nndings of no evidence arc sound. However, the including search slralegy, stalement of objectives, and claily of inclusion crileria. are weak. These reaknesses likely played a .ole in lhe inconclusive lindings.

Therc is no convincing
using

acupunclure/acupressur rcducing body weighl or

Albcrla Ilcrilagc Foundalion forMcdicalRoscarch

Hcalth Tcchnology Asscssmcnt

50

Acupuncturei compilation ofsystematic reviews and meta analyses

AppENDrx C: QuAlrry ASSESSMENT TooLS

Jadad Scale for Evaluating Randomized Controlled Trials

Add
a

23 25 27 2s 30 32-34.4t 42

point for a "yes" and give

no

points for a "no"

1. 2. 3. 4. 5.

Was the study described as randomized (this includes the use of words such as random, randomly and randomization)? Was the study described as double-blind? Was there a description of withdmwals and dropouts? Was the method to generate the sequence of randomization described and was appropriate (table of random numbers, computer-genemted etc.) Was the method of double-blinding described and was it appropriate (identical placebo. active placebo, dummy, etc?)

it

Deduct one point

it

6.

The method to genemte the sequence of randomization was described and it was inappropriate (table of random numbers, computer generated etc)? The method ol double-blinding described and was it appropriate (identical placebo, active, dummy, etc)?
3r.s7.75

7.

Vickers Scale for Evaluating Randomized Controlled Trials

l.

Adequate desc ption ofthe population from which the participants are drawn.

2. Sample size adequate. 3. Random allocation to the treatment arms. 4. P.ognostic vadables adequately assessed. 5. Full description oftest inteNention. 6. Where possible a credible, inactive placebo should be used.
7. 8.

If no placebo is available. standard care may be used (but should have been compared previously to placebo).

The use ofappropriate outcome measures for the condition and the therapy.

Patients blind to treatment allocation and researchers blinded when assessing outcome,

Withdrawal and no-respondents less than 20% of initial sample. 10. Appropriate use ofinference statistics. A ranking of very good receives 4 points, good - 3, fair - 2, not satislactory

9.

poor

and

0. a

Ve.y good reflects considered valid-

study which has adequately met the criteria and the results may be

Alberta Heritage Foundation for Medical Research Health Technology Assessment

51

Acupuncture: Evidnc from Systematic Reviews and Meta analyses

Good reflects that the major

crite a were met and the results

have not been affected

Fair reflects the criteria have not been met fully and that the results have probably not been affected Poor reflects that the criteria have not been met adequately and that the outcome ofthe study has probably been influenced by this.

List of Methodological criteria: ter Riet G, Kleijnen J, Knipschild

28.v-3s

Criteria
C omp arab ility of prognosis A. Homogeneity (1) B. Prestratification (2) C. Randomization D. Comparability of relevant baseline characteristics shown E. U 50 patients per group F. < 20% loss to follow-up (3)

Weight
3 3 12
2

l0
5

Adequate interyention G. Avoidance ofDNIC (4) H. Adequate description ofacupuncture procedure (5) L Mentioning good quality ofthe acupuncturist J. Existing treatment modality in reference group Adequate EIfect Measurctuent

10 15
3

K. Patients blinded L. Evaluator blinded M. Follow-up alter treatment > 3 months N. Pain (7) C)- Use ofmedication P. Activities oldaily living Q. Remark of side-effects Data presentation

10
5

(6)

3
2 3 2

R, Reader is given opportunity to do inferential

statistics

Methodological criteria: Lincle

2e,ar

Method ofallocation to groups

2. Concealment of allocation 3. Baseline comparability 4. Blinding olpatients 5. Blinding of evaluators 6. Likelihood olselection bias after allocation to groups by drop outs
Alberta Heritage Foundation for Medical Research Health Technology Assessment
52

Acupuncture: Evidence from Syslematic Reviews and Meta-analyses

Each item is scored as follows

o=criterion not met or insufffcient information 1=critedon met

Maximum score is 6 An adaptation ofthis scale is used in 2e, where the 6!h and 7ih critefa include the handling of withdrawals, and the handling of inferential statistics. Maximum score is
7
8

Methodological Criteria: Cochrane Collaboration Back Review Group

Patient selection

a. b. c. a. b. c. d. e. a. b. c. d. e. i. ii. f.

were the eligibility criteria specified

treatmentallocation

i.
ii.
lntervention

was the method of mndomization described and adequate was the treatment allocation concealed

were the groups similar at baseline regarding the most important prognostic indicato$ were therapeutic and control interventions operationalized was the care provider blinded was controlled for co-interventions which could explain the results
was the compliance mte (in each group) unlikely to cause bias

was the patient blinded

was the outcome assessor blinded

Outcome measurement was at least one ofthe primary outcome measures applied was there a description ofadverse effects
was the withdrawal/drop-out rate unlikely to cause bias

timing of follow-up measurement perfomed

was a short-term follow-up measurement performed

was a long-term follow-up measurement pe.formed was the timing of the outcome assessment in both groups comparable

Alberta Heritage Foundation for Medical Research Health Technology Assessment

53

Acupuncture: Evidence from Systematic Reviews and Meta-analyses

Statistics

a. b. c.

was the sample size for each group described

did the analysis include an intention-to treat analysis were the point estimates and measures ofvadability presented for the pdmary
outcome measures
26

Methodological criteria: Strauss adapted from Koes

Study Population

a. Description ofinclusion and exclusion criteria (l point) b. Similarity ofrelevant baseline characteristics: the duration olcomplaints,
outcome measures, age, recurence status, radiating complaints (1 point)

value of

c.
d. e.

point) Randomisation procedure adequate: randomisation procedu.e described (2 points). Randomisation prccedure which excludes bias (2 points) Dropouts described for each study group sepa.ately: information from which group and with reason for withdrawal (3 points)
Adequate validity, accuracy, and reliability of diagnosis
(1

Loss to follow'up: <20% loss to follow-up (2 points), <10% loss to follow-up (2

points)

f.
g. h.

Smallest group immediately after randomisation: >50 subjects in the smallest group (6 points), > 100 subjects in the smallest group (6 point).

Interventions
Interventions included in protocol and described adequately: acupuncturc treatment described (5 points)
Pragmatic study: comparison with an existing treatment modality (5 points)

i. j.
k.

Co-interventions avoided: other physical therapy modalities or medical interventions are avoided in the design ofthe study except analgesics (5 points)
Placebo (or sham) contrclled: comparison with a placebo or sham therapy (3 points). Adequate description and use ofan appropriate placebo or sham (2 points).

Good qualification of acupuncturist: mentioning of qualified education and work expedence ofthe acupuncturist (5 points).

Measurement of Effect

i.

Patients blinded: placebo controlled: Attempts for blinding (3 points), blinding evaluated and fully successful (2 points).

Alberta Herirage Foundation for Medical Research Health Technology Assessment

54

Acupuncture: Evidence from Systematic Rviws and Meta analyses

m. Outcome measures relevant: use (measured and reported) of: pain. global measure of improvement, functional status (activities of daily living). spinal mobility, medicine consumption (1 point each). Validity and reliability olinstruments (1

n. o.

Poin0. Blinded outcome assessments: each blinded measurement mentioned under point M. Earns 2 points. Control of observer and subject bias (1 point). Follow-up period adequate: moment of measurement during orjust after trcatment (2 points). Moment of measurement 3 months or longer (2 points) Intention to treat analysis: when loss to follow up is less than 10%: all randomised patients for most important outcome measures and on the most important moments of effect measurement minus missing values, irrespective of non-compliance and cointerventions. When loss to follow-up is greater than 10o%: intention to treat as well as an alternative analysis which accounts for missing values (5 points).
Frequencies ofmost important outcomes presented for each treatment group. For most important outcome measufes, and on the most important moments ofeffect measu.ement. In the case of (semi-) continuous va ablesr presentation ofthe main or median with standard error or percentiles (5 points). Use ofdescriptive as well as inferential statistics. Literature review (1 point), good use ofreferences (1 point)
32

Data Presentation and Analysis

p.

q.

Methodological Criteria: Oxford Pain Validity Scale

This Scale is to be used with trials that are randomized and have an N five main categories with a possible score being between 0 and 16.

10. There are

2. 3. 4. 5. Data Analysis

Blinding maximum 6 points) Size oftrial groups (maximum 3 points) Outcomes (maximum 2 points) Baseline pain and internal sensitivity (maximum
(maximum 4 points)

point)

. . . .

Definition of outcomes
Data presentation: location and dispersion

Statistical testing

Handling oldropouts

Alberta Heritage Foundation for Medical Research Health Technology Assessmen!

55

Acupunclure: compiiation of systematic reviews and meta-analyses

APPENDIX D: CRITERIA FoR EVALUATNG SYSTEMATIc REVIEWS

Papers that summarise other papers (systematic reviews and meta-analyses) by Trisha Greenhalgh 2r:

L
2.

Can you find an important clinical question which the review addressed? Was a thorough search done ofthe appropriate databases and were other potentially important sources explored? Was methodological quality assessed and the trials weighted accordingly?

3. 4. 5.

How sensitive are the results to the way the review has been done?
Have the numerical results been interpreted with common sense and due regard to the broader aspects ofthe problem?

AppENDtx

E: CHEcKlrsr

oF DATA REoUTRED rN A CoMpLETE REpoRT oF

76

ACUPUNCTURE TREATMENT

l.

Patient's posture

2. Number ofneedles* 3. Needle size, manufacturer 4. Rationale and justification for point selection (traditional, tenderness, formulae) 5. Points used (international nomenclature), nonstandard points carefully described 6. Laterality 7. Depth* 8. Stimulation (eg manipulation, electrical, omoxabustion) strength and duration 9. Needle sensation induced
10.
11.

Duration of needling*
Frequency and number of repetitions**

12. Other

simultaneous interventions treatment

13. Subsequent changes to

* where values vary. the median and range should be quoted ** where electrical stimulation is used. details of device, pulse width, waveform, frequency, and strength ofstimulation (eg. presence olmuscle cont.actions) should be

included.

Alberta Heritage Foundation for Medical Research Health Technology Assessment

Acupuncture: compilation of systematic reviews and meta,analyses

REFERENcES

1. 2. 3. 4. 5. 6. 7. 8. 9.
10. 11, 12. 13. 14.

Canadian Institute for Health Informatlon. Health Care in Canada. Canadian Institute for Health Information, editor. Ottawa: Canadian Institute for Health

Information; 2001.
Statistics Canada. Health Reports: Hov, Healthy are Canadians2 Statistics Canada HSD, editor. Ottawa, Ontafio: Statistics Canada; 2001. Vol I2, No 3. Ruedy J. Kaufman DM, Macleod H. Alternative and complementary medicine in Canadian medical schools: a survey. CM,4,r 1999:160(6):816-7. Ernst E, White AR. A review of problems in clinical acupuncture research. American Jounal of Chinese Medicine 1997'.XXV(1):3-11.

Kaptchuk TJ. The Web that has no Weavet. Understanding Chinese Medicine, Chicago, Illinois: Congdon & Weed,Inc.; 1983,
Brooks L. Overview of the regulation of acupuncture in Alberta. HealthTechrrclory Assessment, editor. Edmonton, Albertar Alberta Heritage Foundation for Medical Research: 2001. IP-6.

White A, Ernst E, Birch S, Kaptchuk T, Filshie J. Cummings M, et al. Ernst E, White A. editors. Acupuncture. A scientfic appnisal. United Kingdom: Butterworth Heinemann:1999.
van Tulder MW, Cherkin DC, Berman B, Lao L, Koes BW. Acupuncture for low back pain. Available http://www.cochranelibrary.com 2001.

Mayer DJ. Acupuncture: an evidence-based review of the clinical literature. Arnu

Rev Med 2000:5
1,

:49

-63.

Leake R, Brodedck JE. Treatment efficacy of acupuncture: a review of the research Iiteftt'Jle. IntegfttMedicine 1999;l(3):107-15.

He L, Zhou D, Wu B, Li N. Acupuncture for Bell's palsy lprotocol]. Available: http://www cochranelibtary com 200L

Norheim AJ. Adverse effects ofacupuncture: a study ofthe literature for the years 1981-1994. J Altem Complement Med 1996i2(2):291 7. White A, Hayhoe S, Hart A, Emst E. Adverse events following acupuncture: prospective survey of 32,000 consultations with doctors and physiotherapists. BMl
2001;323:485-6.

MacPhe$on H, Thomas K, Walters S, Fitter M. The York acupuncture safety study: prospective survey of 34,000 treatments by traditional acupuncturists. ,BM/
20011323:486-7.

Alberta Heritage Foundation for Medical Research Health Technolos/ Assessment

57

Acupuncture:Evidence from Systematic Reviews and Meta analyses

15. 16. 17. 18. 19.

Ernst E. Adverse Effects ofAcupuncture. In: Jonas Wayne B, Levin Jefftey S, ed\tots. Essentials of Complementary and Alternative Medicine. Philadelphia, PA: Lippincott Williams & Wilkinis: 1999 p.172-5. Ernst E, White AR. Prospective studies of the safety of acupuncture: a systematic rcview. 4mJ Med200l: I I0:481-5.

ErnstE, WhiteA. Life-threatening adverse reactions after acupuncture? A systematic review. Pain 1957;71(2)1123-6.
Yamashita H, Tsukayama H, Tanno Y, Nighijo K. Adverse events in acupuncture and moxibustion treatment: a six-year survey at a National Clinic in Japan. Altern Complement Med 1999:5(3):229-36.

Badmaev V, Baime MJ, Benor DJ, Block KI, Brody H, Chapman EH. Lippincott Williams & Wilkinis , editot. Essentials of Complementary and Alternative Medicine. Philadelphia, PA:1999. Olsen O, MiddlerolP, Ezzo J, Cotzsche PC, Hadhazy V, Hexheimer A, et al. Quality ofCochrane reviews: assessment of sample from 1998. BMJ200l:323:
829.32.

20. 21. 22. 23. 24. 25. 26. 27. 28.

29.

Greenhalgh T. How to read a paper. Pape$ that summarise other papers (systematic reviews and meta analysis). BMJ 1997:3151672 5.
Rosted P. The use of acupuncture in dentistry: a rcview ol the scientific validity published papers. Onl Dis 1998i4(2):100-4. Park J. White AR, Ernst E. Efftcacy of acupuncture as a treatment for Tinnitus. Arch Otolaryngol Head Neck Surgery2000tl26:489 92.

of

Ernst E. Pittler MH. The effectiveness of acupuncture in treating acute dental pain: a systematic review. h Dent J 1998i184(9)1443-7 . ParkJ, Hopwood V, White AR, Ernst E. Effectiveness ofacupuncture for stroke:a systematic review. J NeutuI 2001;248(7) :558-63.
Strauss AJ. Acupuncture and the treatment of chronic low-back pain: a review of the literature. Chiropractic Jounal of Australia 1999:29(3):112-8.

Berman BM, Ezzo J, Hadhazy V, Swyers JP. Is acupuncture elfective in the treatment of fibromyalgla? J Fam Pnct 1999:48 (3):213-8. Kleijnen J. ter Riet G, Knipschild P. Acupuncture and asthma: a review of controiled trials. Thorax 1991:46(11):799-802.

Linde K, Worku F, Stor W, Wiesner-Zechmeister M, Pothmann R. Weinschutz T. et al. Randomized clinical t.ials ofacupuncture lor asthma a systematic review.
Forsch Komplementamed 1996;3(3):148 55.

Alberta Heritage Foundation for Medical Research Health Technology Assessment

58

Acupuncture: Evidence from Systematic Revie$'s and Meta-analyses

30. 31. 32.

33.

ML, The use of nonpharmacologic techniques to prevent postoperative nausea and vomiting: a meta-analysis. Anesth Analg 1999;88(6):
Lee A, Done 1362-9.

Vickers AJ. Can acupuncture have specific effects on health? A systematic review of acupuncture antiemesis trials. /R SrcMed1996;89(6):303-11. Smith LA. Oldman AD, McQuay HJ. Moore RA. Teasing apart quality and validity in systematic reviews: an example ftom acupuncture trials in chronic neck and back pain. Pain 2000:86(1 2):115-32.

White AR, Emst E. A systematic review of randomized controlled trials of acupuncture for neck pain. Rheumatologl 1999t38:143-7.
Ezzo J, Berman B, Hadhazy VA, Jadad AR. Lao L, Singh BB. Is acupuncture effective lor the treatment ofchronic pain? A systematic review. Pain 2000;86(3)r217 25.

34. 35. 36. 37. 38. 39.

40.

White AR. Resch KL, Ernst E. A meta-analysis of acupuncture techniques for smoking cessation. Tob Contol 1999;8(4) :393 7.
Ernst E, White AR. Acupuncture for back pain: a meta-analysis of randomized controlled trials. Atch Intem Med 1998i158(20)12235-41. Ernst E. Acupuncture,/acupressure for weight reduction? A systematic review.
Wien

Klin

Wochenschr 1997;109(2) :60-2.

ter Riet G, KleijnenJ, Knipschild P. A meta analysis ol studies into the effect of acupuncture on addictlon. BrJ Gen Pract 1990;40(338):379-82.
ter Riet G, Kleinen J, Knipschild P. Acupuncture and chronic pain: a criteria-based meta analysis. J Clin Epideniol l990;43(11):1191-9. Jadad AR. Moore RA, Carroll D, Jenkinson C, Reynolds DJM. Gavaghan DJ, et al. Assessing the quality of.eports ofrandomized clinical trials: is blinding necessary? Controlled Clinical Trials 1996;17:l 12.

41.
42. 43. 44.

Linde K, Melchart D, Fischer P, Berman B, White A, Vicke$ A, et aI. Acupuncturc fot idiopathic headache (Cochrane review), In: The Cochrane Library Issue 2, 1-46. 2001. Oxford: Update Software. Linde K,Jobst K,PantonI. Acupuncture fot chtonic asthma. In;The Cochrane Lib.ary Issue 2, 8 pages, 2000. Oxford: Update Soft\4'are. Smith CA, Crowthet CA. Acupuncture lor induction of labour (Cochrane Review). Cochrane Database Syst Rev:l:CD002962 2001. White AR, Rampes H. Emst E. Acupuncture fo. smoking cessafion. In:The Cochrane Library Issue 2, 10 pages, 2000. Oxford: Update Software.
Consens Statemenr 1997;15 (5):1,34.

45. Acupuncture. NIH

Alberta He.itage Foundation lor Medical Research Health Technology Assessment

59

Acupuncture: Evidence from Systematic Reviews and Meta analyses

46.

Health Care Financing Administration UC. Medicare Coverage Policy - Decisions. Av ail able h t tp ://www. h da. gov /c o v erag e/ 8b 3 c c 1. h tm.
-

47. 48. 49.

50.
51.

Linde K, Vickers A, Hondras M, ter Riet G, Thormahlen J, Berman B, et al Systematic reviews of complementary therapies - an annotated bibliography.Part I: Acupuncture. BMC Complement Alten Med 2001:l(l\:3.

Vicken A. Acupuncture. NHS Centre for Reviews and Dissemination:,200717.

Moroz A. Issues in acupuncture research: the failure of quantitative methodologies and the possibilities for viable, altemative solutions. Am J Acupunct
1999;27(1-2):95-103.

Ernst E. Acupuncture as a symptomatic treatment of osteoarthritis. A systematic teview. Scand J Rheumatol 1997 i26(6)1444 7. Birch S. Issues to consider in determining an adequate treatment in a clinical trial of acupuncture. Complementary Theftpies in Medicine 5,8 12.1997. Vickers AJ, de Craen AJM. Why use placebos in clinical trials? A narrative review of the methodological literature. J Clin Epideniol2000;53:157-61.

52. 53. 54. 55.

56.

Vincent C. Placebo controls for acupuncture studies.

1995r88:199-202.

J Royal Soc

ofMed

Streitberger K, KleinhenzJ. Introducing a placebo needle into acupuncture r esearch. The Lancet 1998:352:364-5.

Irnich D. Behrens N, Molzen H, Konig A, Gleditsch J, Kmuss M, et al. Randomised trial ofacupuncture compared with conventional massage and "sham" laser acupuncture for treatment of chrcnic neck pah. BMJ 2001:322t157 4'5. Araujo MS. Does the choice ofplacebo determine the results ofclinical studies on acupuncture? Forsch K,mplementarmed 1998;5 (Suppl 1):8 11,
Walach H. The efficacy paradox in randomized controlled tdals of CAM and elsewhere: beware ofthe placebo trap. J Altem Complement Med2001;7(3):213-8.

57. 58.

Linde K, Jonas WB. Evaluating complementary and alternative medicine: the balance of rigor and relevance. In: Jonas WB, Levin JS, editors. Essentials of Complementary and Altemative Medicine. Philadelphia. PA: Lippincott Williams & Wiikins; 1999. p.57-65.
Birch S, Systematic reviews of acupuncture - are there prcblems with these? Clinical Acupuncture and Oriental Medicine 2001:2:17 -22. Juni P. Witschi A, Bloch R, Egger M. The hazards of scodng the quality of clinical tdals for meta'analysis. JAMA 1999:282(l l): l0 54-60.
Jadad AR. BMJ Books, editor. Randomised Contolled Trials. A User's Guide. London: BMJ Publishing Group; 1998.

59.
60. 61.

Alberta Heritage Foundation for Medicai Research Health Technology Assessment

60

Acupuncturer Evidence from Syslematic Reviews and Meta analyses

62.

White A, Emst E. Systematic reviews of acupuncture - is a more profitable discussion possible? Clinical Acupuncture and Oriental Medicine 2001;2:lll-5.
Ernst E, White AR. Acupuncture as a treatment for temporomandibular joint dysfunction: a systematic review of randomized trials. Arch Otolaryngol Head Neck Surg I 9 I gl25 (3) :269 -7 2. Emst E, White AR. Acupuncture as an adjuvant therapy in stroke rehabilitation? Wien Med Wochenscht 1996;146(21-22):556-8.

63. 64. 65. 66. 67. 68. 69. 70. 71. 72. 73. 74.
75. 76.

Hopwood V. Acupuncture in strcke recovery:

Therapies in Medicine 1996;4:258 63.

literature review. Complementary

Rosted P. Survey ofrecent clinical studies on the treatment ofskin diseases with acupunctu.e. Am J Acupunct lgg4:22(4):357 -61. Melenger A, Borg-Stein J. Acupuncture and sports medicine. A review of published studies. J An Acad Med Acupunct 2000;tt(2):l 8. South NA. Acupuncture for the treatment of withdrawal symptoms in detoxification processer AADAC, editor. Edmonton, Albertat Alberta Alcohol and Drug Abuse Commissiont 2000. Ashenden R, Silagy CA, Lodge M, Fowler G. A meta-analysis of the effectiveness ofacupuncture in smoking cessation. Drug and Alcohol Revien,1997;16:33 40.
Green CJ, Kazanjian A. Rothon DA. Acupuncture in the management of alcohol and drug dependence @raft). University of British Columbia, editor. Vancouver, B.C.: BC Offtce ofHealth Technology Assessment 2000. 00:17T.

Baillie AJ. Mattick RP, Hall W. Webster P. Meta-analytic review of the efficacy of smoking cessation interventions. Drug and Alcohol Review 1994;13t157 -70. Melchart D, Linde K, Fischer P, White A, Allais G, Vickers A, et al. Acupuncture for recurrent headaches: a systematic review ofrandomized controlled trials. C ep ha Ia lgia lS99;19 (9) t7 7 g 86.
Green S, Buchbinder R, Hall S, Barnsley L, Forbes A, Smidt N, et al. Acupuncture for lateral elbow pain in adults (Protocol for a Cochrane Review). In. The Cochrane Library, Issue 3, 2001. Oxford: Update Software.

Richardson MA, Allen C, EzzoJ, Lao L, Ramirez G. RamirezT,etal. Acupuncture for chemothetapy induced nausea or vomiting among cancer patients (Ptotocol fot a Cochrane Review). In: Oxford: Update Software; 2001. Vickers A, Goyal N, Harland R, Rees R. Do certain countdes produce only positive results? A systematic review of controlled als. Controlled Clinical Trials

.199819:159-66.

White AR, Ernst E. A trial method for assessing the adequacy of acupuncturc treatments. Altenative Thetapies 1998:4(6)r66-71.

Alberta Heritage Foundation for Medical Research Healrh Technology Assessment

61