COMMUNITY HEALTH STUDIES
VOLUME XIK NUMBER 3,1990
THE IMPACT OF HEALTH TECHNOLOGY ASSESSMENT
D.M.Hailey, D.E.Cowley and W.Dmkiw
Health Technology Unit, Australian Inrtitute of Health, Canberra, 2601
Introduction
In this paper we consider matters related to
measurement of the impact of health technology
assessment from the perspective of a central
evaluation unit, and in terms of perceived
influence on policy, on the basis of assessments
undertaken by the National Health Technology
Advisory Panel (NHTAP).
Medical technology has been taken by the U.S.
Office of Technology Assessment (OTA) to apply
to drugs, devices and medical or surgical
procedures used in the CMC of patients.' While
broader definitions of medical technology are
possible, to include for example, health promotion
programs, the activities of the NHTAP have been
directed towards appraisal of devices and
procedures. For the purposes of the terms of
reference of the Panel, 'health technologies' are
d e f i e d as those activities and procedures which
involve the use of devices or equipment to
prevent, diagnose, treat or cure disease and which
contribute substantially to the total cost of healih
care (or potentially could do so). 'Device' means
any instrument. apparatus, implement or machine.
NHTAP does not, in general, assess
chemotherapeutic agents except as necessary in
the course of evaluation of other types of health
technology.
The development of health technology
assessment
The concept of health technology assessment
arose as a result of concerns in many countries at
rises in the cost of health care, and a perception
that such rises were associated with the diffusion
of medical technologies into health care systems.
Technology assessment, including appraisal of the
cost, safety, efficacy, effectiveness and efficiency
of technologies, developed as aid to decision
making by policy makers and providers of health
care.
Benefits through timely and practical
assessment may include:
early warning of new technologies which have
potential to proliferate and generate additional
costs;
HAILEY et al.
advice on the costs, benefits and effects on
infrastructure of technologies as an aid to
rational decision making;
identification of areas where the costs and
impact of technologies are uncertain. and
where further data MC required;
raising the level of awareness of the
community on the costs and benefits of
technologies; and
assistance in the introduction of technologies
in an orderly fashion.
Assessment organisations have been set up by
a number of governments in response to various
concerns on health care technology. Assessment
mechanisms have included literature reviews,
expert committees, consensus conferences,
collegial appraisals and canvassing of professional
. opinion. The U.S.A. is the most active country in
this field, with a number of organisations
undertaking assessments from different
perspectives, both in the public and private
sectom.
Australia's level of health technology
assessment compares favourably with that in most
other couneies of similar size. In Canada, health
technology assessment agencies have been
established only since 1988 (although hospital and
university-based assessments were being
undertaken before then). Among other, smaller
countries, Sweden and the Netherlands are
Particularly active, with national bodies as well as
other Centres involved. The larger industrialised
countries (except the U.S.A.) tend not to have
centralised authorities for health technology
assessment and activity is more diffuse.
Tlmlng and dlfflcultles of assessment
Consideration of the efficacy, effectiveness and
efficiency of a health care.technology needs to be
set in the umtext of its life cycle, the progression
from research through introduction, diffusion and
maturity to eventual obsolescence. The OTA has
linked this to the responsible policy areas which
may become involved at different parts of the
cycle of research and development, evaluation,
regulation and financing.'
223 COMMUNITY HEALTH flUDIES
Figure 1: Evolution of a Medical Technology
.Iffuslon
. Evolution
jNNOVATORS
Market
OPFRATORS Initial use
Initial
ASSESSORS efficacy/
cost data
SOVFRNMENT, Conditional
INSURERS support - reimbursement, - reimbursement ---
grants,
restriction
A key to all h i s activity is evolution of the
place of the technology in the health care
system over a period of time. This is a dynamic,
complex process involving a number of players.
Figure 1 depicts the activities of major sectors
at different points in the cycle. These sectors
might include innovators (typically industry and
university groups), operators (medical
profession, hospitals), assessors (agency,
academic and professional groups) and funding
bodies (governments, insurance agencies). Each
group will undertake different activities as the
technology matures. There will be the hope of
interaction between them in the form of useful
exchange of information.
Each group will of course have different
agendas. Innovators will wish to market as
quickly and widely as possible. Operators will
seek to provide improved health care, with
appropriate reimbursement. Funding agencies
will wish to protect limited budgets and seek
evidence of overall cost savings and benefits.
Some innovations may evolve very rapidly in a
technical sense, putting further strains on the
funding, assimilation and assessment pro-
cesses.
HAILEY et al.
Obso lescence
Modify RepIace
Phase in
useof ___
competing
technology
trials. More competing
Wider Review
licencing?
It is in this complex environment that the
evaluator has to obtain data from innovators and
operators, and seek to influence these groups, and
government and professional bodies, through
analysis and recommendations.
The process of assessment implies the
presence or generation of reasonable data on the
technology in question. Methods available for
gathering data will depend on resources available,
the nature of the technology and the setting in
which it is to be used. Often the most useful
information will be primary data collection that is
relevant to the local health care system. Typically,
it is difficult to secure adequate levels of funding
to support data collection on a range of
technologies. Other problems may include
reluctance of some participants in the health care
system to co-operate in trials. and biases in
interpretation.
Technical problems of health technology
assessment may be severe. The most powerful
methods of primary data collection and analysis
may need major resources and be technically and
logistically difficult. Typically, collection of
sufficient data to permit a statistically strong hid
of a technology will take some time. The ideal of
224 COMMUNlTY HEALTH STUDIES
a randomised trial design may face problems of
compliance. There is limited value in undertaking
lengthy and detailed assessments, if the capability
and nature of the technology changes significantly
during the assessment process. Easier, but less
definitive methods such as the case control
approach will often therefore be used. The more
rapid results obtained will have greater
uncertainty and need of validation. All data and
methodology will be to some extent imperfect,
and any conclusions have to be regarded as
provisional.
Methods of appraisal of primary data may
include the synthesis approach used by NHTAP.
and collegial assessment processes such as that
developed by the American Medical Association.
The time taken to collect information may be
significant, particularly if worthwhile data are to
be obtained on the outcome resulting from use of
technology. This in turn introduces a source of
tension between the evaluator, policy maker. md
interested professional groups. Typically, there
will be a wish for decisions to be made promptly.
Inevitably, government policy decisions on health
technologies will be linked to the inexorable
timetable of the budgetary cycle.
The timing of a technology assessment can
often require careful judgement. As many
technologies evolve rapidly, early assessment of a
new device or procedure runs the risk of not
taking sufficient account of technical
developments (the 'moving target' problem). In
addition, limited data will be available at the early
stages of development of a technology. If, on the
other hand. assessment is deferred for too long,
decisions on the use and support for a technology
may already be taken, and possibly be difficult to
reverse if they prove inappropriate.
The NHTAP work on office pathology testing
provides an example of appropriate assessment
timing. Detailed evaluation of 'dry chemistry
pathology' analysers intended for use outside the
laboratory was taken prior to any widespread
diffusion of the technology in Australia, and
contributed to decisions on the accreditation and
education of users of such equipment.'
The Impact of health technolagy assessment
There is increasing scrutiny of health
technologies and the number and rigour of
relevant papers in medical and other journals
gives some indication of this raised level of
interest. What is less clear is the extent to which
such assessment activity has led to identifiable
changes in health care planning or in usage of
selected technologies. No matter how thorough
the analysis, how compelling the clinical trial or
HAILEY et al.
how eloquent the wording of the recommenda-
tions, health technology assessment will have
little or no influence unless it moves policy
makers or users of technologies to modify their
decisions and methods of practice. Given the
accepted importance of conducting health
technology assessment, and the increased
resources that have been devoted to it, is it now
possible to assess its impact?
For a health technology assessment to have
much chance of making an impact, at least in the
short to medium term,it will nced to be capable
of producing timely results. and to be presented in
a form that is intelligible and accessible to policy
makers and users of technologies. It should also
be conducted in awareness of the realities and
sensitivities of the political process, whether
within government, industry or professional
organisations. While health technology
assessment is a highly complex technical
undertaking, it is also of the essence that it is a
politically related activity. The intention of an
assessment is after all to influence decision
makers on the & p e of support for a technology
and on its appropriate usage md access.
The first problem in attempting to assess the
impact of health technology assessment is that
unique measures may not be possible. I)rpically.
.policy decisions will be influenced by a number
of inputs of which technology assessment will be
only one. If there is a change in the policy on a
technology there will urually be a degree of
uncertainty as to whether the assessment was
responsible, wholly or in part. The simplistic
progression from requirements for information on
a technology, through assessment. provision of
data and recommendations, policy formulation
and subsequent effects of policy on use of the
technology (Figure 2) is not always very close to
real life.
Some effects of technology assessment. for
example influences on expectations and behaviour
pattems in users and providers of services, may be
quite long term. Such effects are hard to measure
and to assign solely to the influence of an
assessment process. Technology assessment is
likely to have an effect on levels of funding; range
of application of the technology; encouragement
for its appropriate use; and encouragement for
competence in its use. It may also influence the
rate of introduction, the control of introduction
and diffusion of a technology.
Measures of impact may include levels of use
of the technology in different applications,
distribution. expenditure and establishment of
accreditation and other quality assurance
programs. One possible approach is to consider
225 COMMUNlTYHEALTH STUDIES
Figure 2: The Polky h e f t ? for a Health Technology
-- Reimbursementrequest
- Procurement b replacement decisions
REQUIREMENT
- Decisions on scope of use

--+ ASSESSMENT - E4Xnomic lmpct

- Safety

- Health authorities
ASSIMILATION - insurers
- Medical profession

- Funding decisions
ACTION - Licensing
- Change in clinical practice

4-7 REVIEW

the results of recommendations made in Australian experience

assessments - whether they were accepted and, if
so. whether any significant action was taken.
Typically the targets for assessment will be
government agencies, professional bodies,
hospital administrators and perhaps
manufacturing industries. The outcome of any
recommendations will, in part, depend on the
origin of the assessment; its intended target;
methodology and content; whether there were
clear-cut recommendations; whether the
recommendations were aimed at, and compatible
with, the target; and whether the target had
appropriate powers and the will to act on the
recommendations.
Figure 3 summarises factors possibly related to
the impact of health technology assessment. the
conditions for such impact to occur and possible
measures of impact. Such impact will be more
likely to occur if there is a relatively central form
of control, whether through government, health
insurance agencies o r professional bodies.
Government influence may extend to importation
controls and provision of capital or recurrent
funding. Professional bodies may have particular
influence on the scope and standards of usage.
If clinical studies of drugs and of the accuracy
of diagnostic tests are excluded, much of the
recent Australian experience in appraisal of health
care technologies relates to the work of the
NHTAP. The Panel, established in 1982, was,
until mid-1989. chaired by the Commonwealth
with representation from States, major health care
sectors and experts in appropriate disciplines.
Support for the Panel is provided by a permanent
secretariat, now based at the Australian Institute
of Health.
It is not possible for any one agency to assess
all health care technologies, given limitations on
resources. Like other assessment bodies, NHTAP
has had to make a judgement as to which topics to
address. Criteria for selection of a health
technology for assessment by the Panel have
included:
costs (both capital and recurrent);
effects on infrastructure and other services;
number of people to whom it might be applied;
availability of competing technologies;
probable significance of the technology in
terms of effects on mortality, morbidity or
quality of life;

HAILEY et al. 226 COMMUNlTY HEALTH STUDIES

Figure 3: HealthTtechnolagy Assessment Impact Consldcntbm
Impact Coditim Measure

Deny Bccess to technology Legislation, political will, Resdabence

(e.g. ban imports)
Effect controlled introduction.
degree of funding
Affect placemend
distribution.
Affect range of application.
Enforce/encomge
competence in use
Encourage appropriate
-ge
possibly professional support.
Government funding (capital costlfees)
professional control of usage.
Government fundin& certificateof
need or licensing.
Government funding,ability to link to
~lidonsandmonitorusage,pcer
review, professional bodies guidelines.
Power to link acucditation to
reimbursanmtor~rcceptance.
Influmtial body with capacity to
prrplae g u i d e b .
of technology.
Numbas of units of
technology o v a time; level
of fees, capital grants.
Distlibution and location
of unitslservices.
Level of usage of techraology
in Merent epplications.
Accreditation and quality
assurance progrmns established.
Guidelines in use,numbas of
~ateprocedures.

level of usage and potential for inappropriate
use;
whether the technology has already been
adequately assessed by another organisation;
and
whether an assessment would be likely to
influence future deployment and usage of the
technology.
Decisions on whether to und.ertake an
assessment of a technology have also been
influenced by the state of development of the
technology and the possibility of significant
further advances, safety considerations, and
implications if its introduction is delayed or a new
use not approved.
Assessments undertaken by the Panel resemble
those of the OTA and the U.S. Office of Health
Technology Assessment in that they represent a
synthesis of available information, drawing on
published literature, expert opinion and work in
progress in industrial, academic and clinical
organisations. Each of the Panel's reports has
typically included technical, clinical and cost
appraisal of a technology. Attention has also been
given to woikforce. safety and social issues,
though not in the sense of comprehensive analysis.
There has been an emphasis on consultation with
professional and industrial organisations, and as
far as possible each published assessment
represents a consensus position.
Inevitably, the consensus reached by the varied
representation on the Panel is not always wholly
compatible with the views of those with interests
in the health technology under consideration. For
example, some proponents of bone mineral
assessment as an aid to the management of
osteoporosis have advocated more extensive
usage than that recommended by the Panel.
The topics considered so far in publications by
NHTAP and its committees are summarised in
Table 1. At the time of writing twenty-four
technologies had been assessed in thirty-five
reports, plus a consensus statement. A review
published separately by AIH in 1989 was intended
to complement an NHTAP study.' Requests for
assessment of technologies were obtained most
frequently from the Commonwealth Department.
with smaller numbers from the States, professional
bodies and industry. In seven cases, NHTAP
initiated the assessment itself.
When the twenty technologies assessed up to
the end of 1988 are considered, in eleven instances
the NHTAP reports appear to have had a
significant influence m the short- to medium-term.
on the basis of acceptance of major
recommendations and subsequent government or
other action. In a further three instances there was
probably some influence on short-term decisions
although some recommendations have yet to be
implemented. Sixteen of the NHTAP reports,
including three where the recommendations had
no obvious short-term effect. have proved useful
as source and educational material. as judged by
requests received by the NHTAP secretariat and
citations in the literature.

HAILEY et al. 227 COMMUNLT HEALTH STUDIES

 TABLE 1
Natlonal Health Technology AdvLsory Panel -summary of activities
~

Health Technology Reference Reports Mqlor Education

(years of short-term WSOUNX
publkatloa) influence material

1. MRImaging mA
P 83.84.87 YeS YeS
88,89.89 plus
consensus statement
2. Medical cyclotrons Ministn 84.85 YeS YeS
3. NMRspecmscopy mA P 85 - YeS
4. ESWL(renal) DCSH 85.87 YeS YeS
5. EsWL(biliary) m
AP 88 YeS YeS
6. Lasersinmdcine NKMRC 86 - YeS
7. Lasersingynaecology RACOG 87 YeS YeS
8. Vestibular function testing WSH 86 YeS -
9. Digital Subhection Angiography DCSH 86 - -
10. 'Office Pathology' tating AHMAC 84.87.88.88.89 Yes YeS
11. Bone mineral assessment DCSH 86.89 YeS Yes
12. Surgical stapling -try 86 - -
13. Portable fluoroscope DCSH 87 YeS -
14. Oxygen ummtrators m
AP 87 - YeS
15. Endoscopy G/E SocietJ 87 - YeS
16. Mammography AHMAC 88 Yes YeS
17. CTScanning mA P 88 Possible YeS
18. Digital Radiography m A P 88 Possible Yes
19. Computerised primetry DCSH 88 Yes YeS
20. Artificialhearts DCSH 88 Possible YeS
21. Brachytherapy DCSH 89 Possible -
22. High energy radiotherapy NSW 89 - -
23. Dynamometry VACC 89 Possible YeS
24. Coronary Angioplasty mA P 89 YeS -
(AIHlit review 89)

Table 2 presents an indication of the response to
major recammendations in NHTAP reports dealing
with the first twenty technologies listed in Table 1.
Three groups of recommendations are considered-
those related to introduction or support for a
technology, those dealing with accreditation,
guidelines or safety matters and those where further
research or study were proposed.
In the first group, seven of twelve
recommendations were accepted by the government
agencies responsible for provision of funding for
the technologies. It is less certain whether support
for oxygen concentrators or lack of funding for
artificial hearts was due, in significant part, to the
Panelk recommendations.
In the second group, five of sixteen
recommendations related to matters such as
guidelines or safety standards were accepted.In two
cases (MRI. office pathology) government agencies
and professional bodies took some action;
professional bodies accepted recommendations in a
h h e r three.(A recommendation in the most recent
NHTAP report, on coronary angioplasty, has also
been accepted by a professional body).
In the third group, government agencies
supported six of thirteen recommendations on

HAILEY et al. 228 COMMUNITY HEALTH STUDIES

 TABLE 2
Results of recommendptions in NHTAP reports

Type of recommendation Accepted Not accepted Uncertain

Related to introduction or 7 2
reimbursement (MRI. mal and biliary (vestibular function
ESWL, bonemineral testing. DSA)
assessmms~m
4- fluoroscope.
comp~-paimeay)
Related to accreditation, 5(1) 2
safety, guidelines (MRI. lasers. office (lasers in gyneecol,
PathOlOW. DSA)
m-ography. cr)
(C-w
=%iOPl.Sty)
Further research or 6 4
study (MRI. cyclotrons, (renal ESWL. bone
biliary ESWL. vestibular mind assessment,
function testing. office staple=%cr)

18 8 14
* Still under discussionhooearly to say

further research or study, and subsequent action
has been undertaken with the active co-operation
of professional groups.
Overall. eighteen of forty recommendations
were accepted, including most of those dealing
with introduction/support. The outcome of most
of the recommendations on accreditation.
guidelines and safety remains uncertain. While
there has been widespread interest and discussion
on preparation of documentation, in a number of
cases professional bodies and others have yet to
put in place these difficult-to-achieve initiatives.
For one area of technology, lasers. them has been
a mixed reaction, with so111e work on national
safety standards proceeding, but perhaps less
action on guidelines at institutional level.
Recommendations on research/study have led
to significant evaluations m several areas. notably
MRI. medical cyclotrons and office pathology
testing. The effect on mammography is less
certain. The NHTAP recommendation on the need
for appraisal of cost effectiveness probably had a
minor influence on decisions taken o n the
AHMAC cancer screening project. The continuing
strong Australian research effort into bone
mineral assessment and osteoporosis has probably
been influenced only to a minor extent by MAP
recommendations.
Follow-up to assessment
A single assessment of a health care
technology will often be insufficient to inform
adequately users and policy makers of relevant
issues. Technologies may evolve rapidly, both in
technical capability and range of application. Data
from assessments and routine usage may
accumulate slowly. For significant technologies,
there will be a need to continue an assessment and
monitoring process ad their life cycles develop.
Such a process should obtain relevant information
on costs and effectiveness, and provide
perspective on the place of the technologies in the
health care system, particularly when competing
methods are available.
As suggested in Figure 1. action in relevant
policy and other sectors will develop as the
technology evolves, and to some extent be
interactive. With wider diffusion of a technology,
the impact of assessment will be increasingly
dependent on the degree of influence of

HAILEY et al. 229 COUUUNITY HEALTH STUDIES

Figure 4: MR Imaging in Australla

DATA ON
ASSESSMENT ASSESSMENT; ' COSTS, USAGE
TECHNOLOGY
MANAGEMENT UPDATE
' CONSENSUS

POLICY
I
PROPOSALS
ACCEPTED
HEALTH
RACR) H DELAY DUE TO BUDGET
PRESSURES(FEDERAL
COORD. DEPARTMENTS]
FEDERAUSTATE
POLICIES ON FEES,
NUMBERS OF UNITS,
APPLICATIONS

PRIVATE SECTOR 1 X PRIVATE MA

-
FURTHER PRIVATE
UNITS,
'
PROPOSALS INSTALLED
OTHER LIMITED GOVT.
NO GOYT. SUPPORT
SUPPORT

Figure 5: Shock Wave Lithotripsy In Australia

- IMMEDIATE, LIMITED, TECHNOLOGY UPDATE

r-------->

INTRODUCTION
- PLACE IN MAJOR HOSPITALS - KEEP UNDER REVIEW
ASSESSMENT - CAUTION ON NEED FOR
- ACCESS TO ALL SUITABLE FUTURE ESWL
PATIENTS
- MONITOR DATA GALLSTONE
ESWL TRIAL

4 BASIS ONLY

DISCUSSION ON
POLICY SECTOR UNITS
FUTURE SERVICES

ONLY LIMITED USAGE AVAILABILITY

DATA. OF SECOND
ACCESS FOR INTERSTATE
- USE OF - GENERATION
OTHER PCN
ESWL
PATIENTSIMPERFECT.

Note: ESWL D enracorporoal ShoCk wave Ilthotrlpsy

PCN = percutaneous nephrolithotorny

HAILEY et al. 230 COMMUNITY HEALTH STUDIES

regulatory authorities and professional bodies and
on the mechanisms for informing users of a
device or procedure of the major recommenda-
tions and significant data.
The developing experience of the NHTAP has
pointed to the need for on-going assessment of
many topics. Of the first nineteen technologies
listed in Table 1. ten have been subject to further
appraisal by NHTAP. and a watching brief is
being maintained on a further five. Such ongoing
assessments place demands on the agencies with
limited resources and effectively reduce the
number of technologies that can be considered.
On the other hand, follow-up appraisal seems
essential for many devices and procedures if there
is to be a reasonable prospect of rational usage
and appropriate diffusion after the initial
introductory phase. Follow-up assessments will
tend to be associated with technologies that have
high costs. are evolving rapidly, have significant
implications for training and infrastructure and
which are applicable to large numbers of people.
Some case studies
Figures 4 and 5 illustrate developments in the
assessment of two major health care technologies.
magnetic resonance imaging (MRI) and shock
wave lithotripsy. In each case the major
recommendations and other assessment action are
compared with policy activities and other relevant
events. The initial assessments were based on
syntheses of published data and expert opinion.
Later recommendations and information on MRI
have emerged through consideration of local
primary data.
Magnetic Resonance Imaging
MRI was assessed by the Panel as it was a
new, high cost, complex diagnostic technology
undergoing rapid change and with potential to be
additive to existing modalities. The NHTAP
recommended limited introduction of MRI linked
to an evaluation program, and no general
reimbursement until the costs and clinical
benefits of the technology were further defiied?
These proposals were accepted by the
Government but there was a delay in putting the
program in place because of budget pressures.
Pressure in turn came on the Government
through entrepreneurs seeking to develop MRI
facilities. This led to a five-centre evaluation
funded by Commonwealth and State governments
and with no reimbursement available for any
system outside the program. Data on cost. usage.
accuracy and management impact, with a
consensus statement, have been provided to
health authorities and professional bodies,
HAILEY et al.
forming a major input to policy on fees, numbers
of units and applications.'
There has been limited introduction of private
MRI facilities in Australia in the absence of
government support. Some government funding is
now beiig provided indirectly to units outside the
program (for examinations which are consistent
with the consensus statement).
The latest phase of assessment is a technology
update drawing on overseas data as well as the
Australian program, and having regard to the
significant developments in. the technology in
terms of lower cost scanners and potential for
wider application (for example, in cardiovascular
imaging).
In this case there has been a major impact of
assessment on government policy, leading to
controlled introduction of the technology in
Australia. Relatively well defined programs are
in place with acceptance by professional bodies.
While there have been technical changes to MRI
during the assessment, the data obtained have
provided important information on the initial use
of this technology in Australia. The influence of
assessment on future policy on MRI remains
unclear. There will be an expectation of widening
usage of this technology. Assessment data would
suggest that any such increased usage would be
most beneficial in the context of tertiary referral
with emphasis on neuro-surgical conditions.
Extracorporeal Shock Wave Lithotripsy
Extracorporeal Shock Wave Lithotripsy
(ESWL) was considered by the Panel as it was a
new, high capital cost therapeutic technology,
with the potential to replace open surgical
procedures. The initial NHTAP assessment
recommended limited introduction of renal
ESWL in Sydney and Melbourne at hospitals
with appropriate facilities.' Not more than three
machines were seen as necessary for Australia,
having regard to expected case load and machine
capacity. It was noted that future availability of
lower cost machines might provide realistic
opportunities for smaller centres to acquire the
technology. Monitoring of usage and
developments in this technology and access for
all suitable patients were also recommended.
Funding was provided for two public sector
machines and government policies have
effectively restricted additional units to a single
private facility. The immediate impact of the
assessment o n usage of this technology in
Australia was significant and there was a direct
influence on policy.
The assessment work has provided direct input
into FederaUState discussions on future services
23 1 COMMUNlTY HEALTH STUDIES
linked to availability of second generation
lithotripters. However, available usage data from
the Australian units have been limited (no funding
was provided for collection) and access to the
units for patients from outside the major States
has not been ideal.
Further assessments have been undertaken - a
technology update and a separate assessment on
Gallstone ESWL which recommended against
routine use and supported one unit on a trial
basis.' This recommendation has also been
accepted and a clinical trial is now in progress in
Melbourne. An NHTAP study dealing with renal
stone therapy is in progress, and will cover both
ESWL and the alternative approaches of
percutaneous nephrolithotomy and laser
ureteroscopy.
Features common to both the MRI and ESWL
assessment activities include effective central
influence on support for the technologies through
government funding (conditional reimbursement
and provision of capital grants), support and
liaison from the bodies concerned and successful
establishment of some facilities independently of
government initiatives. In both cases, there is a
clear need to continue the collection and analysis
of data as the technologies evolve and mature, and
to appropriately adjust policy at both the
government and provider levels.
cTsC&g
In a report on CT scanning published in 1988,
the NHTAP examined trends in the use of this
established technology, and endeavoured to assess
its effect on Australian health care.'
The Panel found that there had been very
rapid growth in the use of CT in Australia. It had
replaced a number of less effective or more
invasive procedures, but the increase in the
number of CT examinations after 1980/81 far
exceeded any decline in the number of alternative
diagnostic procedures. While the efficacy of C T
in appropriate applications had been established,
it was not certain that in routine health care it
was always used in the most cost-effective
manner.
Features of this assessment were the difficulty
in obtaining data on Australian placement and
usage of this expensive technology, and the
limited evaluation data from other countries over
the previous decade. The NHTAP assessment
provided for the first time, a detailed overview of
the use of CT in Australia, but also drew
attention to limited information on some aspects.
The NHTAP made two major recommenda-
tions. The first was that a study should be
undertaken to obtain quantitative data on the
HAILEY et al.
contribution of CT to patient care. As yet, funds
have not been made available to permit such a
study. The second recommendation was that the
relevant professional bodies consider the
development of guidelines for medical
practitioners on the appropriate use of CT. This
recommendation has been taken up by the
NHMRC Health Care Committee, which has
established a working party, which includes
representatives of professional bodies, to prepare
guidelines on the use of CT.
Indlrect indicators of Impact
It is also of interest to consider the indirect
indicators of the impact of NHTAP assessments,
which provide some measure of the extent to
which the work of the Panel has been perceived as
usell.
In a review of the NHTAP it was notable that
the submissions received from government,
industry and professional bodies were
overwhelmingly supportive of the concept of the
Panel and of the range of assessment work
undertaken.lO The Panel has also continued to be
seen as a useful source of advice by governments
and others. An indication is that funding agencies
have continued to request assessments of
particular technologies. Nine of 18 assessments
published since the start of 1988 or currently in
progress have been due to references from health
authorities and insurance bodies. The NHTAP
reports are increasingly being used by universities
and other tertiary education establishments in
course work to provide students with a better
understanding of medical technologies. There is
also increasing usage of the NHTAP publications
by hospital administrations.
The growth in numbers of requests for reports
over the last four years (Table 3) has been
notable. The level of discussion and interest
generated by each of the assessments points to an
aspect of health technology assessment which is
possibly very significant but difficult to quantify.
In the longer term, assessments seem to be
helping to gain acceptance for the concept and
purpose of evaluation from a much wider
audience and encouraging critical consideration of
health technologies. We have the impression that
professional bodies in Australia are more
accepting of the value of assessment than was the
case when NHTAP commenced operation in 1982.
Limitations of the assessment process
There are limitations on the work that can be
undertaken and the impact achieved using the
assessment committee approach adopted with
NHTAP. While there are strengths in the
232 COMMUNlTY HEALTH STllDIES
 TABLE 3
Number ofrcque~tsfor NHTAP reports by Imtitutlw, 1966-1989
1% 1987 1988 1989

Hospitals 7 15 59 68
Companies 5 14 52 40
Medical Practices 3 5 39 29
state Government 5 4 m 31
Individuals 1 1 19 8
Professional Organisations 1 0 18 25
Overseas 0 2 14 20
Universities and Research Institutes 1 3 16 25
Commonwealth Departments 5 4 8 12
Health Insurance Groups 0 2 6 3
Total number of requests 28 50 251 261
~~ ~

(Number of reports requested 141 204 950 1104)

Note: These requests were in addition to reports forwarded to the standard NHTAP distribution
list (about 200 for each publication).

consultation and consensus strategy, obvious
limitations are the restricted amount of time
possible for committee meetings and the relatively
small number of technologies that can be
considered in MYdetail. The policy of seiving to
achieve balanced views and being open to follow-
up assessments, taking account of new data and
valid criticism, is resource-intensive.
To some extent, these limitations could be
eased by provision of additional resources for the
Panel and its secretariat. Gross has suggested a
different avenue of approach for the Panel with
NHTAP acting more as a peak council receiving
reports and recommendations from other groups
including a heavy emphasis on representation
from professional bodies." While there might well
be merit in the use of alternative approaches, for
example consensus development conferences on
appropriate topics, such a suggestion runs into the
difficulty of achieving worthwhile results in a
suitably short period of time, with the level of
resourcing likely to become available. This
proposal might also face problems if applied in
any widespread sense, because of the logistical
difficulty of co-ordinating the operation and
output of a large number of assessment activities
undertaken by other groups. While the consensus
and consultation process is a useful discipline, its
worth will be minimised if it takes too long and
the opportunity to realistically inform policy
decisions is missed.
Some limitations on the impact of health
technology assessment arise from characteristics
in the health care system. Health care is frquendy
a conservative area and practices may be very
slow to change. Decisions on reimbursement, once
taken, are difficult to reverse.
Expanded activities in the areas of health
economics, work force analysis and social
considerations have been suggested for the
NHTAP.'O Without additional resources there are
limited prospects for expanding in these
directions to meet the challenge of integrating
these further perspectives into timely and
appropriate assessments of a realistic range of
technologies.
Conclusions
From the NHTAP experience, it has become
apparent that the Panel's assessments and their
recommendations CM have an important influence
on policy and the subsequent use of a technology
but that much depends on the presence of stable
and receptive policy areas. any significant
changes to the technology and inputs from wider
government. industry and professional-body
interests. It also seems true that the level of the
impact is not necassarily related to the depth of
analysis - timing is crucial.
The impact of health technology assessment
will often depend on support and funding for
significant follow-up evaluation to the original

HAILEY et al. 233 COMMUNITY HEALTH STUDIES

assessment so that data can be generated and
considered as experience is gained with the
technology. Impact on immediate government
funding and licensing decisions may be relatively
quick. The impact of recommendations which
affect professional bodies and concern matters
such as preparation of guidelines may take longer
to become apparent. Impact on the behaviour of
individual medical practitioners will be difficult
to assess, and is a topic which deserves further
attention.
References
1. Office of Technology Assessment.
Assesskg the e f f i q a d 4 e t y of medical
technologies. Washington M3: Congress of
the US, 1978.
2. Office of Technology Assessment. The
implications of cost-effectiveness analysis
of medical technology. Washington DC:
Congress of the US. 1980.
3. National Health Technology Advisory Panel.
Dry chemistry pathology trial. Part 4 -
overview. Canberra: Australian Institute of
Health, February 1989.
4. Rowe M. Angioplasty and other
percutaneous interventional techniques in
the treatment of ischaemic heart disease.
Canberra: Australian Institute of Health,
Septcmk 1989.
5. National Health Technology Advisory Panel.
Nuclear magnetic resonance imaging,
Canberra: Department of Health, June 1983.
HAILEY et al.
Health technology assessment in Australia, and
the work of the NHTAP, is evidently perceived to
be useful by a range of government, professional
and industrial interests, has directly affected
policy on a number of technologies, particularly
in the short term, and has the potential to
influence attitudes to health care technologies in
the longer term. Unequivocal measures of impact
of assessment are lilrely to remain difficult to
achieve given the complexity of health care
systems.
6. Hailey DM. Assessment of MRI in Australia
- a learning experience. H o s p W i c a 1988;
6(7):2941.
7. National Health Tcchnology Advisory Panel.
Shock Wave Lithotripsy. Canberra:
Department of Health, July 1985.
8. National Health Technology Advisory Panel.
Gallstone Lithotripsy. Canberra: Australian
Institute of Health, December 1988.
9. National Health Technology Advisory Panel.
CT scanning in Australia. Canberra:
Australian Institute of Health, June 1988.
10. Report of the committee to review the role
and function of the National Health
Technology Advisory Panel. Canberra:
Department of Community Services and
Health, February 1989.
11. Gross PF. Technology assessment in health
care in Australia. Int J Techno1 Assessment
in Health Care 1989; 5137-183.
234 COMMUNLT HEALTH STUDIES