Panel Testing Condensed Module

PANEL TESTING (Condensed Version)

Purpose: To give participants an insight to the panel testing concept and

implementation principles

Learning At the end of this module, participants will be able to:

objectives:  Define panel testing
 Describe the scheme for a panel testing round
 Explain the principles of the results scoring system
 Analyze panel testing results and provide laboratory
technicians with feedback on their panel testing
performance
 Use the panel testing recording and reporting forms

Content Outline:  What is panel testing?

 What is panel testing used for?
 Panel composition
 Organization of a panel testing round
o Frequency of testing
o Cooperation with intermediate level laboratories
o Instructions for participating laboratories
o Conducting panel testing during a supervisory visit
o Packaging and transportation
 Analysis of results and scoring system
o Classification of errors
o Scoring systems
o Post-validation
 Interpretation and feedback

Handout and
None
exercises:
Appendices: 1. Sample recording and reporting forms:
o PT2. Logbook/Aggregate Results of Test Slide Sets
o PT3. Panel Testing Individual Results and Feedback Form
o PT4. Panel Testing Report of Multiple Laboratories for
District Supervisor & NTP
2. An example of an accompanying letter for sending out panel
testing sets to peripheral laboratories.

1
 Panel Testing Condensed Module

WHAT IS PANEL TESTING?

Panel testing (PT) is one of the options for EQA, it is a system of sending stained and/or
unstained smears from the NRL to peripheral and intermediate laboratories to evaluate
the proficiency of laboratory technicians in performing AFB smear microscopy. This
method tests individual performance of a laboratory worker, not the laboratory overall.

WHAT IS PANEL TESTING USED FOR?

Panel testing is appropriate for countries where little information on laboratory

proficiency is available, and there is a need for rapid assessment in order to prioritize
training and supervisory activities. It is useful to quickly detect problems associated with
very poor performance. Panel testing is also a useful tool to evaluate proficiency of
laboratory technicians prior to and following training: pre-testing helps individualize
specific training needs, while post-test PT results confirm the competency of staff after
training. For laboratories that rarely see positives, a panel test can provide a “refresher”
on the appearance of a positive smear.

Panel testing is useful to:

- supplement a rechecking program;
- provide some data on peripheral laboratory capabilities prior to implementing a
rechecking program;
- monitor performance of individual laboratory technicians in the absence of
adequate funds to implement a rechecking program.

Panel testing is considered a minimal requirement or standard for EQA in settings with
limited resources. Panel testing is considered to be less effective than blinded rechecking
because it does not monitor routine performance. However, in some areas with low
incidence of tuberculosis or very few microscopy problems, a labor-intensive rechecking
program may not be justified to detect only a few errors. Routine panel testing may be
more cost effective in these settings.

PANEL COMPOSITION

Panels can be composed of slides with known results originating from routine work, but
this leads to problems of consistency and it is more difficult to compare performance
between laboratories. Therefore, it is recommended that panel slides are specially
manufactured for this purpose to minimize variation and provide a more uniform test for
all participating labs.

Manufacturing of panel slides as well as validation of panel batches prior to sending

slides out to peripheral laboratories is the responsibility of the NRL. The composition of
the panel sets is also determined by the NRL. Usually a panel set consists of ten slides
which may be stained and/or unstained. Ten slides per panel represents about half the
maximum slides that a technician can examine per working day without loosing quality.
Panel sets can be of different degrees of difficulty, which is mainly determined by the
number of scanty and low-positive smears included into a panel.

2
 Panel Testing Condensed Module

ORGANIZATION OF A PANEL TESTING ROUND

Frequency of testing
Panel testing may be conducted occasionally as a special test or survey or periodically at
least one to two times a year, especially if there is no rechecking system in use. Although
panel testing may be performed in addition to rechecking, its usefulness will be more
limited. However, a panel test can be of great value when investigating the causes of poor
performance suggested by rechecking.

Cooperation with intermediate level laboratories

Organization of the rounds will usually be the responsibility of the NRL, from
preparation of slides and panels to analysis of results and feedback. It is often helpful if
panel testing rounds are organized in close cooperation with intermediate laboratories; the
NRL sends the required number of panel sets to intermediate laboratories and the
intermediate laboratories distribute panels to the peripheral centers, then collect test
results and send them back to the NRL. In this case the intermediate level supervisors
should have an updated list of supervised peripheral laboratories and laboratory
technicians to be included into the panel testing program. This will help estimate the
quantity of panel sets needed and will also help construct a schedule (timeframe) for
panel testing rounds. When intermediate laboratories are engaged to assist with panel
testing, a standard operating procedure (SOP) for the intermediate level supervisor should
be developed to clarify his/her responsibilities and actions within the framework of the
panel testing activities.

Instructions for participating laboratories

Panel slides should be sent out together with a form for standardized reporting (Panel
Testing Individual Results and Feedback Form). The form and/or accompanying letter
should describe the purpose of the panel test and provide basic instructions, i.e. the need
for staining if relevant, and the time allowed for reporting. It is equally important to
explain that, in the absence of further investigations, no incentives, punitive actions or
refresher training, will result from participating in the exercise, in order to obtain more
objective results. The form and/or accompanying letter should also stress that each
individual technician must examine smears and report results separately; the results
should not be shared or compared between technicians; and group work to review the
slides is not allowed. An example of an accompanying letter is provided as an Appendix
to this Module.

Conducting panel testing during a supervisory visit

Although panels can be sent out using any means available, performing panel testing as
part of a supervisory visit may improve the objectivity of the event. This will guarantee
that the panel set will be processed by the laboratory technician for whom the set is
intended and within the allowed time. Moreover, evaluation and corrective action can be
done right away, making this option even more efficient. While in theory panel testing as
part of the supervisory visit appears to be a good option, in routine conditions it may be
impractical. If the test is to be conducted as part of the on-site supervision, the

3
 Panel Testing Condensed Module

supervisor should plan the visit to the peripheral laboratories in a way that allows
sufficient time to conduct routine on-site activities and allocate required time for
laboratory technicians to read panels (time allowed/limited for a laboratory technician to
review panel and report results is a maximum of 2 hours for a stained slide set and 3
hours for an unstained slide set). The supervisor should take into consideration the
number of microscopes available in the laboratory and the number of laboratory
technicians that will participate in panel testing, and ensure that panel testing does not
disrupt routine examination of patient specimens. The peripheral laboratory staff should
be notified in advance to allow them to plan for the extra workload. Since administration
of panel testing during supervisory visits can be impractical in routine conditions, it may
be best reserved for use as part of a special survey.

Packaging and transportation

Options for delivering panel test slides include mail/post, supervisory visits, and courier
systems. Each program will need to determine the best mechanism for delivering slides
based on the services and resources available as well as any relevant regulations in force.
Regardless of the means used to transport panels, appropriate care should be taken to
prevent breakage. Strong plastic slide holders, in which the slides fit tightly (i.e. small
ones designed to hold just 10 slides), or another safe way of packing, will be
indispensable. Each program should also set an appropriate timeline for completing the
test based on the conditions in the country. Reasonable turn around time is expected to be
between one week and one month, depending on the delivery system, staffing and
workload.

ANALYSIS OF RESULTS AND SCORING SYSTEM

Classification of errors
Errors in reading smears and reporting results will be classified as major or minor: false
positive/negative errors related to 1+, 2+ or 3+ results are considered to be major, while
quantification errors (at least a two-grade difference) and errors in the scanty group are
considered to be minor. Summary information on errors classification is presented in the
table below.

Result of Result of Controller

Technician Negative 1-9 AFB/100 f 1+ 2+ 3+
Negative Correct LFN HFN HFN HFN
1-9 AFB/100 f LFP Correct Correct QE QE
1+ HFP Correct Correct Correct QE
2+ HFP QE Correct Correct Correct
3+ HFP QE QE Correct Correct

Correct: No errors
QE Quantification error Minor error
LFN Low False Negative Minor error
LFP Low False Positive Minor error
HFN High False Negative Major error

4
 Panel Testing Condensed Module

HFP High False Positive Major error

Scoring systems
Prior to a panel testing round, the NTP/NRL will develop a scoring system to distinguish
errors and detect problems with technician’s performance. Different scoring systems can
be used depending on the type of errors that are the focus for detection and corrective
action. An example of a scoring system that has different scores for major and minor
errors is presented below:

Set of 10 slides, each slide is worth 10 points, total possible score = 100.
a. HFP and HFN scores 0
b. LFP, LFN and QE scores 5
c. Passing score = 80 – 90

Post-validation
Unfortunately, no system for developing test panels and distributing them to peripheral
sites is entirely without problems. Post-validation is the process of validating panel
slide/batches after receiving aggregate results from all laboratories and before
determining a final score. Post-validation is needed to confirm that discordant results are
due to a laboratory technician’s error and not a problem with a panel slide. If a majority
of technicians fail to report correct results for the same slide/batch, this may represent a
problem with the slide preparation at the NRL. When numerous discrepant results suggest
that problems occurred in the process of preparing the panels at the NRL, the slides
should be returned to the NRL for rereading. This will help the NRL detect the source of
the problem(s), including:
- technical difficulties in preparing individual slides;
- an error in the initial reading of a smear at the NRL;
- incorrect recording of expected results;
- fading of stained smears during transportation to peripheral sites.
The Logbook/Aggregate Results of Test Slide Sets is used for post-validation purposes.

INTERPRETATION AND FEEDBACK

If post-validation confirms that the consistency and quality of the panel set was
acceptable then unacceptable performance in panel testing should result in investigation
to identify the reason, including all sources of technical error. False-positive and false-
negative errors should be considered separately. False-positives may indicate lack of
proficiency or a faulty microscope. If the slides have been returned, it is possible to check
for artifacts that may have caused confusion, but it often will not be possible to determine
the exact cause of the problem without further investigation on-site. False-negatives may
also indicate a lack of proficiency, but may be due to poor stains or staining procedure, a
bad microscope or superficial work in examining the slide. If positives in stained slides
were reported correctly and the errors occur only in smears sent out unstained, the
laboratory stains or staining process may be at fault. This can be verified by checking
and re-staining smears that were returned. If errors are found in both stained and
unstained smears, the reason may be the microscope or lack of technician proficiency.

5
 Panel Testing Condensed Module

Feedback is always required and should be timely and confidential. At a minimum, the
participants should be informed of their own results (Panel Testing Individual Results and
Feedback Form) and how it compares to the aggregate performance of other laboratories.
To gain the greatest benefit and impact from the exercise, a visit to the peripheral centers
with the poorest scores is required for on-site identification of the problems and remedial
action. Thus, technical supervisory visits offer the best opportunity to review results of
panel testing with the technicians in the peripheral laboratories. For this reason, on-site
supervisory visits by experienced staff from the intermediate laboratory or NRL are
recommended once a year and more frequently if significant problems are identified.

Reporting to the NTP coordinators and local health authorities may also be required. As
non-laboratory persons, they may have difficulty in understanding and interpreting the
results, and suggest punitive or disciplinary action. To avoid unintended interpretation of
the results, the report should provide sufficient background information, as well as
carefully considered conclusions and recommendations, not simply a list of scores.
Reports should include both individual results, as well as aggregate performance for all
laboratories tested (Panel Testing Report of Multiple Laboratories for District Supervisor
& NTP). EQA feedback is always confidential: laboratories/laboratory technicians should
be given individual code numbers when presenting aggregate results.

If a majority of laboratories submit unacceptable panel testing results, but the consistency
and quality of panel testing slides is found to be acceptable, this likely represents serious
problems with AFB microscopy. Additional resources should be obtained for supervisory
visits, correction of problems identified in individual laboratories, including replacement
of microscopes (and/or microscope objectives), retraining if needed, and follow-up panel
testing.

Key messages:
 Panel testing is an effective method to quickly obtain information about the
capabilities of individual laboratory technicians to read smears and report results
according to standards approved by NTP.

 An effective system should be established to distribute panels, collect and analyze

data; provide timely feedback to peripheral laboratories.