Clinical Chemistry Chapter 3

Quality Control and Statistics

Introduction some thoughts

When evaluating laboratory results, how do we determine that is normal or acceptable? That is: What is normal or OK? When does a laboratory test result become weird or abnormal ? When do be become uncomfortable with a result? At some point we have to draw a line in the sand on this side of the line youre normal on the other side of the line youre abnormal.
2

Introduction some thoughts

Where and how do we draw the line ?

Introduction some thoughts

Where and how do we draw the line ?

Statistics are used to determine the lines of normal and acceptable.

Introduction some thoughts

Statistics is used to draw lines in the sand for patient specimens, control specimens and calibrators If the results are normal we re comfortable about them and dont worry

But if theyre abnormal, were uncomfortable and we fear that there is something wrong with the patient
or just as bad, something is wrong with the test procedure .

Review of Statistical concepts. Review statistical terms commonly used in chemistry quality control.
Most

definitions in back of book.

Key Terms among those presented by authors

Accuracy* Control specimen* Delta check* Dispersion Histogram Precision Quality Assurance Quality Control Random error Reference method Shift* Standard specimen Trend*

Standard Deviation (SD)* Coefficient of Variation (CV)* Mean* Average* Mode* Median* Gaussian Curve* Westgard Rules* Pre analytic (error) Analytical error Post analytic (error) Range* Levy Jennings Chart* Z - Score

Objectives

Define the Key Terms Discuss how statistics is used to determine normal values and establish QC ranges Given numerical observations, calculate the

Mean, Mode, Median, Range, SD and CV

Recognize random error, shifts and trends on Quality Control Charts

Discuss common Westgard Control Rules

Statistical Concepts (all on lecture guide)

There are lies, dammed lies and statistics. ( Quotation of 19th century British Prime Minister Benjamin Disreili ) Statistics is a (science of )branch of mathematics that collects, analyzes, summarizes and presents information about observations. In the clinical lab, these observations are usually numerical test results A statistical analysis of lab test data can help us to define normal ranges for patients ( normal and abnormal ) and acceptable ranges for control specimens ( in and out of control )

Descriptive statistics information about one group of observation Inferential statistics ways to compare different groups of observations

Review of Statistical concepts. Measures of Central tendency ( how numerical values can be expressed as a central value )
Dispersal

about the central value ( how spread out are the numbers ? ) these two main ideas we can begin to understand how basic statistics are used in clinical chemistry to define normal values and when our instruments are ( or are not ) generating expected numerical results
10

Using

Common Descriptive techniques; expressions of central tendency

Mean

- Average value Median - Middle observation Mode - Most frequent observation

11

Common Descriptive techniques; expressions of central tendency

Mean

- Average value

The sum of all the observations the number of observations

12

Common Descriptive techniques; expressions of central tendency

Median

- Middle observation

List all the observations in order of magnitude and pick the observation thats in the middle In a odd # of observations = Middle observation In an even # of observations = Average of the 2 middle values

13

Common Descriptive techniques; expressions of central tendency Mode - Most frequent observation The observation that occurs most frequently
There

may be more than one mode, or none

at all!

14

Common Descriptive techniques; expressions of central tendency

All three (mean, median and mode) are expressions of a central observation, but they dont say anything about the observations as a whole such as Are they close together? Although we can look at all the individual observations, the mean, median and modes by themselves do not give us any indication about the dispersion of the observations.
15

Common Descriptive techniques; expressions of central tendency So we use additional methods to view data, such as graphical presentations & additional statistical / mathematical manipulation of the data.

16

Graphical presentation
histograms and bar-charts
90 80 70 60 50 40 30 20 10 0 1st 2nd 3rd 4th Qtr Qtr Qtr Qtr

East West North

17

Graphical presentation
charting techniques

Levey Jennings chart

18

Standard Deviation (SD) Is

a mathematical expression of the dispersion of a group of data.

SD

x x
n 1

19

Coefficient of Variation (CV)

a way of expressing standard deviation in terms of average value of the observations used in the calculation

CV% =

Sandard Deviation Mean

100

20

Quality Assurance & Quality Control

Quality Assurance (QA)

Includes

pre-analytic, analytic and post analytic factors (to be discussed) systematic actions necessary to provide adequate confidence that the laboratory services will satisfy given medical needs for patient care. Bishop pg. 69 other words: Quality assurance is an all inclusive / comprehensive system monitoring the accuracy of test results where all steps before, during and after the testing process are considered.
21

All

In

Quality Assurance & Quality Control

Quality Control (QC)

Crudely, it is the system we use in the clinical laboratory to recognize and minimize the analytic errors. QC system is to monitor the analytical process; detect errors during the analysis and prevent reporting of erroneous test results. It uses statistical analysis of test system data Requires following published rules (ie Westgard Rules)

22

Quick Review of Quality Control objectives

Record of precision Early warning of shifts and trends Permits a valid judgment on the accuracy of a measurement Facilitates comparison of test methods Monitor equipment performance Indicates the analytical abilities of a tech Accumulate a body of knowledge to satisfy outside accrediting agencies

23

Accuracy : A measure of how close the

observations are to the true or correct value
Right on target ! Close enough? Keep your day job

In the laboratory we need to report tests with accuracy and precision, but how accurate do we need to be? Its not possible to hit the bulls-eye every time. So how close is close enough?
24

Accuracy - another definition: The relationship between the expected / true result and the result actually achieved from the procedure / analysis.

25

Precision : Observations that are reproducible or repeatable. If you repeatedly test / assay a sample and get wide variation in the assayed results, you have low precision.

26

Precision
3 possible testing outcomes - Hitting the target
x x

x x

Lacks precision and accuracy

Has good precision but poor accuracy x xx x xx

xxx x

Good precision and good accuracy

27

Accuracy versus Precision

The

laboratory must produce results that are both accurate and reproducible.

28

Accuracy versus Precision

If

the results of an assay performed on a single (control) sample are both accurate and reproducible, we would expect to see plotted results:

29

Accuracy versus Precision

If

the procedure lack accuracy or precision, the results of the assay would result in a distorted graph.

30

Classification of Error ***The variables:

Pre-analytical

variables,

right specimen from right patient and in right condition

Analytical

variables

all parts of testing procedure performed properly, controls in range

Post

analytical variables

Correct report to correct person, interpreted correct

All

the phases of the testing process are subject to errors and must be closely monitored, to maintain Quality Assurance.
31

Bias

Bias the amount by which an analysis varies from the correct result.
Example,

If the Expected Value is 50 units, and the result of an analysis is 47, the bias is 3 units.

32

Classification of error
identify examples of:

Pre-analytical error Clerical Poor

collection

Analytical error
Random

or indeterminate Systematic or determinate

Post-analytical error
Clerical
Misinterpreted,

etc.
33

Controls in clinical chemistry

Use of assayed and un-assayed controls in clinical chemistry

Assayed

controls: Un-assayed controls:

Compare

cost vs convenience

34

Establishment of a QC system

Collecting data
Run

assay on control sample & manually enter control results on chart

One chart for each analyte and for each level of control

35

Establishment of a QC system

Collecting data
Many

modern chemistry analyzers have computer program that maintains the QC log.
Like this Dade Dimension

36

Collecting Data for QC

Charting techniques
Levey

Jennings chart is a graph that plots QC values in terms of how many standard deviations each value is from the mean

37

Collecting Data for QC

another LJ chart
Assayed ranges appear above each graph Group mean and SD appear on the left Shaded areas represent 1SD, 2SD and 3SD values

38

Collecting Data for QC

Youden Plot

Allows you to compare the relationship of each levels mean to the group performance Shaded areas represent 1SD, 2SD, 3SD values Group mean of Level I and Level II represented by intersection in center Comparison to group mean represented at intersection of dark lines

39

Collecting Data for QC

Minimum number of determinations

Statistically

should have at least 20 determinations to establish acceptable mean and practical standard deviation.

40

Terms

Standard Deviation (SD) - a measure of the scatter around the mean (barX ) in a Gaussian distribution (Bell curve, or normal frequency distribution)

SD

x x
n 1

41

Formulas for Statistical Terms

Standard Deviation :
SD

x x
n 1

= = =

the number of observations (how many numerical values ) the sum of in this case, the sum of all the the mean value

x
X

= the value of each individual observation

x
x
2

The Standard Deviation is an expression of dispersion the greater the SD, the more spread out the observations are

42

Terms

95% confidence limit 95% of all the results in a Gaussian distribution, also 2 SD

43

Terms
Coefficient of Variation (CV) % =

Sandard Deviation Mean

100

The CV allows us to compare different sets of observations relative to their means. Example:
Compare two different procedures for glucose They use different reagents, have different means, produce different SDs, etc. Since they are different and cant be compared directly, use CV formula.
44

Terms
Coefficient of Variation (CV) % =

Sandard Deviation Mean

100

Again: The CV allows us to compare different sets of observations relative to their means You cant use the SD to compare different groups of data because they are measuring different observations - you cant compare apples to oranges. The CV can turn all groups of observations into a percentage of their relative means - everything gets turned into oranges.

The smaller the CV, the more reproducible the results: more values are closer to the mean.

45

Data interpretation and evaluation

Must be able to calculate standard deviation, range (95% confidence limits), CV, etc. We will have practice exercise(s) in lab.

46

Use of Standard Deviation

Once you have determined the standard deviation, must use the information to evaluate current/ future analysis.
Most labs make use of 2 SD or 95% confidence limit. To put this into a workable form, you must establish the range of the 2 SDs
47

Use of Standard Deviation to obtain the range of acceptable results

mean of group of control values = 104 mg/dL Standard Deviation = 5 mg/dL Determine the Range of 2SD; (which will allow you to evaluate acceptability of performance of the control on subsequent days.) Is a control value of 100 mg/dL acceptable?

48

Using 95% confidence limits, how often will a control be out of range (statically)?

49

Using 95% confidence limits, how often will a control be out of range (statically)?

5% of the time,
That is 1 out of every 20 times!

50

But what if your control specimen is out of control?

Out of control means that there is too much dispersion in your result compared with the rest of the results its weird This suggests that something is wrong with the process that generated that observation Patient test results cannot be reported to physicians when there is something wrong with the testing process that is generating inaccurate reports

Remember No information is better than wrong information

51

But what if your control specimen is out of control?

Things that can go wrong and what to do: ie. Corrective methods
Instrumentation

malfunction ( fix the machine) Reagents deteriorated, contaminated, improperly prepared or simply used up (get new reagents) Tech error (identify error and repeat the test) Control specimen is deteriorated or improperly prepared (get new control)
52

Z Score

An evaluation of any one determination to see how far it is away from the mean.

53

 Example:
Average = 109 mg/dl SD = 20 mg/dl George = 105 mg/dl Z Score = Patient - Mean SD

What is the Z Score of Georges fasting plasma glucose?

105 mg / dl 109 mg / dl For George : 20 mg / dl

0.2 SD

Georges glucose value was 0.2 SD below the mean

54

Use of Standard Deviation to establish Reference Ranges

The same concept used to determine 95% confidence limits for controls is also used to determine reference / normal values for patients.
To establish NV for patients, should use a large sampling of clinical normal representatives.

55

Use of Standard Deviation to establish Reference Ranges

Reference Range the normal or expected value for patients.

56

Establishment of Reference Ranges

Reference ranges the normals

Normal

ranges ( reference ranges ) are defined as being within +- 2 Standard Deviations from the mean

Each lab must establish its own reference ranges based on local population

57

Establishment of Reference Ranges

Factors affecting reference ranges

Age Sex Diet Medications Physical activity Pregnancy Personal habits ( smoking, alcohol ) Geographic location ( altitude ) Body weight Laboratory instrumentation ( methodologies ) Laboratory reagents

58

An Example of the Standard Deviation to determine patient reference range.

A minimum of 20 observations should be sampled in order to obtain valid results ( but Ill use just 6 to save time ) Lets determine the normal range for fasting plasma glucose using 6 people: Johns glucose = Pauls glucose = Georges glucose = Ringos glucose = Micks glucose = Erics glucose = 98 100 105 150 102 101 mg/dl mg/dl mg/dl mg/dl mg/dl mg/dl Average = 109 mg/dl SD = 20.0 mg/dl 2 SD = 40.0 mg/dl

That means that the normal range for this group is from 109 40, or 69 - 149 which is 2.0 SD from the mean

Ringo is considered abnormal if we use this commonly accepted criteria to define normal and abnormal
By the way, the CV for this group of observations is about 18% - a fairly big dispersal about the mean
59

Extra slides

60

61

Statistics Top 10

Mean Median Mode SD CV Z-score Accuracy Precision 2.0 SD

Arithmetic average value measurement of central tendency Middle observation arrange observations first Most frequent observation ( if any ) .., maybe more than 1 Measure of dispersion how spread out observations are SD as a % of the mean compares different data groups Converts observation into its distance from the mean in SDs Closeness to the true or correct result Reproducibility Common acceptable dispersion range

Levy Jennings Chart : Graph of QC data in terms of SDs from the mean

62

63