Defining Accuracy and Precision

Liquid delivery is a common laboratory process and this critical function is often overlooked. As a result, routine research and test results can be in error based on a simple misunderstanding or misapplication of liquid delivery instruments. This article focuses on defining and exploring accuracy and precision, which are fundamental elements of regulatory compliance.

As an example of the importance of accuracy and precision, in January 2007, the Food and Drug Administration [1] (FDA) sent out more than 1,000 letters to drug product owners regarding a “lack of assay reproducibility between original and repeat results,” and in a related warning letter stated that because of lack of reproducibility “the reported concentration results cannot be considered accurate.” The laboratories involved have spent many thousands of hours and tens of millions of dollars responding to this situation. These sorts of events grab headlines and create excitement, but most of us prefer a less thrilling work experience.

What are accuracy and precision?
No measurement is perfect and all measurements have some error associated with them. In the laboratory, two terms often used somewhat interchangeably to describe measurement errors are accuracy and precision. While general usage may be lax at times, in statistical calculations and in regulations, these two terms have different and very distinct meanings. Knowing the precise definitions and importance of each are critical to the development of a sound laboratory quality control program.

Accuracy refers to the deviation of a measurement from a standard or true value of the quantity being measured. We can talk about the accuracy of a single measurement. For example, if a pipette is set to dispense 100 microliters but actually delivers 99 microliters, the accuracy of that particular dispense is off (or the pipette is inaccurate) by -1 microliter. Notice that in this case we know what happened during that last dispense (it was 1 microliter too low), but we don’t have much knowledge about what is likely to happen the next time this pipette is used.

We can also talk about the accuracy of a group of repeated (replicate) measurements. In these cases, we must first determine the mean of the group, and then compare that average value with the standard or true value. Accuracy for a group of measurements refers to the deviation of the group’s mean value from the standard or true value. However, knowing accuracy alone is of limited use. For example, if we know that three replicate measurements averaged exactly 100 microliters, we still can’t predict how likely it is that the next dispense will be within some limits. One pipette might deliver 99, 100 and 101 microliters (pretty good), while a second delivers 80, 100 and 120 (pretty bad). The averages of both sets of data are exactly 100 and both are perfectly accurate, but which one would you use if your life depended on the next volume measurement being very close to 100?

Precision tells us how close a group of measurements are to one another. The closer the data replicates, the more likely the results will be similar in the future. For this reason, good precision has predictive value; it gives us confidence in future results. Precision is usually calculated and discussed in terms of standard deviations and coefficient of variation (CV). A precise or closely-clustered data set has a smaller CV and is generally more reliable than one that is widely scattered.

Because precision is concerned with the closeness of two or more measurements to each other rather than to a standard value, it’s possible for a group of values to be precise without being accurate, or to be accurate without being precise (see Figure 1).

When should accuracy and precision be investigated?
A laboratory should investigate the accuracy and precision of a method when the method is new, the method is questioned because of external quality control data, or the validity of the results is questionable. It’s also a common practice to check the accuracy and precision of laboratory equipment whenever a new instrument is brought into the lab, when equipment is suspected of being damaged, and on a periodic schedule thereafter. In addition, accuracy and precision checks are frequently used as part of the qualification process of a new laboratory technician. The following table shows some examples of errors that influence accuracy and precision.

How should accuracy and precision requirements be determined?
To determine the importance of accuracy and precision of liquid handling and other equipment in a method, you will have to take a close look at the intended use of results, sources of error, and how those errors will affect results. A three-step analysis is involved:

1. Establish the limits of acceptable error in final results
2. Determine each predominant source of error in the method
3. Do a statistical analysis of the effect of these errors on your results.

This analysis will help you to focus efforts on controlling and reducing the largest sources of error in your results.

Why is it necessary to test for accuracy and precision in liquid delivery?
Accuracy and precision are necessary to ensure that results are valid. In general, failure to achieve either accuracy requirement or precision requirements is sufficient to constitute a failed test or calibration.

What is required when equipment accuracy or precision requirements aren’t met?
The FDA sets high standards for ensuring that calibrated equipment meets pre-established accuracy and precision limits:

Subclause b of 21 CFR 820.72 states,“When accuracy and precision limits are not met, there shall be provisions for remedial action to evaluate whether there was any adverse effect on product quality.”

Remedial action requirements include repair and/or recalibration of the equipment and consideration of potential quality effects in three particular areas. According to FDA Quality System Requirements Manual Part 7--"Equipment and Calibration," this includes:

• On the product design or process validation parameters or data
• On the quality of existing components, in-process, or finished products
• Appropriate corrective action.

This regulation means that something as simple as a failed pipette could call into question and place at risk any work done with that device since the last prior calibration, and both validation data and products (in-process or finished) can be held suspect. What constitutes appropriate corrective action can vary depending on the situation. An isolated mechanical failure might be corrected with a simple repair, while a pattern of recurring failures in a particular laboratory may require process reengineering to prevent recurrences or to mitigate the associated risks. Either way, it’s desirable to avoid remedial action. Therefore, regular verification of accuracy and precision are essential.

Lack of accuracy or precision is like a roller coaster ride. Results can be up one minute and down the next. Attention to the accuracy and precision of pipettes and other liquid handling devices improves confidence and quality in laboratory analytical testing. When tests are correct and reliable, production is smooth and laboratories are less likely to be troubled by regulatory compliance concerns. A precise and accurate quality control laboratory is a pleasant place to work—and a smooth-running operation leaves us more time for thrill-seeking outside of work.