With so much focus these days
on collecting and analyzing data, and implementing Six Sigma
and other initiatives to ensure that design goals are met,
it’s possible that critical details such as the data’s
level of accuracy have gone unnoticed. We might not realize
how this inaccuracy translates into false acceptance and
rejection risks associated with measurement-based decisions.
These risks define the probabilities associated with defective
products being accepted while good products are scrapped.
Because measurements in research and development and manufacturing
acquire traceability from calibration and, in turn, often
determine product quality or a product’s target value
through testing, these measurements’ own quality is
extremely important. This is where ISO/IEC 17025 accreditation
comes in. Fundamentally, the standard is about measurement
You might point out that your existing calibration and
testing programs require your providers to produce measurements
“traceable” to some national or international
reference, like those of the National Institute of Standards
and Technology. But what exactly does that guarantee? If,
for example, I calibrate a gage block with a NIST-traceable
ruler, doesn’t that make the gage block “traceable”
to NIST? Although this example exaggerates what happens
in the real world, I’ve actually seen a laboratory
calibrate gage blocks with a micrometer, rather than the
other way around.
Are you beginning to worry about your measurement data
Measurement data should concern us because no matter what
we do, some quantity of unknown error in measured values
will exist because measurement provides only an estimation
of the measurand (i.e., the particular quantity subject
to measurement). This unknown error creates doubt in a measurement’s
result. The probable quantity of unknown error can be defined
only through a further estimate known as measurement uncertainty.
This is calculated in accordance with the ISO Guide to the
Expression of Uncertainty in Measurement (GUM).
GUM defines measurement uncertainty as “parameter,
associated with the result of a measurement, that characterizes
the dispersion of the values that could reasonably be attributed
to the measurand.”
GUM further states, “The word ‘uncertainty’
means doubt, and thus, in the broadest sense, ‘uncertainty
of measurement’ means doubt about the validity of
Measured value = 1.0000022 in.
Expanded uncertainty (U95) = 3.6 µin. (0.0000036
The statement to the left tells us that the laboratory
measured the 1 in. gage block and found it to be 2.2 µin.
over nominal size, with an uncertainty, expressed at the
95 percent level of confidence, of 3.6 µin. In other
words, the lab determined that a 95 percent probability
exists that the measured value is in error by not more than
Recipients of measurement values and compliance statements
based on measurement values are entitled to an estimate
of the amount of uncertainty associated with the measured
values. Without a realistic estimate of measurement uncertainty,
the client can’t judge the measurement result’s
quality. An unknown uncertainty means the measured value
must be considered unreliable and, hence, useless. As NIST’s
Ted Doiron aptly observes about this dilemma, “A calibration
without a valid uncertainty estimate is simply exercise,
the result of which is properly expressed in calories.”
If this information is new to you, it shouldn’t
be. The requirement for an estimate of measurement uncertainty
isn’t new. In fact, ISO 9001:1994 requires it in Section
4.11: “Inspection, measuring and test equipment shall
be used in a manner which ensures that the measurement uncertainty
is known and is consistent with the required measurement
Although ISO/IEC 17025 doesn’t--and indeed can’t--mandate
perfect measurements, it does provide some assurance that
laboratory measurement quality is evaluated, defined and
controlled. It’s then our responsibility to use this
assurance to determine if labs meet our needs.
Upon successful laboratory accreditation, the accreditation
body issues a scope of accreditation that usually defines
either the tests a lab has been accredited to perform or
the best-measurement capability a calibration lab has demonstrated
in accredited measurement disciplines.
The scope of accreditation might not include the measurement
uncertainty that exists for measurements performed on your
devices and/or objects. However, to be compliant with ISO/IEC
17025, the lab must calculate and maintain measurement uncertainty
estimates for all data provided to clients or used in determining
“in tolerance” or “out of tolerance”
status. The standard’s Section 220.127.116.11 states, “A
calibration laboratory, or a testing laboratory performing
its own calibrations, shall have and shall apply a procedure
to estimate the uncertainty of measurement for all calibrations
and types of calibrations.” Section 18.104.22.168 adds,
“Testing laboratories shall have and shall apply procedures
for estimating uncertainty of measurement.” (Certain
exceptions are given for some testing activities.)
The process of uncertainty estimation is somewhat subjective
and requires adequate, unbiased technical review to ensure
credibility. This review is achieved through expert third-party
assessment from accreditation bodies. Because all aspects
of the measurement process feed into measurement uncertainty,
adequately assessing the measurement process is also required.
Combine this need with that of consistency as well as continuous
improvement as realized through compliance to ISO 9001,
and you arrive at ISO/IEC 17025 accreditation.
How many organizations do you know that claim their measurements
are “traceable?” What exactly does that mean?
ISO’s Vocabulary of International Metrology defines
traceability as “property of the result of a measurement
or the value of a standard whereby it can be related to
stated references, usually national or international standards,
through an unbroken chain of comparisons all having stated
This means that for a measurement to be traceable, uncertainty
must be stated. How many certificates have you seen that
claim traceability yet provide no stated uncertainty? ISO/IEC
17025 accreditation resolves this problem by requiring uncertainties
to be stated.
Ideally, all ISO/IEC 17025-accredited laboratories should
be in full compliance to the standard. Unfortunately, my
experience has shown that some aren’t, but considering
the magnitude of the task, labs and accreditation bodies
are doing fairly well and continuously improving. The requirements
are involved, and practical solutions for compliance are
in constant development.
During the interim, you have a better chance of receiving
quality measurements from an accredited laboratory than
from one that doesn’t meet ISO/IEC 17025 requirements.
Moreover, with a bit of knowledge, you can then compare
laboratory capabilities based on their measurement uncertainty.
ISO/IEC 17025 accreditation’s fundamental purpose
is to ensure competency in testing and calibration measurement
processes. However, client involvement is critical to ensure
that adequate services are received.
ISO/IEC 17025 can empower a lab’s client. With this
power comes responsibility, and without adequate knowledge,
clients might still not get the service they need--although
under those circumstances, they’d have no one to blame
but themselves. Thus, it’s important to become familiar
with the requirements in order to use ISO/IEC 17025 to your
advantage--for example, knowing which questions to ask a
The following concepts address several issues that you
should consider when evaluating a testing or calibration
supplier, given the current state of accreditation:
Estimated measurement uncertainty. Inquire as to whether
the lab provides measurement uncertainty with the data.
These values are needed when determining calibration or
test adequacy and subsequent conclusions.
Ask whether the provided “uncertainty” is
really the measurement uncertainty as required by ISO/IEC
17025, or if it’s the reference equipment’s
uncertainty or some other value used to fill in the blank
because the lab hasn’t gotten around to fully meeting
For organizations that have done their homework and understand
how to analyze measurement uncertainty, ask for a copy of
the analysis. If the lab claims it’s proprietary information,
ask if you can view the analysis at its location.
Calibration and/or test data. Prior to submitting a device
or object for testing or calibration, ask for a list of
the measurements that will be taken and a sample data report--especially
if you’re requesting in-tolerance or out-of-tolerance
reporting. For calibration work, you will also want to ensure
the measurements taken during the performance test adequately
cover the functions and ranges you use.
Ensuring quality of calibration and testing results. ISO/IEC
17025 lists many methods that labs can use to ensure the
quality of their calibration and testing results. One of
these, proficiency testing, has since become a requirement
by many accreditation bodies. Others are recommended for
use by labs to add reliability and credibility to measurement
Proficiency testing is a professional service that puts
a lab’s measurement uncertainty claims to the test.
This is done by providing an artifact that has a known value
with an uncertainty smaller than the participating lab’s.
The lab reports its measurement and uncertainty to the provider.
The results are statistically analyzed and reported along
with a “satisfactory” or “unsatisfactory”
If you’re really concerned about the accuracy of
a lab’s estimated measurement uncertainty, ask whether
the given measurement process has been proficiency tested.
Note: To limit initial accreditation costs, many accreditation
bodies allow labs to spread their proficiency testing program
over a four-year period.
Witness some of your contracted tests and/or calibrations.
ISO/IEC 17025 provides lab clients with the right to witness
contracted tests and calibrations, assuming the lab isn’t
violating other clients’ confidentiality. Use this
to your benefit by taking the time to witness the measurements
for yourself. Ask the technician questions and become familiar
with your provider. Although you might not have the technical
expertise to assess the lab’s measurement quality,
you’ll undoubtedly develop an impression that will
either enhance your confidence or cause you some concern
regarding your provider’s capability.
Without ISO/IEC 17025 accreditation, “traceability
to a national standard” is only a phrase printed on
a report. With accreditation, clients can assume some validity
to the measurements provided by their calibration and testing
suppliers, defined via a valid estimate of the measurement
uncertainty. ISO/IEC 17025 accreditation offers a major
step in bringing a valid “traceability chain”
to general industry’s front door.
Because measurements are usually the fundamental basis
of most manufactured product-quality evaluations, the concepts
of measurement quality, traceability and uncertainty put
forth by ISO/IEC 17025 are useful wherever measurement-based
decisions are made. In the manufacturing arena, measurement
uncertainty, combined with process capability and other
quality assurance data, can facilitate measurement-based
decision risk analysis. Risk analysis defines the probabilities
associated with defective product being accepted and good
product being scrapped. These sciences can give quality
departments greater capability in producing proactive, rather
than reactionary, solutions.
Organizations selecting calibration and testing suppliers
must understand the concepts of measurement quality. Because
accreditation is based on periodic assessments rather than
continuous monitoring, clients of these services must take
active roles in monitoring their suppliers. Any apparent
noncompliance observed by recipients of accredited services
should be reported to the labs’ accreditation body.
Let’s not forget that quality is primarily customer-driven,
but customers must be clear about the nature of the quality
James D. Jenkins is president and founder of QUAMETEC,
an organization that assists companies pursuing ISO/IEC
17025 accreditation. Jenkins developed the Uncertainty Toolbox
for Microsoft Excel and is the author of Measurement Uncertainty
Analysis Fundamentals, available at measurementuncertainty.com.
He also chairs the Accreditation Resources Committee for
NCSLI. Letters to the editor regarding this article can
be e-mailed to email@example.com.