Blaine Clapper  |  08/01/2000

How to Select Calibration Management Software

Evaluating your needs will help you select the perfect software package

The need for control over manufacturing processes has never been higher than in today's environment. As this need has increased, so too has the requirement for better management of the equipment used to measure and control manufacturing processes. Fundamental to managing this equipment properly is ensuring that it's correctly calibrated and maintained.

Unfortunately, while many managers are faced with managing a growing number of instruments and increased responsibility, their resources are being reduced. One popular method for minimizing the resources necessary is the implementation of commercial off-the-shelf calibration-management software (CMS).

Desirable CMS Features


Maintains and retrieves master equipment, calibration history and measurement data records


Automatically schedules future calibration due dates


Prints calibration forms


Records signatures and stores documents electronically


Compiles reports for calibration interval analysis


Provides security at multiple levels


Maintains a secure audit trail and standards traceability


Enables field data collection


Archives and restores data


Facilitates customization and scalability

Justifying a move away from legacy calibration-management systems (such as paper-based records or in-house systems based on Microsoft Excel or Access) is usually the easy part of the CMS implementation process. If you need ammunition for justifying the purchase of CMS, here are a few basic advantages you will realize quickly:

Better organization.Paper records are easily lost or misplaced, and they can take up a lot of space as they accumulate over time. In most cases, computer records remain where you put them.

Fewer errors. How are you currently notified of pending calibration or preventive maintenance events? Do you remember all of them, or do you miss some from time to time? Do calibration or maintenance events ever occur without being documented? Have you ever used an instrument that is out-of-calibration? A CMS package should resolve these issues.

Improved productivity. A good CMS package will automatically schedule calibration and maintenance events and notify you when they're due. All of your data will be stored electronically and easily accessible to all of those who need it. Additionally, downtime will be greatly reduced as your equipment is calibrated and maintained on schedule.

Reduced costs. Routine calibration is usually much less expensive than finding out-of-tolerance instruments in the field. Waste is also decreased due to improved product quality.

Compliance with standards and regulations. ISO standards and Food and Drug Administration regulations require timely calibrations and thorough, secure documentation. Your CMS should meet this need.

Once the decision to implement a CMS program is made, the real work begins. Although selecting and implementing a program requires careful planning and effort, it doesn't have to be a painful process as long as a few specific steps are followed.

User requirement specifications

It's common knowledge that the actual manufacturing process is much easier if the appropriate amount of effort is invested in the pre-manufacturing steps of research, design and planning. This rule also applies to selecting and implementing a CMS program.

Many managers in charge of selecting CMS make the mistake of beginning the process by surfing the Web for CMS vendors. Just as quickly as the list appears on-screen, the manager is overwhelmed by the number of options available--different feature sets at widely variable price points from vendors around the world. Save the vendor search for later. Begin the process instead by specifying your facility's specific CMS requirements.

Because you don't have control over the specifications used by the CMS

Three-Step Selection Process


1. Complete user requirement specifications


2. Qualify vendors


3. Evaluate software

designers, you must specify your own requirements up front and then match them to the available CMS packages. To conduct this comparison, it's recommended that you generate user requirement specifications, which describe the application from a functional viewpoint. The emphasis should be on the required functions and not on the method for implementing those functions.

A requirement specification should be written for each function that you'll need the software to perform. The UK Pharmaceutical Industry Group identifies the following guidelines for user requirement specifications in its Good Automated Manufacturing Practices Guide:

• Each requirement statement is to be uniquely referenced and not longer than 250 words.
• Requirement statements should not be duplicated or contradicted.
• The user requirement specifications should express requirements and not design solutions.
• Each requirement should be testable.
• The user requirement specifications must be understood by both the customer and supplier; ambiguity and jargon should be avoided.
• Whenever possible, the user requirement specifications should distinguish between regulatory requirements and desirable features.
• There may be a need for formal review of the user requirement specifications between the customer and supplier to ensure that requirements are understood and have been met in the functional specification.

Specifications will vary from industry to industry and among facilities. In general, all CMS requirements should be able to do the following:

• Add, maintain and retrieve master equipment records and calibration history records
• Automatically schedule future calibration due dates
• Print calibration forms
• Compile reports for calibration interval analysis
• Provide security at multiple levels
• Maintain a secure audit trail to track changes to the database
• Back up and restore data internally

The recommendation is to look for three basic traits in your CMS: flexibility, reliability and ease of use. Flexibility means having the ability to tailor the CMS (e.g., change field names or customize reports) to accommodate your existing processes rather than having to change them and retrain your personnel to meet the software's requirements.

Reliability may be a given, but it's the most difficult trait to assess. You need to know that your data will be secure and that data integrity will be maintained as the volume of calibration records grows. Your information technology department is an excellent resource and will most likely be very willing to provide input. Checking references is an ideal method for evaluating CMS reliability.

Ease of use is vital to effective CMS implementation. Your staff's computer skills should match those required to operate the CMS. You should begin using the software soon after you purchase it, so make sure the implementation process meets your timeframe. Finally, look for features, such as data entry templates and other tools, that speed up the data entry process and eliminate redundant data entry.

It's your responsibility as a buyer to ensure that the software package you select meets your user requirement specifications. A comparison of software specifications to your requirements should be made and documented. A written matrix is a helpful tool in this situation and is discussed in more detail later.

Because not all CMS packages meet every user's requirements, it's also necessary to document discrepancies between the software and requirements, including workarounds or procedures to eliminate the discrepancies. The preparation and verification of the user requirement specifications is vital for showing that the system is suitable for use in its intended application.

CMS vendor qualification

After your user requirement specifications have been completed, approved and signed for by the appropriate individuals, the search for vendors whose CMS packages meet your specifications can begin. However, verifying software functionality is only a portion of the vendor-selection process. You also should verify that the CMS vendor has an appropriate software-development methodology in place--a methodology capable of producing high-quality software.

There are many software development models commonly accepted as capable of producing quality software. Among these are the life cycle model defined by the Pharmaceutical Research and Manufacturers of America's Computer Systems Validation Committee and the standards set by the Institute of Electrical and Electronics Engineers and Good Automated Manufacturing Practices. It's in your best interest to understand which model the vendor uses and whether it's sufficient to develop the quality of software necessary for your application.

Verification of a vendor's software-development process can be accomplished in a number of ways, including vendor surveys, audits and reviews of vendor-supplied documentation. The method with the highest level of assurance is the vendor audit. However, this method is also the most costly and is rarely justified for a CMS package.

You can eliminate the need for an audit by asking vendors to complete a written survey, as well as reviewing documentation supplied by the vendor. Some vendors make copies of their software quality assurance procedures and records available to their customers.

It would be great if the vendor qualification process stopped here, but it doesn't. You should evaluate vendors against others criteria, the first of which is experience. Questions to ask include:

• How long has the vendor been developing CMS?
• What level of calibration expertise does the vendor possess?
• If appropriate, is the vendor knowledgeable about the regulatory requirements under which you operate?

 It's also important to evaluate vendors' reputations. Checking references is an excellent evaluation technique. Ask the vendor to provide a list of organizations similar to yours that are currently using its CMS products. Get the appropriate contact names and phone numbers, and call these organizations.

The reputation check will give you a good feel for each vendor's past performance, but you should also know the following about the future of the CMS:

• When is the next version scheduled for release, and what features will it have?
• Which modules or add-ons are available now, and which are planned for the future?
• Is user feedback considered when developing new versions? How is feedback gathered?

Asking these questions will enable you to determine if the new features and modules are in line with your anticipated needs, as well as the vendors' responsiveness to customer concerns.

The vendor-qualification process can be as limited or as expansive as you wish. If the references you contact are unanimous in their praise of the CMS, that may be all of the information you need. Keep in mind, however, that each facility's needs are unique and that what works well at one company may not work for your operation. If you have the time and resources to conduct a thorough vendor evaluation, make use of those resources. You will be much more comfortable with your purchase if you do.

Software review

By this point you've likely grown tired of conducting research and producing documentation. You know what you need your CMS to do, and you know which vendors can provide quality software. So let the fun begin. It's time to look at the software.

Of course, options exist here too. You may request that the approved CMS vendors come to your facility to demonstrate that their software does everything they say it does, and, more importantly, that it meets your user requirement specifications. While this technique may appear to be the easiest option, remember who's in control during these sessions?the salesperson. Rarely do you get to conduct a real hands-on test of the CMS in this setting.

The preferred evaluation method is obtaining a demonstration copy of the software so that you can try it before you buy it. This enables you to evaluate not only the CMS's functionality but also the actual installation and setup processes. Most vendors offer a free, no-obligation demonstration version of their software and are more than happy to go this route rather than face the high costs associated with sending a sales representative to your facility. Lower sales costs often translate into lower software pricing too.

Remember that matrix mentioned in the user requirement specifications section? It will come in handy now. On a sheet of paper or in a spreadsheet, list your user requirement specification items across the top of the page. List the CMS packages to be evaluated down the left-hand column. As you conduct your evaluations, note on your matrix which features and functions are met by each CMS package. Most important, of course, are the features and functions listed as "requirements." The "nice to haves" should also be researched and may prove to be the deciding factors should multiple CMS packages meet all of the basic requirements.

And don't focus solely on the CMS's capabilities. In addition to software, most vendors also offer lists of services to facilitate the use of their products. Ask the vendors under consideration about the services they offer.

• Do they provide post-sale technical support? Is there a cost for the support, and, if so, what is it?
• What is the industry reputation of the technical support service?
• During what hours is support available?
• What training services are offered? Is customized, on-site training available?
• What services are available to assist with CMS implementation? Data conversion and import, custom report creation, and system validation assistance are common requests.

Closely examine the cost of the vendors' services in comparison to their value. You may find that, while the initial cost (in dollars) of purchasing vendor assistance may be high, the long-term benefit of getting your CMS implemented quickly and your staff trained to use the software correctly is even greater.

Making a decision

Key CMS Traits






Ease of use

Finally, you've arrived at the finish line. The only remaining steps are to select a CMS program, decide which (if any) services you're going to purchase along with the package and then place the order.

If you're lucky, one of the CMS packages listed on your matrix meets all of your user requirement specification items, and the decision is clear. It's more likely, however, that there are some blanks in every row of the matrix. It's up to you to conduct the final product analysis and decide which CMS package will work best overall in your application today and in the future.

The good news is that you'll be comfortable making the decision because you've done your homework up front. You've assembled a thorough list of user requirement specifications and obtained approval from everyone involved, and you've confirmed which CMS packages meet the application requirements documented in the user requirement specifications. You've also determined which vendors meet industry-accepted development criteria and provide CMS and services appropriate for your application.

As long as you do the evaluation steps honestly and objectively, you can be confident that the CMS package you select will be the correct one for your application.



About The Author

Blaine Clapper’s default image

Blaine Clapper

Blaine Clapper is director of marketing and sales for Blue Mountain Quality Resources Inc., a leading supplier of calibration management software. He has a master's degree in business administration from the University of Pittsburgh's Katz Graduate School of Business and has more than 10 years' experience in the software industry.