Survey
Gage Software
Baldrige
ISO & SPC
ISO Software
Gage Primer

 

Article

by Patricia D. Carden, Ph.D.;
Lawrence S. Aft;
and Dirk Dusharme

To see the complete survey results, click here .

Initially intended as a tool to help potential registration clients search for a registrar, our Annual Registrar Customer Satisfaction Survey has since become a tool for registrars, a benchmark against which to check their own continuous improvement efforts. This marks the third year we've conducted the survey and the first year that we've conducted post-survey interviews with registrars regarding the results and their implications.

 To help track survey results from last year to this year, we asked the same 17 questions that presurvey interviews identified as being important to registrars' clients. These questions cover six categories: Customer Loyalty, or whether the customer would recommend the registrar to others or purchase its services again; Communication, that is, how easy was it to contact the auditor/registrar; Value-Added, or added service beyond simply auditing; Administration, or audit scheduling and certificate issuance; Interpersonal Relationships, or how auditors/registrars deal with clients; and Consistent Interpretation.

At the top of the list

 Due to this year's exceptional response rate (see Methodology), several smaller registrars absent from previous surveys made it into this year's results; two of them even near the top of the list. Along with last year's high-ranking registrars, Eagle Registrations and DQS joined the top five registrars, with Eagle scoring at the top of all categories.

 Here's what a few of the high-ranking registrars do to earn top customer-satisfaction scores:

  Eagle. "We practice what we preach," says Chris Shillito, president of Dayton, Ohio-based Eagle Registrations Inc. "We identify the elements in our system that cause us to give the best service, such as timeliness of reports and prompt return of phone calls. The key is to identify what is important to the client."

 Eagle uses internal customer satisfaction surveys along with the Quality Digest Registrar Customer Satisfaction Survey to gauge client satisfaction. Two years ago, both surveys revealed weaknesses in the company's service aspects, which caused Eagle to set service issues as an improvement project and begin developing methods to address them. The result was a marked improvement in subsequent customer satisfaction surveys.

  Smithers . "It's the personal touch; we give good service to our clients," reports Wyn Hardy, director of marketing for Smithers Quality Assessments Inc. of Akron, Ohio. "We communicate frequently, pass along industry updates and trends as we get them, and solicit and act on feedback."

  Quality Certification Bureau. QCB deserves mention here as most-improved registrar. In five out of six categories, QCB jumped from middle of the pack to top 10.

 "We looked at the [Quality Digest] survey results from last year and asked ourselves how we could do a better job," notes Julie Press, vice president of the Edmonton, Canada-based company. "We looked at ways to improve communication and value-added services and ways to help clients through the ISO 9000:2000 transition period."

 QCB's internal surveys showed the company's customer satisfaction scores had improved; our survey showed a huge jump in both the Value-Added and Communication categories.

  Orion. This year, Orion Registrars Inc. of Arvada, Colorado, which had scored at the top of most categories in the previous two years, dropped in the ratings. "No excuses," says Paul Burck, Orion's president. "We had some problems, which we admit, but we've addressed them. It's not our goal to be at the top of the ratings every year. It was nice, and we were happy to be there, but we're happy that we're still in the top 10."

To see the complete survey results, click here .

Customer loyalty

 Customer loyalty—the willingness of a customer to recommend and/or stick with a company—is the ultimate goal of any business. However, it's difficult to really nail down what makes a loyal registrar client. Table 1 shows the correlation between the five categories and the 15 survey items (17 survey questions less the two loyalty questions) and customer loyalty. In other words, how much do these particular items or categories affect customer loyalty? As with previous years, the ability of the auditor to put the client at ease, maintain a professional attitude and deal well with clients has the highest correlation to customer loyalty.

Table 1: Correlation of Survey Categories and Questions to Customer Loyalty

Squaring the correlation number to the right of the item will give you the percentage of the variation in Customer Loyalty that can be explained by that item or category (e.g., the correlation between Customer Loyalty and Communication is 0.42, which means that 18.2 percent of the variation in Customer Loyalty can be explained by Communication).

Survey Items

1) If we had questions or concerns, the auditors helped us to resolve the issue. -- 0.42

2) There was a difference in standards interpretation between the auditor and us.--  0.39

3) The auditors/registrar were hard to pin down to schedule appointments. -- 0.34

4) The auditors had the ability to clearly articulate the standard's requirements into lay terms.--  0.51

5) The auditors were fair. -- 0.57

6) The auditors painted a clear picture of what we needed to do in the future in order to continuously improve. -- 0.50

7) Auditors/registrar were flexible with scheduling.--  0.40

8) Our auditors put us at ease. -- 0.53

9) The audit wasn't as stressful as feared. -- 0.41

10) We were frustrated with the different interpretations of the standard from audit to audit or from auditor to auditor. -- 0.22

11) The auditors took the time to understand and address our concerns.-- 0.59

12) The auditors/registrar followed up on our telephone calls and e-mails to them. -- 0.47

13) We felt as though our auditors were part of a joint effort with us. -- 0.58

14) The auditors/registrar were easy to reach. -- 0.25

15) It took a long time to receive our registration document. -- 0.57

 

Category

Interpersonal Relations -- 0.71

Value Added -- 0.55

Consistent Interpretation --  0.38

Administration -- 0.49

Communication -- 0.42

 

 At first glance, it seems that the smaller registrars (those with fewer than 1,000 registered clients) do better than their larger counterparts when it comes to customer loyalty or any other measure; the top third of all the charts are predominantly small registrars and the lower two-thirds larger registrars. This trend was pointed out by several large registrars in the past and intuitively seems to make sense, although a correlation study conducted on an earlier survey showed no correlation between registrar size and customer satisfaction. However, with the growth that registrars are experiencing (Eagle reports 20-percent to 40-percent growth per year), it won't be long before the smaller registrars become big registrars, and this could become an important consideration.

To see the complete survey results, click here .

 This point isn't lost on the registrars. "We're cognizant of the issue," says Russ Bloom, lead auditor and sales and marketing manager for Smithers, a growing registrar with about 500 registered sites that has been highly ranked by our survey for the past two years. "But as our client base grows, the company will grow as well," he assures. Will that be enough? Bloom admits that growth could affect Smithers if the company isn't careful.

 Maintaining personal service is a concern, notes Press. Some of QCB's 1,400 registered clients defected to QCB from very large registrars. The challenge arises, she says, because increasing the number of auditors to handle the client base also increases the complexity of ensuring that all the auditors are properly trained to offer consistent services and standards interpretation.

Consistent interpretation

 As with previous years, this category, which measures the customer's perception of whether the registrar and/or its auditors provide consistent standards interpretation, was the lowest-scoring category out of the six. In addition, this year's average score for the category was less than last year's.

 There could be several reasons for this year's dip. It could be a quirk unique to this category, or it could be related to some ISO 9000-related events that have occurred in the past year.

 Although ISO 9000:2000 is relatively new, several registrars suggested that some confusion may be creeping into the interpretations due to the revision's release. The major change in the new version—the focus on evaluating the effectiveness of business processes—could be a big contributor, suggests one registrar. "The assessors have to use their judgment to evaluate processes for ISO 9000:2000," he says. "In the old standard, you evaluated the presence of 20 elements; now you have to determine whether the processes are effective." The distinction implies that there's more room for interpretive differences between the auditor and the client or between auditors.

 Another big change is last year's crackdown by the Big Three automakers on the QS-9000 sanctioned interpretations. "The automakers have made it very clear that they want the registrars to follow the sanctioned interpretations exactly," explains the registrar. "And they demand that auditors write up everything." In the past, an assessor doing a QS-9000 audit might make an observation on a weakness in the QMS and issue a warning but not write-up a nonconformance, particularly if the rest of the system was in good shape. Now, all discrepancies must be reported as nonconformances. This new strictness may be catching some automotive suppliers off guard and causing them to score their registrars more harshly in the interpretations category, adversely affecting registrars with a large percentage of automotive industry clients.

 The industry mix of clients being registered to ISO 9000 could also be a contributing factor. One registrar points to the increase in service industry clients over manufacturing clients. Is it possible that interpreting the standard for service organizations introduces complexities that don't exist for manufacturing organizations?

 Overall, the pursuit of consistent interpretation is an ongoing issue for registrars. "It's hard to prepare auditors for every possible situation that they will face in the field," explains Press. "And even if you could train them to be 100-percent consistent, the only way to do that would be to be so rigid in your interpretation that you drop off in the value-added category."

 Mike Fay, Smithers' director of communications, would agree. "Integrity [in standards interpretation] is extremely high on our list," he says. "But we do this without sacrificing customer satisfaction."

To see the complete survey results, click here .

Value-added

 Although a registrars' primary job is to audit a client's QMS, customers are also looking for value beyond merely receiving a certificate.

 Value-added services are a touchy subject among registrars, which must be careful to never cross the line into consulting. We've noticed that the registrars themselves, particularly those that belong to the Independent Association of Accredited Registrars, speak a common language when it comes to what "value-added" means, an issue the IAAR board has addressed, says Press. In fact, the IAAR, which is made up of about 35 registrars, has made value-added part of its strategic planning initiative and is a continual theme in all IAAR sessions.

 "We don't just audit for compliance; we challenge our clients," explains Shillito. "We look for problem patterns. We look for drivers or the lack of a driver, and we always try to look for measures in a client's quality management system that will ultimately improve its financial outcomes." Eagle's auditors try to identify whether there is a measure in place, required or not, that addresses every aspect of a company's QMS. The auditor, as a value-added service, reports a system's shortcomings, even if not required by the standard.

 Press agrees. "If the only thing they get from us is a certificate to hang on the wall and we're not helping them make an impact on the bottom line, then we've failed," she says.

 To do that, Eagle, QCB, Smithers and others look at how their clients are implementing their QMSs and point out areas that could be improved or streamlined, without telling them how to do it.

 "If we think that the way a client is implementing some aspect of its QMS is cumbersome or hard to maintain, we feel it's our responsibility to say something," says Press.

 The most prevalent issue that Press encounters is companies that enter into a kind of ISO 9000 overkill, going beyond what the standard requires in terms of documentation or implementation, in many cases adding documentation or work that adds nothing to the value of their processes and is not required for the standard. Pointing out these areas, even without recommending how to fix them, provides a valuable service to the company, explains Press.

 Another value-added service that many registrars provide is informational updates. Smithers, QCB and Quality Systems Registrar, the nation's first accredited ISO 9000 registrar, offer free informational seminars on ISO 9000:2000. In addition, many registrars offer working documents or sample documents on their Web site as free downloads.

Methodology

 After conducting presurvey interviews with 75 randomly selected ISO 9000 registered companies, Quality Digest met with 15 registrars at Southern Polytechnic University in Marietta, Georgia. Along with SPU professor Patricia D. Carden, Ph.D., and Lawrence S. Aft, we worked with registrars to categorize the critical incidents identified by clients during the interviews.

 After discovering that this year's critical incidents were essentially identical to last year's, we determined to use the same questions and categories this year. This also gave us the opportunity to do an apples-to-apples comparison between the two years.

 As in previous years, the survey instrument was developed using Scantron Technologies eListen software, which allows us to generate unique identifier codes for each survey and host the online survey from Scantron's Web site. The unique identifiers enable us to identify which registrar belongs to each client's survey and prevents ballot stuffing on both the online and faxed surveys.

 Next we pulled from our 51,000-client ISO 9000 North American database all of the registered clients for which we had fax numbers. We then eliminated all duplicate fax numbers (common with multiple-site registrations). We faxed the remaining 35,000 clients the survey instrument. In the past, we simply faxed an invitation to take the survey online. This year, in order to boost the response rate, we elected to include the questions in the faxed survey as well to offer the survey online.

 The result was an astounding increase in the survey responses—6,000 survey returns, or about a 20-percent response rate. Nearly 4,000 responses arrived by fax, while the rest were received at Scantron's eListen site. The faxes were input manually while the eListen surveys were extracted using eListen's Analyzer module. The data was checked for duplicate IDs, patterns and conflicting answers.

 Carden and Aft performed the data analysis using Microsoft Excel, while Quality Digest double-checked the results using Excel statistical add-on Analyze-IT.

 We looked for gross differences in this year's results compared to last year's, but only a few items were readily apparent. We investigated these by interviewing registrars to provide plausible explanations for the differences.

 Quality Digest wishes to thank Southern Polytechnic University for the use of its facilities; Carden and Aft for their expertise and analysis; Scantron Technologies for the use of eListen and its Web site; Laurel Thoennes, our harried ISO 9000 database coordinator; and, of course, the registrars, who bear the survey with a stiff upper lip, keep us on our toes, and provide valuable input that helps us to continuously improve.

To see the complete survey results, click here .

About the authors

 Patricia D. Carden, Ph.D., is department head for Industrial Engineering Technology at Southern Polytechnic University. Lawrence S. Aft is professor and program coordinator for the master of quality assurance program. Dirk Dusharme is Quality Digest's technology editor. E-mail comments about this article to registrarsurvey@qualitydigest.com .

 Interpreting Eagle's Results

A survey of this sort is intended to provide service to readers by giving them useful information when searching for a registrar. The side effect, however, is that it stacks registrars against each other, which could, and probably does, affect their business. We are very aware of this and strive to make sure that the survey is conducted without bias and with the utmost scrutiny in how the data is collected and interpreted. We also try to investigate all factors that could conceivably affect the survey results.

 This year, several registrars were proactive in letting their clients know that the survey was coming and encouraging them to participate. This can be good because it boosts the response, which in turn lowers the margin of error. However, it does raise the question of whether encouraging clients to take the survey causes them to answer differently than they might otherwise have.

 This year's overall high-ranking registrar, Eagle Registrations, is a small registrar that hadn't made it into past surveys because its response rate wasn't high enough to ensure statistical validity. To address that issue, this year Eagle sent a postcard to all of its clients encouraging them to participate. Some might claim that Eagle went to the top because the postcards positively biased the responses. This argument seemed to be a valid concern, so we examined the situation. While not definitive, consider a few points when looking at Eagle's results: Although high, the results are still within reason compared to its nearest competitor. Further, if Eagle's 21-client sample (about 10 percent of their client base) from last year was representative of all of their clients, they would have scored within the top five in all categories had they made it into the survey. Eagle, as with other registrars (e.g., QCB), used past survey results to fine-tune their processes and make advances in customer service as a result, so it isn't unreasonable to assume that these efforts would have propelled them to the top of the charts.

 By supplying that information and a copy of the postcard, which Eagle willingly gave to Quality Digest knowing how we intended to use it, we hope we have provided the reader with adequate information to come to an informed conclusion.

Companies Included in Survey

Clients for the following registrars were contacted for this year's Registrar Customer Satisfaction Survey. Results for bolded companies are in the six main charts included in the article. Results for italicized companies (fewer than 30 responses) are included in the table on page 31. Companies in normal text had only zero or one response.

  • ABS Quality Evaluations Inc.
  • AFAQ-ASCERT International Inc.
  • AIB Registration Services
  • American Quality Assessors (AQA)
  • American Systems Registrar LLC (ASR)
  • AOQC Moody International Inc.
  • AQSR International Inc.
  • ASME International
  • BSI Inc.
  • Bureau Normalisation Quebec (BNQ)
  • BVQI (NA) Inc.
  • Canadian General Standards Board (CGSB)
  • CRS Registrars Inc.
  • Deloitte & Touche Quality Registrar Inc. (D&T)
  • DNV Certification Inc.
  • DQS (North America)
  • Eagle Registrations Inc.
  • Entela Inc.,QSRD
  • Excalibur Registrations Inc.
  • Factory Mutual Research Corp. (FMRC)
  • Global Registrars Inc.
  • Great Western Registrar LLC
  • HSB Registration Services
  • International Certifications Inc.
  • International Management Systems Inc.
  • International Quality System Registrars (IQSR)
  • ITS Intertek Canada
  • ITS Intertek Services
  • KEMA Registered Quality Inc.
  • KPMG Assessment & Registration Services (US)
  • KPMG Canada
  • Lloyd's Register Quality Assurance (LRQA)
  • National Quality Assurance, USA (NQA)
  • National Technical Systems—Certfication Services (NTS)
  • NSAI
  • NSF International Strategic Registrations Ltd. (NSF-ISR)
  • Orion Registrar Inc.
  • OTS Quality Registrars Inc.
  • Performance Review Institute (PRI)
  • Perry Johnson Registrars Inc.
  • Professional Registrar Organization Inc. (PRO)
  • QMI
  • Quality Certification Bureau Inc. (QCB)
  • Quality Services International (QSI)
  • Quality Systems Registrars Inc. (QSR)
  • QUASAR
  • The Registrar Co. Inc.
  • Satra
  • SGS International Certification Services Inc.
  • SGS-ICS Canada Inc.
  • Smithers Quality Assessments Inc.
  • SRI Quality System Registrar Inc.
  • TRA Certification
  • TÜV Essen Quality Systems Division
  • TÜV Management Service/TÜV Product Service
  • TUV Rheinland of North America Inc.
  • Underwriters Laboratories Inc.
  • United Registrar of Systems Ltd. (URS)
  • VCA North America

Menu Level Above 

This Menu LeveL 

Menu  Level Below 

[Contents] [News] [WebLinks] [Columnists]
[Survey] [Gage Software] [Baldrige] [ISO & SPC] [ISO Software] [Gage Primer]
[Results]

Copyright 2000 QCI International. All rights reserved.
Quality Digest can be reached by phone at (530) 893-4095. E-mail:
Click Here

Today's Specials