Three basic evaluation methods exist for any work activity: inspection, compliance auditing and management
auditing. The first method, inspection, measures a process's output against certain characteristics. These characteristics, generally identified as form, fit and function, are specified, and the
process output either possesses those characteristics or it doesn't. As a result, an inspection's outcome is always binary: pass or fail.
In contrast, compliance audits check
on the implementation of written manuals, procedures and work instructions. The compliance audit evolved in the 20th century as business practices became more complex. The first use of compliance
auditing appeared in financial transactions, because tax collectors and bank examiners needed assurance that the financial data were correct. This concept of verifying compliance was picked up by
the quality profession in the 1960s and applied to the military and the nuclear power industry. Compliance audits are still used in high-risk activities, where there is a desire to verify that
the activities are being performed in strict compliance to approved requirements. Third-party registration audits, regulatory inspections and most supplier audits measure compliance. The
application of a compliance audit results in stability and assurance that rules are being followed.
The management audit is a more recent concept. It focuses on results,
evaluating the effectiveness and suitability of controls by challenging underlying rules, procedures and methods. Management audits, which are generally performed internally, are compliance
audits plus cause-and-effect analysis. When performed correctly, they are potentially the most useful of the evaluation methods, because they result in change.
Compliance Audits vs. Management Audits
Whether performing a compliance or a management audit, auditors must obey four basic rules. First, audits must provide information for a defined need, that
is, the customer's need. Second, auditors must be capable of performing their duties. Third, audits must measure performance against agreed criteria. Fourth,
audit conclusions must be based on fact.
Rule 1: Serve your customers
Audits provide information. All affected parties need to know if product, process and system controls are present and being applied, and obviously it
doesn't hurt to know whether these controls actually work. An auditor evaluates the controls against requirements and produces a report. If controls
are present and working, all parties' confidence in the process is increased. If controls are missing or not working, then resources can be applied to fix the problems.
Auditors serve three customers: the auditee, the client and the organization. Auditees' primary goal may be to simply pass the audit, but auditees trying to
derive the most benefit from the audit will also want to know whether the organization is functioning effectively. In this case, an auditor's outside
perspective can be quite valuable. The client (the person who commissions the audit), in contrast to the auditee, is accountable for the auditors' actions and
reports. Committees cannot generally perform this function; an audit boss should schedule the audits and make assignments. Finally, auditors must serve
the organization's needs. Business values are important and the auditors can assist by determining whether the enterprise is actually achieving its goals.
Rule 2: Use qualified people
Auditors must be able to carry out their assignments in an impartial and
objective fashion. This means that they cannot have a vested interest in the activity being audited. If they developed the rules, they cannot impartially
evaluate the effectiveness and application of those rules. Although an auditor can never be totally independent of the auditee, some separation must be
maintained. It's fine to audit within your group, but you can't audit your own job.
Auditors must also be capable of doing their jobs. They need certain
emotional, intellectual and mechanical skills, which they can obtain by attending a course, reading a book or observing others. Often, all three methods are
used. In addition to knowing how to conduct an audit, auditors must be familiar with the technical processes being examined. A good way to demonstrate this
familiarity is to flowchart the activity to be audited--if a person can't flowchart it, he or she can't audit it. Finally, auditors need to be able to communicate well, both orally and in writing.
Rule 3: Measure against agreed criteria
Auditors are not allowed to make up the rules--they must audit against performance standards that are already in place and accepted by the auditee.
This is the planning part of the plan-do-check-act loop. The highest level of requirements includes corporate policies, management system standards and
regulatory requirements. Usually originating from outside the auditee's organization, these requirements establish the goals and objectives to be
achieved. National and international standards, such as QS-9000 and ISO 9001, fall into this highest category. Next comes the local approach, often
called a quality manual or quality plan, for implementing these high-level requirements. It gives the framework for achieving the concepts and should be
fairly compact. This document is then followed by a number of process-specific procedures. Further detail can be provided in work instructions, such as
drawings, traveler sheets and sampling plans. One of an auditor's challenges is to obtain and become familiar with the many levels of requirements forming the basis for the audit.
Rule 4: Use facts to form conclusions
Auditing is fact-based; conclusions are drawn from the data. Facts can be
good (a requirement was met) or bad (a requirement wasn't met), but no judgment or opinion should taint them. These facts, also known as objective
evidence, can come from five sources. They can be physical properties, such as flow rates and dimensions; sensory-derived input from seeing, hearing, smelling
or tasting; documents or records; information drawn from interviews with auditee staff members; or patterns such as percentages or ratios. Auditors use
checklists and other tools to determine the facts to be gathered, and then they perform the fieldwork to gather these facts.
The output of the audit process, be it a management or compliance audit, is a report. The client (audit boss) receives the report from the auditor and delivers
it to the auditee. To prepare a report, the auditor must take all of the positive and negative facts and make some sense of the data. In other words, the auditor must analyze the data.
The first step is to list all of the positive and negative observations (data), then sort those data into controls or problem areas. Generally, there will be a large
number of negative observations associated with just a few control items. This natural chunking of the data allows the auditor to see the patterns, rather than
the individual events. For a compliance audit, these patterns are then reported as either conformities or nonconformities.
Management audits require some additional work. The auditor needs to identify the pain associated with those groups of bad facts. (It's important to
identify business problems, such as scrap, rework and overtime, as pain.) Then the auditor combines the missing control (the system error that's causing the
problems) and the business pain into one statement, called a finding. The finding will reveal cause-and-effect patterns occurring within processes. Because the
business pain is identified, there will be a tremendous desire to do something about it.
By associating the negative facts with missing or weak controls, the auditor
rises to the system level of analysis. This has lasting value, because the system affects the process, which affects the product or service.
Instilling a desire to improve
Audits measure actions against requirements; they examine the product,
process or system against performance standards. This has value when the requirements have been thoroughly tested and scientifically proven, but, unfortunately, this is rarely the case.
Most manuals, procedures and work instructions are imperfect; they're the result of a small number of individuals assembling some rules with limited resources. By focusing on results, the
management audit can determine whether those plans and approaches are any good. If they aren't, the developers and users are compelled to improve their methods because they can
see the adverse consequences of not doing so. When employees and managers begin to see audits as opportunities to improve, they begin to see auditors not
as police officers but as productive members of the organization.
Management Auditor's Rules
About the author
Dennis R. Arter is an independent consultant and trainer. He instructs large and small firms in the fields of management auditing and quality
systems. Arter has served clients in the fields of government, manufacturing, chemicals, energy, research, aerospace, and food processing and is the author of the book
Quality Audits for Improved Performance (ASQ Quality Press, 1994).
Arter is an ASQ Fellow and an active member of the Society's Standards
Group, Customer Supplier Division and Quality Audit Division. He is responsible for coordinating all quality, environmental, dependability and
statistics standards within the ASQ. He was on the team that developed the ASQ Certified Quality Auditor program and holds a CQA charter certificate. He managed the team that developed the
Quality Audit Handbook, published by Quality Press in 1997. E-mail him at email@example.com .