After a Global Pandemic, Are Fully Remote QMS Audits Truly Effective?

Although ISO 19011—“Guidelines for auditing management systems” has included language about remote quality management system (QMS) auditing since the 2018 revision, this became a reality for many of us in March 2020 with the onset of Covid-19 and the mass lockdowns that ensued. Many of us have been exclusively auditing quality management systems remotely for a year and a half, and without doubt will continue to do so.

In the beginning of the pandemic, there were articles circulating to point to the language of ISO 19011:2018 or blogs from consultants that claimed they were able to teach effective remote-auditing methods to ensure that the results were just as effective as their onsite counterparts. However, now that we are a year and a half into this, I wanted to get some opinions about the actual effectiveness of remote auditing as it stands today.

I wanted to know if auditors’ and auditees’ perceptions about the effectiveness of remote auditing had changed during the pandemic, or whether they felt that we were truly meeting the criteria set out in ISO 19011. I wanted to learn more about the techniques that auditors have developed to make fully remote audits more effective, and what the professional community thought we ought to do to continue to close the gap to make them more effective in the future.

The poll

So, to open the discussion, I ran a poll on LinkedIn (a very scientific study, I know) that ran for one week that simply asked the question: “After 1.5 years of global pandemic, do you find that fully remote QMS audits are truly effective?” as well as prompting the respondents to comment with their experiences, techniques, perceptions, and thoughts for the future. Respondents were given the following four options in the poll: 1) Yes, more [effective] than onsite, 2) Yes, as effective as onsite, 3) No, less [effective] than onsite, and 4) Not at all effective. While there were only 96 respondents, the results of the poll were not all that surprising.

Most respondents, 52 percent, felt that remote audits were less effective than onsite audits, and after that, respondents became a bit more divided, with 18 percent taking this a step further and saying that they are not effective at all, and 26 percent believing remote audits that are currently being conducted are as effective as their onsite counterparts. Only 4 percent felt that remote audits were currently more effective.

Although the percentage of respondents saying that remote audits are currently as effective is a bit higher than expected, the data collected certainly suggest that there are some gaps to fill to make remote audits as effective as they should be, as well as effective as they will need to be as we potentially move to a more remote way of working.

What ISO 19011 says

Before digging into the whys and the solutions, I thought it prudent to take a closer look at what ISO 19011:2018 says so we can get a frame of reference from the current standard’s language on the topic.

A search of the term “remote audit” in ISO 19011:2018 will yield just six results, all of which are embedded within the body of the text and appendices. Section 5.4—“Establishing the audit program,” includes the first two.

Paragraph 5.4.3 k) basically states that individuals managing an audit program must determine the extent of the program, with factors such as the availability of information and communication technologies to support remote auditing methods potentially having an impact. Similarly, 5.4.4 g) suggests that the audit program manager must determine resources for the audit program, including the availability of information and communication technologies required to set up a remote audit and support remote collaboration.

These statements seem obvious and don’t yet provide much guidance, but respondents did sometimes echo these statements in their comments on the poll.

One respondent who had commented that he believed remote audits were “just about as effective” was asked what he thought would be needed to help close the gap so they could be as or more effective.

He states: “As strange as it sounds, the next-largest gap was connectivity throughout the plant. We had trouble running live video communication in many parts of operations. To add to the trouble, the noise made it difficult to hear, and it was impossible for the guide to hear. Some elaborate technology setup could help, but it also needs to be simple and reliable.”

This respondent’s sentiment was shared by many others and, indeed, matches closely with some of my own experiences. When planning an audit program, I think many of us are still trying to find the right balance between what needs to be conducted via livestream and what can be done remotely without the use of streaming technology. For the former, I believe that we’re all working to better understand what resources and technologies will be needed to livestream effectively, and what factors and conditions might prevent us from truly meeting audit objectives. If the ability to see and hear the environment on a shop floor is necessary for an audit or site assessment, for instance, do we truly understand how to do that effectively before we engage in the audit activity? If we haven’t worked out all of the bugs in the planning phases, will we really gather the evidence that we need to achieve the audit objectives?

The aforementioned instances in which remote audit is mentioned in ISO 19011:2018 are the only times that the term is used within the body of the clauses. All the remaining occurrences appear in Annex A (informative): Additional guidance for auditors planning and conducting audits.

The third occurrence is in a note in Table A.1, which is a matrix outlining audit methods based on auditor location with these quadrants: onsite with human interaction, onsite with no human interaction, remote with human interaction, and remote with no human interaction, with the fourth in the succeeding paragraphs qualifying feasibility of remote audit activity.

For most of the respondents, it appears that most remote audits being performed are either remote with human interaction or a combination of remote with and without human interaction. The matrix gives several general methods for collecting information such as conducting interviews, observing work with a remote guide, completing checklists and questionnaires, and conducting document review with or without auditee participation, but it’s otherwise not a source of guidance on how to do so effectively.

The fourth occurrence exists in the statement, “The feasibility of remote audit activities can depend on several factors (e.g., the level of risk to achieving the audit objectives, the level of confidence between auditor and auditee’s personnel and regulatory requirements).” It goes on to say, “...it should be ensured that the use of remote... audit methods is suitable and balanced, in order to ensure satisfactory achievement of audit program objectives.” Similarly, this only restates the need to ensure that methods are used and resources determined and employed that ensure that audit objectives are met, but provides no other information on how to do so.

Image above from ISO 19011:2018 used with Fair Use Copyright for the purpose of education and commentary

Closing the gap

We had already established through the poll that most auditors using remote methods don’t feel that they are consistently meeting audit objectives through virtual means, but we have yet to gain much valuable insight on how to close the gap, either from the polling or from the text of ISO 19011.

Of those who felt that remote audits were almost as effective or as effective as onsite audits, when pressed for techniques used, they primarily focused on a combination of the proliferation of new technologies and the reinforcement of widely established auditing techniques that would otherwise be used onsite.

For the use of new technology, it seems to be widely accepted that the increased use in electronic platforms, intranet or cloud documentation solutions, and electronic quality management system (eQMS) solutions provide a huge advantage when virtually auditing.

One respondent commented: “I’ve been working in the field of quality management systems for industrial quality, as well as the digital transformation of paper QMS to software for more than 10 years, and of course this has enabled us to audit the QMS remotely, as every department has all QMS documentation electronically filled out in the system. It is found to be very effective, especially for multi-factory companies. Also, I have to mention, some parts of the electronic QMS are self-audited and followed up on by the system itself.”

It appears that this particular respondent is using primarily an approach that’s remote with no human interaction, and he didn’t respond when pressed about difficulties when human interaction was needed, or when activities at the multiple sites being audited needed to be observed. But his experience with documented elements of a QMS seems to be common, and, indeed, mirror my own.

In my experience, I’ve often used a combination of remote with and remote without human interaction to conduct QMS audits, and so long as the auditee uses and provides access to electronic systems used for document and record control, these portions of the audit can be conducted very effectively, with difficulties typically arising from technological limitations. With the widespread proliferation of eQMS, enterprise resource planning (ERP), and customer relationship management (CRM) technologies in many manufacturing, service, retail, and healthcare industries, it stands to reason that we will continue to see rapid improvement in our ability to audit documentation and records effectively and efficiently remotely.

One comment of note to help improve situations in which human interaction is necessary came in the form of a stark reminder from an experienced director of quality with more than 20 years of experience in operational excellence, and who sits as a current chair for ASQ’s Silicon Valley section:

“I just completed two supplier audits. Both were completed effectively in shorter time. With complete transparency between auditee and auditor, remote audits can be as effective. One should recognize audit is a review of a sample of the QMS, not combing through for 100-percent verification. Instead, I find auditors lacking experience in following the intent of quality auditing.”

This particular comment struck me more than any other. This professional made a statement that is completely in line with quality gurus such as Phil Crosby and W. Edwards Deming. Quality improvement doesn’t always require expensive new technology or equipment, but better utilization of the resources that already exist.

If we commit to our principles of quality improvement, we will note that many of the failings that we currently experience in remote auditing, especially when human interaction is a factor, come not just from technological limitations, but also with the limitations in our auditing abilities. We should remind ourselves that the intent of QMS auditing is merely to sample evidence to assess whether the criteria have been met, and only dig deeper when it seems that the criteria hasn’t been met.

This point isn’t lost, then, when I examine the last two instances in which “remote audit” is mentioned in ISO 19011:2018. They are both in section A.16 Auditing virtual activities and locations, and the section spends nearly equal amounts of time discussing auditor competency as it does enforcing the need to ensure that technological requirements are met.

To make remote audits truly effective, it’s clear that there are still some gaps to fill. Because it appears that remote auditing is here to stay, we owe it to ourselves to understand the gaps and start working toward solutions. I believe that remote auditing does have the potential to be as effective, if not more effective, than the exclusively onsite audits of yesteryear.

There are still some technological hurdles that need to be overcome to ensure connectivity throughout the location being audited. But perhaps more important, we need to invest in training our auditors to audit more effectively in remote scenarios.

Although most respondents unsurprisingly cast their vote stating that remote audits are not yet as effective as they should be, their shared experiences helped to illustrate a not-too-distant future in which they will be.

About The Author