G iven that AI has been this century’s biggest disruptor for nearly every industry in the world, it has been met with justifiable levels of angst, anger, questioning, and skepticism. It’s as if our world has essentially been turned into a version of Jurassic Park, where, instead of dinosaurs, we have computer-generated agentic forces running amok.

As people explore this new technology’s capabilities along with its the implications, we have good reason to fear how it will be used and how much those in power will allow it to shape the world as we know it. Unless, that is, we address not just AI liability, responsibility, or capability but also the issue underlying all of those: risk.

Greg Hutchins, principal engineer of Quality + Engineering (Q+E) and founder of Certified Enterprise Risk Manager (CERM) Academy, argues that the quality industry needs to pivot from quality management and control to a larger discussion of risk. He should know. As the head of the first company certified by the U.S. government for critical infrastructure protection (CIP), he has an insider’s view of how reliant we are as a society on machine-automated processes—and on how much underestimating the inherent risk could cost us.

Hutchins agreed to an interview on what many have simply deemed the “AI Apocalypse” to clarify what’s hyperbolic, what’s hypothetical, and what’s simply a reality that the quality profession needs to address with the available knowledge, experts, teams, and resources.

Megan Wallin-Kerth: Could you state what changes you’ve seen and been a part of in the last year, and what you think people aren’t aware of that maybe they should be?

Greg Hutchins: The profession is looking for new voices, new ways of doing things. Our particular bias—and we have a trade market on it—is that future quality is risk. Why risk? Because we live in a world of uncertainty. We coined the term VUCA: volatility, uncertainty, complexity, and ambiguity. As for solutions, we think the one for the quality profession is to move to risk. Everything is almost rebranding the profession to become a risk professional profession.

Wallin-Kerth: How much of risk assessment do you think should remain strictly human oversight with quality professionals?

Hutchins: Well, that brings up the larger question of AI. Whether we like it or not, that train has left. The AI train has left. The rules of engagement between the human and the machine haven’t been defined. And I say “rules of engagement,” using a military analogy, but it’s still very important. How do we relate to that machine?

[Naturally, my question led to risk management and automation. Who manages the decisions, and when does that require clear human oversight or even intervention in a process? Hutchins talked about discerning how much judgment a machine should have, when humans should intervene, and whether decision-making should be escalated to human oversight.]

Hutchins: Now, if you take that into a business setting, we don’t know. This includes critical infrastructure settings, which is the world I inhabit—power plants, hospitals, nuclear facilities. Obviously, we don’t want the machine to have access to our nuclear launch codes. But what do we want the machine to be able to do, and how will it do it? Lots of questions. We call that the boundary conditions, or safety guardrails, or the rules of nature. And those have not been defined. So, you ask a very deep question. There’s no easy answer.

[The rules of engagement, and the boundary conditions, as he mentioned, are still in the process of being clearly defined.]

Hutchins: And in terms of the quality profession, there’s opportunity for the assurance profession. Because, remember, assurance is the equivalent of trust. We want to have trust with that machine, because the machine’s not going to go away.

Wallin-Kerth: One thing that immediately comes to mind is the metrics that we use for quality assurance. We probably have to expand on them, now that it’s a more complicated question. What metrics would you use, say, in a small manufacturing setting, if you said, “OK, we’re using these AI agents. Let’s look at the metrics to see if what we’re doing is actually safe.”

Hutchins: Good question. So, in the quality profession, especially in ISO-land, we’ve had three levels of documentation, going back to 1987. When ISO 9001 was developed, we had policies for the enterprise, for the strategy, and we had procedures for the process, for the workflow. And then we had work instructions for very hands-on stuff. The analog or equivalent in AI-land is “over the loop.” Human over the loop.

Over the loop means somebody designs the system, the policies, the strategy. You can think of that as quality management or whatever words you want to put in—enterprise risk management. Next level down, at the process workflow level, you have procedures, the equivalent of AI’s human in the loop. And you can break that analog down to anything, right? From therapy to community involvement, or whatever it is.

At the activity level, let’s say you’re my therapist, and the machine is my therapist, too. You’re in the loop. I’m the patient; I’m using AI, and you’re basically facilitating my therapy. You’re in the loop. That’s a work instruction, right? It’s facilitating or designing the—hopefully—mental welfare that I’m getting from this machine. So, if over the loop is policies, then on the loop is procedures. For this type of involvement, in the loop would be work. You’d be in between the machine and the loop. You could take the analog to a manufacturing facility.

[Here, Hutchins drew a detailed comparison between policies, procedures, and work instructions with different levels of human involvement in an AI system. This led to the  following question.]

Wallin-Kerth: What about those already using AI and developing standard operating procedures? We’re already there, and it’s happening in workplaces where processes and procedures are complex, interwoven, and deeply integrated into the workflows and successes of companies and their products.

Hutchins: That would be on the loop.

Wallin-Kerth: So, an SOP (standard operating procedure) would be on the loop.

Hutchins: Right. Actually, the machine flow, the products coming out of that machine, would be a process flow, a workflow. Remember, most of that manufacturing is automated already. You’ve got robotic arms, you have automated SPC, you have a whole bunch of stuff, right? I don’t want to get too technical. You have a workflow. The question is, from a risk point of view, where in that workflow should a human intervene with controls to make sure that the output is conforming to spec, conforming to the drawing?

Wallin-Kerth: So, people at Hexagon are saying that they’re hiring more people, not fewer, as a result of AI, because they need people who specifically track and rein in and check and double-check the standards. Do you think that will become more common? And what about companies that can’t afford that but need AI to keep them competitive, at least as far as the throughput? Let’s say they want productivity—or apparent productivity—but those safety metrics might not be met if they can’t hire or afford to hire more people.

Hutchins: So, now you’ve moved, notice, from the product level to the process level.

Wallin-Kerth: Yes.

Hutchins: When you use the word throughput, that implies you’re looking at it as a process. When you’re looking at the process, you have to design the actual manufacturing for the supplies flow so that it has the right constraints. A constraint is a risk, and a human must be there to oversee it or to evaluate it.

Wallin-Kerth: So this conversation is really about putting AI and risk in perspective. Also, when talking about risk, I mean, there are different levels.

Hutchins: Yes.

Wallin-Kerth: In past conversations on this topic, the consensus has been that the greater the risk and the heavier the implications, the more human oversight is needed within every step.

Hutchins: Yes.

Wallin-Kerth: Is that going to be sustainable if engagement is down in these fields, we’re losing people, and we can’t attract talent?

Hutchins: That’s one of the big issues. Right now, in the current state of manufacturing, we can use agentic decision-making that’s process-based, workflow-based. We have a defined workflow. Think of it as manufacturing—step after step after step. From a quality point of view, the next question is, where should the human intrude into the process? With what type of controls? Intrusion can mean tweaking machines. It can mean verifying and validating something. Intrusion can be comparing the output against the misprocedure to make sure it’s capable. These are techie quality terms.

First of all, is the machine in control? Is it capable of meeting spec? And is it improving, minimizing variation? So, if it’s agentic and workflow-based, no problem. But if we give the machine control to automatically adjust, make adjustments, determine the speed, or determine the throughput, that’s a different question. That’s where litigation and liability come in. Another phrase for liability is things gone wrong. Then the next question is, who pays?

* * *

And there’s the rub: Somebody always pays: intent vs. outcome. Intentions for an easier workflow and a reduction in downtime are all well and good, but without the necessary constraints and a proper understanding of liability—or risk, as Hutchins translated it—the outcome may be a pile of… well, receipts that we must answer for.

But don’t let this conversation fool you into thinking Hutchins is merely pessimistic. Not at all. As he alluded to, we’ll need more people in quality. We need people who are willing to “be the voice of quality,” as he puts it.

Hutchins defined this as people who are willing to see the world in shades of risk assessment, risk management, and risk control. That’s because pursuing quality requires attention to processes in a way that demands we dissect and even question the authority behind each step. And while AI can help us analyze the data that inform those steps, we’ll continue to need people who are willing to manage and mitigate the increasing need to keep AI-driven analysis and implementation from hallucinations, oversteps, and decision-making that ought to be a human’s responsibility, rather than a machine’s.

In other words, before you check the quality of your work with a machine, make sure that humans have checked the quality of work by the machine. And after you check your work with the machine, check the quality of that work with another human.

Welcome to Jurassic Park. Let’s start turning the fences back on with an increased focus on human responsibility to properly translate risk and transform it into regulation.