Sherlock Holmes and Root Cause Analysis

Lessons in RCA from a famous detective

Published: Thursday, April 27, 2017 - 12:02

Although not a quality guru, the fictional detective Sherlock Holmes took a methodical approach to problem solving that can be useful when applied to root cause analysis (RCA) during the investigation of a product or process failure.

Sir Arthur Conan Doyle (1859–1930) was a British physician who created the character Sherlock Holmes. Doyle published 56 short stories and four novels between 1886–1893 and 1903–1927. In addition to Doyle’s original stories, countless authors, including Mark Twain, have also tried their hand at writing Sherlock Holmes stories, and the detective has featured in radio serials, films, and video games.

While enjoying Sherlock Holmes’ adventures, quality professionals can pick up tips for performing root cause analysis. Here are a few from several of Doyle’s stories.

The Copper Beeches

“‘Data! data! data!’ he cried impatiently. ‘I can’t make bricks without clay.’”

An all-too-common mistake during an RCA is to jump to conclusions before data are available. Often, this may be the result of prior experience, such as when a comparable failure has been seen before. Using prior knowledge is not itself a problem, but declaring the root case identified and moving straight to corrective actions is. There may be insufficient data on hand at the start of an RCA, but this can be remedied.

Are production data available from the time the failed part was produced? If so, check the data to see if anything out of the ordinary happened. Are there no data available? Then inspect the part as soon as it is available. Perhaps the part is being shipped from the customer and unavailable at the moment; in such situations, it may be prudent to attempt to re-create the failure under controlled conditions to gain a preliminary understanding of the issue.

Silver Blaze

“‘It is one of those cases where the art of the reasoner should be used rather for the sifting of details than for the acquiring of fresh evidence.’”

During a long and complicated RCA, there may be a point in time in which many reports have been collected, but the root cause remains unknown. This is the time to briefly stop and perform a deep review of the information currently available. It is possible that no one measurement or lab report provides a direct indication as to the root cause; however, a full review of all documents may provide a big-picture view with indications as to where the root cause may truly be.

If measurement data are available, they should be subjected to statistical analysis. For example, multiple reports with the measurement of 10 parts might not show anything of potential relevance; however, a trend may be present if the entire data set is analyzed.

The Sign of Four

“‘No, no: I never guess. It is a shocking habit, destructive to the logical faculty. What seems strange to you is only so because you do not follow my train of thought or observe the small facts upon which large inferences may depend.’”

Any “guess” should be based on the facts on hand. Is there any conclusion that would explain the many little details that are available? For example, one or two facts alone may not lead to any conclusion, but taken as a whole, the facts may contain the key to unlocking the root cause. Such a guess should be viewed as a tentative hypothesis; assume it is true for the sake of testing, and then test the hypothesis. It is OK to be wrong in such situations because the testing may yield new information that drives the investigation forward.

The Adventure of Black Peter

“‘Oh, yes, my dear Watson, I am perfectly satisfied. At the same time, Stanley Hopkins’ methods do not commend themselves to me. I am disappointed in Stanley Hopkins. I had hoped for better things from him. One should always look for a possible alternative and provide against it. It is the first rule of criminal investigation.’”

A critical mistake during an RCA is to form a hypothesis and then seek to defend it instead of challenging it. It is not too difficult to produce evidence in support of an incorrect hypothesis; an inadvertent finger on the scale or accidentally pressing too hard on calipers can produce the desired, yet worse than useless, result. Strong belief in an incorrect hypothesis can lead to failing to identify the root cause, which in turn leads to a failure to implement the type of corrective actions needed to prevent a reoccurrence.

As Margaret Heffernan describes in her book, Willful Blindness: Why We Ignore the Obvious at Our Peril (Walker Books, 2011), it may be advantageous to look for evidence that refutes our belief. Seeking evidence that refutes our favorite hypothesis can result either in rejecting an incorrect hypothesis, or, if we fail to find contradictory evidence, we will have more support for the hypothesis. Subjecting a hypothesis to rigorous testing results in a much stronger and more robust hypothesis. A hypothesis that has survived many challenges is more likely to be a correct hypothesis.

The Valley of Fear

“‘I can see only two things for certain at present—a great brain in London, and a dead man in Sussex. It’s the chain between that we are going to trace.’”

Finding the connection between two seemingly unrelated events may lead to the root cause of the failure under investigation. For example, suppose a product is returned broken and rusted, although it should have survived several more years in service. A break and rust seem unrelated, but perhaps the rust weakened the material until the point where it broke. Or perhaps the break was due to a material problem; the break let in moisture, which in turn resulted in the rust. Following separate lines of evidence until they converge at the cause may also be helpful.

A Study in Scarlet

“‘You don’t seem to give much thought to the matter in hand,’ I said at last, interrupting Holmes’ musical disquisition. ‘No data yet,’ he answered. ‘It is a capital mistake to theorize before you have all the evidence. It biases the judgment.’”

Non-empirical, preconceived notions may lead to seeking confirmation of your hypothesis while disregarding contradictory evidence. Preliminary actions such as inspecting a suspect product can be taken before full information is available, but data are needed before concrete hypotheses can be formed.

Suppose a customer issues a complaint that “length of X is too long.” Parts can be inspected to prevent the shipment of out-of-spec parts, but more details are needed to address the root cause of the issue. Perhaps the machine was set up wrong, resulting in a long part. On the other hand, this could be a set-up part with telltale set-up damage on the end. More information is needed before a good hypothesis can be formed; in this case, a quick look at the part itself would reveal key information. Instead of blindly hypothesizing, get more information. In this situation, asking the customer a few questions could establish the part is 1.4 mm outside of specification, and the end looks damaged.

The Problem of Thorn Bridge

“‘It seemed a certainty when first it flashed across my mind in the cell at Winchester, but one drawback of an active mind is that one can always conceive alternative explanations which would make our scent a false one.’”

Logic alone will not solve the problem; people can always contemplate an alternative explanation that refutes the first idea, or come up with elaborate defenses for a favorite hypothesis. The only way to solve the problem is through the use of empiricism. Instead of second-guessing a hypothesis, go test it and see what the data support or refute.

The Adventure of the Empty House

“‘Ah! My dear Watson, there we come into those realms of conjecture where the most logical mind may be at fault. Each may form his own hypothesis upon the present evidence, and yours is as likely to be correct as mine.’”

We can all conjecture, but we can’t really know without verifying our conjectures. The good Dr. Watson would probably be the one who leads a team in creating a prioritization matrix to vote on the most popular root cause. The detective novel version of this would be, “The team voted; the janitor did it.” Voting on the most popular root cause is not necessarily wrong; vote on how well the evidence is explained by the hypothesis, prioritize the hypotheses based on the voting, and then begin an empirical evaluation of each hypothesis based on the priority it was assigned.

The Adventure of the Norwood Builder

“‘There, that’s enough,’ said Lestrade. ‘I am a practical man, Mr. Holmes, and when I have got my evidence, I come to my conclusions. If you have anything to say, you will find me writing my report in the sitting-room.’”

Detective Lestrade is the kind of quality engineer who starts filling out an 8D report at the first hint of a potential cause. His conclusion would be based on, “It looks like X, so it must be X.” Quality engineer Holmes would actually take the time to ensure the conclusions are indeed correct. Lestrade has a hypothesis, but how can he know if it is correct? The only way to do this is through an empirical evaluation. The objective is not necessarily to be correct; rather, it is to quickly reject incorrect hypotheses so that the investigator can move past the red herrings and arrive at the true root cause.

The Adventure of the Blue Carbuncle

“‘I can see nothing,’ said I, handing it back to my friend. ‘On the contrary, Watson, you can see everything. You fail, however, to reason from what you see. You are too timid in drawing your inferences.’”

Once observations have been made, decide what could account for them. This may not be the root cause, but it can be the push needed to move the investigation along. There may be multiple reasons a failure happened, such as when a diameter is out of specification. Was the tool worn? Was the correct drawing used? Did a scrapped part accidentally get shipped? Consider each possibility as a hypothesis and prioritize based on how well the hypothesis describes the issue, and how probable the cause is believed to be. Then evaluate each hypothesis in order of prioritization.

The Adventure of the Sussex Vampire

“‘One forms provisional theories and waits for time or fuller knowledge to explode them.’”

Sometimes a tentative hypothesis may be clear but not fully supported by the data. In such situations it may be necessary to collect more data before taking other actions. If there are no data to collect, perform an experiment to generate data. For example, attempting to re-create the failure under controlled conditions may provide a better understanding of an issue.

The Adventure of the Cardboard Box

“‘We approached the case, you remember, with an absolutely blank mind, which is always an advantage. We had formed no theories. We were simply there to observe and to draw inferences from our observations.’”

It is best to begin the investigation without preconceived notions. Deciding that you know the root cause before data are available leads to a risk of inadvertently overlooking key details or ignoring contradictory evidence. It is natural to think you know the cause if the failure looks remarkably similar to one that has happened repeatedly in the past; however, this situation may be different, so you must look at the evidence before deciding on a root cause.

The Boscombe Valley Mystery

“‘There is nothing more deceptive than an obvious fact,’ he answered, laughing.”

The answer to the problem may be obvious, but is it the correct answer? Be sure that obvious hypotheses are tested as rigorously as the not-so-obvious ones.

The Adventure of Wisteria Lodge

“‘Is it beyond the limits of human ingenuity to furnish an explanation which would cover both of these big facts? If it were one which would also admit of the mysterious note... why, then it would be worth accepting as a temporary hypothesis. If the fresh facts which come to our knowledge all fit themselves into the scheme, then our hypothesis may gradually become a solution.’”

It may be necessary to form a working hypothesis that best explains the valuable evidence. This hypothesis may prove to be incorrect, but that is OK. Such a hypothesis is the beginning and not the end of the investigation. The working hypothesis should then be evaluated. Does the evaluation support the hypothesis? If not, it should be rejected. Does the hypothesis appear to be partially correct? If so, it can be revised. Did we learn something new during the evaluation? If so, perhaps we can use this new information to form a new working hypothesis.

The Adventure of the Devil’s Foot

“‘Let us walk along the cliffs together and search for flint arrows. We are more likely to find them than clues to this problem. To let the brain work without sufficient material is like racing an engine. It racks itself to pieces. The sea air, sunshine, and patience, Watson—all else will come.’”

There are times when it is best to take a break from the investigation and work on something different. This gives the mind the chance to work in the background. As Atsunori Ariga and Alejandro Lleras write in their article, “Brief and Rare Mental ‘Breaks’ Keep You Focused,” dropping the problem at hand and doing something different gives you a chance to refresh and regain your concentration.

The Adventure of the Blanched Soldier

“...‘When you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth. It may well be that several explanations remain, in which case one tries test after test until one or other of them has a convincing amount of support.’”

A difficult analysis may be greatly simplified by eliminating possible explanations. It may also be easier to confirm something is not the root cause than to find the actual root cause. Such an action results in a shorter list of suspects. Suppose there was a murder investigation with 10 potential suspects. The investigation would be much easier if we could conclusively prove that six of the suspects were out of town when the murder happened. We won’t have the name of the guilty party, but suddenly we have fewer candidates, and therefore fewer alibis to check out. The investigation has been simplified.

The Valley of Fear

“‘The temptation to form premature theories upon insufficient data is the bane of our profession.’”

From such a quote, one could be inclined to think Holmes was a quality engineer. Often, there is an individual who automatically jumps to a conclusion as soon as the investigation has started. They may even be correct, but we can only be certain if we look at the data first and then draw conclusions.

The RCA helix as an approach to root cause analysis

A systematic and structured approach is needed when performing an RCA, and the RCA helix provides a simple and easy-to-apply process for investigating quality failures. The concept is generic enough to apply to many types of issues ranging from inadequate service to the failure of manufactured product. This concept combines aspects of George Box’s iterative inductive-deductive process as described in Statistics for Experimenters: Design, Innovation, and Discovery (Wiley Interscience, 2nd ed. 2005) and W. Edwards Deming’s plan-do-check-act (PDCA).

The first step is to form a tentative hypothesis using the available data. More data should be collected if insufficient data are available. The hypothesis is then evaluated empirically and the results used to modify the hypothesis if it fails the evaluation. If the hypothesis passes, it should then be subjected to conformation testing before improvements are identified (see figure 1). The process starts over if the hypothesis is rejected, and information gained during the evaluation may be used to modify the hypothesis.

Figure 1: RCA helix. © 2016 Matthew Barsalou  

A cause-and-effect diagram should be used to consolidate multiple hypotheses. This is especially useful at the beginning of an issue, when many competing hypotheses must be evaluated. Displaying them graphically can also be helpful when brainstorming as a team the potential causes of a problem because the team can see all the ideas that have been presented. Figure 2 shows a simple cause-and-effect diagram for a murder mystery.

Figure 2: Cause-and-effect diagram based on “The Adventure of the Bruce-Partington Plans”

The various hypotheses are displayed in the cause-and-effect diagram, but it is unclear what actions should be taken, so the hypotheses should be transferred to a spreadsheet such as the one shown in figure 3.

Figure 3: Cause-and-effect spreadsheet. Click here for larger image.

Conclusion

An RCA should be driven by hypotheses that seek to explain the failure using both the evidence at hand, and new data generated though testing and evaluation. The use of a cause-and-effect diagram provides the team with a graphical depiction of the many hypotheses that are under consideration when analyzing a complex issue. Transferring the cause-and-effect diagram to a spreadsheet turns a graphical depiction of hypotheses into an action-item list with a summary of results, thereby making the issue easier to manage and providing a quick overview of results.

This article was originaly presented as a conference paper for the 2016 ASQ World Conference on Quality and Improvement.

About The Author

Matthew Barsalou

Matthew Barsalou is a statistical problem resolution master black belt at BorgWarner Turbo Systems Engineering GmbH. He is an ASQ-certified Six Sigma Black Belt, quality engineer, and quality technician; a TÜV-certified quality manager, quality management representative, and quality auditor; and a Smarter Solutions-certified lean Six Sigma Master Black Belt. He has a bachelor’s degree in industrial sciences, and master’s degrees in engineering, business administration, and liberal studies with emphasis in international business. Barsalou is author of Root Cause Analysis, Statistics for Six Sigma Black Belts, The ASQ Pocket Guide to Statistics for Six Sigma Black Belts, and The Quality Improvement Field Guide.