Book Review: Data Sanity

If you’re involved in quality in any healthcare field, the second edition of Data Sanity is a must-read

Published: Tuesday, September 18, 2018 - 12:00

I must admit, right up front, that this is not a totally unbiased review. I first became aware of Davis Balestracci in 1998, when I received the American Society for Quality (ASQ) Statistics Division Special Publication, Data “Sanity”: Statistical Thinking Applied to Everyday Data. At the time, I was still working in the Navy’s Total Quality Leadership (TQL) schoolhouse, having spearheaded the statistical process control course we taught to Navy Quality Advisors (TQL’s “Black Belts”). I was struck at the time by the apparent depth of his commitment to statistical thinking and data-based decision science, his iconoclastic style, and the simple, clear examples he used to illustrate the points he made in that publication.

A few years later in Minnesota, I was attending the first of many Minnesota Quality conferences, and Davis was a speaker there. Some colleagues introduced us, and I had a chance to hear him speak. If you have not had that opportunity yet, and it should come your way, it’s well worth your time. I can guarantee you will find it to be entertaining and a superb learning experience. We have corresponded over the years, and I have used some of the examples from that early Statistics Division publication in many of my own classes and articles. I count him both a friend and a mentor. He is one of two authors in Quality Digest that I always take the time to read as soon as I see articles by them featured in my inbox (Don Wheeler is the other), and I had bought and read the first edition of Data Sanity, the book.

If you are involved in quality in any of the healthcare fields, the second edition of Data Sanity is a must-read, and should be well-tabbed and dog-eared in a prominent place in your reference library. From the forward (written by no less a healthcare quality luminary than Dr. Donald Berwick) to the appendices, the work is packed with knowledge gleaned from Davis’ long experience in the healthcare field and includes many examples from that field that help drive the concepts home. If you are new to quality in the healthcare field, this book should be on your shelf as a go-to reference for any of the concepts you will be learning. If you’re not in healthcare but actively pursuing quality or continuous improvement, this book will still be a valuable reference... the statistical concepts presented transcend the examples used for illustration.

For instance, his example (carried over from his original 1998 essay) of the three clinics is a classic that should be included in every statistical process control (SPC) training. In that example, the clinic’s records all test well for normality and can’t be differentiated using ANOVA or hypothesis testing, but show up as entirely different from each other when plotted in time order. That example is a clear illustration of the weakness of statistical techniques for enumerative studies, and the real need for more emphasis on analytic studies—both in academia and on the job. Chances are good, by the way, that if you’ve taken stats classes from most universities, they have almost exclusively focused on enumerative tools and techniques.

His major emphasis in the quality tools sections of the book is not statistical theory, however. It’s more about practical tools that can best communicate what’s going on in our day-to-day processes, so we can make them work better. Davis emphasizes collecting the right data for the right reasons, getting the operational definitions and the plan right, and knowing in that plan how you are going to analyze the data, before ever collecting a number. As someone who’s been told time and time again that “we have tons of data” only to find that in those tons there isn’t an ounce of useful insight, I appreciate this emphasis. Data are often collected without anyone ever asking, “What question do we want to answer?” Davis hammers on this question again and again, throughout the book. He takes this simple (but rare) approach to his discussion of statistical tools, as well.

Over and over, he emphasizes the importance of “first plotting the dots” so we can see what’s going on. In Davis’s view, one of the most useful tools in your repertoire is the run chart or line chart, a simple running record over time. Yes, he does eventually get to calculating and plotting limits, but only after many examples that demonstrate the efficacy of a run chart in detecting out-of-control conditions. The book is much more about simple tools that you can learn quickly and put to work right away. The idea is to get people thinking about their processes, collecting good data on important aspects of them, and then plotting the dots to spark insight and conversation about what to do next. This approach makes some of the most important aspects of statistical thinking accessible to any manager or practitioner.

The book is not just about statistical thinking or tools, however. As most of us who ever tried to improve an organization have found, Davis knows that “the soft stuff is the hard stuff.” About half of the book is dedicated to leading the transformation. He includes chapters on the belief system leaders need, a handbook to help leaders get started, some team philosophy, and cultural education. Davis offers substantial nods to more recent change management gurus such as John Kotter and Peter Block, and includes a lot of foundational material, like John Grinnell’s 7-Level Quality Pyramid; but he bases a great deal of his culture change material on the cognitive-therapy work of Albert Ellis and its associated ABC model. This model says that an activating event (A) is filtered through individuals’ beliefs about the event (B), and that sequence triggers the observed behavior or consequences of the event (C). Davis makes a very convincing case that if we extend the model to the organizational results that derive from those consequences, this model explains the failure of a lot of culture change initiatives.

In this simple model, the current culture acts on the current way to do things, accepted and practiced throughout the organization (A1); this is filtered through the belief systems of the individuals in that culture (B1). These belief systems have been long tested and reinforced by observations regarding the consequences of those actions (C1). Managers sometimes convince themselves that if they create a new rule or activating event, beliefs will immediately change and drive people to change behaviors, leading to the organization’s reaping the desired improved results. Davis describes what often happens:

“The executives go off-site for two or three days and come back excited with yet another new version of the mission statement. The first statement usually has something to do with the value of respect, such as, “We shall be a culture of respect where it will be safe for any employee to confront any other employee appropriately to uphold the values of the organization.

“....The executive team has deluded itself that it has indeed created an A2 activating event via an announcement that is going to excite the culture and drive B2 beliefs that yield C2 consequential behaviors and, as a result, desired R2. The workforce embedded in this culture, however, has seen all this before and knows it is just another A1. The executives sense the boredom and then say, “But this time, we really mean it,’ and the workforce says to itself and in conversation with each other, ‘We’ve heard that before, too,’ resulting in R1.”

Davis goes on to make the point that what usually happens is that those who do test the new A2 usually find that they are once again met with the old C1 consequences, which just reinforces the B1 beliefs. A wonderful example of physicians bullying nurses makes this point very clear. In one case, the physician gets away with it (even after the new “respect” mission statement) and the nurse is reprimanded—this reinforces the B1 belief system. Davis discusses a situation in which a hospital created “Code 13.” When a nurse was being browbeaten by a physician, anyone close to the situation would send “Code 13” and the location throughout the hospital via the PA system. This was an open invitation for every supervisor hearing the code to come observe the situation with folded arms, staring the physician down. This actual C2 consequence very shortly built the new B2 belief system that got the hospital new C2 behaviors (physicians stopped bullying the nurses), and the R2 results intended by their A2 activating event.

I would be remiss in my review, however, if I didn’t state that for all its many virtues, this book is not an easy read for the uninitiated. The easy-to-read, accessible parts are all there, but in my humble opinion, Davis’s editors did not do him any favors when the book was in pre-publication. The bones are good, and the information is all well worth the read, but it is uneven in places and doesn’t necessarily flow well or tie together coherently. The few times Davis dips into more arcane statistical concepts (e.g., Fisher’s Exact Method), they seem to come out of nowhere, with little explanation, and would confuse (and might turn off) those with less technical backgrounds. Those brief excursions would be better expanded on and added to a technical appendix (for those with the background and scientific curiosity who want to look “under the hood”).

Having said that, though, I reiterate what I said in the first paragraph. If you are a hospital administrator, you absolutely should read this book; if you are in hospital leadership and are trying to improve the quality of patient care and the quality of your business processes, this is a must-read. For anyone else in the quality profession, I can guarantee you will see things differently after reading this book than you did prior to reading it, and you will no doubt be glad you added it to your reference library.

About The Author

Rip Stauffer

Rip Stauffer uses his extensive experience in total quality and Six Sigma to educate and counsel at all career levels with specific experience in government, manufacturing, medical devices, financial services, and healthcare organizations. A senior consultant at MSI, and CEO of Woodside Quality LLC, Stauffer is an ASQ senior member and statistics division member, a certified quality engineer, a manager of quality and organizational excellence, and a Six Sigma Black Belt and Master Black Belt. He also is an adjunct faculty member at Walden University, teaching graduate and undergraduate business statistics courses and international business courses.