Our PROMISE: Our ads will never cover up content.
Our children thank you.
Davis Balestracci
Published: Tuesday, April 14, 2015 - 17:59 Good data collection requires eight questions to be addressed. The first four involve designing the collection. Most of these issues were discussed in my last column. To summarize: 2. What method(s) will be used for the analysis? 3. What data will be collected? 4. How will the data be measured? Because human variation can compromise the quality of data and render any subsequent analysis virtually useless for project purposes, these questions help to anticipate and minimize its effects. In any process, the collection process itself has the classic six sources of process inputs: people, methods, machines, materials, measurements, and environments. It’s vulnerable to outside variation that compromises its quality. Once you’ve answered the four questions above, you’ve got a great data design. However, one can’t ignore the need to formally plan for data consistency and stability. Especially in this area, the unintended ingenuity of human psychology to sabotage even the best of designs is a most formidable force to overcome. Despite the best design, many problems lurk in the ensuing collection process. It’s only when the human variation in the collection process is also minimized that data can be trusted enough to allow appropriate action on the process being improved. The remaining four questions: 6. Where will the data be collected? To address these next two questions, one must consider even deeper logistics of obtaining the data. For this, understanding the data collectors and their environment is crucial. 8. What training is needed for the data collectors? Seriously consider involving some of the data collectors in the design of any data collection forms. Do the forms allow efficient recording of data with minimum interruption to people’s normal jobs? To allow this, there are further issues for improvement facilitators to keep in mind: Then be sure to answer the following questions that the collectors will naturally ask: Finally, test the data collection process to expose most remaining sources of inappropriate and unintended variation and attempt to remove them. Pilot the forms and instructions on a small scale. After the (brief) test, revise the data collection processes if necessary, using input from the collectors: When possible, sit with the people collecting the data and observe them. As new people enter the collection process, have someone who knows what to do watch the first attempts of novice data collectors. You’re now as ready as you’ll ever be to begin your formal data collection. But again, be forewarned. The human factor is always lurking—via W. Edwards Deming’s funnel experiment rule No. 4: unintentional “random walk” from original intentions. Throughout the collection, audit for missing data, unusual values, and possible bias. Occasionally observe the actual collection to continue improving measurement consistency and stability. And don’t be surprised by anything you observe. Get the respect you deserve. People involved in the actual data collection should have confidence that you and your team know exactly what you’re asking and looking for—and that you’re actually going to do something with the information. People perceive their jobs as already taking up at least 100 percent of their time; they’re doing you a favor, so make it easy for them. Make sure the data collection results ultimately make their work lives easier as well. If you do, your future projects certainly will be easier, too. Quality Digest does not charge readers for its content. We believe that industry news is important for you to do your job, and Quality Digest supports businesses of all types. However, someone has to pay for this content. And that’s where advertising comes in. Most people consider ads a nuisance, but they do serve a useful function besides allowing media companies to stay afloat. They keep you aware of new products and services relevant to your industry. All ads in Quality Digest apply directly to products and services that most of our readers need. You won’t see automobile or health supplement ads. So please consider turning off your ad blocker for our site. Thanks, Davis Balestracci is a past chair of ASQ’s statistics division. He has synthesized W. Edwards Deming’s philosophy as Deming intended—as an approach to leadership—in the second edition of Data Sanity (Medical Group Management Association, 2015), with a foreword by Donald Berwick, M.D. Shipped free or as an ebook, Data Sanity offers a new way of thinking using a common organizational language based in process and understanding variation (data sanity), applied to everyday data and management. It also integrates Balestracci’s 20 years of studying organizational psychology into an “improvement as built in” approach as opposed to most current “quality as bolt-on” programs. Balestracci would love to wake up your conferences with his dynamic style and entertaining insights into the places where process, statistics, organizational culture, and quality meet.A More Robust ‘P’ for Any PDSA Cycle
An excellent plan is essential
1. Why collect the data?
• Is there a clear objective for this collection?
• This should be known even before one piece of data is collected.
• What specific process output(s) do you wish to capture?
• If data are needed from customers, patients, or staff, what is the process for choosing the individuals from whom to obtain the needed data?
• How will you evaluate any output to obtain a consistent number, regardless of who measures it?
• In the case of measuring counts, is the threshold between a nonevent (x = 0) and an event (x = 1) clear (e.g., is Pluto a planet or isn’t it?).
• Remember, there is no “true value,” and it depends on the specific objective.
• Does it allow you to take the actions you wish to take?Four more questions: reducing human variation in collection logistics
5. How often will the data be collected?
• Will any collector’s perception of how often you want the data recorded be a barrier to cooperation, risking incomplete data?
• Are you asking for more data than are needed for the current objective?
• Where is the best leverage point in the process to collect the data?
• Where will job flows suffer minimum interruption?
7. Who will collect the data?
• What effect will their normal job have on the proposed data collection?
• Is this person unbiased, and does she have easy and immediate access to the relevant facts?
• Will the collector’s perception of how the data are going to be used be a barrier to cooperation, or risk data being incomplete?
• What would minimize the risk of data being incomplete?
• Exactly how will the data be recorded?Then there remains the logistics of actually recording the data
• Reducing opportunities for error
• Designing traceability to collector and environment
• Making the data collection form virtually self-explanatory, professional looking, and simple
• Formally training all the people who will be involved. Have a properly completed data collection form available to use as reference. As part of this training, reduce fear by discussing the importance of complete and unbiased information.
• “What is the purpose of the study?”
• “What are the data going to be used for?”
• “Will the results be communicated to us?”
• Do the processes work as expected?
• Are the collection forms filled out properly?
• Do people have differing perceptions of operational definitions?
• Are the processes as easy to use as originally perceived?
Our PROMISE: Quality Digest only displays static ads that never overlay or cover up content. They never get in your way. They are there for you to read, or not.
Quality Digest Discuss
About The Author
Davis Balestracci
© 2023 Quality Digest. Copyright on content held by Quality Digest or by individual authors. Contact Quality Digest for reprint information.
“Quality Digest" is a trademark owned by Quality Circle Institute, Inc.