An article titled “Sharp Drop in Firefighter Fatalities in 2009” appeared in a safety trade journal recently. For the first time in three years, it said, on-the-job firefighter fatalities dropped below 100. The article went on to say that the 82 firefighter deaths in 2009 were substantially below the 10-year average of 98 and down from 105 in 2005.
ADVERTISEMENT |
What prompted me to write this article was the following quote: “While a drop over one year isn’t enough to show a trend… [w]e are hopeful that we will continue to see fewer and fewer firefighter fatalities over the next 10 years.” I was reminded of the old song by Dusty Springfield, “Wishin’ and Hopin’,” in which a young woman sadly admits she has to do more than “wishin’ and hopin’ and thinkin’ and prayin’” to be his. She has to do something differently.
The article referenced a National Fire Protection Association (NFPA) report, and I just had to pull it up and look at the data myself. What I found was 45 pages of “this number is greater than that number” and “that number is less than average.” Every change from year to year was treated as if it was due to special cause variation. Lots of bar charts and pie charts categorized the firefighters’ ages, types of duty, cause of injury, and nature of injury. Good information, perhaps, but I got the feeling the report was written to impress the reader with volume and charts—the wrong charts. Although the report did not contain much data from before 2009, it did contain data for firefighter deaths from 1979–2009 in the form of a bar chart. Unfortunately, data analysis stopped there.
As Davis Balestracci and Donald Wheeler often say, “Plot the data!” So I did. The result was the process behavior chart seen in figure 1:
Connecting the dots
Data that are in time order have an extra dimension that must be taken advantage of. Time-ordered data allow us to “hear” the voice of the process. We must learn to “listen” with the aid of process behavior charts. The chart clearly shows us that firefighter fatalities have indeed decreased, but not in 2009. The so-called “sharp drop” from 2008 to 2009 was due to common cause variation.
However, something significant did happen to reduce firefighter fatalities during the period 1991–1994. The NFPA could benefit by investigating what changes occurred to reduce firefighter deaths from an average of 123 per year to 99. Were there changes in technology, training, health screening? What about 1992 and 1993? These two years should certainly raise some questions. Whatever caused the decrease in firefighter fatalities, it must be identified and continued or enhanced. Otherwise, we can expect that somewhere between 77 and 121 firefighters will die on the job in 2010 and beyond.
A complete and proper analysis of the data is beyond the scope of this article. However, I would like to show one more step in the analysis to help make a point. The NFPA report analyzed the data in terms of fatalities per 100,000 structure fires (see figure 2). A graph was shown with the caption, “While the number of structure fires and deaths at structure fires has dropped, the rate of firefighter deaths at structure fires has not followed the same pattern.”
Another process behavior chart shows that this is not the case. The rate of firefighter deaths decreased twice: once around 1983–1984 and again, more significantly, during 2000–2001. Both these signals of improvement have been missed by the NFPA. Knowledge of what caused these shifts may help save lives in the future.
Ignoring the forest for the trees
Although I have tended to criticize the NPFA report and its lack of proper statistical thinking to gain proper insight for performance improvement, there is one nugget of information that warrants further investigation. According to the report, about 40 percent of firefighter fatalities are sudden cardiac deaths (e.g., heart attacks or strokes), by far the largest category, which underscores “the need for wellness-fitness programs and health screenings for firefighters across the nation.” The NFPA also reports that 44.7 percent of 2,000 firefighters tested in 2005 were found to be obese. Of 1,650 firefighters tested in 2008, 47.5 percent were determined to have high overall coronary risk ratings.
These data may not be scientifically valid—due to sampling protocol—but they are strong evidence that our nation’s firefighters may not be fit for such strenuous duty. Obviously, fitness should be a focus area for the prevention of firefighter deaths on the job. The second largest category of firefighter deaths is vehicle accidents while responding to or returning from alarms.
The only reason to collect data is to turn the information into knowledge (i.e., insight) as a basis for action. We are constantly bombarded by the media with statistics that are misused and abused to tell a story that the author wants to tell. As a result, the insights needed to gain improvement often are lost. In this case, the impression was given that firefighter deaths are trending downward. They are not. Feel-good analysis, coupled with “wishin’ and hopin’” that better times are ahead, will get us nowhere. Our nation’s firefighters deserve better.
Comments
Operational Definitions
Nice article. It amazes me how the media continues to get in wrong (numerical naivete).
One very important thing that is often over-looked when analyzing time series data is the need of Operational Definitions (how the count or measurement is defined). In fact, according to Deming, Shewhart considered this to be more important than his invention of the control chart. I know with the number of adminstrative metrics that are used to compare data from different months, departments and plant sites etc, the importance of OD can't be emphasized enough in my view. For example, if a fire fighter dies from a stroke or heart attach 3 days after fighting a fire, is that included as a job related fatality? I often wonder when I see evidence of improvement, whether the measurement process has changed.
Rich
Another theory of cause
I recently saw results of a study that ranked firefighting as one of the most "mentally" stressful jobs - again a condition that could be contributing to cardiac problems along with or instead of physical shape. I liked your article and the use of the data to strive for improvement instead of just being used to make it "sound" like things are getting better for unstated reasons.
Add new comment