When presented with a collection of data from operations or production, many will start their analysis by computing descriptive statistics and fitting a probability model to the data. But before you do this, there’s an easy test that you need to perform.

ADVERTISEMENT |

This test will quantify the chances that you can successfully fit *any* probability model to your data. By using this simple test to examine the assumptions behind all probability models, you can avoid making serious mistakes. This column will illustrate this test and explain why it works.

### Example 1

The data in Figure 1 are 200 sequential values obtained over time from a single process. The average is 12.86, and the standard deviation statistic is *s* = 3.462. The histogram is shown in Figure 2.

We might be tempted to immediately test these data to see if they’re consistent with a normal probability model. However, before we try to test for a lack of fit with a specific probability model, it’s instructive to test the data to see if they’ll fit *any* probability model.

To do this, we use a test developed and published by Brenda Ramirez and George Runger in 2006. The test statistic compares two different measures of dispersion. The first of these is the standard deviation statistic computed using all of the data. The second of these is the average of the differences between successive values. Here, the time order is given by reading the data in rows. Figure 3 lists the 199 successive differences.

The average of these 199 successive differences is 2.42.*Average successive difference = 2.42*

The Ramirez-Runger test statistic is:

This statistic will be approximately distributed according to an F-distribution having [*n–*1] numerator degrees of freedom and [0.62*(*n–*1)] denominator degrees of freedom. The probability of exceedance (the *p*-value) for this test statistic may be obtained in most spreadsheet programs. In Excel we use the formula:

= FDIST(test stat., num. d.f., denom. d.f.)

= FDIST(2.60, 199, 123)

And Excel will return the probability of exceedance for this statistic. The *p*-value for our test statistic of 2.60 is:

FDIST(2.60, 199, 123) = 0.000 000 013

This value may be interpreted as the probability that *any* probability model can be found that will actually describe the process outcomes. With 13 parts per billion as your chance for success, you can save yourself a lot of time and trouble by simply not trying to fit a probability model to these data. So, what do you do instead? This will be discussed below.

### Example 2

Once again, the data in Figure 4 are 200 sequential values obtained over time from a single process. The average is 10.11, and the standard deviation statistic is *s* = 1.792. The histogram is shown in Figure 5.

So, what are the chances that we can fit a probability model to these data? These data were written in rows, and the 199 successive differences listed in Figure 6 have an average of 1.905.

Thus, the Ramirez-Runger test statistic is

With 199 and 123 degrees of freedom, our test statistic of 1.126 has a *p*-value of:

= FDIST(test stat., num. d.f., denom. d.f.)

= FDIST(1.126, 199, 123) = 0.238

Thus, with a *p*-value of 0.238, it’s not unreasonable to think that you might find some probability model that would fit these data and describe the process outcomes.

### How this test works

Unlike all other tests of fit, the Ramirez-Runger test takes the time-order sequence of the data into account. Why is this important? It goes back to the definition of a probability model.

A probability model is a limiting property for an infinite sequence of random variables that are independent and identically distributed. And a sequence of independent and identically distributed random variables will display the same amount of variation regardless of whether the computation is carried out globally or sequentially.

So the Ramirez-Runger test compares a global estimate of dispersion with a sequential estimate. The global estimate is the usual standard deviation statistic. The sequential estimate is the average of the successive differences divided by its bias correction factor, 1.128. The ratio of these two estimates, when squared, will be approximately distributed according to an *F*-distribution with [*n*–1] and [0.62*(*n*–1)] d.f. The probability of exceedance, or *p*-value, for this test statistic quantifies how reasonable it is to consider the two estimates as being equivalent.

When they’re not equivalent, it’s unreasonable to assume that the data came from a sequence of independent and identically distributed random variables. And when the data show evidence that they didn’t come from a sequence of independent and identically distributed random variables, the notion of a probability model vanishes. At this point, you should abandon all hope of ever fitting a reasonable probability model to your data.

While a large *p*-value doesn’t guarantee that a probability model exists, or tell you which model might work, a small *p*-value provides a red light to save you from wasted effort.

One caveat is needed. Since the Ramirez-Runger test depends upon the time-order sequence of the data, it should always be used with data in their native ordering. Specifically, it can’t be used on data that have been rearranged into a ranking where the values are placed in ascending or descending order.

### So what happens next?

If no probability model exists that will fit your data, what do you do next? If the data have been collected in such a way that they should be homogeneous, then the lack of homogeneity indicated by the Ramirez-Runger test is a signal that some unknown cause is changing your process outcomes. So the question becomes, “How can we identify this unknown cause?”

Aristotle told us that to identify the causes that affect our system, we’ll need to look at those points in time where the system changes. And this simply can’t be done with summary statistics that ignore the time-order sequence within the data. All techniques built on these symmetric statistics implicitly assume that the system isn’t changing in any manner.

Once we know that some unknown cause is changing our system, we must shift from using complex analysis techniques to a more fundamental approach. Rather than trying to compute our way around the lack of homogeneity, we need to find out when and why the process is changing. The primary technique for doing this is a process behavior chart.

As soon as we plot the *XmR* chart for Example 1, we see why no probability model exists for these data. The process is meandering around, with frequent changes in location.

These data are not sufficiently well-behaved to be represented by a single probability model. However, that doesn’t keep your software from drawing a bell-shaped curve over the histogram as in Figure 8.

Ramirez-Runger tells us that, regardless of how pretty Figure 8 may look, there are only 13 chances in a billion that it’s correct. A more realistic representation of the process that produced the data for Example 1 would look something like Figure 9.

Figure 10 shows the *X* chart for Example 2. There, we see no evidence of a lack of homogeneity, which is consistent with the Ramirez-Runger test *p*-value of 0.238. This process shows no evidence of changes during the period covered. The average of 10.1 and standard deviation of 1.8 properly characterize the process outcomes.

### Summary

The Ramirez-Runger test provides a simple numerical test to quantify the lack of homogeneity contained in a data set. Since homogeneity is implicitly assumed by virtually every statistical analysis technique, the Ramirez-Runger test should be a starting point for your analysis.

When the *p*-value for the Ramirez-Runger test is small, you’ll know that you can’t fit a probability model to your data. Neither can you estimate process parameters nor compute confidence intervals, test hypotheses, or use any other statistical analysis techniques. Rather, you’ll need to use a more fine-grained approach, looking for the assignable causes of exceptional variation within the data themselves. And, of course, this will lead to the use of process behavior charts.

When the *p*-value for the Ramirez-Runger test gets larger than 5% or 10%, you’ll have an inconclusive result. The Ramirez-Runger test uses both global and local measures of dispersion, but the fact that these are both based on summary statistics means that they can miss some signals of a lack of homogeneity within the data. Just because Example 2 had a Ramirez-Runger *p*-value of 0.238 didn’t mean that the *X*-chart in Figure 11 was going to show a predictable process.

So, while the process behavior chart remains the final arbiter of when a process is operated unpredictably, the Ramirez-Runger test provides a computation-based alternative that can keep you from making serious mistakes. If you’re not already starting your analysis with a process behavior chart, then the Ramirez-Runger test is the test to use before all other tests.

## Comments

## Errata

You mention Figure 9 twice but mean figure 8 the first time!

Great article. I like to be able to have constructive discussions with colleagues forcing over complicated stats on ppl. And this wil help…. ;-).

## Thanks for pointing out the…

Thanks for pointing out the error. That has been fixed.

## Tie Breaker

I appreciate discovering basically a test where H0 might be that "the process is statistically stable."

I also practice and teach never to compute capability until after observing a "process behavior" chart, because special cause can distort the data to "impersonate" a distribution.

But this test now allows me to handle times when the eyes and brain can't quite nail down a conclusion - when the special cause looks moderate.

Thank you!

## Fantastic

Love it! It's not a test I could see that you would ever bother to use but it gives even more credence to the power of Process Behavior Charts. Very cool.

I assume this is the source? Quantitative Techniques to Evaluate Process Stability, June 2006, Quality Engineering 18(1):53-68

## Add new comment