Where Does the Success-Testing Formula Come From?

Simple formula, not-so-simple answer

Published: Wednesday, December 16, 2015 - 11:55

The planning of environmental or reliability testing becomes a question of sample size at some point. It’s probably the most common question I hear as a reliability engineer: How many samples do we need?

Also, when evaluating supplier-run test results, we need to understand the implications of the results, again based on the number of samples in the test. If the supplier runs 22 samples without failure over a test that replicates the shipping set of stresses, then we need a way to interpret those results.

We often use success testing (no expected or actual failures during the testing) to minimize the number of samples required for a test and still show some level of confidence for a specified reliability level. The basis for success testing is the binomial distribution. The result of the applied stress results in the product either working or not. Binary results.

Recently I received a request to explain where the success-testing sample size formula comes from, or it’s derivation. First here’s the formula:

Where, C is confidence and R is the lower limit of the reliability.

Thus if planning a test and you wanted to demonstrate the product was at least 90-percent reliable with 90-percent confidence, you would need to evaluate 22 units for the equivalent of a lifetime of use. In the shipping example above, the vendor’s testing shows the product would survive the shipping experience with 90-percent reliability with 90-percent confidence, for example.

It’s a simple formula. So, back to where it comes from.

Success-testing sample size formula derivation

C. J. Clopper and E. S. Pearson wrote a paper in 1934 detailing how to determine a confidence interval for a binomial distribution. Let’s start with their result. They used the binomial distribution cumulative distribution expression set equal to 1 minus the confidence, or alpha as many would state it.

C is the confidence, R is the lower limit of reliability give that confidence, n is the number of samples evaluated, and r is the number of failures experienced in the test.

Setting r = 0 is saying there will be or have been no failures, thus a success test. The first two terms after the summation reduce to 1 leaving:

Take the natural log of both sides brings n out of the exponent:

And rearrange to get the sample size formula:

Hope that helps explain where this sample size formula comes from.

About The Author

Fred Schenkelberg

Fred Schenkelberg is an experienced reliability engineering and management consultant with his firm FMS Reliability. His passion is working with teams to create cost-effective reliability programs that solve problems, create durable and reliable products, increase customer satisfaction, and reduce warranty costs. Schenkelberg is developing the site Accendo Reliability, which provides you access to materials that focus on improving your ability to be an effective and influential reliability professional.