‘There are three kinds of lies: lies, damned lies, and statistics.” I’m sure you’ve heard this most vile comment, which was popularized by Mark Twain, among others. This dastardly phrase impugns the reputation of statistics. The implication is that statistics can bolster a weak argument, or that statistics can be used to prove anything.

I’ve had enough of this expression, and here’s the rebuttal! In fact, I’ll make the case that statistics is not the problem, but rather the solution.

In my last article, “Understanding and Using Discrete Distribution,” we looked at different discrete distributions and how you can use them. This time, I’ll show you how to determine whether your data follow a specific discrete distribution.

In my last column, “Detecting the Signature of Information,” I showed how it’s possible to statistically assess the structure of a message and determine its capacity to convey information. We saw how my own words fit the patterns that are present in communications that are optimized for conveying information. However, these were fairly rough assessments to illustrate the fundamentals of information theory.

Statistics can be unintuitive. What’s a large difference? What’s a large sample size? When is something statistically significant? You might think you know, based on experience and intuition, but you really don’t know until you actually run the analysis. You must run the proper statistical tests to know what the data are telling you.

Even experts can get tripped up by their hunches, as we’ll see.