The engineer came into the statistician’s office and asked, “How can I compare a couple of averages? I have 50 values from each machine and want to compare the machines.”

The statistician answered, “That’s easy. We can use a two-sample t-test.”

“How would that work?” asked the engineer.

“We compute a t-statistic based on the difference between the averages, and if the p-value for the t-statistic is small, we’ll have a significant difference,” replied the statistician.

“So a small p-value means we have a big difference?”

“Not exactly. The size of the difference is always relative. It depends on the background variation. A significant difference is one that is detectable in spite of the background noise present. The actual difference may be either large or small. And a small difference may be of no practical importance,” said the statistician.

“So why not call it a detectable difference rather than a significant difference?”

“That’s just the language of statistics. We can’t afford to make things too clear because if we did, people might not seek our help.”

 …