If you’re not a statistician, looking through statistical output can sometimes make you feel a like you’ve wandered into Alice in Wonderland. Suddenly, you find yourself in a fantastical world where strange and mysterious phantasms appear out of nowhere. For example, consider the T and P in your t-test results.
ADVERTISEMENT |
“Curiouser and curiouser!” you might exclaim, like Alice, as you gaze at your output.
What are these values, really? Where do they come from? Even if you’ve used the p-value to interpret the statistical significance of your results umpteen times, its actual origin may remain murky to you.
T & P: the Tweedledee and Tweedledum of a t-test
T and P are inextricably linked. They go arm in arm, like Tweedledee and Tweedledum. Here’s why.
When you perform a t-test, you’re usually trying to find evidence of a significant difference between population means (2-sample t) or between the population mean and a hypothesized value (1-sample t). The t-value measures the size of the difference relative to the variation in your sample data. Put another way, T is simply the calculated difference represented in units of standard error. The greater the magnitude of T (it can be either positive or negative), the greater the evidence against the null hypothesis that there is no significant difference. The closer T is to zero, the more likely there isn’t a significant difference.
Remember, the t-value in your output is calculated from only one sample from the entire population. If you took repeated random samples of data from the same population, you’d get slightly different t-values each time, due to random sampling error (which is really not a mistake of any kind—it’s just the random variation expected in the data).
How different could you expect the t-values from many random samples from the same population to be? And how does the t-value from your sample data compare to those expected t-values?
You can use a t-distribution to find out.
Using a t-distribution to calculate probability
For the sake of illustration, assume that you’re using a 1-sample t-test to determine whether the population mean is greater than a hypothesized value, such as five, based on a sample of 20 observations, as shown in the t-test output in figure 1.
1. In Minitab, choose Graph > Probability Distribution Plot.
2. Select View Probability, then click OK.
3. From Distribution, select t.
4. In Degrees of freedom, enter 19. (For a 1-sample t test, the degrees of freedom equals the sample size minus 1).
5. Click Shaded Area. Select X Value. Select Right Tail.
6. In X Value, enter 2.8 (the t-value), then click OK.
In figure 2 the highest part (peak) of the distribution curve shows you where you can expect most of the t-values to fall. Most of the time, you’d expect to get t-values close to zero. That makes sense, right? Because if you randomly select representative samples from a population, the mean of most of those random samples from the population should be close to the overall population mean, making their differences (and thus the calculated t-values) close to zero.
T values, P values, and poker hands
T values of larger magnitudes (either negative or positive) are less likely. The far left and right “tails” of the distribution curve in figure 2 represent instances of obtaining extreme values of t, far from zero. For example, the shaded region represents the probability of obtaining a t-value of 2.8 or greater. Imagine a magical dart that could be thrown to land randomly anywhere under the distribution curve. What’s the chance it would land in the shaded region? The calculated probability is 0.005712... which rounds to 0.006... which is... the p-value obtained in the t-test results!
In other words, the probability of obtaining a t-value of 2.8 or higher, when sampling from the same population (here, a population with a hypothesized mean of 5), is approximately 0.006.
How likely is that? Not very. For comparison, the probability of being dealt three-of-a-kind in a five-card poker hand is more than three times as high (≈ 0.021).
Given that the probability of obtaining a t-value this high or higher when sampling from this population is so low, what’s more likely? It’s more likely this sample doesn’t come from this population (with the hypothesized mean of five). It’s much more likely that this sample comes from different population, one with a mean greater than five.
To wit: Because the p-value is very low (< alpha level), you reject the null hypothesis and conclude that there’s a statistically significant difference.
In this way, T and P are inextricably linked. Consider them simply different ways to quantify the “extremeness” of your results under the null hypothesis. You can’t change the value of one without changing the other.
The larger the absolute value of the t-value, the smaller the p-value, and the greater the evidence against the null hypothesis. You can verify this by entering lower and higher t values for the t-distribution in step six above.
Try this two-tailed follow up...
The t-distribution example shown in figure 2 is based on a one-tailed t-test to determine whether the mean of the population is greater than a hypothesized value. Therefore the t-distribution example shows the probability associated with the t-value of 2.8 only in one direction (the right tail of the distribution).
How would you use the t-distribution to find the p-value associated with a t-value of 2.8 for two-tailed t-test (in both directions)?
Hint: In Minitab, adjust the options in step 5 to find the probability for both tails. If you don’t have a copy of Minitab, download a free 30-day trial version.
Comments
Why bother with t values
Patrick - I appreciate the humor and your ability to communicate effectively statistical information. Always informative.
Having said that, why do we bother teaching people what a t-value is? P-values are what are critical. If we spent our time teaching those who will never be statisticians what a t-value is, then we get eyeballs rolling in the back of their heads. Rather, every statistical test returns a p-value. I teach all my students, "All this stuff is what a statistician cares about. The p-value is what you should care about". For every test, I teach what statistics it is trying to assess difference and then we search out the p-value(s). You answer the hypothesis test question and then move on about the implications to the business problem. Whether a student knows a t-value is a ratio of mean difference/standard error or a F-test is a ratio of variances doesn't matter. What matters is the p-value. Nothing about the t-statistic is relevant to a business solution. Only the p-value.
Good question!
Thank you for reading the post and for commenting. You make a good point that you can simply use the p-value to interpret the statistical significance of your t-test results without bothering with the t value at all. And certainly, I'd be the first to admit it's possible to lead a rich, fullfilling, and perfectly happy life (and maybe even a *happier* life) never knowing what a t-value is!
So why bother to explain the concept of a t-value?
1) The t-value is in the t-test output. Some people like to know what everything in their output means. Sometimes people will be presenting their statistical results to an audience and someone in the audience will ask them, "what does that T in the results mean?" Of course, one could respond, "I don't know and it's not important anyway-- just use the p-value." But personally I'd rather be able to answer that question--and help others answer that question--if even in a very general, intuitive way. (Someone asked this very question about the t value on the Minitab Group on Linked In, by the way). Now, you're right--the response may cause some to "roll their eyeballs." But I don't personally believe that the rights of the eyeball rollers should supercede the rights of those with stationary eyeballs who want to know.
2) To truly understand the p-value, and to visualize its associated probability, there are concrete, visual thinkers (such as myself) who really need to see what it actually is in relation to a t distribution. Words are often abstract and imprecise signifiers of a statistical concept--and due to their ambiguity they can lead to serious misunderstandings. In fact, the p-value is notorious for being frequently misdefined using words. One small linguistic tweak can warp the meaning.Then you'll find yourself alone in a dark, dead-end alley, surrounded by an angry mob of statisticians...it won't be pretty end for you. But if you really know what something is, then you know what words you can and you can't use to accurately communicate it.
3) To demystify statistics. People without a background in stats are often intimidated by it. And some hate what they fear. That's really a shame. Because oftentimes what's behind all the smoke and mirrors, all the messy nomenclature, all the ugly formulas, all the ifs, ands, buts, is a very clear and intuitive concept. When people get it, they feel much more empowered, much less intimidated. So I guess I like helping people feel that way, if I can.
I'm not sure I'd agree that a focus on practical business interests and deeper knowledge of statistical concepts are incompatible goals. After all, the t-test itself was developed by a brewer for Guiness (Gosset), whose statistical know-how allowed him to successfully create and apply the analytical tools he needed to ensure a quality product (and, hence, a sucessful business).
Easy-to-use statistics packages are making statistical data analysis accessible to more people. That's a good thing. But it also means more people are performing analyses who aren't really certain what their results mean--and what they don't mean. And often they don't know what they don't know, which can lead to poor business decisions.
So every once in awhile, I like to invite readers to journey a little deeper into wilds of StatLand. Hopefully some will find the trip insightful...or at least mildly amusing!
Add new comment