by Dirk Dusharme
Surveys of any type must be viewed with a certain amount of skepticism, or at least with a very clear idea of their limitations. (This from a magazine—and an editor—that does three or four surveys a year.) The problem with surveys isn't so much the data that are collected, or even how they're collected (within certain limitations, of course). The problem is how the results are interpreted. Even a fairly simple survey question such as "What's your favorite color?" is fraught with problems when it comes time to interpret the results. If you're a paint manufacturer trying to decide what the season's hot new colors are, the analysis that 65 percent of people answered "blue" means almost nothing. What color blue—baby blue, sky blue, cobalt blue, or Delft? Including color swatches with the survey might help, but still has problems. Are the swatches being viewed in daylight, incandescent light, fluorescent light, or candlelight? What percentage of the people viewing the swatches are colorblind?
When it comes to interpreting the results of salary surveys, including this one, the problems are even more complex—which is probably why most magazines or organizations that do salary surveys don't delve too deeply into the results. That's also one reason why we always encourage readers to look at the various ways in which data are presented: salary by region, salary by job title, salary by education, and so forth. The best way to look at salary survey data is to look for trends rather than absolute numbers. The one possible exception to this is salary data that come from a local trade organization, credit union, etc., that collects salary data from similar local businesses and job titles as yours. Actual salary figures may then have some real meaning.
For this survey, there are certain trends that we have commented on year after year and will continue to do so, i.e., how education and gender affect your salary, salary differences by region or by industry, and the effect of industry certifications. This year, largely due to a letter from a reader, we are going to delve a little more deeply into our data and take you along for the ride.
The majority (read anecdotal) opinion on the existence of salary differences due to gender is that a problem does exist. Many people cite a 2004 U.S. Census Bureau statistic that women earn 77 cents for every dollar a man makes.
But not everyone agrees. One Quality Digest reader, commenting on an April 3, 2007, Washington Post article ("A Bargain At 77 Cents To a Dollar," Carrie Lukas) said this: "… that women make 77 cents for every dollar that men make is both meaningless and meretricious, being an example of Simpson's paradox in statistics (also known as the problem of the missing parameter). It is impossible to meaningfully compare two generic high-level groups as done in this op-ed. Actual surveys show that for the same job responsibility, industry, geographical region, education, age, experience, travel, overtime, benefits, corporate performance, etc., women actually make one percent more than men."
The reader, although mistaken about the intent of the op-ed, is correct about how the data were used (or misused). The U.S. Census Bureau statistic clearly says that the data compare median earnings for full-time, year-round workers over 15 years old—hardly an apples-to-apples comparison of male vs. female salaries in a specific industry, job type, region, etc.
One of many studies that tend to support the reader's assertion is a July 1999 study released by the National Science Foundation: "How Large Is the Gap in Salaries of Male and Female Engineers?" This study shows that the salary gap between male and female engineers, when controlled for education, sector, region, specialty, degree, and years of experience, dropped from 13 percent to just 2 percent. Other studies have shown similar results. Entire books have been written on the subject, including Warren Farrell's contrarian Why Men Earn More: The Startling Truth Behind the Pay Gap—and What Women Can Do About It (AMACOM, first edition, January 2005).
We have tried to address this issue ourselves in past salary surveys, attempting to cut the salary pie in a variety of ways. Our results always showed that no matter how you slice it, men make more.
Challenged by our reader, we decided to take another crack at it. We were aided this year by having a very large number of respondents. Large numbers help especially when there is no attempt at proper sampling. As in all our surveys, we invited the 65,000 magazine and online readers for whom we had e-mail addresses to take the survey and then did what we could to ensure the largest return rate possible. Because there has been relatively little variation in survey demographics from year to year, we are confident that the 4,300 respondents are a good representation of our readership. Our goal is to look at the data in the same way a casual, nonstatistically minded reader might. By following along with us, you might recognize some of the traps that you can fall into simply by looking at a table. At a later date we will turn over the data to more knowledgeable people than ourselves to perform more detailed analysis.
We encourage you to follow our reasoning and send us your opinions as to how we are parsing the data and the pitfalls of our interpretations.
As a baseline, we first looked at salaries exactly the same way we do every year, focusing first on salary by education. Figure 6a on page 52 shows this year's results, and figure 2 below shows how they compare to previous years.
• Controlling for hours worked and experience. One of the arguments for why women may make less than men is that they work fewer hours to care for young families and may enter the workforce as full-time workers later in life than men. So our first stab at getting more of an apples-to-apples comparison was to look only at respondents with 10 or more years of experience and who worked 40 or more hours per week. As shown in figure 6b on page 52, doing this, we saw what seemed to be a decrease in the salary gap across all levels of education.
• Controlling for age . Even controlling for hours and experience, it occurred to us that age may play a part in salary differences. Women might get started in their careers later than men and may need some time to ramp up their salaries. Or, just because they're working full time this year doesn't mean they were working full time the previous two or three years. Maybe they took time off to have children and stalled their career for awhile.
Figure 6c on page 52 shows that filtering by age seemed to increase the salary gap. Meaning what? That perhaps all we are seeing is natural variation in the data and not real differences? Or, more likely, that until you really get closer to an equal comparison, the data can move in any direction. It could also mean that older women tend to make less than younger women. Also, notice that as we cut the pie thinner, our sample size for each educational level gets smaller and smaller. If our data aren't truly homogenous, then this will also affect our results.
• Controlling for job title. Again, because women may be starting their careers later, given any sample, there may be a higher percentage of women at lower-paying job titles. So we need to add job title to the mix. We chose "managers" because that is the job title that has the greatest number of respondents. At this point, figure 6don page 52shows a significant drop from the baseline. We have to ask the same question. Is that a real drop or just the result of natural variation?
• Controlling for job sector. Finally, because women may be drawn to jobs that typically pay less, we have to filter by job sector. For instance, management jobs in education might be less lucrative than management jobs in manufacturing. This is where we run into some problems. The only way to have enough data to continue our drill-down is take the largest industry sector possible, which in our case is manufacturing—all manufacturing. Because manufacturing can involve everything from high-tech, high-paying jobs to low-tech, low-paying jobs, this may be the weakest link in our chain. It doesn't completely eliminate the possibility that women might be attracted to lower-paying manufacturing sectors (garment manufacturing?) than men, which would tend to increase the apparent salary gap.
Incidentally, to increase the number of respondents, we lowered the age cutoff to 40 years or older for figure 6e belowand 35 years old and older for figure 6f below.
Notice that the percentage of women represented in the data steadily decreases as we narrow our filter. Coincidence? Indicative of something? Nothing?
These last two figures show a significantly smaller gap between male and female salaries than figure 6a above. Without doing any further analysis, and simply looking at the tables shown here, it's easy to conclude that salary differences due to gender, for manufacturing managers with more than a high school education, are much less than we thought.
Based on this data, and our own gut feeling after having run this survey half a dozen times over the years, we're going to guess that there is actually quite a small salary disparity between men and women in the quality field working in manufacturing and with roughly comparable age, experience, hours worked, job title, and education.
Does this seem like a reasonable guess, given everything that we've shown you? Let us know by clicking the feedback link at the bottom of this article. Hopefully by next month, we'll have real statisticians take a crack at the data and see what they think.
By the way, as we point out every year, note that in every case in figure 6, the more education you have, the more money you make. Anything you can do to further your education—whether through traditional schooling, Six Sigma training, or certificates such as those offered by the American Society for Quality—will translate into a better salary.
Quality Digest contacted 66,370 print and e-newsletter subscribers by e-mail and invited them to take the salary survey online. To maximize the response, the invitation was sent twice, with the second invitation not including those who took the survey after the first invitation. We also included an invitation in our Quality Digest Online E-Update and InsideSixSigma e-newsletters. To increase the response rate, participants were entered into a random drawing for an Apple iPod. We received a total of about 4,500 responses. Weeding out invalid, duplicate, or incomplete responses, there were 4,347 valid submissions.
More than 72 percent of the respondents indicated that they were quality professionals. The actual percentage is probably higher because many respondents may have generic titles, such as technician, yet perform a quality function.
Dirk Dusharme is Quality Digest's editor in chief.
|
