Featured Product
This Week in Quality Digest Live
Quality Insider Features
Chris Fox
Rediscovering the mother of all innovation—necessity
Grant Ramaley
Verify ISO QMS certificates using the International Accreditation Forum (IAF) CertSearch database
Drew Calvert
Should firms be required to disclose the negative social impacts of their suppliers?
Innovating Service With Chip Bell
When customers release their imagination to resolve a problem, everyone benefits
MIT News
They can quickly learn to optimize building microclimates for both energy consumption and user preference

More Features

Quality Insider News
An advanced wireless data collection system for acquiring precision measurement data
Realistic variations in glossiness could aid fine art reproduction and the design of prosthetics
All of the manufacturer’s X-ray sources are supported including rotating target technology and 450kV microfocus source
Smart manufacturing includes recyclability and energy and waste reduction, leading to sustainability and cost savings
Ideal for measuring large components or multiple small components, quickly, easily, and accurately
Measure workpieces from 25 mm to 42.5 mm with a measuring range of ±100 µm and a repeatability of ≤0.1 µm
Contactless gauging system for measuring steel wire for tires, copper wire for electrical cables, CO2 welding wire, and more
Too often process enhancements occur in silos where there is little positive impact on the big picture

More News

MIT News

Quality Insider

Improving Recommendation Systems

To predict customers’ preferences, compare similar products

Published: Wednesday, July 13, 2011 - 10:45

Recommendation algorithms are a vital part of doing business on the Internet. They provide the basis of the targeted advertisements that account for most commercial sites’ revenues, and of services such as Pandora, the radio site that tailors song selections to listeners’ declared preferences. The DVD rental site Netflix deemed its recommendation algorithms important enough that it offered a million-dollar prize to anyone who could improve the site’s predictions by 10 percent.

But Devavrat Shah, the Jamieson Career Development Associate Professor of Electrical Engineering and Computer Science in MIT’s Laboratory of Information and Decisions Systems, thinks that the most common approach to recommendation systems is fundamentally flawed. Shah believes that, instead of asking users to rate products on, say, a five-star scale, as Netflix and Amazon do, recommendation systems should ask users to compare products in pairs. Stitching the pairwise rankings into a master list, Shah argues, will offer a more accurate representation of consumers’ preferences.

In a series of papers published during the last few years, Shah, his students Ammar Ammar and Srikanth Jagabathula, and Vivek Farias, an associate professor at the MIT Sloan School of Management, have demonstrated algorithms that put that theory into practice. Besides showing how the algorithms can tailor product recommendations to customers, they’ve also built a website that uses the algorithms to help large groups make collective decisions. And at an Institute for Operations Research and Management Sciences conference in June, they presented a version of their algorithm that had been tested on detailed data about car sales collected over the span of a year by auto dealers around the country. Their algorithm predicted car buyers’ preferences with 20-percent greater accuracy than existing algorithms.

Calibration conundrum

One of the problems with basing recommendations on ratings, Shah explains, is that an individual’s rating scale will tend to fluctuate. “If my mood is bad today, I might give four stars, but tomorrow I’d give five stars,” he says. “But if you ask me to compare two movies, most likely I will remain true to that for a while.”

Similarly, ratings scales may vary between people. “Your three stars might be my five stars, or vice versa,” Shah says. “For that reason, I strongly believe that comparison is the right way to capture this.”

Moreover, Shah explains, anyone who walks into a store and selects one product from among the three displayed on a shelf is making an implicit comparison. So in many contexts, comparison data are actually easier to come by than ratings.

Shah believes that the advantages of using comparison as the basis for recommendation systems are obvious but that the computational complexity of the approach has prevented its wide adoption. The results of thousands—or millions—of pairwise comparisons could, of course, be contradictory: Some people may like Citizen Kane better than The Godfather, but others may like The Godfather better than Citizen Kane. The only sensible way to interpret conflicting comparisons is statistically. But there are more than three million ways to order a ranking of only 10 movies, and every one of them may have some probability, no matter how slight, of representing the ideal ordering of at least one ranker. Increase the number of movies to 20, and there are more ways to order the list than there are atoms in the universe.

Ordering out

So Shah and his colleagues make some assumptions that drastically reduce the number of possible orderings they have to consider. The first is simply to throw out the outliers. For example, Netflix’s movie-rental data assign the Robin Williams vehicle Patch Adams the worst reviews, on average, of any film with a statistically significant number of ratings. So the MIT algorithm would simply disregard all the possible orderings in which Patch Adams ranked highly.

Even with the outliers eliminated, however, a large number of plausible orderings might remain. From that group, the MIT algorithm selects a subset: the smallest group of orderings that fit the available data. This approach can winnow an astronomically large number of orderings down to one that’s within the computational purview of a modern computer.

Finally, when the algorithm has arrived at a reduced number of orderings, it uses a movie’s rank in each of the orderings, combined with the probability of that ordering, to assign the movie an overall score. Those scores determine the final ordering.

Paat Rusmevichientong, an associate professor of information and operations management at the University of Southern California, thinks that the most interesting aspect of Shah’s work is the alternative it provides to so-called parametric models, which are more restrictive. These, he says, were “the state of the art up until 2008, when Professor Shah’s paper first came out.”

“They’ve really, substantially enlarged the class of choice models that you can work with,” Rusmevichientong says. “Before, people never thought that it was possible to have rich, complex choice models like this.”

The next step, Rusmevichientong says, is to test that type of model selection against real-world data. The analysis of car sales is an early example of that kind of testing, and the MIT researchers are currently working up a version of their conference paper for journal publication. “I’ve been waiting to see the paper,” Rusmevichientong says. “That sounds really exciting.”

Article by Larry Hardesty. Reprinted with permission of MIT News.

Discuss

About The Author

MIT News’s picture

MIT News

The MIT News is the Massachusetts Institute of Technology’s (MIT) central hub for news about MIT research, initiatives, and events. It reports MIT news directly and works with journalists around the world to help showcase the achievements of its students, faculty, and staff.

Comments

Good attempt,Author mentioned

Good attempt,Author mentioned that Comparision is better than ranking,however there should be strong parameters defined for comparision,without which it will be a futile attempt. in my opinion, its not worth to depend on ranking system(Though ranking system is a broad concept ranging from Individual ranking system to Organization ranking system- For individual ranking system 0 read "punished by rewards book" and for Organization ranking system- read performance measurement -transformation-by David sptizer).coming to "comparision" method, it just helps you to understand your current state compare to other similar lines.however comparision can not be made accurate as its not at all possible to consider all the factors involved statistically.Hence, The concept of improving recommendation system is not useful. The best metric to know the customer preference is to conduct trend analysis or refer your sales data over a period of time. unless and untill your inventory gets converted into dollors you can not actually realize wat are the customer preferene.after all,its a business..so bottomline should deal with economics.Refer the good article herehttp://www.qualitydigest.com/sept00/html/satisfaction.html,http://jitendranath.palem.in/blog/http://jitendranath.palem.in/blog/