Geoff Lester's two iPods can hold 20,000 songs, so he turned to the iTunes Music Store, which has 1. 5 million online selections. How to choose? After...

Share story


Geoff Lester’s two iPods can hold 20,000 songs, so he turned to the iTunes Music Store, which has 1.5 million online selections. How to choose?


After the 28-year-old Los Angeles resident picked a few tracks, the software algorithms powering iTunes took over, popping out song after song they calculated that he might like.


He did. He estimates that he bought 500 tracks he otherwise wouldn’t have over the past two years. He snapped up songs from The Dining Rooms, Zero 7 and Headset — bands that aren’t exactly mainstream.


“Before, you’d only find out about new music from whatever’s playing on the radio,” said the account manager. “Now, you can find out about all kinds of bands. It’s definitely opened up some genres for me that I would never have found any other way.”


Time was, a trendy friend would offer advice. But in the Digital Age — when customers are a mouse click away from virtually everything — not even the most plugged-in human can keep up with all the choices.


But computers can, and the growing ranks of so-called preference engines try to capture the intricacies of taste to help buyers navigate the plenty.


Like Coldplay? Check out Moby. Fan of Ian McEwan? Try Philip Roth. Those who bought “Harry Potter” also liked “The Chronicles of Narnia.”


“We’re just being flooded with content,” said Erik Brynjolfsson, a Massachusetts Institute of Technology management professor. “And people are increasingly relying on recommenders to help them sort through it all.”


Preference engines emerged in the earliest days of e-commerce to boost sales — the Internet equivalent of “Would you like a belt to go with that?” — but they have improved with technology and incorporated human feedback to more precisely predict what someone might like.


Their spread worries some who fear that preference engines can extract a social price. As consumers are exposed only to things they’re interested in, there’s a danger that their tastes can narrow and that society may balkanize into groups with obscure interests.


“As these things get better and better, nobody has to encounter ideas they don’t already agree with,” said Barry Schwartz, professor of sociology at Swarthmore College. “We lose that sense of community we had when there were shared cultural experiences, even though we may not have liked them. Now we can create our own cocoon and keep all that unpleasant stuff out.”


The most common recommendation tools involve collaborative filtering, a technique that suggests products based on what other people with similar tastes have bought. These tools keep tabs on what people buy, what items they browse, or whether they put items into their shopping carts.


The idea is to divine clusters of taste, based on the actions of thousands of people, so that when a new person arrives, the Web site can match their taste against others in the database and make recommendations.


These systems boost sales. They also solve a problem created by the Internet — the problem of too many choices. Having such a wide selection also makes navigation unwieldy. The original MP3.com site, which offered songs from unsigned artists and bands, exemplifies the challenge.


“It had just so much music on it, it became completely unnavigable,” said Tim Westergren, chief technology officer of Pandora, a company working on a recommendation system for digital music. “After a while, it became, ‘You’re on MP3.com. So what? Who’s going to find you?’ “


Preference engines are useful because they bubble to the surface items that otherwise would be overlooked. At RealNetworks’ Rhapsody music service, 90 percent of the database of more than 1 million songs are played at least once every month, thanks in part to the service’s preference system.


Researchers then wanted to see whether something as complex as taste — for music, movies, books — could become a math formula.


“Human taste is complex, and I’m not sure it can be accounted for,” said Ken Goldberg, a University of California, Berkeley computer-science professor who studied whether a computer program could recommend jokes. “This is one of the biggest challenges we face because preference engines are based on the idea that if we agree on three movies, we will agree on the fourth. But it often doesn’t work that way.”


Preference engines have encountered other problems, such as when people buy gifts.


“If I shop for a present for someone else, the system can get confused,” said Thomas Hofmann, chief scientist at Recommind and computer-science professor at Brown University. A bachelor buying a one-time gift for a baby could, for example, trigger the program to recommend more baby products in the future, when the suggestions are no longer relevant.


Preference engines are also not so good at telling when people’s needs have shifted, said Andreas Wiegend, who was chief scientist at Amazon.com until 2004.


“Say you’re going on a vacation to China and you buy a few guides before you go,” he said. “Once you’re back, you’re no longer interested in guides to China. But the system keeps recommending them to you because it’s too naive to figure that out.”


Recognizing limitations of pure formulas, some companies built recommendation systems around human judgment. Pandora, for one, relies on a large team of musicians to characterize the tone, feel and lyrics of songs.


By letting people put in their two cents, preference engines have a better chance of making surprising recommendations, Weigend said.


“Old-style engines tend to narrow you down,” he said. “But the world is about people. When you let them in, you begin to create the serendipity of discovering truly new stuff.”