THE FUN STARTED

over a year ago, when Amazon.com began selling gallon jugs of milk. An anonymous consumer submitted a review, dubbing it "The Best Milk Ever!" and the tongue-in-cheek comment struck a chord. Since then nearly 900 consumers have written reviews of the $3.99 jug. Comments range from "Worth its weight in gold times infinity" to "My cat is awesome now." One person recalled sending the milk to business contacts over the holidays: "Our clients were super-impressed, and I may be on my way to VP," she raved.

Amazon's now infamous milk-review page parodies a familiar phenomenon: the overwhelming positivity of customer reviews. If a Martian had access to the Internet, he'd conclude that Earth is a consumer paradise where every gadget merits five out of five stars and everyone on eBay is a "superfast A+++ highly recommended seller." In fact, a recent study analyzing more than 585,000 customer-written reviews on Amazon found that the average book title gets 4.2 out of five stars. The same goes for customer write-ups on web sites of companies like Sears, Home Depot and Macy's. Whether they're reviewing cameras or cashmere, more than 80% of consumers award at least four stars. No wonder online retailers are rushing to add customer-comment sections to their sites. For years they spent billions persuading us that all their merchandise was above average; now they can relax and let us convince each other.

You've got to wonder why the ratings skew so high. One possible explanation is that they reflect the truth: Most products on the market today are flat-out wonderful. Ha-ha! The real answer lies in Amazon.com's stapler section, where, believe it or not, a total of 42 customers have commented on stapler purchases. Not surprisingly, 27 granted five stars. A typical write-up: "It works very well and staples many papers together." Five customers really hated their stapler and gave it just one star. But here's the interesting part: Only one reviewer awarded three stars. Does that mean there's no such thing as an average stapler? Hardly. It means that consumers review only products they love or loathe. As Paul A. Pavlou, professor of information systems with the Anderson Graduate School of Management, notes, consumer-generated ratings generally follow a particular distribution: lots of high ratings, some low ratings and few in between.

But why do the very high scores so heavily outnumber the low scores? Christopher Allen of gaming-review site RPGnet.com says it's because most folks review things related to their pet hobbies. Only those who cherish romance novels bother to review them, for instance. And to a fanatic, even a mediocre romance constitutes a pretty good read. Whenever the pool of reviewers is self-selecting, you'll get a positive bias.

Not so at the infamous HotorNot.com, where folks submit photos of themselves for others to rate on a scale of one to 10. Raters view the photos one at a time, and you can't see the next photo until you have rated the one at hand. Site cofounder James Hong reports that as a result, the ratings follow a bell curve, with scores clustering near the average. And the average turns out to be abysmally low. Soon after launching the site, Hong and his partner started goosing the scores to compensate for the overwhelming negativity of the ratings. "People were overly cruel," he says.

In an attempt to balance out the self-selecting reviewer pool, some online sellers are sending product surveys to all their customers. In the meantime, given the Net's grade inflation, look for products rated five out of five stars, minimum. Yes, that rules out Amazon.com's four-star jug of milk, but there are still plenty of staplers to choose from.