Online Reviews

Image via Wikipedia

Like all sensible people, I hate to waste money. So, when I plan on buying something, I like to ensure that I am making a good choice. Looked at philosophically, this is both a value problem (what is best?”)and an epistemic problem (“how do I know?”)  Conveniently many online stores, most famously Amazon, have customer reviews online.  However, as you yourself have probably noticed, these reviews are often not as useful as they might seem.

The first you will see of the typical online review system is stars (or whatever). On the face of it, this might seem to provide a useful assessment of the product. However, it is simply an average (maybe) of all the rankings. As such, it is only as good as the individual rankings. From a critical thinking standpoint, the ranking system is a survey and hence can be assessed by the standards of an inductive generalization.

One obvious problem with the ranking system is that it is based on a biased sample. People who take the time to write a review (or just click stars) will tend to include a disproportionate number of people who have had very good or very bad experiences. This is borne out by the fact that many products have numerous 5 star and 1 star rankings. As such, the stars should be read with due caution.

A second concern is that the rankings are often based on small samples. For example, my own 42 Fallacies on Amazon currently has a 5 star ranking based on one person. While I do agree with the ranking (oh, if only there were six stars), assessing a product on the basis of a small number of reviews would be risky. Of course, even a large sample will still suffer from a bias problem.

A third concern is that people game the system. Since the review processes tend to be rather lacking in regulation and verification, it is very easy for people to load in fake positive or negative reviews. Like plagiarized papers, these are often very easy to spot. If, for example, the “review” reads like company PR, then it is probably a ringer. If, as another example, the review is incredibly negative but praises a competing product at great length, then it is probably someone acting on behalf of that competitor. However, some “hired guns” are probably clever enough to load in reviews while concealing their true nature.

Since the stars are generally not entirely trustworthy, it is natural to turn to the specific reviews.

In some cases, these reviews can be useful. Not surprisingly, assessing reviews is an exercise in critical thinking. As a general rule, I look for reviews that seem to be balanced in assessing the product and note the weaknesses as well as the strengths.  While this does not guarantee that the review is honest, it tends to be a good indicator of a lack of bias. I also look for consistency across the reviews. For example, if reviews for a laptop consistently mention that the screen is not very good, then that serves as some evidence that this is true of the laptop (or that a hired gun has been busy cranking out reviews). Some companies, such as Amazon, link reviewers to their reviews and this can be useful for getting a better picture of the reviewer’s credibility and expertise. For example, if a reviewer has reviewed numerous books in an area and always takes a measured approach in her reviews, then this increases the credibility of her reviews.

Another factor to look for is the time factor. Many reviewers review the product as soon as they get it, which can (in some cases) be a problem. For example, a review of an Android tablet written right after the person opens the box and fires it up will not tell  you much about its actual battery life or ease of use in various tasks. Some reviewers post updates to their reviews, which can be useful.

While five star reviews should be greeted with a critical review, one star reviews often demand special attention. In some cases, of course, the rating is deserved. However, one star reviews are sometimes inflicted unfairly. First, as mentioned above, people try to game the system. Second, the review might be based on an unusual experience with the product that would generally not be a factor for most users. For example, a certain percentage of electronic devices arrive with problems (such as a defective battery) and this should be taken into account when reading a review that gives a product one star for a failed battery. Naturally, if the same problem appears over and over again in reviews, then that makes it a point of concern. Third, one star reviews are sometimes due to a reviewer not using the product properly or not understanding the product. For example, I have seen reviews attacking a product for not doing something that it was never intended to do. Fourth, some one star reviews are criticisms not of the product but of something else, such as the shipping time or the seller. While these can be relevant factors in buying a product from a specific seller, they really are not relevant to assessing the product. A fifth point of concern is that one star ratings are sometimes used in retaliation.

Naturally, you cannot go wrong buying my books. 🙂