A Philosopher's Blog

Online Reviews

Posted in Business, Epistemology, Reasoning/Logic, Technology by Michael LaBossiere on May 30, 2011
This Content of online shopping taken from htt...

Image via Wikipedia

Like all sensible people, I hate to waste money. So, when I plan on buying something, I like to ensure that I am making a good choice. Looked at philosophically, this is both a value problem (what is best?”)and an epistemic problem (“how do I know?”)  Conveniently many online stores, most famously Amazon, have customer reviews online.  However, as you yourself have probably noticed, these reviews are often not as useful as they might seem.

The first you will see of the typical online review system is stars (or whatever). On the face of it, this might seem to provide a useful assessment of the product. However, it is simply an average (maybe) of all the rankings. As such, it is only as good as the individual rankings. From a critical thinking standpoint, the ranking system is a survey and hence can be assessed by the standards of an inductive generalization.

One obvious problem with the ranking system is that it is based on a biased sample. People who take the time to write a review (or just click stars) will tend to include a disproportionate number of people who have had very good or very bad experiences. This is borne out by the fact that many products have numerous 5 star and 1 star rankings. As such, the stars should be read with due caution.

A second concern is that the rankings are often based on small samples. For example, my own 42 Fallacies on Amazon currently has a 5 star ranking based on one person. While I do agree with the ranking (oh, if only there were six stars), assessing a product on the basis of a small number of reviews would be risky. Of course, even a large sample will still suffer from a bias problem.

A third concern is that people game the system. Since the review processes tend to be rather lacking in regulation and verification, it is very easy for people to load in fake positive or negative reviews. Like plagiarized papers, these are often very easy to spot. If, for example, the “review” reads like company PR, then it is probably a ringer. If, as another example, the review is incredibly negative but praises a competing product at great length, then it is probably someone acting on behalf of that competitor. However, some “hired guns” are probably clever enough to load in reviews while concealing their true nature.

Since the stars are generally not entirely trustworthy, it is natural to turn to the specific reviews.

In some cases, these reviews can be useful. Not surprisingly, assessing reviews is an exercise in critical thinking. As a general rule, I look for reviews that seem to be balanced in assessing the product and note the weaknesses as well as the strengths.  While this does not guarantee that the review is honest, it tends to be a good indicator of a lack of bias. I also look for consistency across the reviews. For example, if reviews for a laptop consistently mention that the screen is not very good, then that serves as some evidence that this is true of the laptop (or that a hired gun has been busy cranking out reviews). Some companies, such as Amazon, link reviewers to their reviews and this can be useful for getting a better picture of the reviewer’s credibility and expertise. For example, if a reviewer has reviewed numerous books in an area and always takes a measured approach in her reviews, then this increases the credibility of her reviews.

Another factor to look for is the time factor. Many reviewers review the product as soon as they get it, which can (in some cases) be a problem. For example, a review of an Android tablet written right after the person opens the box and fires it up will not tell  you much about its actual battery life or ease of use in various tasks. Some reviewers post updates to their reviews, which can be useful.

While five star reviews should be greeted with a critical review, one star reviews often demand special attention. In some cases, of course, the rating is deserved. However, one star reviews are sometimes inflicted unfairly. First, as mentioned above, people try to game the system. Second, the review might be based on an unusual experience with the product that would generally not be a factor for most users. For example, a certain percentage of electronic devices arrive with problems (such as a defective battery) and this should be taken into account when reading a review that gives a product one star for a failed battery. Naturally, if the same problem appears over and over again in reviews, then that makes it a point of concern. Third, one star reviews are sometimes due to a reviewer not using the product properly or not understanding the product. For example, I have seen reviews attacking a product for not doing something that it was never intended to do. Fourth, some one star reviews are criticisms not of the product but of something else, such as the shipping time or the seller. While these can be relevant factors in buying a product from a specific seller, they really are not relevant to assessing the product. A fifth point of concern is that one star ratings are sometimes used in retaliation.

Naturally, you cannot go wrong buying my books. 🙂





Enhanced by Zemanta

7 Responses

Subscribe to comments with RSS.

  1. magus71 said, on May 30, 2011 at 9:35 am

    I always read the 1 star reviews first, then go from there. It’s almost impossible to get any fair reviews on political books or anything having to do with the Iraq War.

  2. Equus said, on May 30, 2011 at 12:09 pm

    Imperfection is the curse of human nature baby. Reviews are subjective without a doubt so expect flaws and errors.

  3. FRE said, on May 30, 2011 at 12:33 pm

    After buying something from Amazon, they very soon send an e-mail requesting a review. The problem is that that encourages writing a review prematurely, before there has been time to use the product long enough to evaluate it thoroughly. If a product lacks durability or is unreliably, it can take some time to discover that.

    You can read the detailed reviews to determine of the problems noted are important to you. Or for a book, the objections noted may not be important to you. So, within their limitations, I have found the ratings and reviews helpful, but not totally adequate.

    I bought an LG washing machine which had good reviews, but I found a deficiency which was important to me but apparently not important to the reviewers. It is impossible to use it only to spin a wet item. Instead, one must run it through a short cycle which takes about 10 minutes. Probably that is unimportant to many people, but I have a soakable vest designed to keep motorcycle riders cool in hot weather. After soaking it in water for few minutes, to keep it from dripping before putting it on, it is helpful to be able to spin it for a few seconds but that is impossible with the LG washer, a deficiency which is not noted in the reviews.

    • Michael LaBossiere said, on May 30, 2011 at 4:18 pm

      Good point about books. People tend to evaluate based on what they like/think/feel and regard books that go against this as bad, as opposed to assessing the work fairly. I always have a hard time teaching people that an idea, argument or book can have merit even though they might not like it. Heck, I had a hard time teaching myself to do this so I could grade papers fairly.

      • FRE said, on May 30, 2011 at 4:39 pm

        I just finished reading “America in the Gilded Age” by Cashman, after reading the Amazon reviews. One of the reviewers stated that it contained too much detail and that he found it to be too much like a boring textbook. However, the detail was necessary for the author to back up his points. Also, a really studious reader would appreciate the detail. The point is that people read for different purposes and a book suitable for one reader may be less suitable for another reader. Thus, in addition to looking at the ratings, it is necessary to read the full review to understand the criteria used for the rating.

        Incidentally, the book has an extremely thorough bibliography which could be useful for many readers.

        • frk said, on May 31, 2011 at 10:44 pm

          “Thus, in addition to looking at the ratings, it is necessary to read the full review to under the CRITERIA USED FOR THE RATING.”

          Yes. At the level of judging a pizza: If your good buddy likes only soggy crusts and sweet sauces, and he tells you that Papa Julio’s pizza is great, you may feel safe in ignoring his recommendation of Mama Leone’s Pizza down the street , if you prefer crisp crusts and subtly spiced sauces.

          If your buddy likes poor actors in plotless action films, and your preferences lean to Bergman, you may, again, safely ignore his film recommendations.

          This approach isn’t foolproof. Your friend’s suggestions may surprise and delight you every now and then.

          But, if you have no reference points whatsoever, as is so often the case with online reviews, your chances of disappointment are increased.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: