A Philosopher's Blog

My Old Husky & Philosophy III: Experiments & Studies

Posted in Medicine/Health, Philosophy, Reasoning/Logic by Michael LaBossiere on April 8, 2016

Isis on the GoWhile my husky, Isis, and I have both slowed down since we teamed up in 2004, she is doing remarkably well these days. As I often say, pulling so many years will slow down man and dog. While Isis faced a crisis, most likely due to the wear of time on her spine, the steroids seemed to have addressed the pain and inflammation so that we have resumed our usual adventures. Tail up and bright eyed is the way she is now and the way she should be.

In my previous essay I looked at using causal reasoning on a small sale by applying the methods of difference and agreement. In this essay I will look at thinking critically about experiments and studies.

The gold standard in science is the controlled cause to effect experiment. The objective of this experiment is to determine the effect of a cause. As such, the question is “I wonder what this does?” While the actual conducting of such an experiment can be complicated and difficult, the basic idea is rather simple. The first step is to have a question about a causal agent. For example, it might be wondered what effect steroids have on arthritis in elderly dogs. The second step is to determine the target population, which might already be taken care of in the first step—for example, elderly dogs would be the target population. The third step is to pull a random sample from the target population. This sample needs to be representative (that is, it needs to be like the target population and should ideally be a perfect match in miniature). For example, a sample from the population of elderly dogs would ideally include all breeds of dogs, male dogs, female dogs, and so on for all relevant qualities of dogs. The problem with a biased sample is that the inference drawn from the experiment will be weak because the sample might not be adequately like the general population. The sample also needs to be large enough—a sample that is too small will also fail to adequately support the inference drawn from the experiment.

The fourth step involves splitting the sample into the control group and the experimental group. These groups need to be as similar as possible (and can actually be made of the same individuals). The reason they need to be alike is because in the fifth step the experimenters introduce the cause (such as steroids) to the experimental group and the experiment is run to see what difference this makes between the two groups. The final step is getting the results and determining if the difference is statistically significant. This occurs when the difference between the two groups can be confidently attributed to the presence of the cause (as opposed to chance or other factors). While calculating this properly can be complicated, when assessing an experiment (such as a clinical trial) it is easy enough to compare the number of individuals in the sample to the difference between the experimental and control groups. This handy table from Critical Thinking makes this quite easy and also shows the importance of having a large enough sample.

 

Number in Experimental Group

(with similarly sized control group)

Approximate Figure That the difference Must Exceed

To Be Statistically Significant

(in percentage points)

10 40
25 27
50 19
100 13
250 8
500 6
1,000 4
1,500 3

 

Many “clinical trials” mentioned in articles and blog posts have very small samples sizes and this often makes their results meaningless. This table also shows why anecdotal evidence is fallacious: a sample size of one is all but completely useless when it comes to an experiment.

The above table also assumes that the experiment is run correctly: the sample was representative, the control group was adequately matched to the experimental group, the experimenters were not biased, and so on for all the relevant factors. As such, when considering the results of an experiment it is important to consider those factors as well. If, for example, you are reading an article about an herbal supplement for arthritic dogs and it mentions a clinical trial, you would want to check on the sample size, the difference between the two groups and determine whether the experiment was also properly conducted. Without this information, you would need to rely entirely on the credibility of the source. If the source is credible and claims that the experiment was conducted properly, then it would be reasonable to trust the results. If the source’s credibility is in question, then trust should be withheld. Assessing credibility is a matter of determining expertise and the goal is to avoid being a victim of a fallacious appeal to authority. Here is a short checklist for determining whether a person (or source) is an expert or not:

 

  • The person has sufficient expertise in the subject matter in question.
  • The claim being made by the person is within her area(s) of expertise.
  • There is an adequate degree of agreement among the other experts in the subject in question.
  • The person in question is not significantly biased.
  • The area of expertise is a legitimate area or discipline.
  • The authority in question must be identified.

 

While the experiment is the gold standard, there are times when it cannot be used. In some cases, this is a matter of ethics: exposing people or animals to something potentially dangerous might be deemed morally unacceptable. In other cases, it is a matter of practicality or necessity. In such cases, studies are used.

One type of study is the non-experimental cause to effect study. This is identical to the cause to effect experiment with one rather critical difference: the experimental group is not exposed to the cause by those running the study. For example, a study might be conducted of dogs who recovered from Lyme disease to see what long term effects it has on them.

The study, as would be expected, runs in the same basic way as the experiment and if there is a statistically significant difference between the two groups (and it has been adequately conducted) then it is reasonable to make the relevant inference about the effect of the cause in question.

While useful, this sort of study is weaker than the experiment. This is because those conducting the study have to take what they get—the experimental group is already exposed to the cause and this can create problems in properly sorting out the effect of the cause in question. As such, while a properly run experiment can still get erroneous results, a properly run study is even more likely to have issues.

A second type of study is the effect to cause study. It differs from the cause to effect experiment and study in that the effect is known but the cause is not. Hence, the goal is to infer an unknown cause from the known effect. It also differs from the experiment in that those conducting the study obviously do not introduce the cause.

This study is conducted by comparing the experimental group and the control group (which are, ideally, as similar as possible) to sort out a likely cause by considering the differences between them. As would be expected, this method is far less reliable than the others since those doing the study are trying to backtrack from an effect to a cause. If considerable time has passed since the suspected cause, this can make the matter even more difficult to sort out. The conducting the study also have to work with the experimental group they happen to get and this can introduce many complications into the study, making a strong inference problematic.

An example of this would be a study of elderly dogs who suffer from paw knuckling (the paw flips over so the dog is walking on the top of the paw) to determine the cause of this effect. As one might suspect, finding the cause would be challenging—there would be a multitude of potential causes in the history of the dogs ranging from injury to disease. It is also quite likely that there are many causes in play here, and this would require sorting out the different causes for this same effect. Because of such factors, the effect to cause study is the weakest of the three and supports the lowest level of confidence in its results even when conducted properly. This explains why it can be so difficult for researchers to determine the causes of many problems that, for example, elderly dogs suffer from.

In the case of Isis, the steroids that she is taking have been well-studied, so it is quite reasonable for me to believe that they are a causal factor in her remarkable recovery. I do not, however, know for sure what caused her knuckling—there are so many potential causes for that effect. However, the important thing is that she is now walking normally about 90% of the time and her tail is back in the air, showing that she is a happy husky.

 

My Amazon Author Page

My Paizo Page

My DriveThru RPG Page

Follow Me on Twitter

Tagged with: , , ,

Philosophy & My Old Husky II: Difference & Agreement

Posted in Medicine/Health, Philosophy, Reasoning/Logic by Michael LaBossiere on April 6, 2016

Isis in the mulchAs mentioned in my previous essay, Isis (my Siberian husky) fell victim to the ravages of time. Once a fast sprinting and long running blur of fur, she now merely saunters along. Still, lesser beasts fear her (and to a husky, all creatures are lesser beasts) and the sun is warm—so her life is still good.

Faced with the challenge of keeping her healthy and happy, I have relied a great deal on what I learned as a philosopher. As noted in the preceding essay, I learned to avoid falling victim to the post hoc fallacy and the fallacy of anecdotal evidence. In this essay I will focus on two basic, but extremely useful methods of causal reasoning.

One of the most useful tool for causal reasoning is the method of difference. This method was famously developed by the philosopher John Stuart Mill and has been a staple in critical thinking classes since way before my time. The purpose of the method is figuring out the cause of an effect, such as a husky suffering from a knuckling paw (a paw that folds over, so the dog is walking on the top of the foot rather than the bottom). The method can also be used to try to sort out the effect of a suspected cause, such as the efficacy of an herbal supplement in treating canine arthritis.

Fortunately, the method is quite simple. To use it, you need at least two cases: one in which the effect has occurred and one in which it has not. In terms of working out the cause, more cases are better—although more cases of something bad (like arthritis pain) would certainly be undesirable from other standpoints. The two cases can actually involve the same individual at different times—it need not be different individuals (though it also works in those cases as well). For example, when sorting out Isis’ knuckling problem the case in which the effect occurred was when Isis was suffering from knuckling and the case in which it did not was when Isis was not suffering from this problem. I also looked into other cases in which dogs suffered from knuckling issues and when they did not.

The cases in which the effect is present and those in which it is absent are then compared in order to determine the difference between the cases. The goal is to sort out which factor or factors made the difference. When doing this, it is important to keep in mind that it is easy to fall victim to the post hoc fallacy—to conclude without adequate evidence that a difference is a cause because the effect occurred after that difference. Avoiding this mistake requires considering that the “connection” between the suspected cause and the effect might be purely a matter of coincidence. For example, Isis ate some peanut butter the day she started knuckling, but it is unlikely that had any effect—especially since she has been eating peanut butter her whole life. It is also important to consider that an alleged cause might actually be an effect caused by a factor that is also producing the effect one is concerned about. For example, a person might think that a dog’s limping is causing the knuckling, but they might both be effects of a third factor, such as arthritis or nerve damage. You must also keep in mind the possibility of reversed causation—that the alleged cause is actually the effect. For example, a person might think that the limping is causing the knuckling, but it might turn out that the knuckling is the cause of the limping.

In some cases, sorting out the cause can be very easy. For example, if a dog slips and falls, then has trouble walking, then the most likely cause is the fall (but it could still be something else—perhaps the fall and walking trouble were caused by something else). In other cases, sorting out the cause can be very difficult. It might be because there are many possible causal factors. For example, knuckling can be caused by many things (apparently even Lyme disease). It might also be because there are no clear differences (such as when a dog starts limping with no clear preceding event). One useful approach is to do research using reliable sources. Another, which is a good idea with pet problems, is to refer to an expert—such as a vet. Medical tests, for example, are useful for sorting out the difference and finding a likely cause.

The same basic method can also be used in reverse, such as determining the effectiveness of a dietary supplement for treating canine arthritis. For example, when Isis started slowing down and showing signs of some soreness, I started giving her senior dog food, glucosamine and some extra protein. What followed was an improvement in her mobility and the absence of the signs of soreness. While the change might have been a mere coincidence, it is reasonable to consider that one or more of these factors helped her. After all, there is some scientific evidence that diet can have an influence on these things. From a practical standpoint, I decided to keep to this plan since the cost of the extras is low, they have no harmful side effects, and there is some indication that they work. I do consider that I could be wrong. Fortunately, I do have good evidence that the steroids Isis has been prescribed work—she made a remarkable improvement after starting the steroids and there is solid scientific evidence that they are effective at treating pain and inflammation. As such, it is rational to accept that the steroids are the cause of her improvement—though this could also be a coincidence.

The second method is the method of agreement. Like difference, this requires at least two cases. Unlike difference, the effect is present in all the cases. In this method, the cases exhibiting the effect (such as knuckling) are considered in order to find a common thread in all the cases. For example, each incident of knuckling would be examined to determine what they all have in common. The common factor (or factors) that is the most plausible cause of the effect is what should be taken as the likely cause. As with the method of difference, it is important to consider such factors as coincidence so as to avoid falling into a post hoc fallacy.

The method of agreement is most often used to form a hypothesis about a likely cause. The next step is, if possible, to apply the method of difference by comparing similar cases in which the effect did not occur. Roughly put, the approach would be to ask what all the cases have in common, then determine if that common factor is absent in cases in which the effect is also absent. For example, a person investigating knuckling might begin by considering what all the knuckling cases have in common and then see if that common factor is absent in cases in which knuckling did not occur.

One of the main weaknesses of these methods is that they tend to have very small sample sizes—sometimes just one individual, such as my husky. While these methods are quite useful, they can be supplemented by general causal reasoning in the form of experiments and studies—the subject of the next essay in this series.

 

My Amazon Author Page

My Paizo Page

My DriveThru RPG Page

Follow Me on Twitter

Philosophy & My Old Husky I: Post Hoc & Anecdotal Evidence

Posted in Medicine/Health, Philosophy, Reasoning/Logic by Michael LaBossiere on April 4, 2016

dogpark065My Siberian husky, Isis, joined the pack in 2004 at the age of one. It took her a little while to realize that my house was now her house—she set out to chew all that could be chewed, presumably as part of some sort of imperative of destruction. Eventually, she came to realize that she was chewing her stuff—or so I like to say. More likely, joining me on 8-16 mile runs wore the chew out of her.

As the years went by, we both slowed down. Eventually, she could no longer run with me (despite my slower pace) and we went on slower adventures (one does not walk a husky; one goes adventuring with a husky). Despite her advanced age, she remained active—at least until recently. After an adventure, she seemed slow and sore. She cried once in pain, but then seemed to recover. Then she got worse, requiring a trip to the emergency veterinarian (pets seem to know the regular vet hours and seem to prefer their woes to take place on weekends).

The good news was that the x-rays showed no serious damage—just indication of wear and tear of age. She also had some unusual test results, perhaps indicating cancer. Because of her age, the main concern was with her mobility and pain—as long as she could get about and be happy, then that was what mattered. She was prescribed an assortment of medications and a follow up appointment was scheduled with the regular vet. By then, she had gotten worse in some ways—her right foot was “knuckling” over, making walking difficult. This is often a sign of nerve issues. She was prescribed steroids and had to go through a washout period before starting the new medicine. As might be imagined, neither of us got much sleep during this time.

While all stories eventually end, her story is still ongoing—the steroids seemed to have done the trick. She can go on slow adventures and enjoys basking in the sun—watching the birds and squirrels, willing the squirrels to fall from the tree and into her mouth.

While philosophy is often derided as useless, it was actually very helpful to me during this time and I decided to write about this usefulness as both a defense of philosophy and, perhaps, as something useful for others who face similar circumstances with an aging canine.

Isis’ emergency visit was focused on pain management and one drug she was prescribed was Carprofen (more infamously known by the name Rimadyl). Carprofen is an NSAID that is supposed to be safer for canines than those designed for humans (like aspirin) and is commonly used to manage arthritis in elderly dogs. Being a curious and cautious sort, I researched all the medications (having access to professional journals and a Ph.D.  is handy here). As is often the case with medications, I ran across numerous forums which included people’s sad and often angry stories about how Carprofen killed their pets. The typical story involved what one would expect: a dog was prescribed Carprofen and then died or was found to have cancer shortly thereafter. I found such stories worrisome and was concerned—I did not want my dog to be killed by her medicine. But, I also knew that without medication, she would be in terrible pain and unable to move. I wanted to make the right choice for her and knew this would require making a rational decision.

My regular vet decided to go with the steroid option, one that also has the potential for side effects—complete with the usual horror stories on the web. Once again, it was a matter of choosing between the risks of medication and the consequences of doing without. In addition to my research into the medication, I also investigated various other options for treating arthritis and pain in older dogs. She was already on glucosamine (which might be beneficial, but seems to have no serious side effects), but the web poured forth an abundance of options ranging from acupuncture to herbal remedies. I even ran across the claim that copper bracelets could help pain in dogs.

While some of the alternatives had been subject to actual scientific investigation, the majority of the discussions involved a mix of miracle and horror stories. One person might write glowingly about how an herbal product brought his dog back from death’s door while another might claim that after he gave his dog the product, the dog died because of it. Sorting through all these claims, anecdotes and studies turned out to be a fair amount of work. Fortunately, I had numerous philosophical tools that helped a great deal with such cases, specifically of the sort where it is claimed that “I gave my dog X, then he got better/died and X was the cause.” Knowing about two common fallacies is very useful in these cases.

The first is what is known as Post Hoc Ergo Propter Hoc (“after this, therefore because of this”).  This fallacy has the following form:

 

  1. A occurs before B.
  2. Therefore A is the cause of B.

 

This fallacy is committed when it is concluded that one event causes another simply because the proposed cause occurred before the proposed effect. More formally, the fallacy involves concluding that A causes or caused B because A occurs before B and there is not sufficient evidence to actually warrant such a claim.

While cause does precede effect (at least in the normal flow of time), proper causal reasoning, as will be discussed in an upcoming essay, involves sorting out whether A occurring before B is just a matter of coincidence or not. In the case of medication involving an old dog, it could entirely be a matter of coincidence that the dog died or was diagnosed with cancer after the medicine was administered. That is, the dog might have died anyway or might have already had cancer. Without a proper investigation, simply assuming that the medication was the cause would be an error. The same holds true for beneficial effects. For example, a dog might go lame after a walk and then recover after being given an herbal supplement for several days. While it would be tempting to attribute the recovery to the herbs, they might have had no effect at all. After all, lameness often goes away on its own or some other factor might have been the cause.

This is not to say that such stories should be rejected out of hand—it is to say that they should be approached with due consideration that the reasoning involved is post hoc. In concrete terms, if you are afraid to give your dog medicine she was prescribed because you heard of cases in which a dog had the medicine and then died, you should investigate more (such as talking to your vet) about whether there really is a risk of death. As another example, if someone praises an herbal supplement because her dog perked up after taking it, then you should see if there is evidence for this claim beyond the post hoc situation.

Fortunately, there has been considerable research into medications and treatments that provide a basis for making a rational choice. When considering such data, it is important not to be lured into rejecting data by the seductive power of the Fallacy of Anecdotal Evidence.

This fallacy is committed when a person draws a conclusion about a population based on an anecdote (a story) about one or a very small number of cases. The fallacy is also committed when someone rejects reasonable statistical data supporting a claim in favor of a single example or small number of examples that go against the claim. The fallacy is considered by some to be a variation on hasty generalization.  It has the following forms:

Form One

  1. Anecdote A is told about a member (or small number of members) of Population P.
  2. Conclusion C is drawn about Population P based on Anecdote A.

For example, a person might hear anecdotes about dogs that died after taking a prescribed medication and infer that the medicine is likely to kill dogs.

Form Two

  1. Reasonable statistical evidence S exists for general claim C.
  2. Anecdote A is presented that is an exception to or goes against general claim C.
  3. Conclusion: General claim C is rejected.

For example, the statistical evidence shows that the claim that glucosamine-chondroitin can treat arthritis is, at best, very weakly supported. But, a person might tell a story about how their aging husky “was like a new dog” after she starting getting a daily dose of the supplement. To accept this as proof that the data is wrong would be to fall for this fallacy. That said, I do give my dog glucosamine-chondroitin because it is cheap, has no serious side effects and might have some benefit. I am fully aware of the data and do not reject it—I am gambling that it might do my husky some good.

The way to avoid becoming a victim of anecdotal evidence is to seek reliable, objective statistical data about the matter in question (a vet should be a good source). This can, I hasten to say, can be quite a challenge when it comes to treatments for pets. In many cases, there are no adequate studies or trials that provide statistical data and all the information available is in the form of anecdotes. One option is, of course, to investigate the anecdotes and try to do your own statistics. So, if the majority of anecdotes indicate something harmful (or something beneficial) then this would be weak evidence for the claim. In any case, it is wise to approach anecdotes with due care—a story is not proof.