Fall 2016 - Vol. 11, No. 3

Absence of Evidence Is Not Evidence of Absence

Philosopher and mathematician Bertrand Russell insisted that for claims that cannot be refuted, the burden of proof rests upon those making such claims, not upon those who doubt them. To illustrate, he famously said that even though it could not be refuted, no one should believe him if he claimed, without proof, that a tiny teapot was orbiting the sun between the Earth and Mars. It is often impossible to prove a negative, i.e. that something does not exist.

Russell’s “cosmic teapot” argument was mainly concerned with religious dogma and its skeptics, but Karl Popper applied the related concept of “falsifiability” to demarcate scientific from unscientific thought. What is unfalsifiable is unscientific, and it is pseudoscience to declare an unfalsifiable theory to be true.

Russell and Popper’s distinction between falsifiable and unfalsifiable concepts has been widely accepted, but even so there is confusion, even among the scientifically literate, about what it means when a scientific proposition lacks definitive proof. The public (and the media) too often assume that if proof is hypothetically obtainable but lacking, the proposition must be false. It is this error that makes it so important to emphasize that absence of evidence is not evidence of absence.^*

One of the key reasons the public misinterprets the lack of evidence is its failure to appreciate how difficult it can be to carry out properly controlled studies, especially when it comes to funding and sustaining long-term studies for issues that are perceived as minor ones. Yet, “minor medical issues” can play an important role in our daily lives. To wit, a recent article in The New York Times was headlined “Feeling Guilty About Flossing? Maybe There’s No Need.”¹ The article noted that the Department of Health and Human Services dropped its recommendations about regular flossing, apparently because there are no adequately controlled studies that prove its benefit in preventing caries or periodontal disease. It cited a Cochrane Database Review that found only “very unreliable” evidence that flossing might reduce plaque after one to three months. The article itself was not as misleading as the headline, but the damage was done, nonetheless. The next day, the crushing blow of ridicule was added by an article in the online New York Times about other uses for the floss you no longer needed: as an emergency clothesline or fishing line, for hanging a picture, removing a ring, committing murder by strangulation, etc.²

It should be obvious that 1-3 months is too short an interval to demonstrate the long-term benefit of flossing, but who would fund a 10-year study, and how would it be conducted? As to the Cochrane Review, it can only reveal the results of studies that have been done; it cannot comment on those that haven’t been done for lack of funding or because they are impractical.

In the meantime, it seems sensible to put flossing in the category of those desirable measures for which the absence of evidence is not evidence of absence, and to ignore articles in the lay press that mainly seek to generate headlines. (My previous editorial discussed the media’s penchant for its own type of “data mining,” in which they scan the medical literature for topics that generate striking headlines.³)

It must be noted, however, that without evidence a proposition can be assumed to be either true or untrue. Flossing is an example of the type of error in which the absence of confirmatory evidence is mistakenly assumed to prove a proposition is untrue. The converse error is one in which it is assumed that the lack of falsifying evidence means a proposition must be true. In the current issue of JLGH Dr. Scott Paist’s article on marijuana illustrates this second type of error. Pennsylvania has approved marijuana for “certified medical use” in various “serious medical conditions” even though the claims of effectiveness are not substantiated for most of them.

The public and the legislature have apparently been persuaded by anecdotal evidence because proper studies have not been done, albeit with good reason. First, because of marijuana’s psychotropic effects, it would be hard to design a blinded study in which the participants didn’t know whether they had received the active drug. This would be a problem even for ingested marijuana where the need to duplicate the experience of smoking wouldn’t be necessary. (Dr. Paist informs me that in the future, with vaping and the possibility of isolating cannabinoids that do not produce euphoria, blinded studies might become more feasible.)

Second, marijuana research is uniquely hampered by a lack of supply, because until now only the University of Mississippi was federally approved to grow marijuana for medical research. The supply of legal marijuana was so restricted that it often took years to obtain it, and it was sometimes impossible to get. The DEA and the Obama administration are now lifting that restriction, and more research about medical marijuana should be forthcoming.

After the article about the Obama administration’s action appeared in The Times, a letter in the Science Section (whose readers, one would hope, are more scientifically literate than average) revealed that ignorance of the scientific method is widespread. Mistakenly convinced that marijuana’s medical effectiveness is already proven, the writer was outraged when the Obama administration explained that it was improving supply because there was a need for more studies. He railed: “To say the science isn’t in that cannabis has medical value is the same as saying the science is still out on global warming. Or on cigarettes causing cancer.”

We can only be amused by the suggestion that marijuana’s situation is the same as global warming or cigarettes and cancer. How would one design a randomized study of global warming, not to mention cigarettes and cancer? In those cases, we must use natural experiments, and remain careful how we interpret the data. With marijuana we have the active agent in a form that can be administered easily and without apparent harm. As a bonus, it should be easy to find willing study participants!

Finally, the most prevalent and persistent need to maintain that absence of evidence is not evidence of absence is in the field of evolutionary biology, which provides the most convincing affirmation of the axiom’s validity. Deniers of evolution are continually insisting that there are large holes in the fossil record and a lack of transitional forms, even though new fossils are constantly being found that fill in the gaps in the evolutionary sequence.^** Considering how remarkable it is that any fossils are still recognizable after hundreds of millions of years, it should not be necessary to point out that when it comes to transitional forms in evolution, absence of evidence is most definitely not evidence of absence.

*This axiom is attributed to Martin Rees, a British astrophysicist who is Astronomer Royal. It was popularized in the United States by the late Carl Sagan.

**Even the partial list of transitional forms on Wikipedia contains more than a hundred distinct types, and it is constantly growing. The most famous is probably Archaeopteryx, the “dinosaur with feathers” that lived about 150 million years ago. First described in 1861 (10 more specimens were discovered later), it is traditionally considered the first proper bird, though it is not directly ancestral to modern birds. A famous recent discovery, Tiktaalik, the “lobe-finned fish,” was discovered in 2004 and lived about 375 million years ago. It had fins with bones that have the same basic structure as all four-legged animals (tetrapods), including humans.

REFERENCES
1. Feeling Guilty About Not Flossing? Maybe There’s No Need. NYTimes, August 3, 2016, pg. A10. http://www.nytimes.com/2016/08/03/health/flossing-teeth-cavities.html
2. Dental Floss may not improve your health, but what else can you do with it? NY Times online. August 3, 2016. http://www.nytimes.com/2016/08/04/health/dental-floss-uses.html?action=click&contentCollection=Health&module=RelatedCoverage®ion=Marginalia&pgtype=article
3. Bonchek LI. Can Science Be Trusted? Data Mining, P-Hacking, and Related Problems. Jl Lanc Gen Hosp. 2016 (2); 11:33-34.