Can we even trust epidemiology?

Another class assignment! In this case, just forcing me to do a more thorough review of an article I meant to review anyways. Here it is:

What level of proof is necessary to make a claim? That seems to be the question Taubes is asking in “Do we really know what makes us healthy.” I would posit that a more appropriate question would be ‘how do we interpret our claims?’

Taubes doubts the validity of epidemiological arguments based on his understanding of statistics as “tools . . . that may be unreliable” and “circumstantial evidence” that should be barely trusted, and only until clinical evidence is available. In one sense, he is right: as a science based on probabilities, epidemiology can be done badly and misused, with results that are untrustworthy. What about epidemiology done right? Sampling biases still exist: healthy users, atypical study subjects, compliance. Diagnostics are imperfect and that can bias results. Some confounders are unknown or immeasurable. However, dealing with these issues is one of the purposes of peer review. Taubes claims that the first report is untrustworthy simply because it lacks that peer review, ignoring the review process that is required to make that report. Any report published without peer review is indeed suspect, but that is true in any field. To re-quote John Bailar, “The appropriate question is not whether there are uncertainties about epidemiologic data, rather, it is whether the uncertainties are so great that one cannot draw useful conclusions from the data.”

The problem may come, not from the science itself, but from the understanding and application of the results. Taubes quotes an editorial from the New England Journal of Medicine on the role of the media in this debacle, namely, that the lay media interprets studies wrong. Rather than seeing an association as it is, they insist on a causative – it’s just easier to report. When a majority of these causations fail to materialize (although the associations may still be present), Taubes suggests they should reject epidemiology as an untrustworthy source.

Rejecting epidemiologic studies on the whole because a majority have been refuted is not a sensible choice. For example, many movies and even more books have poor to dismal ratings from tough critics; should I then reject all such media because the majority are considered bad? I would rather focus on the good, even if they are in the minority. That, of course, requires close, skeptical reading on the part of the science reporters.

Taubes champions the experimental study as the savior of epidemiologic conclusions. After all, if we can prove that the epidemiologists were right, we can ‘trust’ them (for a certain value of trust). However, one of the older studies he cites is ethically questionable (Goldberger trying to infect himself and colleagues) and the new studies he considers have plausibility problems of their own (H.R.T. studies choosing different subject types). In many cases, experimental studies are simply not possible, as Taubes admits. Why, then, does he end with a suggestion to wait for clinical trials to back up the epidemiologic associations?

The main problem with this entire debate, however, is the differing viewpoints. Taubes writes to inform the individual readers, who will try to apply the results of epidemiologic studies to their own lives. As any epidemiologist knows, those studies are not meant to predict individual results. To an individual, population-level data is a step removed. Maybe the problem comes simply in the application, stepping down to the micro what is meant to be macro – in other words, the ecologic fallacy.