Posted by Larry Hoover on May 25, 2009, at 8:22:42
In reply to Re: Antidepressants Hardly Help ( Time Magazine) » Larry Hoover, posted by metric on May 13, 2009, at 12:07:04
> The drug industry is also guilty of cherry-picking (negative results are seldom published for reasons not necessarily linked to financial disincentives as well), which has been a major source of controversy. There was a paper published in the New England Journal of Medicine last year pertaining to publication bias in antidepressant trials:
I dispute that conclusion, flat out. There are profound disincentives for publishers/editors including negative or inconclusive papers in their journals. Editors compete for the next scoop, not the next dud. I reiterate this statement, which appeared both in the abstract and the body of the following article:
"There can be many reasons why the results of a study are not published, and we do not know the reasons for nonpublication. Thus, we cannot determine whether the bias observed resulted from a failure to submit manuscripts on the part of authors and sponsors, decisions by journal editors and reviewers not to publish submitted manuscripts, or both."
Blaming the drug companies is biased. Instead, I conclude, as do these authors, that the responsibility for the publication bias cannot be attributed.
> Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. "Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy." N Engl J Med. 2008 Jan 17;358(3):252-260.
I've cherry-picked a few extracts from this paper. I'll fully admit selection bias. The authors of the paper exhibit bias, as I will show. I don't think you can write a paper that isn't biased. You have to apply critical thinking, no matter what you read.
One of the most important details in the whole paper, is the following:
"We wish to clarify that nonsignificance in a single trial does not necessarily indicate lack of efficacy. Each drug, when subjected to meta-analysis, was shown to be superior to placebo."So, the FDA did its job. They had all data, and the drugs were properly approved.
The authors of this paper exhibited bias by declaring certain selection criteria that could only result in a negative skew in the statistics they derived, yet they did not discuss the influence of these decisions on those statistics. Here are a couple of examples:
"Few journal articles used the term primary efficacy outcome or a reasonable equivalent. Therefore, we identified the apparent primary efficacy outcome, or the result highlighted most prominently, as the drugplacebo comparison reported first in the text of the results section or in the table or figure first cited in the text."
"Because we excluded articles covering multiple studies, we probably counted some studies as unpublished that were technically published."In the former example, they limited their analysis to one arbitrarily selected outcome determinant, rather than the body of evidence available to them. The FDA looked at all the evidence.
In the latter example, they create a new category of data "technically published", but do not acknowledge this group in their summary statistics on study publication by category. So, they substitute selection bias, while decrying publication bias.
I don't think there is any doubt that publication bias exists. I do question who is ultimately responsible for seeing to it that full disclosure is made, however. IMHO, it falls to the FDA, and their counterparts in other jurisdictions. They are not constrained by the editorial/publication policies of profit-driven journals. Drug companies have supplied all the data, but they cannot force it to be published. There is only one party that could reasonably be expected to do a better job.
Let's not forget that the whole concept of clinical trials of efficacy is a new construct, one that is constantly evolving. Society changes its expectations over time. It's not that long ago that women weren't allowed to vote. It's not fair to history to abstract prior events, and judge them by standards that did not apply at the time those events occurred. We can learn from studying those events, certainly, but the history is simply that.
In the discussion, the study authors move towards that sort of analysis. They contextualize the observed patterns. Here are some of their comments:
"We do not mean to imply that the primary methods agreed on between sponsors and the FDA are necessarily preferable to alternative methods. Nevertheless, when multiple analyses are conducted, the principle of prespecification controls the rate of false positive findings (type I error), and it prevents HARKing, or hypothesizing after the results are known."So, the construct that existed when these studies were conducted was based on statistical and methodolical variables, not specifically addressing antidepressants. Those methods may not be ideal for antidepressant clinical trials, but these specific studies were done under defined protocols.
"It might be argued that some trials did not merit publication because of methodologic flaws, including problems beyond the control of the investigator."
I'm glad they acknowledged this. I have looked closely at some early clinical trials that were included in this dataset, and there were some real problems in methodology. But it was new methodology twenty years ago, and they had yet to learn how to do things well. If you looked at time sequence (which I have never seen analyzed), I bet you'd find a trend from non-significant to significant trial outcomes from the 1980s to the present.
"However, since the protocols were written according to international guidelines for efficacy studies and were carried out by companies with ample financial and human resources, to be fair to the people who put themselves at risk to participate, a cogent public reason should be given for failure to publish."
Well, according to our current standards of disclosure, I would definitely support that. It's inappropriate to retroactively criticize studies completed according to "international guidelines", however.
I want to close with something I've already said. There is only one body that has full access to all the relevant clinical trial data (not just the papers that summarize the research, but the full dataset), that analyzes each study and the body of evidence as a whole, and that determines if efficacy was in fact demonstrated. That's the FDA, or equivalent. They're the only ones with financial independence, also (I'm going to exclude conspiracy theories).
Lar
poster:Larry Hoover
thread:895119
URL: http://www.dr-bob.org/babble/20090524/msgs/897564.html