The derision and hostility of the audience was palpable, when in a recent cardiology conference in Delhi, I stated that substantial data published in the leading medical journals of the world not only exaggerated drug effects but could also be considered misleading. There was stunned silence when I mentioned the New England Journal of Medicine (NEJM) as one of the leading culprits publishing manipulated data because the NEJM is considered by most as the holy grail of medical journals.
I was compelled to rub matters in by quoting Richard Horton, editor of the Lancet: “Journals have devolved into information laundering operations for the pharmaceutical industry.” In the same year (2004) Marcia Angell, former editor of the NEJM criticized the industry for transforming into a “primarily a marketing machine” and co-opting “every institution that might stand in its way.” Richard Smith, former editor of the British Medical Journal noted that Jerry Kassirer, another former editor of the NEJM had argued that the drug industry has deflected the moral compass of many physicians.
Richard Smith in his excellent article, “Medical Journals are an extension of the marketing arm of pharmaceutical companies published by PLOS medicine, believes that albeit substantial income is earned by medical journals printing advertisements, it is still the least corrupting form of dependence. The advertisements may be misleading (with profits worth millions), but they are visible for all to see and criticize. Moreover, as in every sphere, the public is aware that an advertisement is a larger than life sale pitch.
The larger problem is publication of clinical trials by medical journals. Readers are deeply influenced by these randomized controlled trials, which they believe are the acme of scientific rigor and evidence. A large trial published in a leading medical journal will distributed across the world and may often be covered by the international media. A favorable trial is worth thousands of pages of advertising for a drug company and that’s the reason it is prepared to spend as much as a million dollars to purchase reprints that may be distributed globally.
A study of manufactures supported trials of non steroidal anti-inflammatory drugs in the treatment of arthritis, published in the Archives of Internal Medicine (1994), discovered that not a single trial published negative results out of a total of 56 trials studied. Every trial demonstrated that the company’s drug was superior or as good as control treatment. In 2003 a systemic review of 30 studies comparing outcomes of industry sponsored with non-industry sponsored trials found that overall, studies funded by a company were 4 times more likely to have a favorable result compared to studies funded from elsewhere. It is no wonder that then that between two-thirds and three-quarters of trials published in the major journals – Annals of Internal Medicine, JAMA, Lancet, and the NEJM- are funded by the drug industry. It therefor e becomes imperative that medical journals too declare conflict of interest. A conflict of interest is defined as a set of conditions in which professional judgment regarding a primary interest (patient’s welfare or validity of research) tends to be unduly influenced by a secondary interest (financial advantage). The reality however is that the big medical journals have a serious conflict of interest while dealing with industry trials for fear of losing large incomes earned by sales of reprints if they become too critical.
It is not rare for an editor to get a call from the industry to be told to that massive amounts of reprints will be ordered in the event of the industry sponsored paper getting published. An editor may be compelled to decide whether to earn US$ 100,000 of profit or firing a sub-editor.
In 2004, the BMJ devoted an entire issue on conflicts of interest with a cover page showing doctors dressed as pigs at a banquet accompanied by industry sales people as lizards, the drug industry threatened to withdraw 75,000 British pounds of advertising. The Annals of Internal Medicine lost around US$ 1.5 million in advertising revenue subsequent to publishing a paper critical of advertising by the drug industry.
The NEJM had maintained a solid policy on reviews and editorials: “Because the essence of reviews and editorials is selection and interpretation of the literature, the Journal expects that authors of such articles will not have any financial in a company (or its competitor) that make a product discussed in the article. By 2002,however, the editors of NEJM were whining that it was difficult to find non-conflicted authors. They therefore substantially lowered their benchmark by altering the rule to allow the author gain not more than $10,000! There was no limit for income received from companies whose product was not mentioned.
A 2012 study found that the cost for the median and largest reprint order for the Lancet was 287,383 Pounds and 1.55 million Pounds respectively.
The highest impact factor is carried by the NEJM. The impact factor is the number of citations a paper elicits in a year, in the previous 2 years. Most doctors consider the impact factor as the most prestigious ornament of a medical journal, but Peter Gotzsche is not impressed. Gotzsche in his book, “Deadly Medicines and Organized Crime,” describes a Cochrane review of Pfizer’s antifungal drug, voriconazole, to find 2 studies published in the NEJM, but with both having misleading abstracts. In one of the trials voriconazole was significantly inferior to the comparator medicine, amphotericin B, but the paper concluded that voriconazole was a suitable alternative. There were in fact more deaths in the voriconazole group, and the stated reduction in fungal infection disappeared when the Cochrane reviewers included infections that had been arbitrarily excluded from the study.
The other trial published in the NEJM, had voriconazole being given for 77 days while in comparison amphotericin was provide for a mere 10 days. Amphotercin administration was hampered by absence of pre-medication to reduce drug toxicity nor were fluids given to reduce kidney toxicity in this group. The trial was significantly flawed but the conclusion published by NEJM was, “In patients with invasive aspergillosis, initial therapy with voriconazole led to better responses and improved survival and resulted in fewer side effects than the standard approach of initial therapy with amphotericin B.” The NEJM earned a lot of money form selling the reprints of these 2 articles, and the drug company ensured that the impact factor improved by getting large number of ghost written secondary publications, which cited the original (flawed) papers. Pfizer’s voriconazole trials were cited an amazing 192 and 344 times respectively in the next 3 years. The Cochrane reviewers selected a random sample of 25 references to each of these trials to find that not one paper referred to the flaws in the original trials.
None of the 4 leading medical journals in the Unites States has been wiling to reveal revenue earned from advertising or sale of reprints (NEJM, Annals of Internal Medicine, JAMA, and Archives of Internal Medicine). The Lancet conceded that 41% of their revenue was from reprints while BMJ made less than 7% from reprints. The situation in specialty journals is no better. Gotzsche considers editors (of specialty journals) often have conflicts of interest to companies that submit papers to them, including owning shares and being paid consultants. Many specialty journals publish industry-sponsored symposia. The industry usually pays for getting them published, they are rarely peer reviewed, use brand names instead of generic names, and have misleading titles.
A US Congressional Investigation of spinal device products revealed in 2009 that an orthopedic surgeon had received more than $20 million in patent royalties and more than $2 million in consulting fees from Medtronic during his stint as editor of the Journal of Spinal Disorders and Techniques. Medtronic sells spinal implants and was able to publish in every issue with this orthopedic surgeon as editor a positive paper on their device. The FDA had found an adverse event rate of 10% to 50% but not a single device associated adverse effect was reported in 13 industry-sponsored papers regarding safety and efficacy in 780 patients treated with the device.
Drug companies often employ the following techniques to acquire salutatory results of their product: –
- Conduct a trial of your drug against a treatment known to be inferior.
- Compare your drug against too low a dose of the competitor drug.
- Use multiple endpoints in the trial and select for publication those that show favorable results.
- Conduct multicenter trials and select for publication results from centers that are favorable.
- Conduct subgroup analyses and select for publication those that are favorable.
- Provide results most likely to impress; reduction in relative rather then absolute risk.
The last technique is the most commonly used method of exaggerating effects of a drug in most industry-sponsored articles in cardiology published in the NEJM. The absolute difference would be less than 1% but the data is highlighted in relative terms that albeit statistically significant have little clinical relevance. It is for this reason that the American Statistical Association after becoming increasingly concerned with the obsession of researchers with the “P value”, recently called for a rethink with an official guideline: –
- P values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.
- P values can indicate how incompatible the data are with a specified statistical model.
- Scientific conclusions, and business or policy decisions should not be based only on whether a p value passes specific threshold.
- Proper inference requires full reporting and transparency.
- A p value or statistical significance, does not measure the size of an effect or the importance of a result.
- By itself, a p value does not provide a good measure of evidence regarding a model or hypothesis.
The executive director of the ASA has gone on record that “The p value was never intended to be a substitute for scientific reasoning.” Clearly the ASA statement is intended to navigate research into a “post p <0.05 era.”
In the mean time the festering corruption in medical journals may be tackled by more public funding of trials and stoppage of publication of complete trials as is done now. Medical journals should only provide critique of protocols and results put up for public scrutiny in regulated Web sites. The entire raw data of a trial should be made visible for researchers, doctors, patients, and regulators in order to draw plausible conclusions.