The quotes are categorized as:

Bias in Clinical Trials

“The FDA did approve Prozac for use in children, as two of the three positive studies reviewed by Laughren had come from trials of this drug. But, as many critics have pointed out, from a scientific perspective, there is no reason to think that Prozac is any better than the other SSRIs [Selective Serotonin Reuptake Inhibitor anti-depressants]. The percentage of children who responded to Prozac in the two positive trials was similar to the drug response rate in the twelve failed trials; Eli Lilly simply had been better at using biased trial designs to make it appear that its drug worked. For example, in one of the two Prozac trials, all of the children were initially put on placebo for one week, and if they got better during that period, they were excluded from the study. This helped knock down the placebo response rate. Next, the children who were randomized onto Prozac were evaluated for a week, and only those ‘who adapted well’ to the drug were then enrolled in the study. This helped increase the drug response rate…explained Jonathan Leo, editor in chief of the journal Ethical Human Psychology and Psychiatry, ‘…the placebo group was preselected for nonresponders, while the drug group was preselected for responders.’”

Whitaker R. Anatomy of an epidemic. Crown Publishers. 2010

“Thirty studies of drugs (63%) were supported by pharmaceutical companies, all of which reported positive results. Of the 18 studies (37%) not supported by pharmaceutical companies, 67% reported positive results. The association between pharmaceutical funding and positive findings was statistically significant”

Finucane TE, Boult CE. Association of funding and findings of pharmaceutical research at a meeting of a medical professional society. Am J Med. 2004 Dec 1;117(11):842-5.

“The experimental drug was recommended as treatment of choice in 16% of trials funded by nonprofit organizations, 30% of trials not reporting funding, 35% of trials funded by both nonprofit and for-profit organizations, and 51% of trials funded by for-profit organizations.”

Als-Nielsen B et al. Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events?. JAMA. 2003 Aug 20;290(7):921-8.

“We did…identify a number of negative [of tamiflu/zanamavir influenza antiviral drug] results reported by regulatory bodies as part of drug licensing and approval assessments that had not been published by Roche or GlaxoSmithKline in peer-reviewed journal articles or conference presentations. For example, non-significant primary endpoint data for children with influenza B (WV15758) and for 'at risk' children, with asthma (WV15759/WV15871) were only available from the European Agency for the Evaluation of Medicinal Products (EMEA 2002). Whether these omissions represent true publication bias (failure to publish negative results) or publication lag (extended time from study completion to study publication for negative results) is not clear, although the latter is well known to exaggerate treatment effects in early meta-analyses (Hopewell 2002). In general, both Roche and GlaxoSmithKline were willing to supply conference abstracts/posters and references to published data but (with the exception of a number of clarifications by Roche) would not provide re-analyses or additional data.”

Matheson NJ et al. Neuraminidase inhibitors for preventing and treating influenza in children. Cochrane Database Syst Rev. 2003;3(CD002744).

“We identified 268 trials: 16 (6%) in the Annals of Internal Medicine, 47 (18%) in the BMJ, 46 (17%) in JAMA, 93 (35%) in the Lancet, and 66 (25%) in the New England Journal of Medicine. Of these, 238 (89%) disclosed the source of study support. The nature of the relationship between the authors and the study sponsor was included in 69 of the 100 industry sponsored studies. In the manuscripts that provided this information, the most commonly cited relationships were: employment (30/69, 43%); consultant/honorarium (22, 32%); grants (18, 26%); and stock ownership and participation in a speaker's bureau (7, 10% each). The 30 manuscripts that were coauthored by employees of the industry sponsor represented 30% of published industry-sponsored studies and 11% of all randomised controlled trials in our sample. Only 8 of the 100 industry sponsored studies reported the role of the study sponsor in the methods section, as recommended by the uniform require- ments. Of these studies, 2 manuscripts explicitly stated that the sponsor had no role in the study and 6 described the sponsor's role. Further analysis of these six studies found that the degree of sponsor involvement was highly variable and was usually described with vague wording. For instance, the sponsor's role in data analysis was described in terms ranging from “preliminary evaluation” to “coordinating data collection and statistical analysis.” Although industry involvement in published randomised controlled trials was substantial, the true extent and nature of financial relationships between investigators and industry is difficult to assess because of variable adherence to the disclosure guidelines.”

Gross CP et al. Disclosure of financial competing interests in randomised controlled trials: cross sectional review. BMJ. 2003 Mar 08;326(7388):526-527.
http://bmj.com/cgi/content/full/326/7388/526

“Approximately 1/4 of investigators have industry affiliations, and roughly 2/3 of academic institutions hold equity in start-ups that sponsor research performed at the same institutions. Eight articles, which together evaluated 1140 original studies, assessed the relation between industry sponsorship and outcome in original research. Aggregating the results of these articles showed a statistically significant association between industry sponsorship and pro-industry conclusions (pooled Mantel-Haenszel odds ratio, 3.60; 95% confidence interval, 2.63-4.91). Industry sponsorship was also associated with restrictions on publication and data sharing. The approach to managing financial conflicts varied substantially across academic institutions and peer-reviewed journals.”

Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA. 2003 Jan 22-29;289(4):454-65.
http://jama.ama-assn.org/pi/index.html

“Variables associated with publication were study conclusions favoring treatment (63.5% vs 43.7%), sample size, and measures of treatment effect…Most measures of treatment effect changed slightly from the abstract to the manuscript…In 4.0% of the cases, abstract outcomes were significant but changed to nonsignificant in manuscript form. 8.9% changed from nonsignificant to significant. In 10.8% of studies, the sample size decreased from the abstract to the manuscript, whereas in 48.3% the sample size increased.”

Klassen TP et al. Abstracts of randomized controlled trials presented at the society for pediatric research meeting: an example of publication bias. Arch Pediatr Adolesc Med. 2002 May;156(5):474-9.

“In a survey of editors, 30% of them said they would not publish a meta-analysis that included unpublished research. The meta-analyst is caught in a dilemma as excluding abstracts or other unpublished RCTs means the meta-analysis may be biased toward favoring treatment.”

Klassen TP et al. Abstracts of randomized controlled trials presented at the society for pediatric research meeting: an example of publication bias. Arch Pediatr Adolesc Med. 2002 May;156(5):474-9.

“The free and open sharing of information, data, and materials regarding published research is vital to the replication of published results, the efficient advancement of science, and the education of students. Yet in daily practice, the ideal of free sharing is often breached…84% of geneticists had made at least 1 request in the previous 3 years…for additional information, data, or materials concerning published research…47% [of these] reported that at least 1 request was denied. However, only 12% [reported that they] denied a request they received from another academic researcher…geneticists were no more likely than otherr life scientists to report that their requests were denied…The 2 factors that were significantly associated with an increased likelihood of denying others’ requests were having received a high number of requests in the last 3 years and having engaged in commercial activities”

Campbell EG et al. Data withholding in academic genetics: evidence from a national survey. JAMA. 2002 Jan 23-30;287(4):473-80.

“The 181 peer-reviewed journals examined by the research team published 61,134 original research items in 1997. Of the original research items, the number that contained at least one positive disclosure [as opposed to a declaration of ‘no financial interests’] of an author’s personal financial interests related to the publication was 327 (0.5%)…119 of the 181 journals (65.7%), comprising 58.1% of the articles examined, published no positive disclosures of authors’ personal financial interests. 37 journals (20.4%) had such disclosures in 1% or less of the original research items published in their journals…Based on our sample of 1,396 high impact journals, we found that 15.8% had a published policy on conflicts of interest during 1997. In our sub-sample of 181 peer-reviewed journals with COI [conflict of interest] policies, 87% were medical journals and 13% were science journals. In contrast, medical journals made up only 34% of our original sample…By the mid-1990s one survey found that as many as 34% of all medical journals and 46% of U.S. medical journals with circulation over 1,000 reported they had written COI policies for contributing authors…the vast majority usually publish author disclosure statements suggesting that low rates of personal financial disclosures are either a result of low rates of financial interest (nothing to disclose) or poor compliance among authors to the journals’ COI policies. Based on the previously mentioned pilot study, higher disclosure rates in the template journals, and the growth of commercialization in the biomedical sciences, we believe that poor compliance is the more likely explanation for low disclosure rates in most journals with COI policies.”

Krimsky S, Rothenberg LS. Conflict of interest policies in science and medical journals: editorial practices and author disclosures. Sci Eng Ethics. 2001 Apr;7(2).

“Companies may design studies likely to favor their products…Only 2.1% of subjects in trials of nonsteroidal antiinflammatory drugs were 65 years of age or older, even though these drugs are more commonly used and have a higher incidence of side effects in the elderly…Investigators may receive only portions of the data…companies prefer to retain control over [analysis]…a practice that one might call the nonwriting author-nonauthor writer syndrome has developed…a professional medical writer employed by a drug company, CRO [contract research organization], or medical communications company, who is paid to write an article but is not named as an author; and a clinical investigator, who appears as an author but does not analyze the data or write the manuscript…The nonwriting author…has final control over the manuscript. But many of these authors are busy and may not perform a thorough review.”

Bodenheimer T. Uneasy Alliance -- Clinical Investigators and the Pharmaceutical Industry. N Engl J Med. 2000 May 18;342(20):1539-44.

“Pharmaceutical company-sponsored studies were less likely than nonprofit-sponsored studies to report unfavorable qualitative conclusions...whereas overstatements of quantitative results were not significantly different in pharmaceutical company-sponsored vs nonprofit-sponsored studies [but this may be due to small sample size as 6/20 pharmaceutical company-sponsored trials overstated versus 3/24 nonprofit-sponsored studies]”

Friedberg M et al. Evaluation of Conflict of Interest in Economic Analyses of New Drugs Used in Oncology. JAMA. 1999 Oct 20;282(15):1453-7.

“A Canadian epidemiologist has found a sea of errors in medical drug trials so broad and deep the British science journal Lancet fears it could “cast serious doubts on the validity of current clinical research. “ David Moher’s review of 127 trials done between 1960 and 1995 found the treatment results in medical journals used by doctors was often exaggerated, misinterpreted or flawed””

Evenson B. Medical drug trials badly flawed: study. Calgary Herald. 1998 Aug 27

“Assessment of the quality of reports of RCTs [randomized controlled clinical trials] included in a meta-analysis adds another layer of complexity to the reviewing process. Our results suggest, however, that incorporation of an estimate of the quality of RCTs is important. We found a clinically important and statistically significant 30-50% exaggeration of treatment efficacy when results of lower-quality trials were pooled.”

Moher D et al. Does quality of reports of randomized trials affect estimates of intervention efficacy reported in meta-analyses?. Lancet. 1998 Aug 22;352(9128):609-13.

“The conclusions of review articles are strongly associated with the affiliations of their authors...original research articles that acknowledge sponsorship from the pharmaceutical industry, the chemical industry, or the tobacco industry tend to draw pro-industry conclusions...94% of reviews by tobacco industry-affiliated authors concluded that passive smoking is not harmful, compared with 13% of reviews by authors without tobacco industry affiliations...we identified only 1 review written by an author without any known tobacco industry affiliations that concluded that passive smoking is not harmful to health”

Barnes DE, Bero LA. Why review articles on the health effects of passive smoking reach different conclusions. JAMA. 1998 May 20;279(19):1566-70.

“Authors who supported the use of calcium-channel antagonists were significantly more likely than neutral or critical authors to have financial relationships with manufacturers...(96% vs. 60% and 37%, respectively). Supportive authors were also more likely than neutral or critical authors to have financial relationships with any pharmaceutical manufacturer, irrespective of the product...[but] only 2 of the 70 articles included in the study disclosed the authors’ potential conflicts of interest...[but, the authors wrap up by claiming that these results don’t really mean that researchers are biased, apparently directly contradicting their own research] We believe that the authors we surveyed expressed their own opinions and were not influenced by financial relationships with pharmaceutical manufacturers.”

Stelfox HT et al. Conflict of interest in the debate over calcium-channel antagonists. N Engl J Med. 1998 Jan 8;338(2):101-6.

“Significantly more articles with drug company support (98%; 39 of 40) than without drug company support (79%; 89 of 112) had outcomes favoring the drug of interest”

Cho MK, Bero LA. The quality of drug studies published in symposium proceedings. Ann Intern Med. 1996;124:485-9.

“The most serious concern of a pharmaceutical company-sponsored cost-analysis study is the ethics of the company and whether the investigators were biased... For example, two of Smith’s coauthors work for Burroughs Wellcome...publications that are sponsored by pharmaceutical companies are almost always likely to be positive in the favor of the companies...Some investigators are known [by drug companies] to consciously or inadvertently promote certain products, and these associations can have positive or negative impact in future funding for their trials”

Gulati SC, Bitran JD. Cost-effectiveness analysis: sleeping with an enemy or a friend?. J Clin Oncol. 1995 Sep;13(9):2152-4.

“The reasons given for failure to analyze 32 completed studies were...the sponsoring pharmaceutical company had not released the raw data (14)...The sponsoring pharmaceutical companies were blamed for non-publication in 11%, since they managed the data and were therefore considered responsible for initiating publication [and in another 11% analysis was incomplete, some of which may be due to the same reason]”

Easterbrook RJ et al. Publication bias in clinical research. Lancet. 1991 Apr 13;337(8746):867-72.

“Publication bias is the tendency on the parts of investigators, reviewers, and editors to submit or accept manuscripts for publication based on the direction or strength of the study findings…In medicine, three studies have provided direct evidence for this bias…If treatment decisions are based on the published literature, then the literature must include all available data that is of acceptable quality. Currently, obtaining information regarding all studies undertaken in a given field is difficult, even impossible. Registration of clinical trials, and perhaps other types of studies, is the direction in which the scientific community should move.”

Dickersin K. The existence of publication bias and risk factors for its occurrence. JAMA. 1990 Mar 9;263(10):1385-9.

“In 82 reports (42%) [of clinical trials studied], bias in the conclusion or abstract consistently favored one of the drugs, which was the control drug in only one report and the new drug in the remaining 81 [note that funding for trials is much more likely from the manufacturer of the new drug being tested, rather than the control]”

Gøtzsche PC et al. Methodology and overt and hidden bias in reports of 196 double-blind trials of non-steroidal antiinflammatory drugs in rheumatoid arthritis. Controlled Clin Trials. 1989;10:31-56.

“10 [trials studied] had a neutral section of references, 22 a negative selection [leaning towards a negative view of the study drug] and 44 a positive selection [leading to a more positive view of the study drug than is justifiable]...This study has shown a high frequency of multiple publication and reference bias...If multiple publication is not detected it may cause problems in any pooled analysis of trials (meta-analysis) or mislead the reader of the individual article. In fact, five articles referred to multiple publications by others as if these concerned separate trials”

Gøtzsche PC. Reference bias in reports of drug trials. BMJ. 1987 Sep 12;295(12):654-6.

“An article that reports a statistically significant difference between two treatments is more likely to be published than one which does not...So long as statistical significance is used as a major criterion of acceptability for publication the published results of medical research will contain a high proportion of false positive results”

Newcombe RG. Towards a reduction in publication bias. BMJ. 1987 Sep 12;295(6599):656-9.

“The pooled results presented in this paper illustrate how different conclusions may be reached by selected trials from the published literature [in which ‘negative’ results are less likely to be published] or from a clinical trials registry [where trials are listed when they are initiated, before the type of results they will produce are known]. Whereas published trials of chemotherapy in advanced ovarian cancer demonstrated a significant survival advantage for CC [combination chemotherapy, as compared to the less toxic alkylating agents], the pooled analysis of registered trials did not substantiate this conclusion. In multiple myeloma, both the pooled results of published trials and registered trials suggested a significant survival advantage for CC. However, the estimated magnitude of this survival advantage was less when based on registered rather than published trials...[and] the pooled analysis would not have resulted in significant survival differences if the SWOG [most favourable] study was excluded”

Simes RJ. Publication bias: the case for an international registry of clinical trials. J Clin Oncol. 1986;4(10):1529-41.

“There was a statistically significant association between the source of funding and the outcome of the study, with studies supported by pharmaceutical manufacturers favoring new therapies in comparison with generally supported studies…Among the four pharmaceutically supported studies favoring traditional therapies, one suggested that the therapy, while not improving overall survival, '...may be helpful early in the course of the disease and in certain subgroups of patients.' Another compared two therapies manufactured by the sponsoring drug company. The other two studies…were funded by companies that did not produce any of the drugs investigated. In no cases was a therapeutic agent manufactured by the sponsoring company found to be inferior to an alternative product manufactured by another company.”

Davidson RA. Source of funding and outcome of clinical trials. J Gen Intern Med. 1986;1:155-8.

Trial Design and Reporting

“Only 48% (49/102) of trialists responded to the questionnaire regarding unreported outcomes, 86% (42/49) of whom initially denied the existence of such outcomes prior to receiving our list of unreported outcomes.However, all 42 of these trials had clear evidence of unreported outcomes in their protocols and in the pub-lished articles…Among trials that measured efficacy or harm outcomes, 71% (70/99) and 60% (43/72) had at least one unreported efficacy or harm outcome, respectively…Among 78 trials with any unreported outcome (efficacy or harm or both), we received only 24 survey responses (31%) that provided reasons for not reporting outcomes for efficacy (23 trials) or harm (10 trials) in their published articles. The most common reasons for not reporting efficacy outcomes were lack of statistical significance (7/23 trials), journal space restrictions (7/23), and lack of clinical importance (7/23)…92% (91/99) of trials had at least 1 incompletely reported efficacy outcome, while 81% (58/72) had at least one incompletely reported harm outcome. Primary outcomes were specified for 63 of the published trials, but for 17 (27%) of these trials at least 1 primary outcome was incompletely reported.”

Chan AW et al. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004 May 26;291(20):2457-65.

“the usual procedure used in clinical trials of collecting adverse event data only while patients are taking study therapy and not collecting adverse event data after patients discontinue study treatment may need to be reevaluated. It is entirely possible that treatments may cause toxicities (e.g., liver damage or acceleration of atherosclerosis) that may not be apparent while the patient is taking the therapy and may only be manifest many months or years later. A rigorous intention-to-treat analysis for adverse event data as was employed here could provide important data with respect to establishing the toxicity profile of different regimens. Also, longer-term follow- up is necessary as many adverse events are not likely to emerge in the limited follow-up carried out for treatment trials for licensure.”

Reisler RB et al. Grade 4 events are as important as AIDS events in the era of HAART. J Acquir Immune Defic Syndr. 2003 Dec 1;34(4):379-86.

“[PhRMA] in its "Principles on Conduct of Clinical Trials and Communication of Clinical Trial Results" released in June 2002 have accepted the principle of investigator freedom to publish research findings…the PhRMA Principles are explictly voluntary. There is no mechanism for enforcement, or to punish companies that do not comply.”

DuVal G. Institutional ethics review of clinical study agreements. JME. 2003;28(2).

“unfortunately, even when an investigator has had substantial input into trial design and data interpretation, the results of the finished trial may be buried rather than published if they are unfavorable to the sponsor’s product. Such issues are not theoretical”

Davidoff F et al. Sponsorship, authorship, and accountability. N Engl J Med. 2001 Sep 13;345(11):825-7.

“Meta-analysis, done properly is a systematic effort to search for and winnow out all the best evidence and show how well a given intervention works. It is crucially dependent on the identification of all available data from clinical trials. In 1989, Gotzsche, who was performing a meta-analysis of 244 trials of nonsteroidal anti-inflammatory drugs in rheumatoid arthritis drew attention to a practice that seemed to subvert the normal process of publication as well as of meta-analysis. Excluding abstract, letters and brief versions, Gotzsche found 44 multiple publications of 31 of the clinical trials, 20 trials published twice, 10 three times and 1 trial 5 times, with the overall proportion of multiple publications being at least 18%. The fact that the data had been published elsewhere was not noted in 32 of the 44 articles. Indeed, in about half of them, the first author and number of authors were different, and in half there were important discrepancies between the various versions of the same trial…In 1996, Huston and Moher found the same problem when they attempted to perform a meta-analysis of the effects of the antipsychotic agent risperidone.”

Rennie D. Editorial: Fair Conduct and Fair Reporting of Clinical Trials. JAMA. 1999 Nov 10;282(18).

“The presentation of results can also be rather unethical. Results expressed as relative risk reduction should be avoided. Misoprostol reduces by 51% (relative risk reduction) serious gastrointestinal events in patients receiving NSAID [non-steroidal anti-inflammatory drugs], compared to the placebo. This reduction is statistically significant and sounds clinically pertinent. But the absolute risk reduction is only 3.8 per thousand (% of events: placebo 7.4 per thousand; misoprostol 3.6 per thousand). Much more informative is the number of patients needed for treatment (reciprocal of the absolute risk reduction). 263 patients are needed for treatment with misoprostol over 6 months to avoid one case of a serious gastrointestinal event. At the same time, for every 13 patients treated, one patient will experience diarrhoea or abdominal pain sever enough to stop the drug…[by contrast] Anticoagulant therapy reduces by 53% compared to placebo, the risk of recurrence of stroke in patients with atrial fibrillation. Absolute risk reduction is 9%. Thus, only 11 patients treated for one yeare are needed to avoid one case of recurrence of stroke.”

Chassany O, Duracinsky M. Ethics and clinical trials. Fundam Clin Pharmacol. 1999;13(4):437-44.

“Articles studied included simple random samples of reports of original research (including meta-analyses but not other types of reviews) appearing in 5 medical journals between July 1, 1996, and June 30, 1997 (Annals of Internal Medicine, BMJ, JAMA, Lancet, and New England Journal of Medicine); all articles appearing in a sixth journal CMAJ (Canadian Medical Association Journal), between July 1, 1996, and August 15, 1997, were also studied…The proportion of deficient abstracts ranged from a low of 18% [in one journal, not named] to a high of 68%. Inconsistency between abstract and body was generally more common than omitted data (ie, data in the abstract not found in the body). A substantial proportion of deficient abstracts contained both kinds of defects (25/104; 24%)…These findings are all the more surprising considering that the journals studied are all prominent and highly regarded general medical publications whose editors were founding members of the International Committee of Medical Journal Editors, a respected standard-setting body. These journals have full-time professional staffs who can be presumed to devote a good deal of time and energy to editorial and production processes.”

Pitkin RM et al. Accuracy of Data in Abstracts of Published Research Articles. JAMA. 1999 Mar 24/31;281(12):1110-1.

“The authors believe that the scientific community and the public will be best served by open publication of financial disclosures for readers and reviewers to evaluate”

Krimsky S, Rothenberg LS. Financial interest and its disclosure in scientific publications. JAMA. 1998 Jul 15;280(3):225-6.

“a tendency exists among investigators, peer reviewers, and journal editors to allow the direction and statistical significance of research findings to influence their decisions regarding submission and publication, and that about one in two trials initially reported in summary form is never reported in sufficient detail to permit an informed judgment about the validity of its results.”

Chalmers I. Underreporting research is scientific misconduct. JAMA. 1990 Mar 9;263(10):1405-8.

“For the 102 trials that had been completed but for which a report had not been submitted, the lack of a beneficial effect of the treatment was the most frequently cited reason for not publishing (34%)…Negative trial outcome was stated as the reason for rejection of 7 of the 23 submitted papers (30%) or 7 of the 13 papers (54%) for which peer review was complete.”

Dickersin K et al. Publication bias and clinical trials. Control Clin Trials. 1987 Dec;8(4):343-53.

“A double-blind, randomised, multicentre trial design was developed in December, 1983, by a drug company to compare their investigational synthetic prostaglandin analogue with ranitidine in patients with endoscopically proven gastric ulcer…In March, 1986, the Danish investigators [one of the trial centers] asked for the trial report and were informed by local company officials that some data computing still needed to be done but that preliminary analyses had demonstrated that the investigational drug produced significantly lower healing rates than ranitidine…Healing rates in the other centres were similar except in one country…and that group…planned to submit their own results for independent publication…In April, 1987, the Danish investigators had still not received the complete report…Meanwhile registration was being sought with several national drug licensing authorities”

Lauritsen K et al. Withholding unfavourable results in drug company sponsored clinical trials. Lancet. 1987 May 9;1(8541):1091.

“The pooled analysis for the published trials [comparing alkylating agents to combination chemotherapy for treatment of ovarian cancer] with survival data…[shows] a statistically significant survival advantage for combination chemotherapy…The estimated improvement in median survival was 16% to 19% [a few months]…a clear, albeit modest, enhancement of survival…By contrast to the pooled results of published trials, the registered trials show, in general, no statistically significant survival advantage for combination chemotherapy…Furthermore, the estimated media survival ratios are all close to unity [i.e. survival with either treatment was, on average, about the same]…Hence, a pooled analysis based on the registered ovarian trials shows no clear advantage for combination chemotherapy…The positive trials appeared in prominent journals such as New England Journal of Medicine and Cancer…Less widely read journals contained only negative trials. Studies with significant results were published within 5 years of study closure. Several negative trials were published more than 5 years after closure or have yet to be published…[However, based on the prominently published data] In May 1985 in the National Cancer Institute Database PDQ (Physician Data Query), several forms of combination chemotherapy were recommended as ‘standard treatment’ for advanced ovarian cancer.”

Simes RJ. Confronting publication bias: a cohort design for meta-analysis. Stat Med. 1987 Jan-Feb;6(1):11-29.

“we consider 11 important aspects of [clinical trial] design and analysis, we surveyed all 67 clinical trials published in the New England Journal of Medicine, the Lancet, and the British Medical Journal from July through December 1979 and in the Journal of the American Medical Association from July 1979 through June 1980. Of all 11 items in the 67 trials published in all four journals, 56% were clearly reported, 10% were ambiguously mentioned, and 34% were not reported at all...only 19% reported the method of randomization...Although information about whether patients were blind to treatment was given in 55%, information whether there was blind assessment of outcome was reported in only 30%. The statistical power of the trial to detect treatment effects was discussed in only 12% of the articles”

DerSimonian R et al. Reporting on methods in clinical trials. N Engl J Med. 1982 Jun 3;306(22):1332-7.

Ethics of Clinical Trials

Ross JS et al. Guest authorship and ghostwriting in publications related to rofecoxib. JAMA. 2008;299(15):1800-12.

“Patients died prematurely in two failed clinical trials at Seattle’s Fred Hutchinson Cancer Research Center - experiments using drugs in which the center and its doctors had a financial interest. The patients and their families were never told about those connections, nor were they fully and properly informed about the risks of the experiment”

Wilson D, Heath D. Uninformed consent: what patients at ‘The Hutch’ weren’t told about the experiments in which they died. Seattle Times. 2001 Mar 11-15
http://seattletimes.nwsource.com/uninformed_consent/

“We noted in the November 18 issue (1) that Dr. Vera Price, the author of a Drug Therapy article published in the September 23 issue, (2) had received major research support through her university from companies that manufactured drugs discussed in her article and that she had recently consulted for those companies. Although she informed us of these arrangements at the time her article was solicited, we failed to respond in accordance with our conflict-of-interest policy, which would have disqualified her. Because of the inconsistency between our stated policy and practice in this case, we reviewed the files and spoke with the authors of the Drug Therapy articles published since January 1, 1997, to determine whether there were similar inconsistencies. We have now completed our review. We found 18 instances in which one or more authors of Drug Therapy articles received major research support through their institutions from relevant companies or served as consultants for those companies at the time they were invited to prepare their articles. The authors had informed us of these arrangements, and the editors in Boston were aware of them and approved them. The list is as follows:

Lemon S, Thomas DL. Vaccines to prevent viral hepatitis. N Engl J Med 1997;336:196-204. Dr. Lemon received grants from SmithKline Beecham Pharmaceuticals.

Hoofnagle JH, Di Bisceglie AM. The treatment of chronic viral hepatitis. N Engl J Med 1997;336:347-56. Drs. Hoofnagle and Di Bisceglie received grants and other support from Abbott Laboratories, Chiron, Eli Lilly, Glaxo Wellcome, Roche, Schering-Plough, and Viratek.

Delmas PD, Meunier PJ. The management of Paget's disease of bone. N Engl J Med 1997;336:558-66. Drs. Delmas and Meunier received grants from Boehringer Ingelheim, Merck, Novartis, Procter & Gamble, Rhone–Poulenc Rorer Pharmaceuticals, and Sanofi.

Quagliarello VJ, Scheld WM. Treatment of bacterial meningitis. N Engl J Med 1997;336:708-16. Drs. Quagliarello and Scheld received grants from Pfizer and Roche.

Holleman F, Hoekstra JBL. Insulin lispro. N Engl J Med 1997;337:176-83. Dr. Holleman received grants from Eli Lilly and Hoeschst Marion Roussel.

Rudick RA, Cohen JA, Weinstock-Guttman B, Kinkel RP, Ransohoff RM. Management of multiple sclerosis. N Engl J Med 1997;337:1604-11. Drs. Rudick, Cohen, Kinkel, and Ransohoff received grants from Athena Neurosciences, Berlex Laboratories, Biogen, Connective Therapeutics, and Pharmacia–Upjohn.

Eastell R. Treatment of postmenopausal osteoporosis. N Engl J Med 1998;338:736-46. Dr. Eastell received grants from and served as an advisor to Eli Lilly and Procter & Gamble and served as an advisor to Novartis and SmithKline Beecham Pharmaceuticals.

Flexner C. HIV-protease inhibitors. N Engl J Med 1998; 338:1281-92. Dr. Flexner received grants from Abbott Laboratories and Agouron Pharmaceuticals.

Hortobagyi GN. Treatment of breast cancer. N Engl J Med 1998;339:974-84. Dr. Hortobagyi received grants from Bristol-Myers Squibb and Targeted Genetics.

Alward WLM. Medical management of glaucoma. N Engl J Med 1998;339:1298-307. Dr. Alward received a grant from Alcon Laboratories.

Osborne CK. Tamoxifen in the treatment of breast cancer. N Engl J Med 1998;339:1609-18. Dr. Osborne received a grant from and served as a consultant to AstraZeneca.

Drazen JM, Israel E, O'Byrne PM. Treatment of asthma with drugs modifying the leukotriene pathway. N Engl J Med 1999;340:197-206. Drs. Drazen, Israel, and O'Byrne received grants from and served as advisors to AstraZeneca, Boehringer Ingelheim, Glaxo Wellcome, Merck, Schering-Plough, Sepracor, SmithKline Beecham Pharmaceuticals, and 3M.

Elia J, Ambrosini PJ, Rapoport JL. Treatment of attention-deficit–hyperactivity disorder. N Engl J Med 1999;340:780-8. Dr. Elia received grant support from Celegene.

Steinberg MH. Management of sickle cell disease. N Engl J Med 1999;340:1021-30. Dr. Steinberg received a grant from Theradex.

Swift RM. Drug therapy for alcohol dependence. N Engl J Med 1999;340:1482-90. Dr. Swift received grants from Dupont, Lipha, and Pfizer.

Knopp R. Drug treatment of lipid disorders. N Engl J Med 1999;341:498-511. Dr. Knopp received grants from Bayer, Bristol-Myers Squibb, Kos Pharmaceuticals, Merck, Parke-Davis, Pfizer, and Wyeth–Ayerst.

Ferris FL III, Davis MD, Aiello LM. Treatment of diabetic retinopathy. N Engl J Med 1999;341:667-78. Dr. Davis received grants from Agouron Pharmaceuticals, Eli Lilly, Miravant Medical Technologies, Novartis, and Roche, and Dr. Aiello received a grant from Eli Lilly.

Vance ML, Mauras N. Growth hormone therapy in adults and children. N Engl J Med 1999;341:1206-16. Dr. Vance received grants from Eli Lilly and Pharmacia–Upjohn, and Dr. Mauras received grants from Alkermes, Eli Lilly, Genentech, Merck, Pharmacia–Upjohn, and Wyeth–Ayerst.

We regret our failure to apply our policy correctly. Steps have been taken to ensure against any recurrence.

Marcia Angell, M.D. Robert D. Utiger, M.D. Alastair J.J. Wood, M.D. References 1. Price VH, Angell M, Wood AJJ. Authors' conflicts of interest: a disclosure and editors' reply. N Engl J Med 1999;341:1618-9. 2. Price VH. Treatment of hair loss. N Engl J Med 1999;341:964-73.”

Disclosure of Authors’ Conflicts of Interest: A Follow-Up. N Engl J Med. 2000 Feb 24;342(8):586-7.

“The efficacy of drugs has to be assessed by clnical trials. But, a trial can be unethical if its scientific and methodological quality is poor, such as for example: the trial is not comparative, not randomised, or is a without-blind procedure (even though it is feasible), there is no calculation of the number of patients to be included, the trial is not in compliance with Good Clinical Practice, or has a flawed analysis strategy (e.g. 'per protocol' analysis instead of 'intent to treat analysis'). The French National Ethics Committees reaffirmed in 1986: 'What is not scientific is not ethical.'”

Chassany O, Duracinsky M. Ethics and clinical trials. Fundam Clin Pharmacol. 1999;13(4):437-44.

“With audiences of lesser expertise [than scientific peers], or the general public, the obligation to reveal the limitations and weaknesses in data or methodology are much greater. There is a clear duty to provide assessment and explanation with appropriate reservations about the preliminary state of evidence and the fact that it may change with more data. This is an obligation all too often honored only in the breach, especially when the possibility of a new treatment, however remote, is present. Understandably, those afflicted with a serious illness and their families grasp desperately at any suggestion of cure or amelioration. They often listen selectively. They tend to filter out the negatives and overestimate the potential benefits. Such subjects are exceedingly vulnerable and exploitable, all of which sharpen the obligation to provide evidence with ethical circumspection.”

Pellegrino ED. The ethical use of evidence in biomedicine. Eval Health Prof. 1999 Mar;22(1):33-43.

Clinical Trials and Markeeting

“The pharmaceutical industry spends more time and resources on generation, collation, and dissemination of medical information than it does on production of medicines.”

Collier J, Iheanacho I. The pharmaceutical industry as an informant. Lancet. 2002 Nov 2;360(9343):1405-9.

“We all know that there is no such thing as a free drugs company lunch, but, what the hell, the food's good, those marketing people seem ever so nice, and, anyway, there doesn't really seem to be any catch. The catch, however, need no longer be a mystery. For the marketers have set out explicitly the methods they use to "educate" doctors into championing their products.

The May edition of Pharmaceutical Marketing magazine explains what it calls "the tricks of the trade." These include developing "opinion leaders" (key doctors or other health professionals or even patient group representatives who are likely to be able to influence many others); building successful relationships with them ("how you can best get them to know and like you and your products"); and skills training ("ensuring your product champions communicate effectively on your behalf").

These "tricks" appear in a 24 page supplementary guide under the heading "Effective medical education." The guide, written by people from the industry, describes medical education as "a potent weapon to be used by the marketer in supporting promotional activities."

This may at first not seem all that surprising, but Joe Collier, professor of medicines policy at St George's Hospital and Medical School, London, said, "Doctors are being cultivated because they are valued as promotional tools, and that is new. Are they being paid out of scientific funds or out of promotional funds?"

He said that what the marketers meant by "medical education" was effectively advertising and so should be covered by advertising regulations: "The use of opinion leaders hasn't previously been viewed as advertising, but now it has to be brought into that fold." He added, "The marketers are talking about educating people about their medicines, so `medical' and `medicines' are being used interchangeably, which is a trick." Professor Collier said that the guide set out "a detailed method of manipulation, a concept of using and duping that is very underhand." He added, "It's very blatant. Doctors are visibly being used as pawns."

So what exactly does this guide say? It advises marketers, in identifying opinion leaders, not to "risk wasting money" on doctors "who you eventually hear have no credibility with their peers." Instead, marketers should aim for those who are "on the editorial boards of key publications for ultimate target audiences," on scientific committees, members of key professional societies, representatives of national or international guideline committees, and key players on formulary committees. "The key aim," says the guide, "is to ensure that you are working with a mix of people who can ultimately be called upon to communicate on your behalf in different situations."

Marketers are advised not to overuse people: "If you front the same people to speak at your symposia, write publications etc, they will inevitably be seen as being in your pocket. No one wants to have a reputation for only having a handful of supporters."

Opinion leaders who are not prepared to overtly support a product are also worth cultivating. "Remember the halo effect," says the guide. "Being seen to merely work with you is good enough."

Marketers are advised to ask opinion leaders for their advice. "The advisory process is one of the most powerful means of getting close to people and of influencing them. Not only does it help shape medical education overall, it can help in the process of evaluating how individuals can best be used, motivate them to want to work with you and with subliminal selling of key messages ongoing all the while."

At one point, the guide starts referring to the opinion leader as a "trainee." Professor Collier said, "To call someone a trainee when we are talking about professor so-and-so is a bit rich."

Hugh Gosling, editor of Pharmaceutical Marketing, denied that there had been any departure in the way that the pharmaceutical industry dealt with opinion leaders. "I think it's exactly the same as it always has been. I don't feel that there has been a turning point in any way."

He said that the boundaries between what doctors and drug companies thought was meant by medical education had become more blurred in the past couple of years. "Medical education used to be seen as purely non-promotional. That is no longer the case," he said, adding that medical education and promotional activities worked well together.

But he denied that there was anything sinister or any duping taking place and felt strongly that the pharmaceutical industry acted ethically in its dealings with opinion leaders. "No one wants to be used," he said.”

Jackson T. Are you being duped? How drug companies use opinion leaders. BMJ. 2001 May 26;322(7297):1312.

“More than $11 billion is spent each year by pharmaceutical companies in promotion and marketing, $5 billion of which goes to sales representatives. It has been estimated that $8000 to $13,000 is spent per year on each physician”

Wazana A. Physicians and the pharamceutical industry: is a gift ever a gift?. JAMA. 2000 Jan 19;283(3):373-80.

“Much previous research of physician drug decision-making has been based on physicians’ own assessments of what forces influence their prescribing behavior; the present study suggests that this is not a reliable measure. We chose two drugs groups (cerebral vasodilators and propoxyphene products) for which the information about efficacy conveyed by scientific sources differs markedly from that conveyed by commercial sources...Although the vast majority of practitioners perceived themselves as paying little attention to drug advertisements and detail men [drug salesmen], as compared with papers in the scientific literature, their beliefs about the effectiveness of the index drugs revealed quite the opposite pattern of influence in large segments of the sample...In the absence of mandatory postgraduate education or recertification, pharmaceutical advertising becomes...the major source of continuing education for American physicians”

Avorn J, Chen M, Harltey R. Scientific versus commercial sources of influence on the prescribing behavior of physicians. Am J Med. 1982;73:4-8.

Peer Review

“Does a science peer review system based on secret submission policies benefit the American public who fund science? A review by this author of correspondence between the prestigious Proceedings of the National Academy of Sciences of the United States of America – the print weekly and online daily research journal (paid subscription) of the National Academy of Sciences – and the authors of several recent scientific papers, most eventually published by PNAS, reveals a nasty back story about submission procedures that in some cases work against the best interests of the public as well as sound science.”

Mazur S. Turmoil at the NAS. CounterPunch. 2010 Jan 7
http://counterpunch.org/mazur01072010.html

“This week the peer review system has been in the newspapers, after a survey of scientists suggested it had some problems. That is barely news. Peer review – where articles submitted to an academic journal are reviewed by other scientists from the same field for an opinion on their quality – has always been recognised as problematic. It is time-consuming, it could be open to corruption, and it cannot prevent fraud, plagiarism, or duplicate publication, although in a more obvious case it might.”

Goldacre B. Peer review is flawed but the best we’ve got. The Guardian. 2009 Sep 12

“for some authors, the disclosure [of financial conflicts] is completely transparent to all involved in the manuscript evaluation process, including peer reviewers; but for other authors, the disclosure is made public only at the time of publication. In addition, some authors continue to misunderstand what is expected and provide inaccurate or incomplete disclosures that are discovered after publication and result in a published correction or letter of explanation.”

Flanagin A et al. Update on JAMA's Conflict of Interest Policy. JAMA. 2006 Jul 12;296(2):220-221.

“Medical journals have many problems and need reform. They are overinfluenced by the pharmaceutical industry, too fond of the mass media, and yet neglectful of patients. The research they contain is hard to interpret and prone to bias, and peer review, the process at the heart of journals and all of science, is deeply flawed. It’s increasingly apparent that many of the studies journals contain are fraudulent, and yet the scientific community has not responded adequately to the problem of fraud. Editors themselves also misbehave. The [nominal] authors of the studies in journals have often had little [to] do with the work they are reporting and many have conflicts of interest that are not declared. And the whole business of medical journals is corrupt because owners are making money from restricting access to important research, most of it funded by public money. All this matters to everybody because medical journals have a strong influence on their healthcare and lives”

Smith R. The trouble with medical journals. Royal Society of Medicine Press. 2006

“Recent disclosures of fraudulent or flawed studies in medical and scientific journals have called into question as never before the merits of their peer-review system. The system is based on journals inviting independent experts to critique submitted manuscripts. The stated aim is to weed out sloppy and bad research, ensuring the integrity of the what it has published. Because findings published in peer-reviewed journals affect patient care, public policy and the authors’ academic promotions, journal editors contend that new scientific information should be published in a peer-reviewed journal before it is presented to doctors and the public. That message, however, has created a widespread misimpression that passing peer review is the scientific equivalent of the Good Housekeeping seal of approval. Virtually every major scientific and medical journal has been humbled recently by publishing findings that are later discredited. The flurry of episodes has led many people to ask why authors, editors and independent expert reviewers all failed to detect the problems before publication.…even the system’s most ardent supporters acknowledge that peer review does not eliminate mediocre and inferior papers and has never passed the very test for which it is used. Studies have found that journals publish findings based on sloppy statistics. If peer review were a drug, it would never be marketed, say critics, including journal editors.”

Altman L. For science's gatekeepers, a credibility gap. NY Times. 2006 May 2

“most of the evidence in what doctors believe to be “evidence-based medicine” is more infomercial than dispassionate science. It’s vital to protect the integrity of our medical knowledge. But the current peer review system alone can’t do the job. The journals, and the peer reviewers they rely on, are in the untenable position of having to trust that corporate sponsors have accurately and completely reported their findings. At present, journal editors and peer reviewers typically are not allowed unrestricted access to the data from commercially sponsored research. Amazingly, many drug company-funded researchers who write the articles are also not allowed access to all of the data the company has collected. There is no better cautionary tale than the unwarranted success of Vioxx”

Abramson J. Drug profits infect medical studies. Los Angeles Times. 2006 Jan 7

“Once upon a time there was a little-known patent clerk in Bern [Einstein] who received a disappointing annual performance review in [19]’05…[This paper reprints peer review rejections by the following leading computer scientists. In all cases the papers being reviewed are directly related to what they are now famous for developing.]…E.W. Dijkstra [an important theoretician of computer science who also created structured programming which is used universally today. Many modern computer languages or development tools directly support this] ‘Goto Statment Considered Harmful’…The author is a proponent of the so-called “structured programming” style, in which, if I get it right, gotos are replaced by indentation. Structured programming is a nice academic exercise, which works well for small examples, but I doubt that any real-world program will ever be written in such a style [new computer languages generally do not have a goto statement but support structured programming in numerous ways] .…E.F. Codd [The father of Relational Databases] “A Relational Model of Data for Large Shared Data Banks.”…The paper contains no real-world example to convince us that any model of practical interest can be cast in it [Oracle is a company based entirely on Relational Databases and has made the founder, Larry Ellison, fabulously wealthy]…A. Turing [The father of computing as a science and critical to the British WW II Engima code machine cracking project] “On Computable Numbers, with an Application to the Entscheidungsproblem.”…This is a bizarre paper. It begins by defining a computing device absolutely unlike anything I have seen, then proceeds to show—I haven’t quite followed the needlessly complicated formalism—that there are numbers that it can’t compute [google ‘Turing Machine’]…C.E. Shannon [Creator of information theory]…“A Mathematical Theory of Communication.” This paper is poorly motivated and excessively abstract.…these days, television is sexy and color television is even sexier. Discrete channels with a finite number of symbols are good for telegraphy, but telegraphy is 100 years old, hardly a good research topic. [Shannon was describing digital communications which is almost universal today (except for analog television and other quickly vanishing relatives)]…R.L. Rivest, A. Shamir, and L. Adelman [Creators of the widely used RSA security system] “A Method for Obtaining Digital Signatures and Public-Key Cryptosystems.”…I doubt that a system such as this one will ever be practical…Further, as the authors acknowledge, a data encryption standard already exists…True, the IBM method presents the problem of distributing the encryption key, but their method is a standard and we must live with it…Finally, there is the question of the application. Electronic mail on the Arpanet [now known as the internet] is indeed a nice gizmo, but it is unlikely it will ever be diffused outside academic circles and public laboratories— environments in which the need to maintain confidentiality is scarcely pressing…Granted, we are seeing the appearance of so-called microcomputers, such as the recently announced Apple II, but their limitations are so great that neither they nor their descendants will have the power necessary to communicate through a network.”

Santini S. We are sorry to inform you…. IEEE Computer. 2005 Dec

“There is increasing concern that in modern research, false findings may be the majority of even the vast majority of published research claims…Conflicts of interest are very common in biomedical research, and typically they are inadequately and sparsely reported…Prestigious investigators may suppress via the peer review process the appearance and dissemination of findings that refute their findings, thus condemning their field to perpetuate false dogma. Empirical evidence on expert opinion shows that it is extremely unreliable…Highly prejudiced stakeholders may even create a barrier that aborts efforts at obtaining and disseminating opposing results…investigators working in any field are likely to resist accepting that the whole field in which they have spent their careers is a ‘null field’. However, other lines of evidence, or advances in technology and experimentation, may lead eventually to the dismantling of a scientific field”

Ioannidis JP. Why most published research findings are false. PLoS Med. 2005 Aug;2(8):e124.

“The BMJ is one of the four best-respected general medical journals in the world, with the Lancet, the New England Journal of Medicine and the Journal of the American Medical Association. They send out scientific papers for peer review by academics in the same field as the authors. This process gives the paper a sort of validation or stamp of authority in the medical world. But Smith and other journal editors have become disenchanted with it. ”Peer review is a very flawed practice,” he says. “It is slow and expensive, a lottery, and prone to abuse and bias. Much of the time it doesn’t pick up errors.””

Boseley S. Science/Medical journals are immoral, a former BMJ editor tells Sarah Boseley. The Guardian Weekly. 2005 Jul 8

“One such prodigy was John Darsee, a Harvard researcher who had, by the age of 33, published more than 125 research articles, book chapters, abstracts, and other papers. But it turned out that he had fabricated data in scores of papers. Some of the problems with his data should have been obvious at the time. In one paper he described a family in which a 17-year-old male had four children--the eldest of which was 8 years old. Do the maths. These and other glaring problems eluded not only the peer reviewers and editors, but also, apparently, his coauthors.”

Judson HF. The great betrayal: Science in the culture of fraud. Harcourt. 2004

“The well-researched practice of concealing the identities of peer-reviewers or authors, while laborious and expensive, appears to have little effect on the outcome of the quality assessment process (9 studies). Checklists and other standardisation media have little reliable evidence to support their use (2 studies). There is no evidence that referees' training has any effect on the quality of the outcome (2 studies). Electronic communication media do not appear to have an effect on quality (2 studies). On the basis of one study little can be said about the ability of the peer-review process to detect bias against unconventional drugs. Validity of peer-review was tested by only one small study in a specialist area. Editorial peer-review appears to make papers more readable and improve the general quality of reporting (2 studies), but the evidence for this may be of limited generalisability…At present there is little empirical evidence to support the use of editorial peer-review as a mechanism to ensure quality of biomedical research, despite its widespread use and costs.”

Jefferson TO et al. Editorial peer-review for improving the quality of reports of biomedical studies . Cochrane Methodology Reviews. 2003
http://www.cochrane.org/reviews/en/mr000016.html

“The peer review system derives its power from the little known practice of governments to deputize their authority to distribute funds for research to committees of “experts”. These experts are academic researchers distinguished by outstanding contributions to the current establishment. They alone review the merits of research applications from their peers, and they have the right to elect each other to review committees. Outwardly, this “peer review system” appears to the unsuspecting government and taxpayer as the equivalent of a jury system – free of all conflicts of interest. But, in view of the many professional and commercial investments in and benefits from their expertise, and even of the rewards from their universities and institutions for the corresponding overheads and partnerships – all legal in the US since president Reagan – “peer reviewers” do not fund applications that challenge their own interests. Since “peer review” is protected by anonymity, does not allow the applicant personal representation or an independent representative, nor a say or even a veto in the selection of the “jury”, and does not allow an appeal, its powers to defend the orthodoxy are unlimited. The corporate equivalent of academia’s “peer review system” would be to give General Motors and Ford the authority to review and veto all innovations by less established carmakers competing for the consumer. Even the professional journals and the science writers of the public media comply with the interests of government- funded majorities because they depend on their monthly “scientific breakthroughs”, the lucrative advertisements from their companies, and the opinion of their subscribers. For example, an early precursor of this article was written in response to an open invitation from a pharmacology-journal over 3 years ago. But, after considerable pressure on the journal from anonymous “AIDS experts”, the editor requested a reduced article, which was neither accepted nor rejected. Instead, the editor simply dropped all further correspondence. Subsequently, the editor of a prestigious German-based science journal invited another precursor of this article 2 years ago, which received two favourable reviews in short order. But before the manuscript could be revised, the editor informed us that the publisher was concerned about losing subscribers if our paper were published and ceased all further correspondence. It is this passive resistance that can grind down even the most determined truth seeker.”

Duesberg P et al. The chemical bases of the various AIDS epidemics: recreational drugs, anti-viral chemotherapy and malnutrition. J Biosci. 2003 Jun;28(4):383-412.

“Editorial peer review is used world-wide as a tool to assess and improve the quality of submissions to paper and electronic biomedical journals. As the information revolution gathers pace, an empirically proven method of quality assurance is of paramount importance. The increasing availability of empirical research on the possible effects of peer review led us to carry out a review of current evidence on the efficacy of editorial peer review. We found few studies of reasonable quality, and most of these were concerned with the effects of blinding reviewers and/or authors to each others’ identity. We could not identify any methodologically convincing studies assessing the core effects of peer review. Major research is urgently needed.”

Editorial peer review for improving the quality of reports of biomedical studies. Cochrane Database Syst Rev. 2003

“Pulitzer Prize winner John Crewdson’s revelations are startling: a malevolent autocrat in his little fiefdom perched at the top of the biomedical hierarchy burning with obsession for a Nobel Prize; and rivals, like himself, sabotaging each other’s work, stealing credit from subordinates (peer review is an opportunity to steal). Superiors at the National Institutes of Health and other institutions allow these top “scientists” to act with impunity, insuring that their pronouncements, no matter how unlikely, be received as gospel by credulous scientists and doctors below them in the pecking order that is big science. ”

Crewdson J. Science fictions: A scientific mystery, a massive cover-up, and the dark legacy of Robert Gallo. Little, Brown. 2002
http://www.sciencefictions.net

“Many readers seem to assume that articles published in peer-reviewed journals are scientifically sound, despite much evidence to the contrary. It is important, therefore, that misleading work be identified after publication…Recent draft recommendations from the World Association of Medical Editors say that "[e]ditors should promote self-correction in science and participate in efforts to improve the practice of scientific investigation by…publishing corrections, retractions, and letters critical or articles published in their own journal."…Editors should give special attention to letters making criticism of methodology. They should do one of the following: satisfy themselves (perhaps by having the letter peer reviewed) that the criticisms are unfounded or unimportant, agree to publish the letter and invite the authors to respond, or invite a response from the authors and then decide whether to publish…Time limitation on correspondence denies readers the opportunity to draw attention to methodological deficiencies.…In effect, [in the 6 major medical journals used as examples] there is a statute of limitations by which authors of articles in these journals are immune to disclosure of methodological weaknesses once some arbitrary (short) period has elapsed, which cannot be right.”

Altman DG. Poor-quality medical research: What can journals do?. JAMA. 2002 Jun 5;387(21):2765-7.

“But some journals do send the disclosure statements to referees for good reason, argues Krimsky. "Personal financial interest is a relevant factor in raising the scepticism of peer reviewers," he claims. "There are too many articles that are not sufficiently rigorously reviewed, so the flaws in them are only disclosed after publication." [obviously many journals are not sending financial conflict of information to reviewers]”

Kolfschooten KF. Conflicts of interest: can you believe what you read?. Nature. 2002 Mar 28;416(6879):360.

“Science bases its presumed authority in the world on the reliability and objectivity of the evidence that is produced. If the pronouncements of science are to be greeted with public confidence...it should be able to demonstrate that peer review, ‘one of the sacred pillars of the scientific edifice’, is a process that has been validated objectively as a reliable process for putting a stamp of approval on work that has been done. Peer review should also have been validated as a reliable method for making appropriate choices as to what work should be done. Yet when one looks for that evidence it is simply not there”

Horrobin DF. Something rotten at the core of science?. Trends in Pharmacological Science. 2001 Feb;22(2):51-2.

“We studied two journals in which manuscripts were routinely assessed by two reviewers, and two conferences in which abstracts were routinely scored by multiple reviewers. Agreement between the reviewers as to whether manuscripts should be accepted, revised or rejected was not significantly greater than that expected by chance”

Rothwell PM, Martyn CN. Reproducibility of peer review in clinical neuroscience. Is agreement between reviewers any greater than would be expected by chance alone?. Brain. 2000 Sep;123 ( Pt 9):1964-9.

“[Comments to a National Academy of Sciences panel on peer review include] 'I have been on study sections and have seen members who clearly lacked expertise review proposals and grade proposals in a biased, or self-serving, or bad scientific manner', 'Under the present 'culture', which focuses on fault finding and amplification of minor errors and discouraging innovative research, nearly all NIH funding has gone into confirming, reconfirming, and reinventing what is already known, by individuals of very little insight or talent', 'I have seen the results of ideas being stolen. Who will be believed the experienced peer [who may have stolen the ideas] or the new investigator?', 'Unscientific grant review rhetoric never receives objective scrutiny', 'The AIDS and Related Research Study Section was composed of individuals with widely different areas of expertise…For the most part, we couldn't understand the reviews written by other members of the panel'”

Agnew B. NIH Eyes Sweeping Reform of Peer Review. Science. 1999 Nov 5;286:1074-6.

“History suggests a near universal rule-that innovation comes from an unexpected direction, and that it is usually opposed by leading authorities in the field...I suspect that peer reviewers sometimes subconsciously give harsh reviews to clinical projects, ostensibly because the science is poor, but in reality because they fear successful outcomes-the advent of cures would herald the demise of most research funds in their field...Peer review, as at present constituted, encourages lying and favours the corrupt”

Horrobin DF. Peer review of grant applications: a harbinger for mediocrity in clinical research?. Lancet. 1996 Nov 9;348(9037):1293-5.

“Peer review can be performed successfully only if those involved have a clear idea as to its fundamental purpose. Most authors of articles on the subject assume that the purpose of peer review is quality control. This is an inadequate answer. The fundamental purpose of peer review in the biomedical sciences must be consistent with that of medicine itself, to cure sometimes, to relieve often, to comfort always. Peer review must therefore aim to facilitate the introduction into medicine of improved ways of curing, relieving, and comforting patients. The fulfillment of this aim requires both quality control and the encouragement of innovation. If an appropriate balance between the two is lost, then peer review will fail to fulfill its purpose.”

Horrobin DF. The philosophical basis of peer review and the suppression of innovation. JAMA. 1990;263(10):1438-41.

“a tendency exists among investigators, peer reviewers, and journal editors to allow the direction and statistical significance of research findings to influence their decisions regarding submission and publication, and that about one in two trials initially reported in summary form is never reported in sufficient detail to permit an informed judgment about the validity of its results.”

Chalmers I. Underreporting research is scientific misconduct. JAMA. 1990 Mar 9;263(10):1405-8.

“For the 102 trials that had been completed but for which a report had not been submitted, the lack of a beneficial effect of the treatment was the most frequently cited reason for not publishing (34%)…Negative trial outcome was stated as the reason for rejection of 7 of the 23 submitted papers (30%) or 7 of the 13 papers (54%) for which peer review was complete.”

Dickersin K et al. Publication bias and clinical trials. Control Clin Trials. 1987 Dec;8(4):343-53.

“any substantial peer review in the field ceased with Prusiner's first prion paper. From then on, when Prusiner submitted articles to journals, he recommended they not be refereed by his competitors--and few were. His competitors, in turn, suggested that their work not be refereed by Prusiner. One journal, Cell, for whatever reason, published only papers from Prusiner and his colleagues, whereas Nature, for instance, published mostly papers from the competition.”

Taubes G. The game of the name is fame. But is it science?. Discover. 1986 Dec
http://slate.msn.com/id/2096/sidebar/42786/

The Placebo

“when an experimental drug [Nevirapine in this case] is found to be superior to a control that is not harmful [AZT in this case] (thus replacing a placebo), the effectiveness of the experimental drug is thereby established.”

Jackson B, Fleming T. A drug is effective if better than a harmless control. Nature. 2005 Apr 28;434(7037):1067.

“The level of side effects experienced tips them off: those taking the active [anti-depressant] medication are more likely to experience the standard side effects–dry mouth, weight loss or gain, dizziness, headache, nausea, insomnia, and so on–clear signals they are taking a powerful drug, while those taking the sugar pill are not. As a result, the 'double-blind' study is immediately unblinded for those rating outcomes…The so-called blind procedures in the Emslie study were at best visually impaired, subject to allegiance effects and experimenter bias”

Sparks JA, Duncan BL. The ethics and science of medicating children. Ethical Hum Psychol Psychiatry. 2004 Spring;6(1):25-39.

“Jackson felt strongly that he didn't want to drop the placebo part of his protocol. Testing the two drugs against nothing, instead of only against each other, was the only way to make a valid scientific assessment of the worth of both medications.”

Swingle AB. The pathologist who struck gold. Hopkins Medical News. 2001 Spring/Summer

Temple R, Ellenberg SS. Placebo-Controlled Trials and Active-Control Trials in the Evaluation of New Treatments; Part 1: Ethical and Scientific Issues. Ann Intern Med. 2000 Sep 19;133(6):455-63.

Ellenberg SS, Temple R. Placebo-Controlled Trials and Active-Control Trials in the Evaluation of New Treatments; Part 2: Practical Issues and Specific Cases. Ann Intern Med. 2000 Sep 19;133(6):455-63.

“In general a trial comparing a new drug to a reference treatment results in a higher effect whatever the treatment group, probably because investigators are more confident to inform their patients that they will receive, whatever the treatment allocation, an active treatment. In contrast, a placebo-controlled trial generally leads to a smaller effect whatever the treatment arm, and reflects more accurately the true effect of the drug. The use of placebo is certainly unethical in life-threatening diseases and when an effective proved drug exists, for instance in severe infectious disease, myocardial infarction, pulmonary embolism. [but this is a purely ethical argument, and earlier the authors stated 'What is not scientific is not ethical', and assuming that any treatment for a life-threatening disease is better than no treatment is surely not scientific when there is no proven therapy, or when the proof of the therapy is marred by questions about the trial or the broadness of its applicability]”

Chassany O, Duracinsky M. Ethics and clinical trials. Fundam Clin Pharmacol. 1999;13(4):437-44.

“Double-blind placebo-controlled trials offer the best option for a rapid and scientifically valid assessment of antiretroviral drugs in reducing mother to child transmission of HIV”

Dabis F et al. Methodology of intervention trials to reduce mother-to-child transmission of HIV-1 with special reference to developing countries. AIDS. 1995;9(Suppl A):67-74.

“Trials for which no double-blinding [of the placebo] was reported yielded estimates of Odds Ratios that were exaggerated by 17%, on average”

Schulz KF et al. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995 Feb 1;273(5):408-12.

“Many operations for cancer have not been properly evaluated by randomised controlled trials, and there are situations where the placebo effect is likely to be strong but the specific effect may even be harmful…A review of several studies of internal mammary artery ligation showed that the magnitude of the placebo effect of surgery is about the same as for other placebo responses-ie, 35%…For surgical procedures, as for drugs, the placebo effect must always be taken into account if any assessment is to be objective”

Johnson AG. Surgery as a placebo. Lancet. 1994 Oct 22;344(8930):1140-1.

“the placebo still exerts an important influence on our practice…The placebo effect in general practice is the power of the doctor alone to make the patient feel better, irrespective of medication”

Thomas KB. The placebo in general practice. Lancet. 1994 Oct 15;344(8929):1066-7.

“In addition to comparisons between active drugs, 29 trials (15%) [of non-steroidal anti-inflammatory drugs] also involved comparisons with a group treated with a double-blind placebo...Compared with baseline [health measurements at the beginning of the trial], placebo patients deteriorated in 3 trials but improved in 14...Thus the common statement in trials without a placebo that the drug(s) had been effective is questionable.”

Gøtzsche PC et al. Methodology and overt and hidden bias in reports of 196 double-blind trials of non-steroidal antiinflammatory drugs in rheumatoid arthritis. Controlled Clin Trials. 1989;10:31-56.

“In both surgical and medical randomized controlled trials, those studies using a placebo control had a greater gain than similar trials using an active therapy as the standard”

Miller JN et al. How study design affects outcomes in comparison of therapy, II: surgical. Stat Med. 1989 Apr;8(4):455-66.

Randomization of Participations

“Here we consider the importance of concealing the treatment allocation until the patient is entered into the trial.”

Altman DG, Schulz KF. Concealing treatment allocation in randomised trials. BMJ. 2001 Aug 25;323:446-7.

“The randomization process failed to create similar groups in six of the eight trials of mammographic screening [for breast cancer]”

Gøtzsche PC, Olsen O. Is screening for breast cancer with mammography justifiable?. Lancet. 2000 Jan 8;355:129-34.

“Trials with unclear allocation concealment [hiding which participant was assigned to which group] still yielded exaggerated estimates of treatment effects after accounting for [several factors]...Odds Ratios [risk of no treatment effect occurring with test drug versus placebo] were, on average, 30% lower...estimating larger treatment effects.”

Schulz KF et al. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995 Feb 1;273(5):408-12.

“Randomization eliminates selection biases in controlled trials. Unfortunately, investigators often address randomization improperly in the design and implementation phases of trials and neglect it in published reports. Moreover, an analysis of prominent general journals revealed that among trials in which unrestricted randomization was used, the sample sizes in the two comparison groups were more similar than would be expected by chance. Furthermore, results of only 4% of hypothesis tests comparing baseline characteristics were significant at the 5% level...Only 32% of the reports described an adequate method for generating a sequence of random numbers, and only 23% contained information showing that steps had been taken to conceal assignment until the point of treatment allocation. A mere 9% described both sequence generation and allocation concealment. In reports of trials that had apparently used unrestricted randomization, the differences in sample sizes between treatment and control groups were much smaller than would be expected due to chance. In reports of trials in which hypothesis tests had been used to compare baseline characteristics, only 2% of reported test results were statistically significant, lower than the expected rate of 5%.”

Schulz KF et al. Assessing the Quality of Randomization From Reports of Controlled Trials Published in Obstetrics and Gynecology Journals. JAMA. 1994 Jul 13;272(2):125-8.

“80 reports of randomised clinical trials in 4 leading general medical journals were reviewed. The reporting of the methodology of randomisation was inadequate. In 30% of trials there was no clear evidence that the groups had been randomised. Among trials that used simple randomisation the sample sizes in the two groups were too often similar, and there was an unexpected small bias in favour of there being fewer patients in the experimental group. The handling of comparisons of baseline characteristics was inadequate in 41% of the trials...Randomisation allocates treatments without bias, but does not necessarily produce groups that are similar in important prognostic factors”

Altman DG, Doré CJ. Randomization and baseline comparisons in clinical trials. Lancet. 1990 Jan 20;335(8682):149-53.

“In this study of 128 evaluations reported in journals selected from four medical disciplines, half of the investigations used a randomized design. The overall likelihood of success reported from studies, of innovations over standard therapy as measured by the Mann-Whitney statistics, was substantially greater for the non-random studies than for the randomized studies.”

Colditz GA et al. How study design affects outcomes in comparison of therapy, I: medical. Stat Med. 1989 Apr;8(4):441-54.

“In this study we observed that non-randomized [surgical] studies tended to report larger gains than did the randomized studies...Gilbert et al investigated the gains attributed to innovations in surgery and reviewed randomized and non-randomized controlled trials. They found average differences in the proportion of treatment successes of 1.3% for primary therapies and 0.4% for secondary therapies. These results contrast with corresponding gains of 12.5% and 6.0% in our study”

Miller JN et al. How study design affects outcomes in comparison of therapy, II: surgical. Stat Med. 1989 Apr;8(4):455-66.

“In the study of Bakos & Backstrom...those [expectant mothers] with even [birth]dates were assigned to oxytocin [infusion of this hormone] and those with uneven dates to amniotomy [breaking of the amniotic membrane to induce labor]. This implies that clinicians could know which method of induction would be used for each individual woman a long time before they needed to decide whether induction of labor was necessary or not...The null hypothesis, that clinicians’ preconceived ideas had not resulted in a different policy toward induction of labor for women destined to undergo amniotomy than for those destined to receive oxytocin, was rejected at p<0.00025. Also the null hypothesis, that such biases would end after the women had been entered into the trial and would thus not influence further adherence to the study protocol, which required a further intervention in women with less than acceptable progress 4 h after the first intervention, was rejected at p<0.0005.”

Keirse MJNC. Amniotomy or oxytocin for induction of labor: re-analysis of a randomized controlled trial. Acta Obstet Gynecol Scand. 1988;67:731-5.

“Trials that are historically controlled achieve more statistically significant results, whereas studies in which controls are selected at random obtain fewer significant differences between therapies tested...The nonrandomized studies differ from those in the blinded-randomization group in the smaller number of variables reported [as baseline risk factors]...Chance is...a very unlikely explanation for the 34.4% of reported tests significantly different at the 5% level in the non-randomized studies [and 81.4% of the variables were mal-distributed to favour the treatment group in these studies, versus 56.1% in blinded-randomized studies]...bias in treatment assignment could be a more important determinant of outcome than the treatments under investigation. The differences in the distribution of prognostic variables are smaller when randomization is not [?] blinded, but the trend is there.”

Chalmers TC et al. Bias in treatment assignment in controlled clinical trials. N Engl J Med. 1983 Dec 1;309(27):1358-61.

“The thromboembolism rate was significantly lower in the anticoagulated group in all the 22 studies in which it was reported. The average absolute decrease was 12.1% in the nonrandom trials with historical controls, 10.7% in those with alternatee or similar controls and 7.9% in the random control trials…[although this author believes that random controlled trials are superior] The reports of the random control trials have defects in addition to [inadequate size] and [lack of] blinding…Most lack assurances that randomization was a properly blinded procedure; compliance and withdrawals were not adequately documented in most [although the author does not believe that these defects negate the trial’s conclusions]”

Chalmers TC et al. Evidence favoring the use of anticoagulants in the hospital phase of acute myocardial infarction. N Engl J Med. 1977 Nov 17;297(20):1091-6.

“The reliance on randomization to control all extraneous risk factors has softened to the extent that investigators are accustomed to examining the distribution of known risk indicators among the treatment groups to determine whether randomization has succeeded in its goal…randomization is indeed unreliable in achieving the goal of eliminating confounding…Confounding exists if the underlying probability for the outcome of interest in the absence of treatment differs among the treatment groups…statistical significance tests amount to a futile exercise in verifying the adequacy of the randomization process…[although] it is conceivably of interest to learn whether there is some credibility to the notion that randomization was effective in controlling unknown risk factors…a factor which has a lopsided distribution among treatment groups, but which has only a marginal influence on the outcome, will not create substantial confounding. Another factor that strongly influences the outcome may account for considerable confounding with only a slightly uneven distribution among groups…the latter factor may not be found to be 'significantly' imbalanced. Tests of [statistical] significance are thus poor measures of confounding”

Rothman KJ. Epidemiologic methods in clinical trials. Cancer. 1977 Apr;39(4 Suppl):1771-5.