Are Your Medications Safe?
The FDA buries evidence of fraud in medical trials. My students and I dug it up.
By Charles Seife February 9, 2015
Agents of the Food and Drug Administration know better than anyone else just how bad scientific misbehavior can get. Reading the FDA’s inspection files feels almost like watching a highlights reel from a Scientists Gone Wild video. It’s a seemingly endless stream of lurid vignettes—each of which catches a medical researcher in an unguarded moment, succumbing to the temptation to do things he knows he really shouldn’t be doing. Faked X-ray reports. Forged retinal scans. Phony lab tests. Secretly amputated limbs. All done in the name of science when researchers thought that nobody was watching.
That misconduct happens isn’t shocking. What is: When the FDA finds scientific fraud or misconduct, the agency doesn’t notify the public, the medical establishment, or even the scientific community that the results of a medical experiment are not to be trusted. On the contrary. For more than a decade, the FDA has shown a pattern of burying the details of misconduct. As a result, nobody ever finds out which data is bogus, which experiments are tainted, and which drugs might be on the market under false pretenses. The FDA has repeatedly hidden evidence of scientific fraud not just from the public, but also from its most trusted scientific advisers, even as they were deciding whether or not a new drug should be allowed on the market. Even a congressional panel investigating a case of fraud regarding a dangerous drug couldn't get forthright answers. For an agency devoted to protecting the public from bogus medical science, the FDA seems to be spending an awful lot of effort protecting the perpetrators of bogus science from the public.
Much of my research has to do with follies, foibles, and fraud in science, and I knew that the FDA wasn’t exactly bending over backward to correct the scientific record when its inspectors found problems during clinical trials. So as part of my investigative reporting class at New York University, my students and I set out to find out just how bad the problem was—and how much important information the FDA was keeping under wraps.
The silence is unbroken even when the FDA itself seems shocked at the degree of fraud and misconduct.
We didn’t have to search very hard to find FDA burying evidence of research misconduct. Just look at any document related to an FDA inspection. As part of the new drug application process, or, more rarely, when the agency gets a tipoff of wrongdoing, the FDA sends a bunch of inspectors out to clinical sites to make sure that everything is done by the book. When there are problems, the FDA generates a lot of paperwork—what are called form 483s, Establishment Inspection Reports, and in the worst cases, what are known as Warning Letters. If you manage to get your hands on these documents, you’ll see that, most of the time, key portions are redacted: information that describes what drug the researcher was studying, the name of the study, and precisely how the misconduct affected the quality of the data are all blacked out. These redactions make it all but impossible to figure out which study is tainted. My students and I looked at FDA documents relating to roughly 600 clinical trials in which one of the researchers running the trial failed an FDA inspection. In only roughly 100 cases were we able to figure out which study, which drug, and which pharmaceutical company were involved. (We cracked a bunch of the redactions by cross-referencing the documents with clinical trials data, checking various other databases, and using carefully crafted Google searches.) For the other 500, the FDA was successfully able to shield the drugmaker (and the study sponsor) from public exposure.
It’s not just the public that’s in the dark. It’s researchers, too. And your doctor. As I describe in the current issue of JAMA Internal Medicine, my students and I were able to track down some 78 scientific publications resulting from a tainted study—a clinical trial in which FDA inspectors found significant problems with the conduct of the trial, up to and including fraud. In only three cases did we find any hint in the peer-reviewed literature of problems found by the FDA inspection. The other publications were not retracted, corrected, or highlighted in any way. In other words, the FDA knows about dozens of scientific papers floating about whose data are questionable—and has said nothing, leaving physicians and medical researchers completely unaware. The silence is unbroken even when the FDA itself seems shocked at the degree of fraud and misconduct in a clinical trial.
Such was the case with the so-called RECORD 4 study. RECORD 4 was one of four large clinical trials that involved thousands of patients who were recruited at scores of clinical sites in more than a dozen countries around the world. The trial was used as evidence that a new anti-blood-clotting agent, rivaroxaban, was safe and effective. The FDA inspected or had access to external audits of 16 of the RECORD 4 sites. The trial was a fiasco. At Dr. Craig Loucks’ site in Colorado, the FDA found falsified data. At Dr. Ricardo Esquivel’s site in Mexico, there was “systematic discarding of medical records” that made it impossible to tell whether the study drug was given to the patients. At half of the sites that drew FDA scrutiny—eight out of 16—there was misconduct, fraud, fishy behavior, or other practices so objectionable that the data had to be thrown out. The problems were so bad and so widespread that, contrary to its usual practice, the FDA declared the entire study to be “unreliable.” Yet if you look in the medical journals, the results from RECORD 4 sit quietly in The Lancet without any hint in the literature about falsification, misconduct, or chaos behind the scenes. This means that physicians around the world are basing life-and-death medical decisions on a study that the FDA knows is simply not credible.
It’s not just one study, either. The FDA found major problems with sites involved in the other three clinical trials that were used to demonstrate rivaroxaban’s safety and effectiveness. RECORD 2, for example, was nearly as awful as RECORD 4: Four out of 10 sites that the FDA inspected showed evidence of misconduct, or other issues grave enough to render the site’s data worthless—including clear evidence of data falsification at one site. In aggregate, these problems raise serious doubts about the quality of all four key rivaroxaban studies—and, by extension, doubts about how seriously we should take the claim that rivaroxaban is safe and effective. The FDA is keeping mum, even as wrongful-death lawsuits begin to multiply.
The FDA’s failure to notify the public is not merely a sin of omission. In March 2009, the FDA convened a committee of outside scientific experts to mull the “robustness and meaningfulness” of the results from the four rivaroxaban trials, RECORDs 1, 2, 3, and 4. (The agency regularly calls in advisers to get advice, or, more cynically, to get cover, about a decision the agency has to make.) When the agency briefed the committee, it was (to put it mildly) coy about the problems it was finding. It said only that inspectors had found “significant issues” at two clinical sites involved in the RECORD 4 study—and that data from one of them was included in the analysis. Inspections were still ongoing, so it’s not easy to say precisely what the agency knew at that point, but it’s clear that the FDA wasn’t admitting to everything it knew. A bunch of inspections had been completed a month prior to the meeting, and we know for certain that the agency was fully aware of major issues beyond the two it revealed to the advisory committee. In a memo dated three days before the advisory committee meeting convened, the FDA detailed “falsification of data by a subinvestigator” at a RECORD 2 site. The advisory committee was not told.
By itself, this might seem like a miscommunication or an oversight, but the FDA has a history of not notifying the public about the misconduct it finds. About a decade ago, the agency got into trouble over a newly approved antibiotic, Ketek. Inspectors had found extensive problems (including fraud) affecting key clinical trials of the drug. Yet the agency did its best to hide the problems from even its most trusted advisers. As David Ross, the FDA official in charge of reviewing Ketek’s safety, put it, “In January 2003, over reviewers’ protests, FDA managers hid the evidence of fraud and misconduct from the advisory committee, which was fooled into voting for approval.” However, when the reports of misconduct at one clinical site began appearing in the press—along with stories of liver damage and blurred vision associated with the new drug—Congress stepped in, demanding information from the agency about the fraud.
But even the Senate couldn’t wring key information about the misconduct out of the FDA. “Every excuse under the sun has been used to create roadblocks,” complained an indignant Sen. Charles Grassley, “even in the face of congressional subpoenas requesting information and access to FDA employees.” The head of the FDA, Andrew von Eschenbach, attempted to explain to Congress why the agency didn’t tell its advisory committee about the problems in the Ketek study: “After considering the fact that the investigation results were preliminary ... FDA decided to hold the Advisory Committee meeting as planned ...” without notifying the committee of the potential problems. But Rep. Bart Stupak quickly pointed to an email, which, he argued, contradicted von Eschenbach's testimony. “So either you are not being forthright with us, when I believe you are, but whoever is doing your work is trying to lead this committee down the wrong path.” And the correct path showed that site after site involved in study 3014, as well as other key Ketek studies, were tainted as well.
In the decade since the Ketek affair, it’s hard to see any change in behavior by the agency. On occasion, the FDA has even actively approved and promoted statements about drugs that, according to its own inspectors, are based upon falsehoods. At the end of 2011, the FDA learned that an audit of a Chinese site involved in a key clinical trial of a different anti-clotting agent, apixaban, had turned up evidence of fraud: Personnel had apparently been fiddling with patient records. Worse yet, the fraud appeared to invalidate one key finding of the study. Just three months earlier, the researchers running the trial proudly announced in the New England Journal of Medicine that there was a “significant reduction in mortality” among patients who took apixaban compared with those who took the old standby, warfarin. Alas, the moment you exclude the data from the Chinese fraud site, as per standard FDA procedure, that statement went out the window. Yet look at the label for apixaban—the one approved by the FDA after the fraud was discovered—and you read that “treatment resulted in a significantly lower rate of all-cause death ... than did treatment with warfarin,” backed up by the data set with the Chinese site included. In other words, the label is carrying a claim that the FDA knows is based upon fraud. In a written response to my questions on this subject, the FDA stated that, “The FDA extended the drug’s review period to address the concerns. However, the review team did conclude concluded [sic] that the data at that site and other sites in China did reflect meaningful clinical information; that was not what was considered unreliable.”
Again, this isn’t an isolated incident. I had previously encountered bogus data on FDA-approved labels when a colleague and I were looking into a massive case of scientific misconduct —a research firm named Cetero had been caught faking data from more than 1,400 drug trials. That suddenly worthless data had been used to establish the safety or effectiveness of roughly 100 drugs, mostly generics, that were being sold in the United States. But even after the agency exposed the problem, we found fraud-tainted data on FDA-approved drug labels. (The FDA still maintains its silence about the Cetero affair. To this day, the agency refuses to release the names of the 100-odd drugs whose approval data were undermined by fraud.)
And the FDA covers up drug-related misconduct in other, more subtle ways, too. For example, the agency publishes the canonical listing of generic drugs in the United States, known as the “Orange Book.” Prescription drugs in this book are often given what’s called a “therapeutic equivalence code.” This code is a two-letter designation that signals the quality of the scientific evidence that a generic is “bioequivalent” to the name-brand drug. The code “AB,” for example, tells pharmacists and physicians that there are solid scientific studies proving that bioequivalence. Another code, “BX,” signals that there isn’t sufficient data to prove the generic is bioequivalent to the name brand.
When the Cetero misconduct was uncovered, key bioequivalence studies for scores of generic drugs turned out to be worthless. By rights, some of those drugs should have had their designation downgraded from AB to BX. But even though the FDA updates the Orange Book monthly, there was no rash of drugs losing their AB rating in the months after the Cetero affair broke. In the year and a half after the Cetero fraud was first announced, I was able to identify a grand total of four generic drugs (in various dosages) that were downgraded to BX, none of which appeared to be linked to the Cetero problem. On the other hand, the one prescription generic drug that I knew for sure had been hit hard by the Cetero fraud—both key studies supporting its bioequivalence to the name brand were declared worthless—had no change in its designation. The FDA apparently allowed the drug to keep its AB badge for months without any valid data backing the drug’s bioequivalence. When asked, point blank, whether the agency had downgraded the bioequivalence code of any products due to the Cetero affair, officials promptly dodged the question. A written statement issued by the agency’s press office in response to my queries noted that the FDA requested additional data from the companies whose drugs were implicated in the Cetero affair and that “If the data were not provided within 6 months or the data provided did not support a finding of bioequivalence, FDA said it would consider changing the generic product’s therapeutic equivalence rating in the Orange Book from AB to BX.” Not a word about a single bioequivalence rating actually being changed.
Why does the FDA stay silent about fraud and misconduct in scientific studies of medicine?
This, too, is a pattern of behavior rather than a one-off. In the past few weeks, another major Cetero-type case began to emerge—this time, having to do with GVK Biosciences, a firm in Hyderabad, India. The European Medicines Agency, the European equivalent of the FDA, examined more than 1,000 drugs in various dosages affected by GVK’s “data manipulations” and has suggested pulling 700 off the market. You can find the full list on the EMA website; to their credit, the Europeans are being relatively transparent as the crisis develops. Not so much on this side of the pond, alas. So far from the FDA, we’ve heard precious little, even though there are drugs on the U.S. market that rely entirely on GVK’s tests. In a written statement, the FDA admitted that there were some 40-odd drugs whose approval depended upon GVK-run studies. Which ones? The agency is keeping mum, as it did with Cetero and with other similar cases. However, the agency assures us that it inspected GVK's facility and found nothing to be concerned about; if the situation changes, “FDA will take swift and appropriate action to ensure that the drug products available to American consumers are safe and effective.”
Why does the FDA stay silent about fraud and misconduct in scientific studies of pharmaceuticals? Why would the agency allow claims that have been undermined by fraud to appear on drug labels? And why on earth would it throw up roadblocks to prevent the public, the medical community, its advisory panels, and even Congress from finding out about the extent of medical misconduct? The answers the FDA gives are fascinating—they show how an agency full of well-meaning people can do intellectual backflips to try to justify secrecy.
The most common excuse the agency gives is that exposing the details about scientific wrongdoing—naming the trials that were undermined by research misconduct, or revealing which drugs’ approvals relied upon tainted data—would compromise “confidential commercial information” that would hurt drug companies if revealed. This claim falls apart under scrutiny. The courts have ruled that when information is provided by companies involuntarily, such as the information that an FDA inspector finds, “commercial confidential information” refers to proprietary material that causes substantial, specific harm when it falls into the hands of a competitor. It doesn’t cover embarrassing peccadilloes—or misconduct that might cause bad publicity when word gets out.
Another excuse I’ve heard from the FDA is that it doesn’t want to confuse the public by telling us about problems, especially when, in the FDA’s judgment, the misconduct doesn’t pose an immediate risk to public health. For example, when my colleague and I asked the director of FDA’s Center for Drug Evaluation and Research why the agency wouldn’t name the drugs affected by the Cetero fraud, she told us that the matter “did not rise to the level where the public should be notified. We felt it would result in misunderstanding and inappropriate actions.” But even the most paternalistic philosophy of public health can’t explain why the FDA would allow drug companies to put data on its labels that the agency knows are worthless, or to fail to flag bioequivalence problems in a publication that is specifically designed for the purpose of flagging those very problems.
Charles Seife is a journalism professor at New York University. His most recent books are Sun in a Bottle: The Strange History of Fusion and the Science of Wishful Thinking and Proofiness: The Dark Arts of Mathematical Deception
http://www.slate.com/articles/health_and_science/science/2015/02/fda_inspections_fraud_fabrication_and_scientific_misconduct_are_hidden_from.single.html
