Linda Bren
Published: Wednesday, November 19, 2008 - 16:25
From an antacid to a new cancer treatment, every drug must be proven safe and effective in controlled clinical trials before the U.S. Food and Drug Administration (FDA) allows it to be sold in the United States.
When federal law first required controlled clinical trials in 1962, most people didn’t know how to do them and they were rampant with problems, according Dr. Robert Temple, director of the FDA’s Office of Medical Policy. Over the years, FDA experts evaluated the many shortcomings of the clinical trials they saw and worked to fix them. To this end, the FDA established regulations and guidance to ensure that sponsors of clinical trials, medical researchers (investigators), and ethics committees (institutional review boards) understand how they can effectively carry out their responsibilities and comply with federal law.
"As FDA expertise evolved, so did controlled clinical trials, and the quality of their design, conduct, analysis, and interpretation improved," says Robert T. O’Neill, Ph.D., director of the FDA’s Office of Biostatistics, who has seen much of the evolution of controlled trials since he joined the FDA in 1971. "FDA has been the major force behind the development of good principles for the design and interpretation of controlled trials. We’ve promoted, fed, and cared for controlled clinical trials as a critical force in drug development, and we continue to do so today."
What Is a Controlled Clinical Trial?
"If the course of illness were always predictable, you would simply observe the treated participants, see how they fared after treatment, and decide whether the treatment helped and what its bad effects were," says Temple.
But most diseases have a variable course, so to find out the effect of a medical product, you must compare a group that got the experimental product (test group) with a group as similar as possible to the test group, but that received an inactive substance (placebo) or a treatment known to be effective. This group is called the control group, thereby making the clinical trial "controlled." Any differences in results between the test group and the control group can then be attributed to the experimental product.
A shortcoming of many early clinical trials was failure to decrease the possibility of bias. Bias refers to any factor that distorts the true outcome of a study, leading to overestimating or underestimating the effect of the product under investigation.
Bias may be introduced when investigators interpret results in one group more favorably than results in the other group, even if they are the same. Or bias may occur when investigators use their knowledge of a participant’s condition to assign that person to a specific group; for example, people who are less ill may be assigned to the investigational treatment group.
Randomization to treatment groups and double-blinding are two ways used to minimize bias in clinical trials. Randomization, essentially a coin toss by a computer, means participants are assigned by chance to a group so that neither group will be healthier or more likely to improve than the other.
Blinding is used in conjunction with randomization. Single-blinding means the participant doesn’t know whether he or she is receiving the experimental product, an established treatment for that disease, or a placebo—but the research team does know what the participant is receiving. Clinical trials are usually double-blinding, meaning that neither the participant nor the research team knows during the trial which participants receive the experimental product.
Blinding is done to make sure that such factors as investigators’ preferences or expectations, or participants’ desire to please investigators or hopes of improvement, cannot influence and distort results.
When a company wants to market a drug, it must submit an application to the FDA that includes data from controlled clinical trials. FDA experts review the application to decide whether the clinical trials are well-controlled, whether they show the effectiveness of the product, and whether all available data show that the product is safe enough to allow it on the market.
Proof of Safety
The FDA now interprets safety to mean that the benefits of a product outweigh its risks. "As drugs always have side effects and are never completely free of risk, it is hard to interpret safety any other way," says Temple. "But from 1938 to 1962, the FDA had little good evidence of benefit to weigh against risk."
The FD&C Act gave the FDA the authority to refuse to approve a drug application on specific grounds that are still valid. "The legal requirements for safety set forth in 1938 were very broadly written and are absolutely unchanged today," says Temple, "although we have learned a great deal about how to fulfill and interpret them."
"We weren’t very good at safety evaluation in the early days," adds Temple, who has reviewed data from numerous clinical trials since he joined the FDA in 1972. "There were no standards, no controlled trials, and no post-marketing surveillance. But we got better over time. And in the age of effectiveness, we also got better at safety."
Evidence of Effectiveness
It took an international crisis to lead to an important change in drug law. In 46 countries, more than 10,000 deformed babies were born to mothers who had taken a drug called thalidomide to help them sleep at night and to quell their morning sickness. The United States averted widespread tragedy because, although the manufacturer submitted an application to market the drug in this country, the FDA would not approve the drug based on the sketchy safety data provided.
The thalidomide disaster led to a law that was directed not only at drug safety, but also at drug effectiveness. The landmark drug law, the 1962 Kefauver–Harris Amendments to the FD&C Act of 1938, required new drugs to be shown effective as well as safe. It was based on several years of testimony before Sen. Carey Estes Kefauver about how poorly the effectiveness of drugs was evaluated and strong urging that drugs be studied in a way that would provide meaningful evidence of effectiveness.
But how was effectiveness to be determined? The law said there had to be "substantial evidence" that a drug will have the effect it is claimed to have under proposed labeling.
"Before this law, the country operated on opinion-based medicine," says O’Neill. "Prior to 1962, a doctor would be pictured with a drug on the back of Look magazine saying, ‘This is the best thing since sliced bread.’ The 1962 law said, ‘Opinion is not the only thing that matters. Now you have to show evidence.’"
The law also specifically stated that the only source of substantial evidence would be evidence derived from adequate and well-controlled studies. "It was really not the effectiveness requirement that was radical," says Temple. "But the requirement for adequate and well-controlled studies changed all of medical science."
The 1962 amendments also incorporated important ethical features regarding drug experimentation on humans, including a requirement for informed consent from participants.
Poor Studies, Ineffective Products
The law stipulated that the FDA must conduct an extensive review of all the drugs that the agency had approved between 1938 and 1962 to ensure that they also were effective. If they were not, the FDA was to get them off the market under what was known as the drug efficacy study implementation (DESI) program.
With the help of the National Research Council of the National Academy of Sciences, the FDA reviewed the evidence of effectiveness for thousands of drugs, finding that many of them were not effective for any of their claimed uses. More than 1,100 drug products that had been FDA-approved were withdrawn from the market as part of the DESI program.
The more than 20-year DESI effort was, "an instruction in every way a study could be badly designed, analyzed, and reported," according to Temple, who reviewed many of the DESI drugs. In 1970, for the first time, the FDA described in regulations the characteristics of an adequate and well-controlled study.
The DESI effort had another significant effect, says O’Neill. "Because of that experience, and reflecting on what went right and wrong in clinical trials, we were able to write guidances for industry beginning in the mid-1980s on how to report clinical trials, which influenced the design, conduct, and interpretation of clinical trials."
Unlike a regulation, a guidance is not enforceable, but companies have a strong incentive to follow it because it represents a pathway the agency has said is acceptable. If clinical trial investigators don’t follow FDA guidance, they must show that their alternative approach meets the requirements, says O’Neill. "The guidance does a lot of the hard work for them."
As the following examples show, when new problems have arisen in clinical trials, FDA experts have worked to solve them and to translate the lessons learned into new guidances and regulations.
No Data Fishing Allowed
"They’d go sifting through the data, breaking patients into lots of groups and picking the one that showed the drug worked!" says Temple. "If you go fishing through data like that, you’ll always find something positive."
A fundamental, but crucial, concept, called prespecification, arose from this type of distortion, says O’Neill. "A clear statement of the study’s objectives is required and study investigators must specify in advance how they will judge a trial’s success or failure." Study objectives must be included in a study plan, called a protocol, which describes what study procedures will be done, when, and by whom.
Counting All People
"ART seemed a model effort," says Temple. It was double-blinded, randomized, and placebo-controlled, and it involved about 1,600 participants.
But there were problems. Although published reports did not indicate it, nine participants who had died—eight of them on Anturane, and one on a placebo—were excluded from the trial results.
"The drug company decided that three patients had not been ‘compliant’—they found all the drugs in their rooms after they died—and that they weren’t eligible for the trial after all," says Temple. "When you included these patients, there was no evidence of an effect from the drug."
The FDA discovered another fault with the ART. The investigators attempted to distinguish and classify deaths using various definitions they developed. "They had concluded that their drug prevented sudden death," says Temple. "But to get to that conclusion, the cause of death assignment had to be distorted. In many cases, when a death occurred on their drug, they called it something other than sudden, for example, a heart attack. But if a death that looked the same occurred on placebo, it was often called sudden death."
As a result of the ART experience, FDA regulations require any new drug application to account for all participants, including those who dropped out of the trial or died. In addition, determination of cause of death must be done by investigators blind to the investigational product.
This article was originally published in the March/April 2007 issue of FDA Consumer magazine.
Linda Bren is a staff writer for FDA Consumer.
FDA Compliance The Advancement of Controlled Clinical Trials
A clinical trial uses an investigational product in human volunteers to examine its effects. An investigational product may be an experimental drug, medical device, or biologic factor, such as a vaccine, blood product, or gene therapy.
Before 1938, manufacturers could market a drug without submitting any information to the FDA or any other agency. The Federal Food, Drug, and Cosmetic Act (FD&C Act) of 1938 was passed when more than 100 children died from taking a sulfa drug that had not been tested in people. This law required companies intending to market a new drug to submit an application to the FDA that included data to demonstrate that the drug was safe for its intended use.
"Crises usually create the opportunities for changes," says O’Neill.
The 1962 law made it clear that companies had to prove their drugs were both safe and effective if they wanted to market them. But what about the more than 3,000 prescription drugs already on the market that had been approved based on safety alone?
In one early clinical trial involving a drug intended to treat pain, investigators tested the drug in a group of people with pain without finding any effects, says Temple. So they broke the participants into groups—those with moderate or severe pain, and those with mild pain—and found an effect in the moderate-to-severe group. In the next study, they repeated the process, finding an effect only in the participants with mild pain.
The need to account for all participants in a trial had not been an issue for the FDA until the Anturane Reinfarction Trial (ART) of 1980, says Temple. The drug Anturane (sulfinpyrazone), according to the drug company, helped to prevent sudden death during the first six months after a heart attack (myocardial infarction).
About The Author
Linda Bren
© 2023 Quality Digest. Copyright on content held by Quality Digest or by individual authors. Contact Quality Digest for reprint information.
“Quality Digest" is a trademark owned by Quality Circle Institute, Inc.
