Medical Device Daily Washington Editor
SAN FRANCISCO — One of the sessions at this year's edition of Transcatheter Cardiovascular Therapeutics dealt with the Holy Grail of statistical science: making use of Bayesian statistical analysis.
The question of how Bayesian and frequentist approaches to clinical trial designs match up is no small matter inasmuch as the Centers for Medicare & Medicaid Services recently held a meeting of the Medicare Coverage and Evidence Development Advisory Committee on the topic. The FDA has issued a guidance on the subject as well, but the attitudes of device advisory committees toward Bayesian trials are difficult to pin down at times.
In one instance, the advisory meeting for the PMA for the Downstream oxygen therapy system, made by TherOx (Irvine, California) indicates some aversion to the Bayesian approach (Medical Device Daily, March 20). Among the complaints voiced by this panel was a higher mortality in the Downstream arm of the pivotal study, but several panelists expressed discomfort with the smaller data set allowed by the use of a Bayesian model.
A different outcome was seen in an advisory committee hearing for the Navistar ablation catheter, which was examined by an advisory committee last November (MDD, Nov. 24, 2008). The most conspicuous feature of that application was that the device's sponsor, Biosense Webster (Diamond Bar, California), switched from a frequentist to a Bayesian analysis after having come to an agreement with FDA on the protocol for the trial, although it should be pointed out that this application was for an expanded indication, from atrial flutter to atrial fibrillation.
Giving the statistician's perspective on the matter was John Boscardin, PhD, an assistant professor of medicine and biostatistics at the University of California San Francisco (San Francisco), who opened by asserting, "the first important perspective is that there's a huge opportunity to do Bayesian."
Boscardin said that the Bayesian guidance from the Center for Devices and Radiological Health "reflects years of efforts" on the center's part, and was augmented by a follow-up paper by CDRH's Gene Pennello and Laura Thompson in the Journal of Biopharmaceutical Statistics in 2008, in which they purported to have clarified the question of adjustments for multiplicity in device trials.
Boscardin said those who want to employ Bayesian inference must "have a prior belief about unknown parameters" regarding the device's performance. "The problem I think everyone feels about Bayesian" is the prior belief, however, setting up a dilemma that can quickly grow more complicated.
"So you collect new data and they're not very suggestive" about treatment effect, Boscardin remarked, but the use of Bayesian analysis can nonetheless lead to exclusion of a null treatment effect. "You can think about this as a type 1 error," he said, a reference to the inference of a treatment effect when in fact there is none.
So how does a sponsor use favorable prior information to build a Bayesian trial without accidentally biasing toward a favorable outcome?
Boscardin said Bayesian statisticians do not believe in a type 1 error, but "from a practical standpoint, Bayesian statisticians need to believe in a type 1 error" for talks with FDA and with clinicians. On the other hand, hierarchical models, which permit analysis of subgroup data, can overcome the type 1 error paradox thanks to the availability of those multiple data sets. "If I collect new data that produce roughly the same qualitative result," the overall analysis is more reliable, he said. However, any inconsistency with prior data sets will leave a sponsor back in a state of uncertainty.
Boscardin added a cautionary note to his discussion. "Bayesian trials are welcome, but FDA focus will include frequentist characteristics, including type 1 error," he said. Hence, while Bayesian trials "will definitely be judged using frequentist ideas," hierarchical models can be used to overcome such problems.
Mark McCarty, 703-966-3694;