WASHINGTON _ Sepsis researchers and biotechnology companieshave spent the last 12 years watching as a succession of no less than19 Phase II and III clinical trials testing new therapies have failed toproduce an approvable drug. These failures were all the more bitterbecause they occurred relatively late in the drug developmentprocess, at a stage when most business models would assume thatrisk was decreasing.
Despite the disappointments, investigators and companies alike aretrying to pick up the pieces and move on. Top clinical trial experts,who gathered here on Monday and Tuesday at a CambridgeHealthtech Institute conference, "Designing better drugs and clinicaltrials for sepsis," placed part of the blame for the failures on faultyanalytic and statistical methods.
One of their conclusions: using subgroup analyses to interpretclinical trial results and develop hypotheses for further clinicaltesting is a hazardous business. This practice, more cynically referredto as "data dredging," has been routine in the analysis of clinicaltrials of sepsis, according to experts.
Other serious problems have plagued the design of sepsis trials,including the heterogenous nature of the syndrome, drug candidatesthat didn't do what they were designed to do, the lack of clinicallyrelevant animal models and scant knowledge about the complexhuman immune system and just exactly how it goes awry duringsepsis. While some of these factors may be unique to sepsis, thephenomenon of subgroup analysis is common to clinical trials inmany diseases.
Most drug studies are prospectively designed to test one hypothesis,such as, does drug A work in patient group B? When the answer is"no," as it has been all too often recently in the biotechnologyindustry, companies legitimately look more closely at their data tosee if the results could generate any new hypotheses for furthertesting. In statistician's jargon, this process is known as conductingan "exploratory analysis."
The Death Of A Cherished Hypothesis
The problem is that if patient populations are sliced and dicedenough times, one can always find a result of some kind. Forcompany executives who must face angry shareholders and forresearchers who can't accept the death of a cherished hypothesis,exploring data retrospectively is a dicey business. The danger: ashaky new hypothesis can be generated from studying a small subsetof patients.
Subgroup analysis is not a disease that afflicts only sepsis trials.Indeed, a recent Wall Street Journal article suggested that Kirkland,Wash.-based ProCyte Corp., whose copper-based gel for treatingdiabetic foot ulcers failed to outperform a placebo in a pivotal PhaseIII trial last October, was guilty of just such a faulty subgroupanalysis. The company based its Phase III trial design on a benefitseen in a small subset of patients from a Phase II trial (seven patientsout of 81).
Patrick Scannon, chief scientific and medical officer at XOMACorp., Berkeley, Calif., chronicled the course of his company'ssepsis drug, E5, and told conference attendees on Tuesday he hasconcluded that subgroup analysis "introduces confusion" and shouldbe avoided when analyzing clinical trial data. "Stick to the targetpopulation," he advised.
Scannon speaks from a decade of experience. XOMA launched itsfirst Phase III trial of E5 in the mid-1980s, testing the drug inapproximately 500 patients with gram-negative infection. Although E5 produced no significant survivalbenefit in those patients, the company found a subgroup of patients,those with gram-negative infections who were not in refractoryshock (meaning they were less severely ill), in which the drugappeared to work. On that basis, a second Phase III trial enrollingapproximately 850 patients was initiated.
Once again, when the overall data were analyzed, E5 failed toproduce a benefit in the targeted patients. Yet another subgroup ofpatients _ this time those that had reversible organ dysfunction,such as pulmonary or renal failure _ did appear to benefit. Scannonsaid the company spent 11 months analyzing data from the two trialsbefore it and marketing partner Pfizer Inc. decided to test the drug inanother clinical trial.
The third Phase III trial testing E5, which is being funded by Pfizer,is currently under way. Scannon said the company hopes to provethat E5 will benefit patients with sepsis and organ dysfunction. Thetrial must enroll 1,700 patients in order to achieve sufficientstatistical power.
Centocor Inc., of Malvern, Pa., and Synergen Inc., of Boulder, Colo.,two other biotechnology companies that have seen their sepsis drugsfail, also relied upon subgroup analyses in designing clinical trials. Inboth cases, the second Phase III trial failed to confirm subgroupeffects seen in the first.
Steven Opal, an associate professor of medicine at Brown UniversitySchool of Medicine, of Providence, R.I., who has worked closelywith numerous biotechnology companies developing sepsis products,provided perhaps the most damning evidence of subgroup analysesgone awry. He examined the clinical trial outcomes of eight novelsepsis therapies (including E5, Centocor's HA-1A and Synergen'sAntril) tested in similar patient populations. In each case, a differentsubgroup seemed to benefit. Worse yet, when the drugs were testedin the subgroups in a second or "confirmatory" clinical trial, not oneof them worked.
One researcher at the meeting who asked not to be identified put itthis way, "Retrospective subgroup analysis is like shooting an arrowat a wall, drawing a circle around it and calling it a bulls eye," hesaid. "You have no idea how to interpret the skill of the archer in thatcase."
For consumers of clinical trial information _ stockholders, analystsand journalists _ the "skill of the archer" is critical information.These consumers must rely, somewhat blindly, on the wisdom ofcompany executives to make drug development decisions such aswhether Phase II data truly supports advancing to Phase III trials orwhether serial Phase III trials will be worthwhile. The message ofresearchers at this week's sepsis meeting was that subgroup analysiscan often cause companies to miss the mark. n
-- Lisa Piercey Washington Editor
(c) 1997 American Health Consultants. All rights reserved.