Medical Device Daily Washington Editor
GAITHERSBURG, Maryland — FDA has decisions to make about clinical trials for digitized images of pathology slides, but Friday's second-day session of the hematology and pathology devices advisory committee gave industry few hints about what the agency will require in those trials.
The panel heard plenty on the first day regarding the different types of monitors used to view whole slide images (Medical Device Daily, Oct. 23, 2009), but the panel seemed split on a couple of essential questions, including whether a trial should examine variability in interpretation between readers versus variability exhibited by one reader upon a second examination of a pathology slide.
The session drew more than half a dozen speakers for the open public hearing, including Andrew Evans, MD, of the University Health Network (UHN; Toronto, Canada). Evans said that up to 2004, a single pathologist had handled pathology slides from the several centers operated by UHN, which required that the pathologist scramble from one center to another to retrieve and review slides. This situation, he said, disrupted the pathologist's workflow and "has resulted in a compromise in diagnostic accuracy."
Evans said UHN purchased a robotic imaging system in 2004 that was an improvement, but that UHN spent "approximately 18 months doing an in-house training and a validation study" of the system. He said the system rendered "an acceptable deferral rate" to review of the original slide, but that the system was slow, requiring about 10 minutes for each slide.
UHN opted in 2006 to upgrade to a whole slide imaging (WSI) system that offered "diagnostic accuracy rates that will match anything in the public literature" for slides, Evans said. As for image quality, he stated, "less than 2% of our cases require rescanning." "We found that WSI has been a Godsend to us," he said, adding that UHN has "found it to be safe, reliable and accurate."
Paul Valenstein of the College of American Pathologists (CAP; Northfield, Illinois) gave the panel a few recommendations. "We believe as a society that WSI has important implications," he said, noting that the technology "is moving from research into clinical diagnosis." He said also that the electronic portability of such technology can "provide access to underserved areas."
On the other hand, if accuracy decreases by only 1% for breast biopsies, "an additional 6,000 patients will be misdiagnosed annually," Valenstein said. Hence, he said, "the stakes are quite high," although he stated further that CAP "is agnostic with respect to this technology" because it sees the evidence as equivocal.
CAP's position is that FDA "should focus on how diagnosis is actually rendered" in ordinary use, Valenstein said, adding that the society recommends that any clinical trial should be designed for non-inferiority and "should determine whether the diagnosis arrived at ... is the same as arrived at by the same pathologist" viewing only the original slide.
"We do not recommend a study designed for diagnostic accuracy," Valenstein remarked, stating that a study should establish whether the use of WSIs exerts a differential effect on the diagnosis compared to microscopic reviews of slides. Valenstein said "validation procedures should be specific for specimen types," but he opined that it is "not practical to approve systems for specific diagnoses."
Giving an overview of potential considerations for clinical trials was Tan Nguyen, MD, a medical officer at the Center for Devices and Radiological Health at FDA, who said that among FDA's chief concerns "is how we can reasonably demonstrate ... that any competent pathologist" can use WIS without compromising diagnosis. Another consideration for FDA, he said, is that "an out of focus or even partially out of focus" image may miss "a single cell" and result in misdiagnosis.
Nguyen said a prospective study can suppress bias caused by case selection and can be built around real-world practice, but that such a design will not eliminate the problems of non-uniform specimens and variability in the quality of glass slides. He also indicated that FDA is not utterly averse to the use of existing slides to bolster a PMA. "A retrospective study ... if thoughtfully designed, may allow the opportunity to use archival cases," he remarked.
A retrospective study intended to characterize receiver operator interpretation based on a multiple reader/multiple case data set "has been successfully deployed and proved its effectiveness," Nguyen said, and "can provide the practical ability" to compare diagnoses across readers, so long as those readers have similar levels of training and experience.
The first question FDA posed to the panel was whether an application should be limited to image compression schemes that retain the full data set (non-lossy) or should allow compression schemes, such as jpegs, that filter out some data (lossy). Panel chairwoman Dorothy Adcock, MD, of Esoterix Labs (Englewood, Colorado), summarized that the panel "generally feels that this issue can't be answered with a yes or no and perhaps such a decision ... should be left to the manufacturer." FDA should also "be aware of the alternatives," such as non-lossy scanning followed by decompression of the image, she said.
The second question dealt with the idea of allowing a system's use to be validated for all organ systems by extrapolation from a single organ or organ system. Opinions dotted the chart on this question, with one panelist arguing that there is "too much difference to allow one to extrapolate" while another made the case that "the issue is whether it can generate enough information," and that "its not necessary" to have organ-specific data. Adcock summarized that the panel "feels that is difficult or impossible to use an organ approach, or even a disease approach," but that such a specifications might be necessary if only to allow the design of a study.
The panel could not agree on whether a study should look at variability between readers versus variability within one reader, and was also hung on the verdict regarding the use of an expert panel for clinical trials.
Mark McCarty, 703-268-5690