Medical Device Daily Washington Editor
GAITHERSBURG, Maryland — The second day of the two-day meeting of the radiological devices advisory committee was convened to review 510(k) standards for computer-assisted detection (CADe) devices, and as was the case with the previous day's hearing (Medical Device Daily, Nov. 19, 2009), the agency's reviewers were on the receiving end of a number of disapproving comments from a variety of sources.
The negative comments from industry were to be anticipated, but once again, academe weighed in. In one instance during which FDA seemed to insist that CADe systems be tested in a way as to eliminate any variation in the interpretation of imaging scans, Maryellen Giger, PhD, of the University of Chicago (Chicago), asked the panel, "why do we keep realizing over and over again that a radiologist's training is excruciating in detail?"
The stated topic of the meeting was deceptively simple, given that such software is used in a range of applications, including mammography, lung examinations, and colonography performed via computed tomography. Robert Smith, MD, one of the reviewers at the radiological devices branch at FDA's Office of Device Evaluation, said that the risks, benefits and clinical significance of CADe are "specific to device, organ and disease." On the other hand, he acknowledged that the proposed CADe guidance "provide[s] general guidelines and was not written to be specific to organs and/or diseases."
Smith also warned that FDA "does not clear or approve general CADe technology," but rather "clears or approves individual, specific CADe medical devices based on safety and effectiveness for intended patients."
Smith noted that the panel's conclusion in its March 2008 meeting that reader studies "should be the primary analysis is unchallenged and the opinion of the committee," a quote attributed to then-panel chairman Leonard Glassman, MD of Washington Radiology Associates (Alexandria, Virginia). However, Glassman, who still serves on the panel, seemed to back off that statement in Wednesday's hearing. He quipped that the chairman of the 2008 panel – indeed, Glassman himself – "should be executed" for making such a statement.
FDA asked the panel whether stand-alone performance would suffice in a 510(k) application as opposed to a clinical evaluation regime. The panel discussed the question at length, and if there was any consensus at all, it was that a clinical study would be mandated if the stand-alone evaluation of the new CADe software offered fewer true positive diagnoses.
The panel seemed a bit less certain as to whether more false positives should prompt a clinical evaluation, with some comments reflecting the view that the additional highlighted images on a scan could distract a reader from more closely examining legitimate findings. The panel's approximate consensus seemed to be that a higher rate of false positives might be acceptable so long as they did not exceed a certain statistical boundary, but the panel left it to FDA to establish precisely where such a boundary might fall.
During an opening question addressing whether FDA has adequately captured the comments heard during the 2008 meeting, Glassman made a case for reclassifying CADe software for mammography. He said, "I would echo the fact that the difference between breast and other CADs is regulatory and artificial."
Glassman added, "the class II process seems reasonable for something that is well understood, fairly well researched. I think breast [CADe] belongs in class II," adding that "the sooner the regulatory burden" can be tied to risk, the sooner the benefits of digital mammography can be enjoyed by more women.
Janine Morris, a scientific officer at the Center for Devices and Radiological Health who served as the agency's representative on the panel, responded, "if we want to discuss changes in classification ... the agency will explore that," but redirected the panel to address "what do we do this minute?" with applications.
Tech assessment institute proposed
During her remarks, Geiger also observed that she is "concerned about the timeliness and consistency of" translation of research into clinical practice. In addition to her observation that the agency seems to need to be routinely reminded that radiologists are highly trained, she said "we want standardization of testing" as well as a way to "maintain the integrity of test sets" for benchmarking a new system.
Geiger added, "we need to determine a performance-level standard based on published data and a cooperative study" that "would tell us what is needed as the minimum bar of a computer stand-alone performance for sensitivity and false positives" for CADe software.
"The way to do this is with an independent technology assessment institute" which she said could be overseen by the American Association of Physicists in Medicine (AAPM; College Park, Maryland). She added that this "would give us a least burdensome approach" and would "allow for a case mix of enrichment and matching" and "random samples of the population."
Several speakers during the afternoon public hearing took exception to the agency's failure to more promptly deliver a guidance for CADe software, but a representative of the Medical Imaging Technology Alliance (MITA; Arlington, Virginia), warned that investors may start fleeing the U.S. market.
Stephen Vastagh, director of industry programs at MITA, addressed the panel, but his remarks were more aimed at FDA. He noted that industry was not afforded a reasonable amount of time to examine the questions put to the panel inasmuch as those questions "were announced just two working days before the meeting."
Vastagh warned, "it should not be a foregone conclusion that industry will continue to develop" CADe products for the U.S. market given the increasing regulatory hurdles. He noted that "there has not been a new CAD product cleared or approved in the U.S. since October 2006, and not for lack of trying." He said that MITA has estimated that between 10 and 20 applications were made since the October 2006 success, none of which were approved or cleared.
Vastagh alleged further that "it became clear at yesterday's FFDM panel meeting that delays within FDA were caused by a split within the professional staff" at the agency. He said "a handful of the minority" at FDA "has been able to tie up a large portion of new medical imaging technologies" over the past three years "by insisting that every, or nearly every, submission requires clinical studies."
Vastagh added that the regulatory environment "discourages new investment" and blunts advances in a variety of technological areas, including biomarkers. He asserted that it is "amazing that so few persons have been able to cause all this."
Mark McCarty, 703-268-5690