The evidence on evidence-based medicine speaks for itself. The approach is a vast improvement over everything that came before, and over current rival approaches. It’s often enough to give its practitioners a bit of hubris –which is why it’s so important to remember that evidence-based medicine can’t tell us everything we want to know, either.

Possible pitfalls and limitations of evidence-based medicine were on stark display at a press conference of the American Society of Hematology last week, where not one, but two studies reported results that were the opposite of what one might suspect from looking at one part of the evidence.

Ralph Meyer and colleagues at the NCIC Clinical Trials Group and the Eastern Cooperative Oncology Group demonstrated this rather starkly in a long-term study that compared chemotherapy alone to a radiation-based regimen for the treatment of early stage Hodgkin’s lymphoma. They found that the more intensive radiation-based treatment did decrease the recurrence of the Hodgkin’s disease itself. But over the course more than a decade, there were twice as many deaths in patients receiving radiation as those receiving chemotherapy.

The findings come with an additional twist: extensive radiation is no longer the standard treatment for Hodgkin’s lymphoma; newer regimens use less radiation, where the caveats of extensive radiation may or may not apply.

Meyer noted that there is no easy way out of this conundrum. “What we want is long-term data on the newest treatments, which is a mathematical impossibility.” But in the absence of such data, it is important to remember that “treatments that are given to patients have their own-long-term effects.”

And an editorial that appeared alongside the paper concluded that “these results support the view that the relapse rate is not a reliable surrogate for long-term survival, which is the most important treatment outcome” – and which is rarely looked at.

At the same press conference, the opposite relationship between initial impression and long-term effects was also on display. Updated information form a clinical trial of PCI-32765 showed that the Bruton’s tyrosine kinase inhibitor has ever-increasing effects against chronic lymphocytic leukemia – after an early sharp rise in blood levels of leukocytes when treatment is initiated.

Such a rise, of course, is the opposite of what an oncologist hopes for when treating a leukemia patient. Indeed, a few patients were taken off the drug early on, presumably because their doctors assumed they were progressing as their leukocyte counts rose. (And some patients do progress ‑ but nevertheless, response rates in one trial are now at 70 percent, up from about 25 percent a year ago.)

And so, modern medicine may be a marvel – but it’s good to bear in mind that whether our initial reaction to a given dataset might be “gee whiz” or “oh shoot, ” what those data mean is not always obvious at first glance.