We’ve seen shorter abstracts, and we’ve seen abstracts with more curious findings, but we’ve never seen a shorter abstract with more curious findings than this one, done by Dignity Health and financed by the California Healthcare Foundation, evaluating a gadget made by Propeller Health. (Along with financing the study, the California Healthcare Foundation, as coincidence would have it, is also an investor in Propeller Health.)
Let’s turn to the abstract. The study group’s use of inpatient care for asthma declined by 62% vs. the control group. You might think this result violates Dr. John Ioannidis’ well-known conclusion that large treatment effects are usually wrong, but you’d be mistaken. You see, there was no treatment here. There was only an effect. Dr. Ioannidis’ result applies only to actual comparisons of effects due to different treatments, not to random changes in effects using the same treatment. In this study, the actual treatment was the same and even the inhalers were the same.
Even the asthma control improvements that were achieved in this study were the same.
Yes, the most impressive part of the study was that this 62% reduction in inpatient days (and comparable cost reduction, as we’ll see) was achieved even though actual scores for asthma control improved only by a statistically insignificant 2% in the study groups vs. the control groups. In other words, there was no effect on asthma control scores, or to use the abstract’s language, overall asthma control improved similarly” in both groups. (Of course that “improvement” was simply regression to the mean due to the selection criteria favoring previously high utilizers, but we are focusing on the differences between the groups, not the groups compared to the baseline.)
Putting all this information in a table makes it clear that Dr. Ioannidis is going to have to re-examine his conclusions because apparently it is indeed possible to achieve a massive reduction in events and costs even if nothing else changes:
|Drug Regimen||Inhaler||Impact on Asthma Control||Impact on Events and Costs|
Still, we’ll give them credit for that 2%. Avid readers of Intelligent Design Awards and Why Nobody Believes the Numbers may recall that the ratio of event reduction to quality score improvement is called the “Wishful Thinking Multiplier,” and Propeller Health’s Multiplier of 31x just shattered Viverae’s record of 17x. (The real answer, of course, is that on a good day event changes match quality improvement changes, but there are many reasons even that 1x assumption may be high, as described in the book.)
We’re also not sure where exactly Dr. Merchant found these asthmatics but, wow, are they ever sick. During the baseline they were running 326 asthma inpatient days per 1000 asthmatics. A typical group of 1000 asthmatics will have about a quarter that many days. Perhaps that’s because when these folks show up in the ER they almost always get admitted. Whereas the typical asthmatic has a ratio of about 5-to-2 of ER visits to inpatient days, these folks were averaging less than 1-to-2. Neither of these statistical anomalies seemed to bother Dr. Merchant enough to explain why his population might be such an outlier.
All this leads up to the savings figures. Annualized gross savings were $688 in the study group. Assuming a typical commercial health plan has a 4% asthma rate, that equates to about a $27 PMPY savings. Nice work when you consider that we’ve never seen a commercial health plan that spends more than $20 PMPY on total asthma ER and IP events, and most spend about $15