Response to Milliman’s Non-Response to “Questioning the Widely Reported Savings Reported for North Carolina Medicaid”
(Note: AMJC policy does not allow the author to publish a rebuttal)
I appreciate Mr. Cosway finding time in his busy schedule five months following publication to respond to my commentary, although a fuller response to an article entitled “Questioning the Widely Publicized Savings Reported for North Carolina Medicaid” would have answered the actual questions about the savings and ethics posed in the article. Obviously to everybody except apparently Mr. Cosway, the whole point of a response to an article with that title should be to answer the questions.
- Why did Mr. Cosway feel that the Agency for Health Research and Quality’s (AHRQ) Medicaid admissions data, considered the gold standard by health services researchers, which showed precisely the opposite of his conclusions (no change in children’s hospital admissions) not be worthy of mention in his extensive report, even if only to say why it was wrong? Couldn’t this omission be construed as an attempt to mislead legislators into believing that no such data existed?
- Why did Mr. Cosway also feel that the MACPAC Report to Congress showing North Carolina’s per capita Medicaid costs to be substantially higher than the average for the surrounding states in the two beneficiary categories in which the medical home model predominated was similarly not worthy of mention, even though one could easily conclude from this report that the medical home had driven the state’s Medicaid costs much higher? Couldn’t this also be construed as an attempt to mislead legislators who may not be aware of this data into believing that this data does not exist?
- Why did he also feel that it was not worthy of mention that the particular diagnosis categories specifically targeted by CCNC for admissions reduction, such as asthma and diabetes, showed no reduction in admissions relative to other states? Did Mr. Cosway not feel it to be relevant that a program targeted, at great expense, at reducing chronic disease admissions did not in fact reduce chronic disease admissions?
- Why, with almost the entire children’s Medicaid population in the medical home, thus giving North Carolina the luxury of comparison to a “natural control” consisting of the two other states (South Carolina and Tennessee) that also reported statewide children’s admissions and costs to AHRQ, did he elect not to mention that those states’ children’s Medicaid admissions rates were essentially identical to North Carolina’s even absent any large medical home investment on their part? Couldn’t that also be construed as an attempt to make legislators believe that no such comparative data existed, because if they found out about it, they would wonder why North Carolina’s performance was so mediocre?
Another question, not asked in the commentary due to space reasons, is what makes Community Care of North Carolina uniquely able to generate massive savings through its medical home model when all the other literature, such as this one in the prestigious Annals of Internal Medicine show either no effect or are inconclusive? Since Mr. Cosway’s finds so much more savings that all other researchers, shouldn’t he have indicated that as a study limitation? Otherwise, couldn’t legislators be misled into believing these models of care are widely believed to dramatically reduce costs?
Mr. Cosway also suffers from two memory lapses. Most importantly, he says now that his $250 million savings estimate was based on cost categories other than admissions, since there were no reductions in admissions according to AHRQ. (And admissions could have fallen to zero without accounting for even half of that $250 million.) Before making that statement, Mr. Cosway would have been well-advised to re-read page 3 of his original report, which says (italics mine):
“This medical home model…has a cost, as members receive more primary care services and prescription drugs. Also the medical home model has direct costs… It is assumed that these costs would be more than offset by emergency room visits, inpatient hospital admissions and other [unspecified] services.”
Mr. Cosway’s report therefore states precisely the opposite of what Mr. Cosway is saying now: admissions would have accounted for more than 100% of the $250,000,000 net savings. (ER visits showed no change.) Partly because he listed no dollar amount of the increase in preventive/administrative costs, my analysis graciously and generously assumed that prevention and administration costs hadn’t increased even though his report acknowledged that they did. If one were instead to take Mr. Cosway at his word, the admissions reduction needed to make his numbers true up would have been even higher than the already-impossible $250,000,000.
Memory lapse #2 concerns the baseline. Probably uniquely in all formal and most informal population health outcomes reporting, he also forgot to put a baseline year in the original report. Obviously a baseline is needed to compare two numbers. Recognizing this, I noted in my commentary that it wouldn’t matter what year was the baseline from which the alleged program impact started, because the baseline admissions rate has been mostly flat both absolutely and relative to neighboring states for years, essentially mooting his entire objection. (Also, why an author would want to draw attention to his having forgotten to put in a baseline puzzles me.)
Given all these inconsistencies, impossibilities and unanswered questions, it is no surprise that, subsequent to my commentary, other unaffiliated outcomes experts have come to similar conclusions, commenting on this report in the lay media with phrases like "fatally flawed," "professionally embarrassing," and "would be laughed off the stage".
These items above also represent the reasons I offered Mr. Cosway a chance to settle this dispute with a $50,000 bet on the validity of his report -- plus a donation of $25,000 to CCNC and a public apology if I lost — and also explain why Mr. Cosway declined.
Having said that, two apologies on my end are required now.
First, it was imprecise of me to write “per year” for a specific savings figure ($250,000,000) attributable only to a single calendar year. Alleged savings in the other years were different.
Second, I was confused about the previous consulting firm report. That report, from Mercer, claimed a 47% reduction in admissions in 2006 vs. the 2000 baseline (note that Mercer did manage to specify a baseline). The invalidity of this claimed reduction, though not the focal point of my commentary, was addressed in my award-winning book Why Nobody Believes the Numbers along with many other Mercer claims. The interplay between the two consulting firms’ savings figures (Mercer covering 2000 to 2006, and Milliman covering 2007 to 2010) has never been addressed by the state. Perhaps I made an incorrect assumption that Mercer’s report was not retracted, since I could not find any press release by the state disavowing it. That's why I assumed that the Milliman subsequent period savings was additive with the Mercer prior period savings. I apologize for that assumption.
However, no interpretation of the interplay of the two savings figures or the North Carolina’s retraction/endorsement of the earlier report would change the following conclusion: AHRQ data for the state as a whole shows no significant reduction in children’s admissions or admissions in the chronic disease categories targeted by the North Carolina program, either absolutely or relative to the "natural control" states, while the specific questions raised in the article about Milliman’s selective use and omission of source data remain unanswered.
To conclude on a positive note, I would invite Mr. Cosway to register for my next course in outcomes report analysis. It costs only $500 to take the population health management industry’s course in outcomes report analysis and (if the test is passed) receive Certification in Critical Outcomes Report Analysis, and perhaps North Carolina taxpayers, who spent in excess of $500,000 on Mr. Cosway’s outcomes report, might prefer in the future that Mr. Cosway be trained and possibly certified in Critical Outcomes Report Analysis before collecting more of their money to analyze more outcomes reports.