Well, the results are in.
Based on a survey of New Hampshire voters, Rick Perry and Rick Santorum will receive NO votes in the New Hampshire primaries. Jon Huntsman and Mitt Romney will receive the exact same number of votes and will be tied for first place. Ron Paul and Newt Gingrich will receive the exact same number of votes and will be tied for second place.
And, in the general elections, Barack Obama will receive exactly three times as many votes as either Ron Paul or Newt Gingrich and will receive exactly 50% more votes than Mitt Romney or Jon Huntsman.
You see, in the New Hampshire town of Dixville Notch, nine registered voters casted their votes in the Republican primary. The tallies of the votes were:
Newt Gingrich: 1
Jon Huntsman: 2
Ron Paul: 1
Rick Perry: 0
Mitt Romney: 2
Rick Santorum: 0
Barack Obama: 3
Applying those statistical breakdowns to the population of New Hampshire, and to the country at large, one can only come to the conclusion that Jon Huntsman and Mitt Romney will receive the exact same number of votes in the New Hampshire primaries. Those are the statistics and statistics don’t lie.
No problem, right?
Is anyone willing to conclude that a breakdown of the statistics from Dixville Notch is an accurate representation – down to one tenth of a percent – of the percentage of voters supporting each candidate in New Hampshire? Anyone? Any hospital administrators? Any hospital board members? Any CMS officials? Bueller? Bueller?
Didn’t think so.
Yet administrators and boards of directors of many multimillion dollar hospitals make similarly irrational and childish conclusions every day.
Let’s change the voting situation above just slightly.
Instead of making this voting a presidential primary, let’s make the voting a patient satisfaction survey.
Instead of voting for presidential candidates, let’s make the votes a rating of health care providers.
Press Ganey surveys allow patients to rate physicians on various metrics based on a scale of 1 through 5.
Rather than using the actual scores of 1-5 to create the statistics, Press Ganey confuses the statistics by increasing the scores twentyfold. A score of “1” is converted to a score of “ZERO” on a 0-100 scale, “2” is converted to “25,” “3” is converted to “50,” “4” is converted to “75” and a score of “5” is converted to a score of “100”. After all, if the mean comparisons were “4.285 and 4.105” instead of between “85.7 and 82.1,” many people might just realize that the comparisons end up being insignificant.
Once the conversions to higher values have taken place, then an “overall mean score” is calculated. That mean score is then compared with scores from hundreds of other hospitals and physicians around the country in order to rank providers. If two patients give a physician a score of “very good” (“5”) in every category and one patient gives a physician a score of “good” (“4”) in every category, Press Ganey calculates the average (“mean”) score by adding the scores 100+100+75, dividing by 3 patients, and then assigns that physician a combined score of 91.6 which gives a physician a percentile rank in the high 90s. However, if the third patient gives the physician a score of “3” instead of “4”, the “mean” score drops to 83.3 which is an abysmal percentile rank – 21st percentile in the printout below. That is the problem with low sample sizes. A change of one patient’s opinion from “good” to “fair” can change a physician’s Press Ganey rank from excellent to abysmal.
In the sample Press Ganey report below, note that the “overall mean scores” are in the 80-85% range – which is equivalent to a mix of “4”s and “5”s on the surveys. The scores below were calculated on a total of 15 patient surveys out of over a thousand patient visits. Based on the opinions of those 15 patients, Press Ganey sent a report to hospital administrators asserting that all 1000+ patients that visited the hospital in that month would vote EXACTLY the same way as those 15 patients.
Hospital boards and administrators merely look at the “current percentile rank” number for their institution, demand that the percentile rank for their institution become higher than some other higher arbitrary value, then chastise providers when they don’t hit those arbitrary goals. Make our percentile rank 90% or else.
Actions against providers don’t stop at being scolded, though. Some hospitals and physician contract groups make provider pay dependent upon the percentile ranks. If your scores arbitrarily decrease, you get paid less. Get too low of a percentile rank for too long and you can even lose your job.
There are many issues with Press Ganey survey statistics. Inappropriately extrapolating statistics from low sample sizes is just one of the problems.
As this post hopefully demonstrates, we need to question the competence of hospital administrators and hospital board members who chuckle at the notion of Rick Perry receiving no votes in the New Hampshire presidential primary yet who treat Press Ganey rankings as indisputable truth.