Home / Healthcare Updates / Presidential Voting and Press Ganey

Presidential Voting and Press Ganey

Well, the results are in.

Based on a survey of New Hampshire voters, Rick Perry and Rick Santorum will receive NO votes in the New Hampshire primaries. Jon Huntsman and Mitt Romney will receive the exact same number of votes and will be tied for first place. Ron Paul and Newt Gingrich will receive the exact same number of votes and will be tied for second place.

And, in the general elections, Barack Obama will receive exactly three times as many votes as either Ron Paul or Newt Gingrich and will receive exactly 50% more votes than Mitt Romney or Jon Huntsman.

You see, in the New Hampshire town of Dixville Notch, nine registered voters casted their votes in the Republican primary. The tallies of the votes were:
Newt Gingrich: 1
Jon Huntsman: 2
Ron Paul: 1
Rick Perry: 0
Mitt Romney: 2
Rick Santorum: 0
Barack Obama: 3

Applying those statistical breakdowns to the population of New Hampshire, and to the country at large, one can only come to the conclusion that Jon Huntsman and Mitt Romney will receive the exact same number of votes in the New Hampshire primaries. Those are the statistics and statistics don’t lie.

No problem, right?

Is anyone willing to conclude that a breakdown of the statistics from Dixville Notch is an accurate representation – down to one tenth of a percent – of the percentage of voters supporting each candidate in New Hampshire? Anyone? Any hospital administrators? Any hospital board members? Any CMS officials? Bueller? Bueller?

Didn’t think so.

Yet administrators and boards of directors of many multimillion dollar hospitals make similarly irrational and childish conclusions every day.

How?

Let’s change the voting situation above just slightly.

Instead of making this voting a presidential primary, let’s make the voting a patient satisfaction survey.
Instead of voting for presidential candidates, let’s make the votes a rating of health care providers.

Press Ganey surveys allow patients to rate physicians on various metrics based on a scale of 1 through 5.


Rather than using the actual scores of 1-5 to create the statistics, Press Ganey confuses the statistics by increasing the scores twentyfold. A score of “1” is converted to a score of “ZERO” on a 0-100 scale, “2” is converted to “25,” “3” is converted to “50,” “4” is converted to “75” and a score of “5” is converted to a score of “100”. After all, if the mean comparisons were “4.285 and 4.105” instead of between “85.7 and 82.1,” many people might just realize that the comparisons end up being insignificant.

Once the conversions to higher values have taken place, then an “overall mean score” is calculated. That mean score is then compared with scores from hundreds of other hospitals and physicians around the country in order to rank providers. If two patients give a physician a score of “very good” (“5”) in every category and one patient gives a physician a score of “good” (“4”) in every category, Press Ganey calculates the average (“mean”) score by adding the scores 100+100+75, dividing by 3 patients, and then¬†assigns that physician a combined score of 91.6 which gives a physician a percentile rank in the high 90s. However, if the third patient gives the physician a score of “3” instead of “4”, the “mean” score drops to 83.3 which is an abysmal percentile rank – 21st percentile in the printout below. That is the problem with low sample sizes. A change of one patient’s opinion from “good” to “fair” can change a physician’s Press Ganey rank from excellent to abysmal.

In the sample Press Ganey report below, note that the “overall mean scores” are in the 80-85% range – which is equivalent to a mix of “4”s and “5”s on the surveys. The scores below were calculated on a total of 15 patient surveys out of over a thousand patient visits. Based on the opinions of those 15 patients, Press Ganey sent a report to hospital administrators asserting that all 1000+ patients that visited the hospital in that month would vote EXACTLY the same way as those 15 patients.


Most hospital administrators don’t question the statistics in the least.
Most hospital board members don’t question the statistics in the least.

Hospital boards and administrators merely look at the “current percentile rank” number for their institution, demand that the percentile rank for their institution become higher than some other higher arbitrary value, then chastise providers when they don’t hit those arbitrary goals. Make our percentile rank 90% or else.

Actions against providers don’t stop at being scolded, though. Some hospitals and physician contract groups make provider pay dependent upon the percentile ranks. If your scores arbitrarily decrease, you get paid less. Get too low of a percentile rank for too long and you can even lose your job.

There are many issues with Press Ganey survey statistics. Inappropriately extrapolating statistics from low sample sizes is just one of the problems.

As this post hopefully demonstrates, we need to question the competence of hospital administrators and hospital board members who chuckle at the notion of Rick Perry receiving no votes in the New Hampshire presidential primary yet who treat Press Ganey rankings as indisputable truth.

See Part 2 of this post here.

9 comments

  1. Oh you are wicked. But so very right.

  2. This is why I hate these surveys so much. Making unit or hospital wide policy changes based on such low samples is insane.

    I would also wager that those who have negative things to say are more likely to send the survey back vs. those who had a good experience. It seems like the patients who were happy with their care send us cookies and forget about the survey!

  3. People who vote usually fall into two groups: the increasingly miniscule number who actually consider it their civic duty and people who have an axe to grind. Same principle applies here, I suspect.

    The difference? Political volunteers get fed cheap pizza instead of homemade cookies. :) Never mind how I know this.

  4. A more stark comparison: If you proposed Press Ganey methodology as a way to evaluate an intervention, be it a therapy or a education technique, you’d be laughed out of the room. Doctors would never expose patients to therapies supported only by data as flimsy and erratic as Press Ganey scores.

    The fact that doctors’ employment is being subjected to these scores shows just how little we matter in the scheme of healthcare today.

    When you look at a chart like this, though, it makes sense: http://www.masscare.org/wp-content/uploads/2007/04/GrowthPhysicians&Admin.jpg

  5. I was just very delighted to hear that most likely in the near future, patient satisfaction scores will no longer influence our pay directly. No more trying to shoot for the 75% percentile. I guess the idea is that if we just see patients faster, the scores will go up in general. I agree on this since the vast majority of people rank us poorly because they had to wait too long.

  6. Nothing will change until doctors join together and demand a stop to this madness.

  7. Wonderful article. Simple and to the point. This comparison displays, without a doubt, that these survey “results” are absurd. Beautiful, man, beautiful.

  8. Our facility actually biases the samples further by only sending surveys to patients discharged from the ED (that means mostly non-emergent patients for those following at home). Admits and transfers (the true emergencies) get either no survey, or a survey at discharge that covers the entire hospitalization, and does not (to my knowledge) single out the ED for attention.
    Knuckleheads.

  9. Thank you very much for writing this article. When I bring this up at my hospital I get looks of confusion. Part of the problem is that the people making and implementing the policies in response to Press Ganey survey numbers have no background in statistics, even high school-level. Just as an example, our ED nursing charts are audited every month for completion of the Braden scores. Until recently the method of audit was for a nurse to pull four charts at random from a single day in a given month. Those four charts from a single day represented the entire month’s audit. They saw no problem with this method or size of sampling.

    One other issue I have with the Press Ganey percentile scores is their volatility. From month to month the scores will fluctuate wildly from high 80s to below 10. Do our administrators really believe that from one month to the next the quality of our care goes from better than 4 out of 5 hospitals nationally to worse than 9 out of 10? If the think these scores are reliable, they must think so. It’s very depressing to be faced with the overwhelming success of this groupthink.

Leave a Reply to Sarah G Cancel reply

Your email address will not be published. Required fields are marked *

*