Charles Reese, MDYesterday we looked at a startup concept which would hopefully enable us to replace our current, large, traditional Patient Satisfaction service.  Hallelujah!  The new system would be email based, similar to Survey Monkey in style and execution, thus customizable, inexpensive, able to carry video or other content, and able to inexpensively reach perhaps 70% of our discharged patients. 

With the traditional vendor, we would send out thousands of paper surveys but get back only one or two hundred returns.  The paper based process is time consuming and the survey is very long, so only the most motivated patients take the time to fill it out.  Since some of the “most motivated” people are also the most unhappy, the process selects for unhappy campers.   And the small sample size makes it almost impossible to make valid assessments of individual providers, a fact that has sometimes led to unjust and inappropriate actions against physicians.

Despite all the questions on the survey, and the very detailed analysis that comes back, the reports are of very little use in actually managing the department.  Say this week you test a new procedure or remind the staff to say this or that thing to improve communications, and you want to assess how this impacted the patient’s experience.  Because of the sampling errors noted above, and because the surveys come back incrementally over months, your ability to measure change is basically nil.  What you need is something that is as close to real time as possible, a new instrument altogether.  But with the traditional system, you are “forbidden” to use any other survey because it skews the results of the former.

And then there is the much misused benchmarking function… don’t get me started!  Finding an “apples to apples” compare group is tough enough (you can’t really trust what they tell you and need to investigate on your own), but then you have the very narrow distribution curve where changing your score of 85 to 83 can move you from the 70^th to the 30^th percentile.  Suddenly your COO or Board thinks there has been some catastrophe, how could this have happened… but really this is much more likely a statistical blip and due to random variability in the process.  In this context, managing to percentile rank is an exercise in futility, if not downright silliness.

I actually think the best way to use the large/national/traditional service is to have a quick  look about once a year, including benchmark comparisons, and see how you compare to the most valid peer group you can find.  But for the rest of the year, give me something that is flexible, adaptable, as automated as possible, above all something that actually helps us manage the many challenges we face on a day to day basis.  The new concept noted above may have some intrinsic biases… for instance, selecting only for people who use email regularly… but I think the information will be much more valid and powerful, and we will have much much better control over how the data is used and analyzed.  I really hope it works out, and that our years of political maneuvering will allow us to opt out of the traditional system which is, truly, a white elephant whose time has come, and gone.