Of Survey Data and Human Biases

“I just spoke with three patients last week who told me how much they love the care I provide. This week, two other patients told me this is the highest quality care they have ever had. I don’t understand why my Press Ganey scores aren’t better! I don’t think the data are valid.”

When I speak with physicians, and hear the above argument, I understand their confusion and frustration, and why this would lead them to question the accuracy of the data. The data are valid, but the reactions are natural.

Humans aren’t computers; we are social animals. Human brains are designed to allow large amounts of social information to be encoded, stored, and retrieved in an efficient and useful manner. These highly developed processes, however, are not without their drawbacks. Making a judgment, for example, is a highly complex process. To be accurate, the person making the judgment needs to consider all relevant information. A physician making a statement about his/her patients’ satisfaction needs to consider the viewpoint of all (or a representative sample) of his/her patients. Moreover, the physician would need to consider all of the various facets of those patients’ satisfaction. Finally, he/she would need to develop some method of summarizing the data so that it is actionable – or at least understandable.

Because we literally cannot do this, the human brain has developed shortcuts. All people use these shortcuts, but very few are actually aware of them. For example, the availability bias is the tendency to make a judgment based on information that is readily available in memory. For example, when patients give a doctor high praise, he/she is likely to remember this. Later, when the doctor is making a judgment about patient ratings of care, he/she is likely to easily recall this information and base judgments on it.

A doctor might argue that we are equally likely to easily recall patients’ criticisms. But consider the availability bias in conjunction with the confirmation bias, which is the tendency to search for information that confirms one’s preconceptions. Even if patients’ criticisms are as equally available in memory as patients’ praise, physicians, without realizing, are more likely to seek information from patients that confirms his/her belief of being a good physician (patients’ praise).

Now consider both of these in light of the self-serving bias, which is the tendency to take credit for successes and deny responsibility for failures. Thus – again, without realizing – a physician will have the tendency to take credit for the patient’s successes and deny responsibility for failures. “I explained it to him a thousand times; he just never listens.” So, put altogether, the physician will tend to deny failures, seek information confirming successes, and make judgments based on this information, which will likely be readily available in memory. Re-read the quote at the beginning of this post with those biases in mind.

Remember there are two members of this dyad – the physician and the patient. The patient is also susceptible to biases. Consider expectancy effects, which occur when someone subtly communicates to another the kind of behavior he/she expects to find, thereby creating the expected reaction. The manner in which a physician asks the patient a question, for example, can create an expectancy effect. “Do you feel satisfied with your visit today?” This is a simple question, but how do you think the patient will respond? The patient knows very well the physician wants a “yes” response. And unless the patient wants to be considered negatively by the care provider and spend extra time explaining issues, the patient will say yes.

Consider the following question as an alternative: “How could I have made your visit better?” What expectancy effect does this question create?

Another issue is the tendency of patients to reply in a manner that will be viewed favorably by their physicians. This is called the social desirability bias. Patients lie to avoid negative outcomes. “Have you been keeping on your exercise routine?” “Why yes, doctor.” “You’ve been taking your medicine regularly, right?” “Yep, haven’t missed a dose.” “Do you have any questions that I haven’t answered?” “Nope.” “Do you understand the plan for your new diet and exercise routine?” “Yep, I got it.” “Overall, are you satisfied with your visit today?” “Oh absolutely!”

Environmental effects occur when the physical environment affects feelings, thoughts, or behavior or impacts people in other ways. Many people are intimidated by the sheer presence of a physician. Being in a medical environment – with physicians, nurses, medical equipment, other patients, etc. – can be overwhelming to some. Many patients will feel anxiety about being in the situation and will do whatever they can to minimize that anxiety. So, put altogether, the patient is in an intimidating environment, is well aware of what the physician is expecting him/her to say/do, and has a tendency to lie to reduce anxiety and negative consequences. On top of that, the physician will tend to deny failures, seek information confirming successes, and make judgments based on this information which will likely be readily available in memory. Again, re-read the quote at the top of the page keeping this in mind.

These are just a few of the many human cognitive biases; many others are at play. This is why the physician-patient interaction is so complex – much more complex than most people realize. This is also why Press Ganey employs sound survey development techniques, and sound research methods like random sampling, to avoid these types of biases. Without a doubt the data are not perfect; there is error in all data.

However, considering all of this, which would you consider a more valid, unbiased measure of patient satisfaction – survey data or physicians’ opinions?