Blinking lights don't make a better knee brace – fighting cognitive biases in testing orthopedic devices

As a researcher in a health-care-related field, I am keenly aware of how frequently economics enters the discussion these days. I am a biomedical engineer^[1] who works with patients using orthopedic devices: prosthetics, such as an artificial limb; and orthotics, which help improve the function of an intact limb, like a knee brace or custom shoe insert. In recent years, these devices have become far more complex and technologically advanced^[2], to the point where they can sense walking patterns and modify their function accordingly. Others contain motorized components to add power.

With increased complexity comes increased cost, so it has become important from a health-care economics perspective to measure and document improved outcomes related to more expensive devices. Those of us who do this research know there’s an added layer of scrutiny. If my lab research determines that patients have a more biomechanically sound walking pattern with knee brace A versus brace B, not only are clinicians interested, but so are insurance companies.

The comment that made me question everything

Several years ago, I had helped organize a group of experts to assess the state of the science in microprocessor-controlled prosthetic knees, which adjust the stiffness in the knee joint depending on what the user is doing. A comment from one orthopedic surgeon on the panel made me realize I’d been missing part of the equation in my own studies.

He raised the potential for something called confirmation bias – the tendency for a person to actually experience what he or she expects – when a user is wearing an advanced prosthetic knee. The remark made me aware just how possible it is for a patient using an advanced, expensive orthopedic device to report that it’s better just because it’s more advanced – regardless of its actual function. Patients might even move differently while using it, all because they think it is better. Suddenly, I questioned the way I measure outcomes in my own research.

When in doubt, test

Of course, as a researcher, my solution to a question like this is to launch a study. Along with a master’s student in our lab, Brittany Balsamo, I decided to find out what would happen if people thought they were wearing an advanced orthopedic device when, actually, it was just a standard knee brace. I’d never done a project involving deception before, so I was, perhaps perversely, more than a little excited.

We tested confirmation bias in 18 healthy young adults^[3] wearing two identical, standard, off-the-shelf hinged knee braces to see whether biases might affect their perception and their movement. Though the two braces were the same, we jazzed up the appearance of one of them and added an LED circuit, switch and USB port.

We told the healthy volunteers that we were testing a prototype of a new knee brace designed to dynamically alter its stiffness based on the wearer’s movement patterns. We showed them a mock-up flyer (entirely false) showing some baseline data on the new brace and listing its benefits. We gave them an idea of how much more it would cost, and told them we’d be measuring their walking in the new brace compared to the same manufacturer’s standard model. Then, we gave them a survey to assess their expectations.

Before using the braces, the majority of the subjects expressed a preference for the so-called “computerized” brace. They expected it to be better in appearance, stabilization, comfort and function in various types of sports. Next, we measured their walking in the lab in both braces using a multi-camera system that gives detailed information about 3D movement. After the trial, the subjects expressed an even stronger preference for the “computerized” brace, even though, unbeknownst to them, both braces were functionally identical.

But did they walk differently?

When we analyzed the gait data, we found that despite their perceptions, the subjects walked remarkably similarly in both types of braces. I was shocked with how identical the walking patterns were. Average stride length was exactly the same – to the millimeter. The amount of knee bending was only one-tenth of one degree different.

Because the subjects were healthy young adults, we did not expect major differences in gait. What was most interesting was that 83% of subjects thought the “computerized” brace performed better, even though they had walked identically in both. Moreover, subjects were less concerned with the purported increased cost of the “computerized” brace after they used it, thinking the benefits were worth it, even when there were none.

In the end, I realized just how important it is to consider cognitive biases when testing advanced orthopedic devices, particularly if I’m using patient-reported outcomes. Established techniques like placebos and blinding are challenging to implement with these devices, but it helps to know that confirmation bias may be present with orthopedic devices. And as complexity grows, it will be more important to find ways to avoid it.

[ Like what you’ve read? Want more? Sign up for The Conversation’s daily newsletter^[4]. ]

References

1. ^{^} I am a biomedical engineer (scholar.google.com)
2. ^{^} complex and technologically advanced (www.opcareers.org)
3. ^{^} tested confirmation bias in 18 healthy young adults (doi.org)
4. ^{^} Sign up for The Conversation’s daily newsletter (theconversation.com)