A recent article by a Washington Post blogger commented on an analysis of the effectiveness of over 3,000 different medical interventions. The analysis was performed by the British Medical Journals Clinical Effectiveness project.
The results indicate that only 35% of medical treatments are likely to be beneficial and without negative side effects. Fully 50% of these medical interventions are complete unknowns in terms of their effectiveness. Basically, we do not know if over half of our medical interventions are likely to work or not.
When I first saw this article my knee jerk reaction was that this shows the incompetence of the medical profession as a whole. Chiropractors are often the doctor of last resort, so we see the mistakes of the medical profession more often than their successes.It must be easy for sanitation workers to develop a negative view of the people they clean up after. After all, they spend their whole day cleaning up our smelly garbage.
Then I thought about how extremely knowledgeable and competent many of the medical doctors I know are, and how much they genuinely care for their patients and seek to provide them the best care possible. I remembered the arguments I have had with friends who went to medical school and how passionate they are about saving lives. In the end it’s not always the mistakes of medical doctors that chiropractors see in their offices. Usually those doctors did a great job, it’s just that those patients would have been better served with chiropractic first.
So I decided to look at the study and see what it really says. The results weren’t so surprising when I looked at how they structured their investigation.
The study was an analysis of Randomized Controlled Trials – the gold standard of evidence in the evidence based medicine research hierarchy. These studies are notoriously difficult to design and perform and are extremely expensive. They attempt to take an intervention performed in the chaotic real world and strip it down to a few small variables that can be controlled and studied.
For some interventions this isn’t terribly difficult. For others it is near impossible. For example, how does one develop a control group for an intervention that isn’t well understood – chiropractic, acupuncture, massage? If you do not know how it works exactly, how can you develop a fake treatment that won’t work to compare against? How do you perform a believable fake massage or chiropractic adjustment? These difficulties, coupled with the great expense involved, make performing adequate numbers of Randomized Controlled Trials on these interventions unlikely.
And there is the rub. If there weren’t an adequate number of those trials for an intervention, they were lumped into the unknown effectiveness group.
Then there is the Randomized Controlled Trial (RCT) itself. Another article by the British Medical Journal analyzed the effectiveness of parachutes on the basis of the Randomized Controlled Trial. As there have been no RCT’s of parachutes (you can’t throw live, blindfolded people out of planes without a parachute) they determined that the effectiveness of the parachute could not be established by the Gold Standard of evidence based medicine.
That article was a spoof, a bit of British humor meant to entertain doctors stuck at the hospital during the Christmas season. They do a spoof article every Christmas and they are all very entertaining to anyone who has undergone a healthcare education. What isn’t funny about this parachute article is how similar it was to the article studying 3,000 real medical interventions. During times of a budget crunch studies like this can be used to justify restricting treatments or not covering them through private or public health insurance plans. As Medicare consumes an ever increasing percentage of our nations GDP, you can be certain that studies like this will be performed to choose where to trim the fat. When the political pen-stroke falls, it is likely to cut out some very beneficial treatments, simply because they are not conducive to the gold standard study design.
My point is to take exaggerated stories like this with a grain of salt, realize that science has its limitations and when merged with public policy decisions those limitations can take on a life of their own. In the end effectiveness is measured by the individual, not by bureaucrats and their paper or scientists in carefully controlled laboratories that in no way resemble real life. Science is a tool meant to serve us. We are not its slaves, as any paratrooper can be thankful for.
Peter Fox DC