Thursday, February 10, 2011

How biased is commercially supported CME?

All CME is biased to an extent. Everyone who's knowledgeable on a subject has a point of view, which will invariably sneak into presentations and writings whether or not there is commercial support. The real question in the raging family fight (as blogger Billy Rubin puts it) over CME funding is whether the bias in industry supported CME is worse than that of non-supported offerings in a meaningful way. It's a debate that has real consequences because those who argue in the affirmative seek policy change which would eventually end CME funding by industry. Though probably in the minority and backed with little more than a collection of anecdotes and popular belief (which, as Thomas Stossel recently pointed out, often takes on the intensity of religious fervor) they seem to be winning in terms of policy change (policy change, mind you, not evidence or logic).

Until recently there was no research quality evidence (yes, there was a lot of soft science on the psychology of influence and performance indicators attributable to drug rep promotion but nothing about outcomes from CME) to guide the discussion. (For convenience here I'll loosely use the terms pharmascold and pharmapologist to describe the proponents and opponents, respectively, of restrictive policy change). Absent such research quality evidence the debate usually followed a pattern. The pharmascolds made their appeal to popular belief (industry money is dirty, so if it supports CME it has to be degrading) and numerous anecdotes. (For a nice collection of anecdotes read Daniel Carlat's blog; he's got a bunch of them). Aside from the fact that collections of anecdotes do not equal data, there are anecdotes to support either side. I could cite plenty of examples of non-supported CME offerings that reflect presenters' biases. The government sponsored NCCAM CME offerings, for example, reflect a pervasive bias about what scientific standard ought to be applied to health claims. Even the highly respected UptoDate, from which I often obtain CME credit, is biased, containing many articles which conclude with authors' recommendations based on what they prefer at their institutions. On the other side the pharmapologists argued about unintended consequences and said “show me the evidence to justify policy change and its negative consequences.”

Which leads me to a post from yesterday by Thomas Sullivan (for the sake of discussion we'll call him a pharmapologist) at Policy and Medicine concerning his point-counterpoint with Dr. Howard Brody who takes the pharmascold position (his main post in question being here.) Much of the discussion was over recent research data, a huge new database which, in the form of three published studies, looked to see if there was a difference in perceived bias between industry and non-industry funded activities. (There wasn't. You can find links to the studies in the Thomas Sullivan post). He provided a detailed point by point summary of the exchange, so I'll just make a couple of observations. I read with interest this paragraph from his post:

CME providers are concerned about the decreasing support for CME programs because it means fewer programs for health care practitioners, less innovative and collaborative programs, greater inconvenience for doctors in both timing and geography, larger and less interactive programs, and broader programs that do not address the specific needs of target audiences.

In addition, with waning industry support offerings are becoming more expensive. Increasing distance to meeting sites inflates travel expenses. If you get an educational stipend those expenses can eat it up pretty fast. In my own case the last 15 CME hours for 2010 were out of pocket at around $100 an hour. (For that particular meeting, which may be on its last legs, the course directors have had to fork up some of their own money just for its sponsoring institution to break even.)

Do I think I'm entitled to industry support? Not at all. But the fact remains that as support diminishes my options for CME are more and more restricted. Like the health care system that provides my stipend, I'm on a budget. I have competing financial demands. When money's not an object choices increase. When it becomes an object they diminish. I live with that fact with humor, not rancor or self pity. As a matter of fact I really, really get a laugh out of one tired pharmascold argument: that by giving up industry support doctors somehow “take control” of CME. I'm not sure how that's supposed to work, because as industry support slips away I'm less and less in control of my options.

And about those three studies? Dr. Brody dismisses them with a bit of circular reasoning:

What did the studies show? When physicians attend CME programs, they have to check off boxes on an evaluation sheet, stating whether they do or don't think that the presentation they just listened to showed inappropriate or excessive commercial bias. What all three studies showed is that the vast majority of docs, most all the time, check the NO box. To me that suggests that either the docs are lazy about what boxes they check, or else that they may be unable to detect bias when it might actually exist.

The reasoning is circular because its conclusion is assumed in its premise. It goes something like: “commercial CME is excessively biased compared to non-industry CME. These docs didn't report that. Ergo all these docs (well over a million in the studies, by the way) are either too lazy to give appropriate responses or lacked the ability to detect bias.” It's as if the idea of inappropriate bias attributable to commercial support in comparison to non-supported CME is so self evident as to be axiomatic. External evidence be danged.

Dr. Brody then offers up the straw man and shifts the burden of proof with this:

To suggest that a study that consists of these data show positively that no bias exists in CME programs seems a far stretch. (There might in fact be no commercial bias in CME programs, but you'd need far better methods than in these three studies to know that.)

No one's claiming that no bias exists in CME programs. As to the burden of proof, shouldn't that be on the shoulders of those who want major policy change with all its potential unintended consequences?


Billy Rubin said...

Hi Dr. RW--

Thanks for the post and the shout-out. I have never heard the terms "pharmascold" and "pharmapologist" before. If they are your pet terms, nicely done, and I'll henceforth credit you with those terms for the six people who read my blog.

There is a lot to unpack, but in the interests of time & space I just want to focus on one point you make about "bias." You note that non-industry sponsored CME events are "biased" and that you have read Up To Date pieces that "[contain information] which conclude with authors' recommendations based on what they prefer at their institutions."

To me, that's a slippery use of the term. Everyone acknowledges that individuals may have particular biases, but that's very different than industry bias. Dr. X's bias may be due to how his or her institution's always done business, or their specialty, or because of a personal experience, or an anecdote heard during an impressionable time, etc. We all possess these biases and the best we can do is be aware of them, be circumspect about them, and rely as heavily as possible on the best evidence that evidence-based medicine can provide.

But personal bias has a very different flavor than the bias your average pharmascold inveighs against--that's a conscious bias with a particular goal in mind, namely, the increase in revenue for a given company. When drug companies set up MECCs and the like as part of CME, they very much know what they're doing and why they're doing it. You state that pharmascolds produce little more than anecdotal evidence to support this, and forgive me for sounding snotty as I'm really trying to be cordial, but I find that assertion preposterous.

It's not just big-bad industry that is capable of this kind of commercial bias. You may be familiar with the case of Jesse Gelsinger, the teenager with OTC deficiency who died as part of a gene therapy trial at UPenn in 1999. Dr. James Wilson, the lead researcher, was found by the FDA to have committed a variety of violations of research conduct, including their failure to disclose to patients and families that he had a financial stake in the research should it prove successful. Would a huge financial incentive entice even a fine researcher as I'm sure Dr. Wilson is to--without malice aforethought--cut corners? And if that's true, then what of these mega-million dollar corporations who are beholden to stockholders.

One last disclosure: one of my major research projects involves working intimately with a private corporation on a diagnostic device. I want them to succeed! If that means that their CEO, CFO, and the rest of the crew gets rich in the process, bully for them...but to keep my research as pure as possible, I can't drink from that well. It doesn't mean that I don't have biases that might influence my research, but it does minimize them. Best, Billy

R. W. Donnell said...

As far as I know the term "pharmascold" was coined by Thomas Stossel. "Pharmapologist" originated at Dr. Brody's blog, which is linked in my post. Neither is original with me.

Of course industry is motivated to increase revenue. There may have been a time not too many years ago when drug companies knew they could directly influence doctors towards their products through biased "CME." Personally I think they're smart enough to know that they can't very well do that anymore with today's firewalls that keep them from directly influencing content. (I'm not talking about those restaurant meetings and on-site lunches with key opinion leaders, which are no longer CME accredited, at least in my state).

On a related note, Dan Carlat himself recently predicted (see post linked here that the pressure to move drug companies out of accredited CME will result in more focus on non-accredited and purely promotional offerings. That would be an unfortunate development.

Whether CME is industry supported or not it's very difficult in today's environment to consciously introduce bias. One way or another, it sneaks in.

Bias can lead away from scientific medicine in many ways. If you claim that it's somehow more harmful to the CME product when proprietary interest is involved (as opposed to other types of interest) you should back that up with evidence.

Does bias reach the final CME product in a more meaningful way with industry supported offerings than with other offerings? That's the question that demands proof.

Preposterous assertion? All I can say is I've followed this debate for quite a while, looked for the proof, and there just isn't any.