The Arora Internal Medicine Board Review controversy revisited

The American Board of Internal Medicine and the new ethos of academic honesty

I blogged several times in 2010 concerning this controversy and ended the series with a summary post here. Recently Mayo Clinic Proceedings published two articles on the topic. Among the authors of the first paper is the immediate past president and CEO of ABIM Christine K. Cassel, MD. Both papers deal with intellectual property rights of the ABIM and larger issues of academic honesty. Both papers attempt to delineate the boundaries of propriety. The Cassel piece makes it clear:

There is no ambiguity in what we ask physicians to agree to:

•Each physician who registers to take an ABIM examination is directed not to communicate examination content.

•Before admission to the examination, at the start of each examination, and immediately following the examination, physicians agree to a pledge of honesty to “not disclose, copy, or reproduce any portion of the material contained in this examination,” and they are warned that “ABIM will impose severe penalties on any candidate involved in efforts to provide specific examination question content to others.”

Not to disclose..any portion of the material.. OK. Reasonable people might disagree on exactly what that means. But it purports to be unambiguous and the sentence about not disclosing any portion has no qualifiers. As such it forces a certain interpretation. After taking the exam it's OK to say “Boy that was hard!” But statements like “I couldn't believe all the lupus questions on the test” speak to content. If you want to play it safe consider yourself gagged. ABIM means business.

The other paper, by Ruhnke and Doukas, suggests there are reasonable boundaries that can be drawn as to what is appropriate in exam discussions but ultimately fails to make a clear distinction. More below about that.


What happened---here's a brief summary for those new to the controversy:

About three years ago ABIM accused an internal medicine board review company, Arora, of harvesting board exam content from recent attendees and using such content, alleged to consist of actual exam questions, to teach subsequent courses. A suit filed against Arora was settled in ABIM's favor. It was not pleasant for ABIM diplomats who took the Arora course:

Over two thousand diplomats of the ABIM who had attended an Arora course (approved for 42 AMA category 1 CME hours) got a letter from ABIM expressing disappointment in their ethics for not blowing the whistle. A copy of this letter is to be maintained in those diplomats' ABIM files...

..emails between course attendees and Dr. Arora were examined, and a smaller number of physicians, 135 or so, were targeted for more severe sanctions (more than just the letter). In addition, a few of that latter group were sued by ABIM.

Meanwhile ABIM got into a tiff with another board review company, Frontrunners, over similar accusations of teaching from the test. There was a long and tortuous legal trail which I have not been able to follow with clarity. (As far as I can tell neither side is the clear winner yet). One of the issues is whether ABIM in partnering with the American College of Physicians (ACP) is trying to monopolize the board prep industry.

In the discussion that follows I will cover five aspects of the controversy: 1) general observations about board review resources; 2) conflicts of interest, real or perceived, driving ABIM's actions against board review companies; 3) ABIM's proprietary interest and violations of its intellectual property rights; 4) broader issues pertaining to academic honesty; and 5) where do you draw the boundaries?

General observations about board prep:

Though there are differences in degree all board review courses, to one extent or another, are guilty of what ABIM has accused Frontrunners and Arora of. This includes those that are unabashedly commercial like Frontrunners and those that are affiliated with academic institutions like Mayo, Hopkins, etc. Having attended several in the latter category I have heard speakers who have written board questions, recently taken the exam themselves and gotten feedback from residents. Without question they use the information thus gained in their teaching points. I am left wondering which courses ABIM will target next.


Conflicts of interest, real or perceived, on the part of the ABIM:

ABIM has partnered with ACP in the board prep and recertification process. Is that a conflict of interest that motivated ABIM to go after competing board review companies? I would find it difficult to believe. I really don't know. But if the conflict isn't real it is widely perceived. And perceived conflicts are important as well as real ones in the minds of many ethicists. Public perception, they might argue, impacts public trust. If the public doesn't care practicing internists do. Internists in the rank-and-file need to feel that the ABIM is working for them and not against them. But confidence in the Board among practicing internists seems low these days.


ABIM's proprietary interest and copyright infringement:

I must say I was new to the idea of exam content as intellectual property when this controversy first erupted. It's clear, nevertheless, that the Board deems it so. ABIM has every right to defend its intellectual property. Candidates who took the exam signed a pledge and were bound by it. Those who disclosed exam content were wrong for violating the pledge. If it stopped there it would be easy enough to understand, but the Board didn't drop the matter there, choosing instead to elevate the discussion to the subject of academic honesty. That's where things get a little fuzzy.


Broader issues of academic honesty:

Let's be clear again: if a candidate for certification signed a non-disclosure pledge and then violated it they were wrong. But the ABIM goes a step further and calls it cheating. That warrants further discussion.

During the long educational journey we travel to become physicians we gradually learn the rules and values of academic honesty. These values are internalized to form an ethos that is agreed upon and shared among peers. We all know you can't copy from someone else's exam paper, bring unauthorized notes to the exam or plagiarize. Those activities constitute cheating. But what about studying from past exam content? Is that cheating as the Board claims? Apart from any consideration of stealing content (which is a legitimate concern of the Board) if studying from old exams is inherently wrong that's a new ethos. It's not part of the one we learned during our higher education. Using old exams as study material is common in graduate and post-graduate education. This is not the “it's OK because everybody's doing it” argument because professors have long recognized the process and even encourage it by posting their old exams on the web. Here is just one example from MIT. So look at what is going on. The Board is pointing to a practice long accepted by institutions of higher learning and suddenly calling it cheating. One could argue that it is or it isn't but if it is it's just been redefined. (It's a bit of a stretch so if you think I'm building a straw man read here). Why did the Board shift the discussion? They conflated two issues (copyright and academic honesty) that should be discussed separately. (Note that the Ruhnke paper, as I have quoted below, acknowledges and implicitly legitimizes the practice in higher education of studying from old exam materials).


Fuzzy boundaries:

If, as the Board claims, this is really about professionalism and academic honesty then any meaningful (not just verbatim) conveyance of exam content for teaching purposes would be a violation. Any form of teaching to or from the test would be a violation. That means every board prep resource out there, whether private or academically affiliated, is suspect. Indeed the very designation “board review” is suspect. It's simply what board review courses do. They teach the test. All of them.

The Ruhnke article makes that very point (even implicating ACP's board prep resources!), suggests there are boundaries we can draw to address the issue, but then fails to delineate them:

The distinction between cheating and guided study is crucial. Historical test questions are routinely used throughout higher education,9 and recent examination experience is commonly used to create focused study materials. The American College of Physicians produces the Medical Knowledge Self-Assessment Program, which includes a summary of high-yield information likely to appear on the examination.10 The introduction states, “The content was turned over to 11 carefully selected chief residents and fellows who had recently passed the certification exam. These physicians strained the essential testable points.” The editor continues, “As a frequent lecturer on Board preparation, I rely on input from hundreds of post-examination residents. With their ideas in mind, I refined the outline of Board Basics to target important topics and eliminate nonessential information.”10 Used by medical students for over 2 decades, the First Aid series is updated annually based on examinee reports. Using a focused study guide and sample questions created with examinees' input bears some similarity to using practice questions that may have been on a recent examination as a vehicle for targeted learning. The difference lies in the detail and specificity of the information conveyed to future examinees. Nevertheless, from the public's perspective, both practices represent shortcuts for examination success.

So what are the appropriate boundaries? It all depends on whether this about copyright or the larger issue of academic honesty. If it's about academic integrity as the Board claims, then all existing board review courses should come under scrutiny because they teach from past exam content.