Diagnostic time out

What is a diagnostic time out? Succinctly defined, it’s a deliberate exercise in differential diagnosis and systematic clinical reasoning in the care of an individual patient. But wait, I hear someone say… isn’t that what we do already? Well, no. We’re all familiar with the traditional model for clinical reasoning that we’re taught in medical school but those of us in the real world of practice nowadays, if we’re honest, realize that it seldom happens. There’s just not enough time when you’re forced to see too many patients each day. And hospitalist incentives, with their emphasis on speed and quick adoption of specific diagnostic labels, run in opposition. What do we as hospitalists do instead? Well, aside from all the care pathways and metric incentives that tell us what to do, we rely on clinical instincts and rules of thumb. Because they bypass formal analysis, they save time. They serve as cognitive shortcuts. We call these heuristics. This method of thinking (fast, instinctive, intuitive) is sometimes known as system 1 thinking. It has the advantages of being efficient and fast and sometimes, in critical situations, life saving. But it comes at the cost of a certain error rate. In order to better understand the process of system 1 thinking we have given the various heuristics names and categories. I recently listed some of those in this post. 

If system 1 is our usual measure of processing to get around time constraints the alternative is system 2: formal clinical reasoning .  System 2 thinking was the topic of a recent paper in CriticalCare Clinics. Although based on a survey of people working in a NICU the article has general applicability. The authors contrast system 1 and system 2 thinking in this manner:

Dual process theory holds that individuals engaging in medical decision-making use one of 2 distinct cognitive processes: a system 1 process based on heuristics – the use of rapid pattern recognition and rules of thumb – or a system 2 process, based on deliberate analytical modeling and hypothesis generation. While invoking system one processes individuals can think fast and reflexively and can even operate at a subconscious level, using pattern recognition to sort vast amounts of clinical information quickly before an illness script that allows for the rapid elaboration of a differential diagnosis. In contrast system 2 processes require focused attention and are purposefully analytical, relying on deliberate counter-factual reasoning to generate hypotheses regarding the pathophysiologic mechanisms by which a patient’s symptoms are produced.

The authors introduced the concept of the diagnostic time out to describe this shift of thinking because it requires deliberate effort. It’s not going to arise spontaneously in the natural course of the ward routine. (The authors were not the first ones to use this term). The diagnostic time out can be considered the cognitive equivalent of the better known procedural time out.

Why is a diagnostic time out needed? Research on diagnostic error has indicated that while some instances are due to system problems (such as failure to communicate test results) most are cognitive errors. These can be linked to the heuristics of system 1 thinking. The diagnostic time out, or the deliberate exercise of system 2 thinking, is a way to complement these cognitive shortcuts with a more analytical process.

Some opinion leaders in the field of diagnostic error have suggested universal adoption of system 2 thinking. This is problematic due to time constraints. Besides, there are some essential benefits of system 1 thinking, particularly in acute life-threatening situations. The real trick is how best to selectively employ system 2 thinking. In other words what are the situations in which system 2 thinking should be used? The authors suggest handoff situations in complex patients including ER to hospitalist, off service/on service and ICU to ward transfers.

How does it work? The authors propose a template but it’s really just the traditional clinical reasoning process. One of their points really got my attention: during the time out diagnostic labels should be removed and replaced by signs, symptoms, manifestations and clinical concerns. This of course is the opposite of what your coders and hospitalist leaders want you to do.

What are some of the barriers to implementation? In addition to time constraints, fear of ambiguity is an important factor. We are afraid to admit what we don’t know. One thing you will never hear a hospitalist say out loud is “I’ll have to think about that.”

A little more on metacognition

This article from Academic Emergency Medicine, published in 2002, remains applicable today. It makes the point that heuristics in medicine are valuable even though they can lead to error. The article also makes the statement:

The increasing use of clinical decision rules, as well as other aids that reduce uncertainty and cognitive load, e.g., computerized clinical decision support,will improve certain aspects of clinical decision making, but much flesh-and-blood clinical decision making will remain and there will always be a place for intuition and clinical acumen.

It presents an exhaustive list with detailed descriptions of the various cognitive shortcuts.

Indulge me in a little metacognition

I found an interesting post about cognitive shortcuts in medicine. I have a minor objection to the title of the post which is Cognitive Errors. Cognitive shortcuts, known as heuristics, which are examples of fast instinctive thinking (system one), often lead to error. In some cases, however, they can be useful because they are efficient and time saving. There is an up side as well as a down side to system one thinking in medicine.

Let’s go down the list. I’ve skipped some of them.

The first example given is affective error. This refers to an emotional response surpassing objectivity.

Next is aggregate bias. I struggle with this one. The author says that the aggregate bias is the belief that data in the aggregate don’t apply to the patient in front of you. My understanding (maybe I’m wrong) is that aggregate bias, otherwise known as the ecological fallacy, is the opposite. That is, it refers to inappropriate application of population data to an individual. It has more to do with treatment decisions than diagnostic error. Remember, one of the first principles of evidence-based medicine is that clinical reasoning decision making starts with the unique aspects of the individual patient. After looking at a variety of references, it would appear that both definitions have been used. Most medical references define aggregation bias the way the blog author does. Those outside of medicine define it as inappropriate extrapolation.

The ambiguity effect is really a bias against ambiguity. So we tend to stick with things we are more familiar with. That may cause us to ignore other possibilities and take too narrow a view of things. As originally conceived it had to do with probability. That is, people have a tendency to gravitate toward choices in which the probability is known or explicitly stated. Of note, the ambiguity effect was first described by Daniel Ellsberg.

The anchoring heuristic is one of the better known cognitive biases. This refers to the tendency to stick with one’s initial hunch despite new evidence to the contrary. You may be so proud of your initial hunch that you ignore new information. Confirmation bias and diagnostic momentum are related concepts.

Ascertainment bias, as the author points out, is an umbrella category. It encompasses a lot of stereotypes and biases. In essence it’s just—-well, bias. It’s not very useful as a unique category in discussions of cognitive error.

Availability bias is one of the better known cognitive shortcuts. This refers to the influence of prior experience. This causes bias toward the first thing that comes to your mind. For example, if you’ve been burned by having missed a case of aortic dissection you may tend to be over concerned about aortic dissection in every future case of chest pain. The flip side is you may fail to consider things you haven’t seen in a long time.

Base rate neglect is a cognitive shortcut that may be considered harmful and wasteful in ambulatory medicine but may be your friend in the arena of hospital and emergency medicine. It’s a failing to consider the true prevalence of diseases in clinical reasoning. It ignores the old aphorism “common things happen most often.” In the high acuity world of the hospital, where you really need to be risk-averse, base rate neglect may be beneficial. Put another way you and and your patient may be better off if you consider worst case scenario.

Then there’s belief bias. I’m not sure this belongs in a discussion of diagnostic shortcuts as it has more to do with treatment recommendations. I cringe when I hear somebody say they “believe“ in a particular treatment, implying that belief surpasses reasoning from evidence .

Blind spot bias is similar to the Dunning Kruger effect in which we think we're smarter than we really are. Humility is the remedy here. Does this lead to a form of cognitive shortcut? Maybe in that we fail to pause and consider carefully that we might be wrong.

Confirmation bias is akin to anchoring. This is the tendency to be selective in what type of accumulating evidence you consider. That is, you consider mainly evidence that supports your original hunch.

The framing heuristic is another well known shortcut. We are biased toward diagnostic possibilities in accordance with the way the initial presentation is framed. Though it can be useful it restricts our differential diagnosis in a way that excludes a wide range of possibilities. Not every returning travel with fever has a parasite, for example.

The gamblers fallacy, according to the blog author, is “the erroneous belief that chance his self correcting.“ This is a cognitive error that tends in the opposite direction to the availability heuristic.

The order effect is something I was vaguely aware of but had not considered as a cognitive error category. It refers to the tendency to focus on information that is proximate in time and to do so at the expense of the totality of events over time. This typically occurs at the point of hand off in a patient who has had a very long hospital course.

Premature closure is just what it says. It’s a tendency for thinking to stop once a tentative diagnosis has been made. It overlaps with other categories such as anchoring. There is probably a subtle difference between premature closure and anchoring. Anchoring implies an emotional attachment to a diagnosis whereas premature closure implies diagnostic laziness.

Representativeness restraint has also been known as a representativeness heuristic. It is a cognitive shortcut characterized by focusing too much on the prototypical manifestations of a disease. This may cause the clinician to miss atypical presentations.

Search satisfaction is another example of laziness in clinical reasoning. It’s a tendency to stop searching once an answer has been found. The author gives the example of missing a second fracture on an x-ray once the first one is identified.

Sunk cost fallacy is a type of emotional heuristic as well as diagnostic laziness. It is the tendency to ignore new information and not consider alternative diagnoses once the original diagnosis has been arrived at after a great time effort and expense (the sunk cost).

Sutton’s slip might be the dark side of Sutton’s law (going where the money is). Pursuing the obvious might lead to error because of other possibilities being ignored.

Zebra retreat is the avoidance of rare diagnoses to a fault. It’s an opposite of base rate neglect.

Are residents getting enough training in managing crashing patients in the hospital?

From a recent study in the Journal of Hospital Medicine:

BACKGROUND: Internal Medicine (IM) residency graduates should be able to manage hospital emergencies, but the rare and critical nature of such events poses an educational challenge. IM residents’ exposure to inpatient acute clinical events is currently unknown.

OBJECTIVE: We developed an instrument to assess IM residents’ exposure to and confidence in managing hospital acute clinical events.

METHODS: We administered a survey to all IM residents at our institution assessing their exposure to and confidence in managing 50 inpatient acute clinical events. Exposures assessed included mannequin-based simulation or management of hospital-based events as a part of a team or independently in a leadership role. Confidence was rated on a five-point scale and dichotomized to “confident” versus “not confident.” Results were analyzed by multivariable logistic regression to assess the relationship between exposure and confidence accounting for year in training.

RESULTS: A total of 140 of 170 IM residents (82%) responded. Postgraduate year 1 (PGY-1) residents had managed 31.3% of acute events independently vs 71.7% of events for PGY-3/4 residents (P less than .0001). In multivariable analysis, residents’ confidence increased with level of training (PGY-1 residents were confident to manage 24.9% of events vs 72.5% of events for PGY-3/4 residents, P less than .0001) and level of exposure, independent of training year (P = .001). Events with the lowest levels of exposure and confidence for graduating residents were identified.

CONCLUSIONS: IM residents’ confidence in managing inpatient acute events correlated with level of training and clinical exposure. We identified events with low levels of resident exposure and confidence that can serve as targets for future curriculum development.

Case based textbook reading in the residency curriculum

From a recent paper:

Textbook reading plays a foundational role in a resident’s knowledge base. Many residency programs place residents on identical reading schedules, regardless of the clinical work or rotation the resident is doing. We sought to develop a reading curriculum that takes into account the clinical work a resident is doing so their reading curriculum corresponds with their clinical work. Preliminary data suggests an increased amount of resident reading and an increased interest in reading as a result of this change to their reading curriculum.

Long live the textbook. After evidence based medicine “became a thing” there were some who went so far as to recommend against reading textbooks. The idea was that there should be a singular focus on looking up answers to focused clinical questions at the point of care, a process known as “foreground reading.” Textbook reading is considered background reading. I contend both are important and dealt with the question in more detail here.

Quackademic medicine metastasizing to accredited CME

Not that there’s anything new about this but it seems to be getting worse.

An ABIM leader enters the debate over MOC

Defenders of the board certification establishment have been largely silent amidst the onslaught of criticism so it’s noteworthy when one of the speaks out. Here’s a viewpoint piece in JAMA. Heavy on unfounded assumptions, light on evidence,  unconvincing to me.

MOC or Drexit?

An interesting read at Kevin MD.

Metacognition: from study habits to clinical reasoning

This article was published in a pharmaceutical education journal but it has general applicability.

What are residency programs doing to mitigate the July effect?

Here’s a report from the green journal:

Results

The response rate was 16% (65/418 programs); however, a total of 262 respondents from all 50 states where residency programs are located were included. Most respondents (n = 201; 77%) indicated that errors occur more frequently in July compared with other months. The most common identified errors included incorrect or delayed orders (n = 183, 70% and n = 167, 64%, respectively), errors in discharge medications (n = 144, 55%), and inadequate information exchange at handoffs (n = 143, 55%). Limited trainee experience (n = 208, 79%), lack of understanding hospital workflow, and difficulty using electronic medical record systems (n = 194; 74% and n = 188; 72%, respectively) were reported as the most common factors contributing to these errors. Programs reported instituting several efforts to prevent harm in July: for interns, additional electronic medical record training (n = 178; 68%) and education on handoffs and discharge processes (n = 176; 67% and n = 108; 41%, respectively) were introduced. Similarly, for senior residents, teaching sessions on how to lead a team (n = 158; 60%) and preferential placement of certain residents on harder rotations (n = 103; 39%) were also reported. Most respondents (n = 140; 53%) also solicited specific “July attendings” using a volunteer system or highest teaching ratings.

Difficulties with EMRs definitely contribute to the problem.

IU critical care

A collection of videos in critical care medicine.

Red flags for in residency candidates for negative outcomes

From a recent study:

Results

From a dataset of 260 residents who completed their residency over a 19-year period, 26 (10%) were osteopaths and 33 (13%) were international medical school graduates A leave of absence during medical school (p less than .001), failure to send a thank-you note (p=.008), a failing score on United States Medical Licensing Examination Step I (p=.002), and a prior career in health (p=.034) were factors associated with greater likelihood of a negative outcome. All four residents with a “red flag” during their medicine clerkships experienced a negative outcome (p less than .001).

Conclusion

“Red flags” during EM clerkships, a leave of absence during medical school for any reason and failure to send post-interview thank-you notes may be associated with negative outcomes during an EM residency.

This was a study of emergency medicine residents. The applicability to other specialties is unknown.

Using FOAM in the residency curriculum

Free full text article.

What is a master clinician-teacher?

Via Nephron Power.

The master clinician’s approach

Definitely worth the read.

Male and female resident evaluations in emergency medicine

From JAMA Internal Medicine:

Importance Although implicit bias in medical training has long been suspected, it has been difficult to study using objective measures, and the influence of sex and gender in the evaluation of medical trainees is unknown. The emergency medicine (EM) milestones provide a standardized framework for longitudinal resident assessment, allowing for analysis of resident performance across all years and programs at a scope and level of detail never previously possible.

Objective To compare faculty-observed training milestone attainment of male vs female residency training

Design, Setting, and Participants This multicenter, longitudinal, retrospective cohort study took place at 8 community and academic EM training programs across the United States from July 1, 2013, to July 1, 2015, using a real-time, mobile-based, direct-observation evaluation tool. The study examined 33 456 direct-observation subcompetency evaluations of 359 EM residents by 285 faculty members.

Main Outcomes and Measures Milestone attainment for male and female EM residents as observed by male and female faculty throughout residency and analyzed using multilevel mixed-effects linear regression modeling.

Results A total of 33 456 direct-observation evaluations were collected from 359 EM residents (237 men [66.0%] and 122 women [34.0%]) by 285 faculty members (194 men [68.1%] and 91 women [31.9%]) during the study period. Female and male residents achieved similar milestone levels during the first year of residency. However, the rate of milestone attainment was 12.7% (0.07 levels per year) higher for male residents through all of residency (95% CI, 0.04-0.09). By graduation, men scored approximately 0.15 milestone levels higher than women, which is equivalent to 3 to 4 months of additional training, given that the average resident gains approximately 0.52 levels per year using our model (95% CI, 0.49-0.54). No statistically significant differences in scores were found based on faculty evaluator gender (effect size difference, 0.02 milestone levels; 95% CI for males, −0.09 to 0.11) or evaluator-evaluatee gender pairing (effect size difference, −0.02 milestone levels; 95% CI for interaction, −0.05 to 0.01).

Conclusions and Relevance Although male and female residents receive similar evaluations at the beginning of residency, the rate of milestone attainment throughout training was higher for male than female residents across all EM subcompetencies, leading to a gender gap in evaluations that continues until graduation. Faculty should be cognizant of possible gender bias when evaluating medical trainees.

Gender bias or true differences in milestone attainment?

Hospitalists’ enthusiasm for MOC: less than overwhelming

Survey results.

Can FOAM be incorporated into medical student and resident education?

It matches the educational needs of millennials according to this article.

Ace those board exams

A lot of it is about attitude.

Pimping: is it good teaching?

Academic Life in Emergency Medicine presents a hypothetical case of pimping on rounds that didn't go well. The follow up discussion lacked clarity because of confusion on just what pimping is. One of the discussants, for example, expounded what was wrong with pimping then went on to suggest ways to pimp more effectively. If pimping is merely teaching by asking questions then it is the Socratic method. It can be either abusive or constructive. Here's a discussion of Socratic teaching from The Pathology Guy.