Checklist aims to overcome biases in medical training

Host Amber Smith: Upstate Medical University in Syracuse, New York, invites you to be The Informed Patient with the podcast that features experts from Central New York's only academic medical center. I'm your host, Amber Smith. Bias is when you have an inclination or prejudice for, or against, something, not based on evidence. Today, I'm talking about preventing bias with Dr. Amy Caruso Brown. She's a pediatric oncologist and bioethicist at Upstate who has done quite a bit of research in the area of racial, ethnic and cultural disparities in medical treatment and the provision of health care to diverse populations. And she also teaches medical students. She's created a bias checklist that medical schools are using to help produce curriculums that are, hopefully, free of bias. Welcome to "The Informed Patient," Dr. Brown.

Amy Caruso Brown, MD: Thank you for having me. I'm so happy to be here with you.

Host Amber Smith: We spoke about this subject a few years ago, and now your bias checklist is being used by other medical schools. And I understand you've received some grant funding as well. What will the grant money allow you to do?

Amy Caruso Brown, MD: Yes, we just received a grant, a president's grant, from the Macy Foundation, which supports medical education and educational innovation. The main purpose of that grant is to support building a website, a multimedia tool kit with all of the different components that we've developed to support schools that are adopting checklist approaches to reviewing curricula for bias, the purpose being, we have so many resources that we've created, and I've been asked to give many workshops around the country now. And we really felt the need to have everything in one place, where it was readily accessible and where users could identify themselves by their role.

Are you someone who's using the checklist because you're creating a lecture of your own? Are you a dean who wants to adopt this for your whole school? Are you a community member, an advocate who would like to see this used in your community? And the website will help us quickly route people to the right resources for their role.

Host Amber Smith: So how many medical schools have professors who are using your bias checklist? Do you know?

Amy Caruso Brown, MD: I don't know the a bsolute number. I don't know the a know I have been contacted by somewhere between 25 and 30 institutions. It probably increases every week. I think I hear from someone new at least once a week who's heard about the checklist in one way or another. And we have people from 15 schools who meet with us regularly.

We meet every other month just to troubleshoot and talk through how we're using it in our respective institutions. What problems or barriers we're running into? What ways in which we think we might want to change the tool or additional things we want to adopt to help support use of the tool. And I have heard from, at this point, one veterinary school (that) contacted me and said, "Do you think this could be adapted for veterinary medicine?"

And I said, "I think it could. Do I know how? That's not my area of expertise, but I'd love to see you go for it."

Host Amber Smith: Well, I was going to ask: The checklist is designed, or it originally was designed, for medical school. Is it expanding to other health professions, too?

Amy Caruso Brown, MD: Yes. I guess to go back a little bit, what happened is we developed the checklist initially on paper and then in electronic version, which we made publicly facing and publicly accessible so that anyone could go to it and use it online. And we published some studies on that. And then we just started hearing from people who had found it one way or another, who wanted to work with us. And pretty quickly we started hearing from other health professionals. I have heard from nutritionists. We have many people from nursing programs. There's at least I think four nursing schools that are really committed to this work around the country. And they've spun off their own subgroup that meets regularly because the needs are a little bit different.

Bias is not specific to any one profession. It's interprofessional, these things we see, like undertreating the pain of black patients, and even to black children, who get their pain routinely undertreated in our emergency rooms. That's not coming from any one member of the health care team. That's the biases in society affecting the education of the whole team and then how they practice.

But the needs are a little bit different in terms of how you review curricula. We see, in medical education, for instance, that there's a lot of siloing. So the person who gives one lecture may not know what the next lecturer in line is talking about. The person who's coming in after them may or may not know anything about that person's content, where, what we see when we've worked with the nursing programs, is they tend to have one faculty member who is running the entire course, giving most of the lectures, leading most of the sessions and knows exactly what's going on. So it's a little bit different in that way.

Host Amber Smith: I have more questions, but first let me ask you, what is the "siloing" that you mentioned?

Amy Caruso Brown, MD: Siloing in curricula are when the curriculum is broken up into small pieces that are taught by lots of different people. There may not be enough coordination between those different educators.

So that, say, you have someone who's coming in to talk about the pathophysiology of a particular organ, what happens when this organ is not working well. And then later you may have a clinician, a physician or a nurse or another health professional, come in to talk about treating patients with diseases of that organ.

And ideally, those people would get together and have coordinated so that their material complements each other's, but when you have curricula that are very "siloed," there may not be that coordination. I think where it's really important is that the only person who experiences an entire, say, MD program from start to finish, is the student. Everyone else is just seeing their pieces. And hopefully those pieces are big enough that we don't have a lot of problem with siloing, but even when the pieces are as big as courses or clerkships, which are the rotations the students go through, only the student is really going to see the whole spectrum of how bias manifested in that curriculum. Maybe I'll just add the best example, which is when we were starting our work with the checklist,I asked a student to review a case I was working on, and it was a bilingual, Spanish-speaking family, set in New York City. And they're making end-of-life decisions for a father with Alzheimer's disease. And they're Catholic, and we talk a little bit about, you know, religion and faith and how does that matter at the end of life?

But the student, after reviewing it, said to me, "It's really, really nice to see a Latino family that is not undocumented or migrant farm workers." And when I talked to her, she told me all of the cases she had had over the year where the patient was described as Spanish speaking, the patient was also a migrant farm worker. And it wasn't any one faculty member doing all of these cases. It was many different educators thinking, "Hey, this is an underserved group with important health care needs. We should talk about it," but because they weren't coordinated with each other, because they were siloed, what we end up with is a really clear bias over the whole curriculum.

Host Amber Smith: Can you walk us through how the checklist works?

Amy Caruso Brown, MD: Sure. If you go to the public-facing version, we have a tiny URL to make it accessible, but it's housed on REDCap, which is a platform for this kind of data collection. So if you go there, you will see sort of an introduction, a link that says if you've never used this before, and you don't know anything about this, you might want to go here and read the FAQ (frequently asked questions), and then there's some other resources. And eventually that will link to the website that the grant is supporting, so that we'll be able to steer people in very specific directions.

But right now there's just an FAQ and a glossary -- that is helpful. Then you go into a section that asks about who you are and why you're using it. And that's mostly for our quality improvement and review that we want to know who our users are. We designed it for anyone in an education role, and that might include a student teaching peers, or a resident (physician) teaching medical students, anyone in an education role, to be able to use, to review their own content and to do that independently. I kind of want to save it as in the privacy of their own home or office, just to do that sort of deep self-reflection work about "What am I teaching?" and "How might all these biases that are prevalent in our society, the systemic racism, the sexism and misogyny, how might that be influencing what I'm teaching?"

But we also see that people want to use and do use the checklist in other ways, that course directors sometimes want to use it to review the content of the faculty who are lecturing in their course, mentors might want to look at something their mentee is preparing.

Students might want to assess something they saw presented, a lecture they've already sat through or a small group case that they participated in. So they might want to go back and say, "Something in that content worried me, or I'm not sure about that. And I'd like to use the checklist."

So the first part asks who you are, why you're here, what you're using it (for). If you are from Upstate, we have a whole bunch of other questions that help us pinpoint what part of the Upstate curriculum you're reviewing. And again, that's for our internal tracking, to know how much of the Upstate content is being reviewed.

And then it's broken into domains, and each domain represents an area that we know where we know bias in health care and in the treatment of patients and families is a real problem. And that's coming from the medical and the health professions' literature.

First one is race and ethnicity. That's one a lot of people know a lot about, but there's sections on gender, on sex, on sexuality, on sexual orientation, all these different pieces. On age: there's a lot of age bias in medicine. On disability: another area we don't talk about, but where it's really pervasive, we find that patients with disabilities get fewer preventive health screenings than able-bodied patients, and that seems to come from some, maybe not even consciously, but some unconscious assumptions about the well-being and quality of life of disabled people, of assuming, say, that they're not having sex, and therefore you may not need to do the counseling you would provide other patients. And of course that's not true.

So there's all those domains. Each domain has a question that says, "Is this something you're teaching about in your session?" If you answer yes, you will see some more questions that are really common ways that we have seen bias show up in health care professions and health care professions' education. For instance, for gender, you will see one that asks whether you are using the term "atypical" or "variant" to talk about symptoms in women, because we see that that is very common in the medical literature and medical language, that diseases do present differently in women, in people with female anatomy or female hormones than they do in inpatients with male hormones or male anatomy. And because a lot of studies were done with the idea of the middle-aged man of a certain size as the default patient, the way symptoms show up in women gets labeled atypical or variant, even though anything that's showing up some way in 50% of the population is really not atypical and not variant, but that kind of language is going to shape how future doctors and health care providers think about symptoms and what they look for, what their default is, what they're expecting, and what they consider unexpected. For each domain, there's questions like that.

If you say, "No, I don't talk about this," there is a question that just asks the user to pause and reflect on whether it's something they should be talking about. If you say you don't talk about disability in medicine, where disability is in almost everything we do, the checklist will ask you, "Well, do you think maybe you should?" And the answer is not always yes. There's always a box that you can check that says, "I'm just not sure. I don't know what the right answer is here." And then there's a question about whether the ways in which you're talking about this topic could perpetuate shame, bias, stereotypes or stigma. And that includes a drop-down box with some of the common ways we've seen this show up. So things like fat shaming, equating a particular body size with health; that's very common, or a particular BMI (body mass index) with health, even though again, not backed up in the (medical) literature.So all of these examples are things that we know are pretty common.

They show up a lot in medical education and health professions education. And we also know from studies that have been done across our fields, that they're not true, or they're not supported by the science, or they're not useful in caring for patients. A lot of the things that I learned, learning to associate certain diseases with epidemiology around race, so knowing that certain -- I'm a pediatric oncologist -- that certain malignancies show up more in white children than black children or vice versa. And yet that is never helpful to me. That's not how I make diagnoses. You're always going to see patients who don't look or sound or act the way the stereotypical patient in the textbook does.

And at the end of the day, we're looking at the whole picture. We're very often looking at their physical exam and labs. And for me, I, in the end, I'm making diagnoses based on pathology. If someone has a tumor, I'm going to send the tumor to the lab and ask them to tell me what it is. I'm not going to say, "This patient is this race, therefore, it's less likely that it's this." That's how we write standardized test questions. That's not how we practice. And that's not how we want our students to practice.

So, after answering all of those questions and going through each domain, and it's set up to try to be efficient so that people can quickly answer no for things that don't apply to their content and move on.

And then there's logic embedded into the checklist that will analyze the answers and give a box for each one that says, "You should be a little bit worried about bias." If that's the case, you may not see the box, but if that's the case, the box will drop down and say, "You should think about this a little bit more. Here are some resources. Here's my contact information, or other schools that have adopted this have their own internal leadership point of contact because I really, when I talk to educators, encourage them when they're using it to feel really comfortable asking questions and not see this as the beginning and the end, but just as a beginning to doing this work into reflecting on their content.

Host Amber Smith: This is Upstate's "The Informed Patient" podcast. I'm your host, Amber Smith, talking with Dr. Amy Caruso Brown about the bias checklist she created for medical and health education. So it's not just focused on race or gender, but do you have a sense, since people have been using the checklist, of the most common types of bias that are bubbling up?

Amy Caruso Brown, MD: Yeah, we do, but they're actually all pretty common. Some are just harder to identify than others, and I think part of what we're seeing is that, as a profession, we're getting better at kinds of bias that have gotten a lot of attention, but not better across the board. I was just talking a little bit about, racial, essentialism and epidemiology associated with race without an explanation. That might be the single most common form of bias. We love to put up graphs and say, look, here are the populations where these diseases occur.But we don't often follow that up with relevant information that is helpful to the future health care practitioner. So, great to know that this disease occurs more in people with this ethnic background than people with that ethnic background, but how does that actually help me take care of the patients? So we still see that a lot. And what we've tried to work on with faculty is saying, it doesn't mean you always need to cut that out. Sometimes it's irrelevant and you should just cut it out of your lecture, but often the students want more information, and they need more information, they need to know is that association occurring because people from certain populations have been historically disadvantaged. They have experienced intergenerational trauma, toxic stress from racism, so they haven't accumulated the generational wealth that other population groups have. And that is of course, particularly true for black Americans. And so they have less access to care. And maybe that's why we're seeing this disease association.

Sometimes it is genetics. It's just not genetic as much as the way we've been teaching implies. I think sickle cell disease is a great example because we know a lot about why sickle cell disease occurs in the populations that it occurs. And we can explain all of that to students. So what we've tried to do with those examples in other areas is to say, we just need to give students a little more content and context. We need to help them understand why this association is occurring.

And then, what are they going to use that information for? Are we teaching to the test? Is this a fact to spit out later that they'll never actually use when they go out and treat patients? Or is this a really important area of inequity, where they need to know it because they need to work with the communities that they're going into to help get rid of this inequity, to help resolve that health disparity.

But then I think other areas have gotten much less attention, and there's maybe a little more ambiguity around them. So, gender as a spectrum is a newer concept for a lot of our faculty, very few of our students. Most of our students are very, very comfortable with gender not being binary, with it being a spectrum where people may identify as male, female, in between, neither.

And so we see some disagreements between students and faculty over how that should be presented, how that should be taught. And then, I think disability and weight are still really neglected as areas of bias, that we see a lot of them, but they don't get as much attention, and it can be hard to get traction around the idea that weight isn't a simple medical concept, and losing weight is not just as easy as saying, "Well, you should lose weight, and then you'll be healthier."

Host Amber Smith: Have students identified biases that aren't necessarily covered in your checklist?

Amy Caruso Brown, MD: You asked about the most pervasive kinds of bias. And I didn't mention the one that the students independently bring up the most, which is interprofessional bias, that they see how different specialties criticize each other. And they see how nurses may talk behind the doctors' backs, the doctors say, "Oh, my God, that nurse," and it really troubles the students. They haven't been absorbed into the culture enough to ignore it, but I just saw on a Facebook group I belong to, a physician moms group, a surgeon saying, "Please stop telling students this; it's not true. Please stop telling students that they can't be surgeons and be involved moms and dads," involved parents in general. And yet, I still hear it all the time. We still see a gender gap in who goes into surgery; some of it may be preferences, but I think a lot of it is mentoring, that we're still telling students, you know, you should pick this field or that field because of gender biases. That's really frustrating, but the students definitely notice that interprofessional bias in ways that I think faculty sometimes miss.

Host Amber Smith: I'm curious about how this will ultimately affect patients or whether patients will even notice anything different in their medical appointments. If we fast-forward a few years, when the students who went through these curriculums that have been looked at for bias, will they be different providers and will patients recognize that?

Amy Caruso Brown, MD: Yes. So that's our hope. We work very closely with the office of evaluation, assessment and research with Dr. Lauren Germain, and I think one of the challenges is that we can report things like how many areas of the curriculum am I finding bias? What are the common areas of bias? How often do people who use the checklist make changes? And I'm very interested in all of that data. We're starting to look at, does routine use of the checklist change educators' comfort level, talking about these topics? So if you're regularly reviewing your material for bias and thinking about these different domains and issues, does that make you more comfortable talking to a colleague or a student about racism? Does it make you more comfortable incorporating teaching about racism and health into your content?

So those are the things we've started to figure out how to measure. We're starting to look at students' perceptions of the learning environment, but of course the goalis patient-oriented outcomes. What we really want to see is that students who learn in a less biased environment who receive a curriculum that doesn't perpetuate these myths and biases and stereotypes, that when they go out and see patients, they're better prepared to take care of patients from diverse backgrounds, that they're not defaulting to stereotypes and algorithms that will lead them to misdiagnoses, because we know that's a major source of medical error, is missing a diagnosis, not doing the right test or not thinking that the test is necessary.

Much of that probably comes from biases we teach. We saw that over the last several decades with women and heart attacks, right? That one's gotten a lot of attention, that women's heart attacks were being missed because physicians have learned a particular presentation that was typical of male patients and wasn't the way female patients were presenting.

So the goal being, if we do this across the whole curriculum, maybe we can address that problem from the top down, instead of addressing it with each individual disease, where we finally get enough evidence to say, "OK, we've been teaching that wrong," or "We haven't been teaching people to look for this, and we should."

But that's far down the line, and it's a hard thing to measure. How much does one, within a society that is still really struggling with these biases, where I think there's a lot of agreement within medicine and a lot of agreement in some parts of the country, but still a lot of disagreement in other parts of the country and reluctance to even acknowledge that these are real problems.

So how much does our tool make a difference in that situation? I think that's going to be hard to measure. I still hope we'll get thereand that we'll see that. As one student put it, "Standardized test questions teach students to make snap judgments and not look back, with very little information."

And I think what we're trying to do with the tool and what I'm trying to do in the classes I teach is push back on that and say, we need to always be asking ourselves: What biases am I bringing into this room? And I'm not just thinking, even then, about racial, ethnic, or cultural biases, but also, have I heard that this family is difficult? Have I heard that, hey, they're not giving the medications right, or they don't understand anything? Because I'm bringing all of that into the room. And how can I learn to put that aside, so that I can come in and see this family with a clean slate, acknowledging my biases, because they're going to be there, and being able to recognize them, but then being able to put them aside.

Host Amber Smith: It seems like a higher level of professionalism, really, what you're describing.

Amy Caruso Brown, MD: Exactly, exactly.

Host Amber Smith: So your checklists may be able to shape the education of people who are going to become doctors in the coming years. What about the ones who are already practicing medicine?

I wonder if there are effective ways for those doctors to recognize their biases.

Amy Caruso Brown, MD: Yes, that's a great point and a great question. Certainly for the checklist itself, we encourage the use across the spectrum of medical education, so that could be a speaker who is doing continuing medical education, not just for students and residents, but for faculty. I use this on my own presentations all the time. We certainly encourage it in residency education, where there are more opportunities, but the only thing that we have found we really can't apply a checklist to is, unfortunately, the kind of informal education that happens all the time. The teaching we do on rounds, which is often directed at our residents, but there's also a lot of teaching happening between practicing physicians, where someone calls me as a hematology-oncology consultant and has a question about anemia or wonders if this could be childhood cancer, and I'm doing education back with that person in that conversation. And it's maybe one of my colleagues or partners or a physician at another hospital. That's an informal situation in which it's hard to apply a checklist to something you're not planning ahead of time, but what we're looking to see, and we're hoping to see, is that many of us do formal education as well, that if we use the checklist regularly, what people are telling us is that they start to ask themselves the questions when they're talking in informal situations, that they find that it started to shape their thinking and becoming a habit so that maybe they're not clicking the boxes, in the way that they would if they were preparing a lecture or writing a small group case for students to discuss. But they're still thinking about the questions as they're providing that informal education. And I also hope as a reflective tool, you may, as an educator or practicing physician or a nurse or a physical therapist, sit down with a checklist, not knowing anything, going through, thinking, "Hmm, I think this may apply. What else can I learn about this?" So I hope that it's a trigger for continuing education for a lot of people who are in practice and who are using it themselves. They may start out thinking, "Oh, my goal is to make sure that I don't pass on bias to the students," but they may end up learning new things about their practice that changes how they take care of their patients.

Host Amber Smith: Well, Dr. Brown, thank you for making time to tell us about this. I appreciate it.

Amy Caruso Brown, MD: You're welcome. Thank you for having me.

Host Amber Smith: My guest has been Dr. Amy Caruso Brown. She's a pediatrician and bioethicist who also teaches medical students at Upstate. "The Informed Patient" is a podcast covering health, science and medicine, brought to you by Upstate Medical University in Syracuse, New York, and produced by Jim Howe. Find our archive of previous episodes at upstate.edu/informed.

This is your host, Amber Smith, thanking you for listening.