Do marathons kill people who aren't in the race? Does when you're born make you more likely to get the flu? And what's the difference between a good doctor and a bad one? These are some of the questions Anupam Bapu Jena of Harvard University and EconTalk host Russ Roberts take up as they discuss Jena's book, Random Acts of Medicine.
Watch this podcast episode on YouTube:
This week's guest:
This week's focus:
Additional ideas and people mentioned in this podcast episode:
Vinay Prasad on Pharmaceuticals, the FDA, and the Death of Duty. EconTalk. Patrick House on Consciousness. EconTalk.A few more readings and background resources:
Adam Mastroianni on the Brain, the Ears, and How We Learn. EconTalk.A few more EconTalk podcast episodes:
Robert Aronowitz on Risky Medicine. EconTalk. Megan McArdle on Failure, Success, and the Up Side of Down. EconTalk. Tim Harford on Adapt and the Virtues of Failure. EconTalk. Robert Frank on Success and Luck. EconTalk. Luca Dellanna on Risk, Ruin, and Ergodicity EconTalk.More related EconTalk podcast episodes, by Category:
* As an Amazon Associate, Econlib earns from qualifying purchases.
| Time | Podcast Episode Highlights |
|---|---|
| 0:37 | Intro. [Recording date: August 22, 2023.] Russ Roberts: Today is August 22nd, 2023, and my guest is physician and economist, Anupam Bapu Jena. He is the Joseph P. Newhouse professor of healthcare policy at Harvard Medical School and a physician in the Department of Medicine at Massachusetts General Hospital. He has an M.D. and Ph.D. in economics from the University of Chicago, and he hosts the Freakonomics, M.D. podcast. He is also the author, along with Christopher Worsham, of the book Random Acts of Medicine: The Hidden Forces that Sway Doctors, Impact Patients, and Shape Our Health, which is our topic for today. Bapu, welcome to EconTalk. Anupam Bapu Jena: Thank you for having me. |
| 1:17 | Russ Roberts: This is a fantastic book of very interesting empirical work on very interesting questions. I found the questions as interesting as the answers, and I'm sure that's often the case, but they're both fantastic. But, you also raise a lot of issues along the way, besides the questions that you examine from your research and that of others. And so, I hope we're going to get into both those things, results from the book, and the implications for healthcare policy and medical care generally. Let's start with what's at the heart of the book, which is something we haven't talked about in a while on this program, which are natural experiments. Talk about what those are and how they come up in medical care and healthcare policy. Anupam Bapu Jena: Sure. So, maybe the best way to describe them is to start with something that might be familiar to many of us, which is something called a randomized trial. We see that a lot in medicine. We see that in other places as well. But, whenever you take a drug, ideally, we know whether or not that drug works in a population of people because it's been tested in a trial, where a bunch of people are randomized to receive the drug and another group are randomized to receive a different drug or maybe a placebo. And the idea is that everything about those two groups of people is otherwise similar. And so, any differences in outcomes, health benefits, or health effects that we see in the treated group versus the control group can be attributed to the drug itself. So, it says something about the causal effect of the drug on the outcome. Now, those sorts of questions come up all over the place in public policy, in medicine, in the natural sciences as well. In economics, a lot of economists have, for decades now, been using natural experiments. We don't see them being used as often in medicine. And, the idea behind a natural experiment is that a group of people are exposed by chance to something--not by the hands of an investigator, but by some natural phenomenon. They're living their lives in a group of people who are exposed, for example, to a hurricane or not, or to a public policy based on an income threshold or their age, and another group of otherwise similar people are not exposed to that thing. And so that allows you to say something about the causal effect of a particular intervention on some outcome that you care about. |
| 3:45 | Russ Roberts: So, let's start with an example from the book which I found extremely interesting. You talk about the Boston Marathon, where tragically there was a terrorist attack there, where--in the past--where three people were killed and hundreds of others were injured. And I think it was your wife who had the deep insight that that was not the only effect of a marathon. So, talk about why marathons can be bad for your health even when there aren't terrorist attacks, and how you might go about trying to figure out if that matters or not. Anupam Bapu Jena: Sure. So, a few years ago, my wife ran this race. It was called the Race to Remember, and it started in one part of Boston and went to Cambridge. And, it passed by the hospital where I work, and she wanted me to watch her on the race route. So, I said, 'Okay. All right, I'm going to do that, but I'm going to park at the hospital because I've got a parking spot there.' So, I'm driving along the main thoroughfare to go to the hospital, but the exit is blocked. And so, I turn around and I go back home. And, the exit was blocked because of the race. And so, I see my wife a few hours later, and I tell her what happened, and she says to me in this offhand way, 'Well, what happened to everybody that needed to get to the hospital that day?' And, that was just a totally offhand comment that she made. And then, fast forward seven, eight months later, we published a paper in the New England Journal of Medicine just a week before the Boston Marathon in that year. And, we put together data from about 10 or 11 cities over 10, 11 years. We identified where marathons were running the routes, and we figured out who was living close to the marathon route versus just outside of the marathon route in that same city or town. And then, all we did is we look at the mortality rate of people who have a cardiac arrest, which is when your heart stops or a heart attack. So, these are very serious conditions. You don't choose to have them any particular time. And, we looked at what happens to those conditions: the mortality rate on the day of a marathon compared to the surrounding days in the affected areas--these are areas that are along the marathon route--and then the unaffected areas, which are just outside of the marathon route, on those same days, the marathon day and the non-marathon days. It's sort of a second control group. And, the main findings are twofold. One is that mortality goes up quite a bit on the day of a marathon compared to the surrounding days in the areas that are right on the marathon route. And the second thing is that ambulance transport times go up by about 30% on the mornings and the early afternoons of the marathons. But then in the evening, when the roads reopen, there is no more delay. And, you might ask, Russ, 'Well, is this because of people running the marathon and keeling over in the street?' No, it's not that. The people that we're looking at are late 70s and above. If you look at people who have dementia or who are on dialysis, a smaller group, those are not running. Those people are not running marathons. And, yet you see a very similar pattern in that group as well. So, it's something about the delay in care that older Americans who have cardiac conditions experience because the roads are closed on the dates of the marathon. So, that's the main finding. |
| 7:10 | Russ Roberts: And, in all of your studies, one of the things I really appreciated is that when these natural experiments occur--in this case, the natural experiment is there's a marathon. And, of course, it's unlikely but possible that people have cardiac events on the day of a marathon because for reasons you can't observe, not because they're excited about the race. It just so happens that on certain days where the marathon happened to be, it happened to be picked on a day when it was extra hot or extra cold, or extra something. And, these are, as you identify in the book, these are confounding factors, and you often want to check and make sure that the group that you're calling your control--because it's not a controlled randomized trial--that they are similar. So, almost every study you did in this book, at least I think every study, you look and see whether at least on the things you can measure, whether the traits that might affect outcomes--income, for example, education--and various other related--age--whether they're similar in the two groups. I'm curious: Often you just say, 'We checked it and turned out they were,' but it's a little more complicated than that. So, why don't you talk a little bit about how you try to assess that and what do you do if it turns out not to be the case? Anupam Bapu Jena: Good question. So, in any randomized trial, and let's say in a medical study, the first exhibit that is presented in a scientific paper is typically this comparison of the treatment group and the control group to establish that they look very similar, and you would expect that because they've been randomized to an intervention versus not. So, it's not a surprise, but you always have to show that. And so, in any study that we do where we are arguing that people are by chance exposed to something--in this case, a marathon versus not--the first thing that we show is that on all observable characteristics that we have, that the groups are similar. And, we go through painstaking detail to show this. We show on demographics that they're similar. We show that in terms of the chronic conditions that they have, that they are similar. For example, rates of high cholesterol, high blood pressure, diabetes, atrial fibrillation, heart failure. I mean, I could go on with a long list of medical conditions. If I noticed that some of those conditions were more prevalent among people who had a heart attack on marathon day versus other days, I'd have some pause because I'd say, 'Wait, okay, maybe these groups are not as good as randomized. Maybe there is something different about the people who are having a heart attack on marathon day versus not.' We also look at things like the types of procedures that might be performed on marathon versus non-marathon days. So, I sort of gave you a story of: Look, there are delays in care that are happening. But, what if, instead, what happens is that there are just different doctors and different nurses in the hospital, or that they are short-staffed on marathon day because healthcare workers are at the scene, or they decide they can't come to the hospital that day because the roads are closed? We can look at that, as well, by looking at staffing levels. We can look at the types of procedures that are performed in the hospital. We can look at the types of conditions for which people are hospitalized. So, the main thing when you're trying to say that there's a natural experiment that's happening--an experiment in nature--is to try and suss out all of these different ways in which the groups might be different and just to make sure that they are the same. The other thing, though, that it's really important is that even when you have a natural experiment, you can say that the effect that we observe is because of the marathon. But, an interesting thing, and I don't think we think about this enough even in randomized trials, is it doesn't tell you the mechanism by which this is happening. So, marathon day is random. Let's say we can establish that. But, is the effect because of road closures? Is the effect because of--we talked about staffing problems in the hospital? Is it because people are upset that--you joked about people being excited, but maybe people are upset because the roads are closed and that stress and anxiety provokes a heart problem. Maybe people know that there's a marathon happening, and then they delay calling 911. So, it's not actually that there's an ambulance delay because they're delayed because the roads are closed, but they just call 911 later than they otherwise would have. That, by the way, we can look at. We actually have the timestamps of when 911 is activated, and we don't see that there's any delays in when they actually call the ambulance. So, there's all of these sorts of things that we want to be thoughtful of when we do this study. |
| 12:02 | Russ Roberts: The other thing I think that's important that often gets lost in this kind of work in academic life--less so in a book for the general public--but it's a very important distinction between the way economists use the word 'significant' and the way normal human beings use the word. So, economists use the word 'significant' in this kind of a study to mean we have a pretty good likelihood that this did not occur by chance but was due to the marathon. In other words, the numbers are always going to be different because of randomness. No two days are alike. But to attribute it to the marathon, it has to be sufficiently large, given the size of the sample. This is a standard thing that leads to a published paper. But it doesn't mean it's significant in the everyday sense of the word. And, in this particular case, and I don't know if you have it off the top of your head, but the magnitudes matter a lot. Is this one extra person? Is it ten? Is it 10% of the heart attacks turn out worse? And, you're very careful in the book, usually, to share that measure of everyday use of the word 'significant.' So, in this case--the case of the marathon--how bad is it for people who go into cardiac arrest on marathon day? Anupam Bapu Jena: Yeah, great question. So, a couple thoughts. So, one is you're absolutely right. We always want to distinguish between a pure statistical significance versus economic significance; and, very closely related to economic significance, when you're talking about medical issues, I'd call that clinical significance. So, is this a clinically significant finding? And, I think it is for a few reasons. So, one is that--just percentage terms, the mortality rate goes up by about 20% on marathon days compared to non-marathon days. Now, so that's a meaningful increase. That's the first data point for you. It's not reassuring, but it's meaningful. The second data point is that--and you started off by describing the Boston Marathon bombings, injured a lot of people, killed several people; that event is salient in most people's minds who live in Boston and probably outside of Boston--but, more people would be projected to die from marathon-related road closures in any given city in any given year than died in the Boston Marathon bombings. So that gives a sense of, 'Well, it's meaningful,'--you know, what do we mean by meaningful? I think that's meaningful in terms of interpretation, as well. And then, the other thing, which I always think about as well, is: Sometimes a finding can be meaningful or economically or clinically significant, but you always want to have--there's got to be, like, a sniff test. Because sometimes it's too large. And, if it's too large, you've got to say, 'Okay, what's going on here? Either there's something that I'm probably missing,' if it's too large. Either there's a literal data error--some calculation problem in the analysis--or there may be something that you actually are not observing, a true confounder. One of those two things is going on. And so, whenever we have a study like this where there is sort of a clinical dovetail, you know, the story is about delay. So, there is an estimate that could come from our study about how much do minutes of delays matter in terms of mortality? And, if that is consistent with what we might see in other non-randomized settings, ballpark, that's more of a reassuring thing to see. Russ Roberts: It's a great example. |
| 15:42 | Russ Roberts: Now, when you're presenting these results, you mention, 'Well, we're not going to not hold marathons.' But, you could argue--you can make a legitimate argument for it. It doesn't mean it's decisive. But, if it turns out that your view holds up--this empirical finding--I would give it some serious thought. But, more importantly, I would give some thought to how we route marathons and what roads we close. And, you would start to think about whether there are other facilities we ought to make available on marathon days. I mean, the Boston Marathon has to be a somewhat iconic route. It's been run for a long time. It would be emotionally, I think, challenging for the people of Boston to have a different route. But, certainly, you could route part of it maybe away from the hospital. You could, again, put facilities in other places for that day, as an emergency. I don't know--have people talked about that? Have you talked about it? Anupam Bapu Jena: They have not as much as I would have thought. So, I remember when this paper first came out, we actually called a bunch of emergency medical service agencies in different cities, and some of them were aware that this might be an issue, and others were not really aware. Because, what they typically focus on--and this is not just true of marathons: marathons are an extreme example because there's so many miles of road that are shut down--but it could be something like a Taylor Swift concert, which leads to a couple of hours of delay in a major metropolitan area. Any sort of large public gathering--concert, July 4th celebration, whatever--may have that same effect. And I think we focus on the safety and health of the participants of those events, not so much the bystanders. And I think, so, some agencies are not as familiar with that idea as they probably should be. But I think where they could come in and say, 'All right, look, well: Here's how we would be able to address it.' So, a simple example--maybe not simple--but one thing would be: if you're not willing to bisect the marathon route, then maybe what you do is you have sort of pods--areas where on one side of the marathon route there's an open field. So, if you have a cardiac arrest--a heart attack--you can be driven there, and there are multiple helicopters there waiting to transport you right above the route to the place where you need to go. Now, I don't know how feasible that would be. Certainly, there's a cost consideration, but you could do that cost-effectiveness analysis and figure out whether or not it makes sense to do that. But, the key thing is that people should be thinking about that issue in general. And, the other thing I'd say, Russ, is, so that's sort of--like, what do you take away from this paper?--is: Okay, that's one thing. How do you structure these events? But, there's another thing that I think is also important, which is it firmly establishes how time matters when it comes to acute medical problems. It's something that, in medicine, we all think is true, but it's hard to show that it's true. And, one of the most important questions that any of us face when it comes to medical care--whether you're on the receiving end or the deliverer of that care--is, 'Do I need to do something now?' You know, you've got a kid who is three years old at home in the middle of the night, and they've got a fever and headache. Do you need to call the pediatrician now? Can you wait a few hours? Do you take them to the emergency room? You're never going to conduct a randomized trial that says, 'All right, a thousand people who have chest pain, go to the ER [Emergency Room] immediately, and the rest of you tune into this podcast.' You can't do that. But here, nature gives us that information. Russ Roberts: Yeah, it's incredibly powerful, actually, in thinking about exactly the question of, 'How much effort you should put into being timely in various situations?' Like you say, we all know intuitively sooner is better than later. But, in many, many things, sooner is not particularly better, and later is a lot cheaper-- Anupam Bapu Jena: Yeah, exactly-- Russ Roberts: because you don't really want to go to the hospital in the middle of the night. Anupam Bapu Jena: Exactly. Sooner is almost always more costly. Yeah. Russ Roberts: Yeah. And, by the way, there's also weather, would be another example. I always wonder about--in snowier climates, I assume it takes longer to get to the hospital, and there are often snow events that are not well dealt with in certain climates. As an economist, I'm willing to entertain the idea that people with heart issues are more likely to avoid snowy areas so they can get to the hospital more quickly. But, I don't think it's a big effect. But I don't think it's a big effect. So, I think the delays due to weather would be another obvious example of where you, on those days, you might want to have some alternative procedures. And, another way to deal with it, as you mention, is to deliver care on the spot rather than getting them to the hospital, which is another way to cope with this, the delay challenge. Anupam Bapu Jena: In some places, they actually try to do that. And, I think it depends on the particular medical care that is being administered. In some European countries, they have almost these hospitals in an ambulance. And, for some things, that might work. But, I could imagine for something like an infection--let's say pneumonia or sepsis--here's an area where you could certainly imagine, if you thought this person had sepsis, giving them antibiotics early. And there, would a matter of 30 minutes or an hour matter? I think they might. It's hard to show empirically, but it is a sort of question where the implications of getting that answer are quite large. If you could show credibly that the administration of antibiotics 30 minutes to an hour earlier at the scene of where you first see a patient in their home or a nursing home improves outcomes, then that's something we should be doing, and we certainly could be doing quite well. But, you know, weather is another interesting area because it is random oftentimes. So, it's good for that, and it also affects a lot of different things. I was having a discussion with someone the other day, and they were talking about the effect of rain; and it could be the case that, for example, storms reduce mortality. So, one of these is they increase mortality because of delays and the direct effects of the storms. But, if people drive on the roads less often because of storms, which is totally reasonable, you might see actually a reduction in traffic fatalities. Or people drive more carefully. So, if you have as your denominator the number of people on the road, car injuries might go down, not only because there's fewer cars in the road, which is one effect. But, also, people are vigilant. They know what to expect, and so they drive more safely accordingly. Russ Roberts: Yeah. A related question would be whether, how many ambulances crash in their zeal to reduce time to the hospital? I assume it happens--I've never seen one--but I assume it happens every day in America: an ambulance crashes somewhere. Anupam Bapu Jena: Yeah, it must happen. It is also an interesting question about whether or not they should go with lights and sirens, because those things can be distracting, and what happens to other cars who are trying to evade the ambulance? I mean, these are interesting questions. |
| 22:57 | Russ Roberts: One of the things that kept echoing for me in your book--and I'll just mention, I didn't say it before: a lot of the work you're reporting is work you've done with various co-authors using these kind of techniques. There are many, many things in life that can't be measured, and there are many things that we desperately want to measure that we can't measure. I thought about this both in terms of doctors, which we'll talk about, but also patients, where we know some things about a patient--we might know their age, we might have a lot of their prior medical history--but we might not know how healthy they really are. In fact, we don't know. And, we're using these various measures that we have data on as proxies. And, I kept thinking about this. So, let's talk about a couple examples that I thought were really fascinating. You ask a profound question in one chapter: 'What makes a good doctor?' And, you look at older versus younger doctors, and you look at doctors from prestigious medical schools versus not-prestigious. In an earlier chapter, you looked at doctors who were doing research and are in theory on the cutting edge who didn't go to conferences, and the doctors who were left behind to take care of their patients, who were not as--in theory--not as experienced, not as skilled, not as sophisticated. And, you find very interesting things in those cases. So, try to go through each of those and talk about what you've learned in the studies you've done. Anupam Bapu Jena: So, let me start with the second one because there's things about what is it that makes a good doctor; and then there's the things that doctors do more or less of. And, I think that can separate why we're finding these different things here. So, in the second example, there's a chapter, 'What happens when all the cardiologists leave town?' And, what we showed was during the dates of these major cardiology conferences--like the American Heart Association or American College of Cardiology--patients who have cardiac problems, acute cardiac problems, they actually do better on the dates of those meetings, if they by chance are hospitalized when the cardiologists or some cardiologists are out of town. Russ Roberts: So, their regular doctor is out of town. They're stuck with the kid--the young upstart who couldn't get a paper accepted at the conference--and they have lower mortality rates during the conference. It's shocking, but possibly true. Anupam Bapu Jena: Yeah, exactly. And, it's a type of thing where I actually had the opposite instinct. When I thought about the idea, I thought, 'Okay, staffing is going to be lower, patients are going to do worse because the staffing is lower, and maybe the staffing of the better cardiologists might be lower.' And then we find this finding, which is the opposite of what we thought. And then, this goes back to our earlier discussion: 'Well, what do you do then?' Well, the first thing you got to do is establish: 'Is this a natural experiment?' Show that these characteristics of these patients are identical on cardiology meeting days and non-meeting days. All right, check. We do that. Now we're into the: 'What is the mechanism that explains this finding?'--land. And, there is an additional data point that we figured out, which I think tells a story, which is that: Rates of certain procedures fall. They fall by about 30% on the day of these meetings versus the non-meeting days. So, how do you put that together? The way that I put it together--and I'll give you maybe two stories--is as follows. So, the first is a story about a young man and an older woman. The man is 40 years old. He's a construction worker. He has chest pain while he's working at a construction site. No other medical problems. He gets rushed to the emergency department. They do an electrocardiogram--an EKG--of his heart. He has a heart attack. They rush him to the cardiac stenting lab. They put in a stent. He's discharged from the hospital in two days, lives a long, happy life. Great. Then, there's another woman. There's another person who is a woman. She's 90 years old. She lives in a nursing home. She's got 10 different medical problems because she's lived a long life. She's 90 years old. And, she has the exact same chest pain. She is brought to the emergency department. She gets an EKG--and she has the exact same EKG. And, she also has a heart attack. So, they take her to the cardiac stenting lab. She also gets a stent, but she dies within two weeks of that procedure because of the complications of the procedure. That sort of story I think will resonate with anybody, whether you're in or not in medicine. And, the idea there was that for the first guy, it was black-and-white whether he should get a stent or not. In the second person, we might act like it's black-and-white in part because either we don't have the data to tell us otherwise or because we have an inclination to want to do something to help people. And, maybe there's financial things at play as well. Though, I think that's less important than the desire to do something to help somebody who has a physiologic problem that we think we understand and that we can solve. But, those types of people, we're actually probably in either the gray area of medicine or maybe the other portion of black-and-white where you don't do anything. And, the second story is the following. If you were to take any given cardiologist and say to them, 'All right, I want you to think about all of the procedures that you did in the last year'--let's say a cardiologist did 400 procedures--and you said to them, 'Pick the hundred procedures where you're most confident that you delivered a procedure for which the benefit exceeded the risk.' I'm pretty sure that they could give you close to 100 people where that was actually true. Right? Maybe not perfectly, but for the most part. But, the challenge is: in the real world, they're not constrained to doing it in 100 people. They can go all the way down the distribution of risk/benefit; and they may end up in a place where the benefit is outweighed by the risk. And, they don't know that. They don't know that that's true. Otherwise, they wouldn't do it, I presume. But, the challenge in what I think these sorts of studies can illuminate is: All right, something might work on average. We know that cardiovascular care is on average good. But, it is also true that on the margins, there may be people for whom the benefit is outweighed by the risk. How do you show that? How do you quantify the magnitude? And then, for me, the most important question is: 'Is it even possible that cardiologists could figure out who those people are?' And I think what these types of studies say, is, 'Yeah, they probably can. If push comes to shove, they can prioritize.' So, on the dates of these meetings, there are people who don't get procedures. But that's not random. The people who don't get procedures are not randomly chosen to not be given a procedure. They are selected intentionally by the cardiologists in the hospital to not receive that procedure. And that, to me, is very powerful. Russ Roberts: And then, the question is, 'Why is it that when the conference isn't going and the high-level research cardiologist is in charge--not the stand-in colleague replacement--why in those settings the cardiologist is more likely to order those procedures?' Because that's the claim, right? Anupam Bapu Jena: Yeah, that is the claim. The short answer there is, I can't tell you. I don't know. I think there's a few possibilities. So, one is that there is an incentive to do it. Now, the question is, 'What is the incentive? Is it a financial incentive?' A lot of times when I give a talk about something like this, people say, 'Well, in America, we're paid to do more. The cardiologists have an incentive to do more.' And, the short answer there is, I don't think that that's driving it. For a few reasons. One is, in most of these settings, we're looking at academic teaching hospitals where their doctors are not often paid the same way that they might be paid outside of that setting. So, it is true that there is some fee-for-service type payment, but it's not nearly as big a deal as in other places. And so, I don't think that that's what's happening. And also, there's other evidence suggests that the responsiveness of procedures or the type of care that you provide relative to what you're paid, it's there, but it's small. It doesn't explain this. So, I think what's going on is that there's just a desire to do something, right? Think about what's happening here. There's a woman who has a blockage of her heart. We know that. If you unblock something that is blocking an artery, wouldn't that make it better? Yeah, it makes sense that it would make it better. And, unless you had convincing evidence to the contrary, you would do what you think physiologically makes sense. And, I think that's a big driver. We operate in medicine, in an area where we have some physiologic information or data or evidence, but it's not perfect. And, that is where the art of medicine comes into play. It's to know: Who is this person in front of me? Do I think the risk and the benefit are aligned, or are they wildly different in one direction or another? |
| 32:19 | Russ Roberts: Yeah. We had a recent episode--it hasn't aired yet, but by the time our conversation is aired, it will have been released--with Vinay Prasad--on this question of screening and the natural--not just the financial--incentive, but the natural emotional need that we all have to find out and to rule out or to feel comforted. And, I think that's part of it. But I'm going to give a different answer, that I like to think about. It's not an answer: it's a speculation. And I'd love your reaction to it. So, you come out of medical school, and you think everything works because you've drunk the Kool-Aid and you're in love with your profession, and it's extraordinary. And, at some point, along the line, I think you probably start to wonder whether some of the things that you're told are indicated in this setting or maybe not the right thing. But, if you're really successful and you're doing the 400 or maybe more procedures a year, I would suspect that when you're at the top of your career, in the peak of your skill, you kind of get a little overconfident. And maybe rightfully so. You're really good at these things. You've done them a lot. And, although a young person might not be confident doing a stent on a 90-year-old woman, the veteran's done it so many times: 'I think I can handle. I think she's going to be okay.' And, they convince themselves. And then, there's probably some point--there's a U-shape or inverted-U here--where you start to think, maybe later on, you start wondering, 'All those times I did that procedure, maybe they did not work out the way I thought they did.' And, you don't get a lot of direct feedback. Your patients die. No one sends you a card three years later and say, 'Well, they made it three years. You thought it was five, but just want to let you know.' So, there's not a lot of direct evidence. And, ego plays a role. And you also want to convince yourself--you've been doing this procedure for so long to so many people--the idea that it might not be always indicated, it's kind of unnerving. So, I don't know. What do you think of that? Anupam Bapu Jena: So, I think it's perfect. I completely agree. I think that is definitely going on. And, the way I would think about it--another way to say it--is a following: is, if you've got a sick person in front of you, if you do a procedure and they benefit--or you've got a sick person in front of you, if you do a procedure and they, quote-unquote, "do well,"--you pat yourself on the back. You say, 'I made a good decision. I intervened at the right time. This person benefited.' If you do that to the same sick person in front of you and you do a procedure, they do poorly, you don't think to yourself: 'Wow, the procedure caused that.' Well, you know what? 'This was a really high-risk person. The likelihood of this thing working out was not that high because of all these 10 other medical problems that this person had, and she's 90 years old. These things happen.' So, there's ways that people can rationalize what they want to be true. There's another way to kind of get at this question. I do think you're right: that, as you get more experience, confidence does rationally build. There might be irrational components as well, but I think first order probably is a rational response, which is that you get more experience, you get better at things, and you can do things in places where you perhaps couldn't have done them before. But, that might come with some trade-offs that you don't recognize. One idea that I've thought a lot about is whether or not doctors are able to actually suss out the difference between what they did and what something else did. So, here's a good example. Imagine you have a cancer patient who is unlikely to respond to a treatment. And, you give them a medication. And they do really, really--a cancer medication--and they do really, really well. That could be totally random, right? But, my guess is what you would see is that doctors who see patients who are subsequently very similar to that patient would be more likely to give that medication because they ascribe the benefit to the medication--as opposed to something else. Like this person just--even though they looked like they weren't doing well, physiologically, they were quite strong, they're vigorous, they're not that frail compared to other people. So, they might have been able to do better anyway. Maybe it's through the medication, but maybe not. And so, there's all of these instances where doctors are exposed--first of all, they're often not exposed to any feedback. But sometimes they get feedback, but they might interpret it in different ways. Russ Roberts: Sure, yeah. I like the--one of my favorite doctors, a friend of mine, was--he's very aware that most things get better on their own, and he's very careful not to attribute his own intervention or the drug or whatever it is. And, it creates a natural skepticism that I--as listeners will not be surprised to hear--I find I'm very biased toward believing that. But I don't know how widespread that is in the profession. But, that's a person who's older and wiser. |
| 37:26 | Russ Roberts: But, I want to try something different on you. And this is kind of a crazy idea. Because I got thinking about when you say, 'What makes a good doctor?' Our system--the system in America--is actually designed to produce good students, not good doctors. We like to think they're related, but they're increasingly--that relationship is increasingly unimportant in a world with more AI and a world of the Internet, right? So, the fact that a really smart person with a really good memory--that used to be a really helpful thing in diagnosing, in diagnostics, and possibly in treatment. But, in a world of AI [Artificial Intelligence] and a world of Google, those skills are not so important. There are other things. And I'm going to pick one for fun. You didn't talk about this, but you raise a wonderful question in the book. You see somebody come through the door to be your doctor. How do you feel when you find out it's male/female, black/white, fancy medical school/not fancy? Etc., etc? But, I would suspect that the best doctors aren't easily measured by any of those things. And, age--all the factors that you write about in the book--that there's something intangible we might call--I'm going to call it intuition, a term that I've struggled with. I used to sneer at it, but after hearing Patrick House talk about it--the neuroscientist in the episode we did with him--where he sees intuition, he says, 'It's not your gut. Your gut is good for dissolving food.' Intuition is when--it's the part of your brain that processes a lot of data and evidence and things that have happened, but without your conscious knowledge. And, that's what you're drawing on when you're really a great diagnostician, I would think. So, we don't spend, I assume, any effort in applying and looking at candidates for medical school to find out what might correlate with great intuition. Or other characteristics--bedside manner being an obvious example that is important but not generally cared about in admissions. Have you thought about how we might improve that? Instead of just picking the doctors who got the best grades, we actually picked the ones who would be the best doctors? Anupam Bapu Jena: Yes-- Russ Roberts: It's two different things-- Anupam Bapu Jena: I've thought a lot of--I mean, there's six different ways we could go with that. But, remind me at the end, I do have a thought experiment. I'd be curious to get your answer on it because it sort of meshes with this idea of intuition, or what I might call the art of medicine, which I think are related. But, yes: I think that the way that we pick doctors is--the way that we pick doctors and the way that we train doctors is arcane. And, one way you could show that empirically is I bet that if you looked at the gradient between, sort of, doctors who went to certain medical schools--quote/unquote "the best ones," versus "the worst ones"--50 years ago versus now, the gradient in outcomes, however we might measure it, is probably much narrower now because there's a lot of other factors that go into a good clinical outcome besides whether or not the doctor had an encyclopedic knowledge. Which might've mattered before there was things like computers where you could look up information. We teach a lot of things in medical school that are completely irrelevant to the practice of medicine. A lot of things, actually. And then, we do not teach things which are extraordinarily irrelevant to the practice of medicine. So, I think one of the--and I think Vinay talks a lot about it actually--is that one of the key things that doctors have to be able to do now is interpret evidence. Interpret data. And, a new clinical study comes out: Is it a high-quality study? Is it a low-quality study? Do you allow it to influence your decision-making in some sense? That is a key skill that is required, I think, of physicians and other providers today that we don't teach at all in medical school. Which is why I think in many instances we see very low-quality studies being used, produced, and relied upon. So, that's one thing. Then, there's another thing which relates to, sort of the art of diagnosis. I don't think we spend enough time, in medical school, training people to be good diagnosticians. Because ultimately, that's the core--one of the core purposes of a doctor is to be able to make that right diagnosis. And only in the last 10 years or so have we sort of understood that one of the key quality problems in this country is misdiagnosis. We focus a lot on safety errors--like, amputate the wrong leg, or you operate on the wrong part of the body. Those sorts of things. You leave a tool in the body during surgery. Those are sort of obvious problems. The less obvious ones are: Someone came in with chest pain. And you thought about a clot in the lungs, you thought about a heart attack, you thought about a ruptured lung; but you didn't think about a ruptured aorta. Why didn't you think about that? What is it that prevented you from making that correct diagnosis? So, that is an area where I think the training could be a lot better. Now, in terms of selection--you know, how would you choose medical students differently? I mean, I still think you want to try to get as smart as people as possible to be in medicine. And, is the testing going to be any different? Like, we have standardized tests. Would you change them in any way to attract a different type of person? I don't know. I think probably what we want is really smart minds. So, the selection I think may not be that different, but the training might need to be very different. So, I think there's a lot there. But, let me get to your response to that, then I'll give you a thought experiment. Russ Roberts: No, I was just going to say that I don't think we want the smartest people to be doctors. We certainly don't act that way. We get the best students to be doctors-- Anupam Bapu Jena: That's true-- Russ Roberts: They're not the smartest. We get the people who are the most--there's a lot of virtue to this by the way. We get the most diligent, the ones who are the most focused on success. And, there are a lot of characteristics that come along with that, that might turn out to be extremely important. Obviously--we might not get to it--but at various points in the book tou talked about the role of focus. And, staying focused when you're exhausted, staying focused when it's your birthday, staying focused when you're worried about your stock portfolio. These are all examples you write about and how doctors' outcomes--how patient outcomes--differ or not. In many cases, they don't differ. Because doctors are able to put those things aside. And, I think that's extraordinary. And, it could be that some of these criteria that we use for medical school admission--and training, weeding people out--is selecting for those characteristics, not just the ability to get an A in a really hard chemistry class. Which really, on the surface, seems to be the wrong criterion. |
| 44:44 | Russ Roberts: But, anyway, I don't mean to say that we should select for people who have good intuition. I actually think your point about training--the importance of training and how we train people--especially with respect to uncertainty, risk, data, and evidence, I think is extremely important. And, I think we ought to be thinking about how to select people who can do that well: who can be trained in that style, who are susceptible to that, who are flexible in certain intellectual ways. And, I have no thoughts on how to find them or what tests we would do in advance. But, anyway, I think those two things interact. The kind of people who come to medical training and how you train them probably hasn't been thought about very much. Tell me your thought experiment. Anupam Bapu Jena: Well, so, yeah, let me just respond to that and give my thought-experiment. Russ Roberts: Yes, go ahead. Anupam Bapu Jena: And, there's this phrase, common things being common, right? There's a lot of medical ailments that are very common and that are very commonly misdiagnosed--because they're common, but also because they present in ways that it's not obvious to the diagnostician. You could imagine a situation where we train people--the majority of people--to be diagnostically excellent when it comes to common medical problems and focus their mental bandwidth on being really good in those areas. And then there's other people who become really specialists, who can help solve the zebras. So, if we don't know what's going on here, we quickly allocate you to somebody who has diagnostic excellence in a more limited domain of diagnosis. And, that's something--but, what we do in medical school is not that. We go ad nauseam about the zebras. And so, if you're teaching in the medical rounds and you present something to a medical student, they're going to give you all the zebras as well. But they might miss one of the common things that you never want to miss because we've trained them to learn about all the zebras. Russ Roberts: Yeah. And that's, of course, the bread and butter of many medical shows on TV, which you also write about in passing. We may not get to that again, but interested readers, listeners will find that in the book as well. Go ahead, what's the thought experiment? Anupam Bapu Jena: All right. So, here's the thought--the thought experiment is following. All right. So, the listers may have a--we'll give a 10-second pause to think about what they would say. Sorry. So, there's three types of doctors, okay? So, the first doctor is a doctor who does not know any of the evidence. They're not aware of the most recent clinical trials. They don't know the most recent guidelines. They just practice based on their experience, what they learned some time ago. So, that's the first doctor. And, that doctor has certain types of health outcomes for their patients. And, the second doctor is the polar opposite. That doctor, he or she, they know the most recent clinical trials, they know the most recent clinical guidelines, and they follow that information like a cookbook. They never deviate from it. And they have certain health outcomes associated with their care. And then you've got a third type of doctor who--let's say, 70% of the time, if you were to observe their patterns of a practice in a very detailed way, 70% of the time, what they're doing appears to follow the cookbook. They follow the clinical guidelines, the most recent clinical trials. But then, 30% of the time, they're doing something different. You can't square it very clearly with the guidelines based on at least whatever information you're able to obsess[?] as an outside observer. But, nonetheless, they're doing something differently. And, the thought experiment is: which of those three types of doctors do we think would have the best outcomes? Let's say, outcomes is like mortality or health improvement. Which of those three types of doctors do we think would have the best outcomes? I'm curious, what do you think? And, by the way, I don't know the answer to this question. It's a question that I want to study, and I think we can study it, but I have my own intuition of who might be the best. Russ Roberts: Well, it reminds me of: Who do you want to win a chess game on your behalf: a human being, a program, or a human being working with a program? And, I think we want the latter. I think they do the best, at least for now. It may turn out not to be the case eventually. And, what's fascinating about it is that, as human beings, I think we struggle to accept algorithmic advice based on data with no art to it. And, if we're not careful, we will overrule and override the algorithm when we shouldn't. And, that's the risk of category three. Category three, though, has a deep appeal because category three--the thoughtful category three doctor, the doctor who uses evidence almost all the time but understands that there are times when it does not apply and uses this mystical, dangerous, but sometimes powerful idea of intuition senses something that is often missed by the algorithm in the patient's eyes. And of course, eventually the algorithm will look at the patient's eyes, too, and scan the retina, and get everything right. And, we like to think that. But, going back to the original question that you asked that I repeated recently: 'Who do you want to see walk through the door?' The first person scares me--the person who doesn't keep up. Anupam Bapu Jena: Yeah. 'What is lipitor?' Russ Roberts: The second person also scares me. And, the third one is like my brother, sister. But, I worry that they picked the wrong 30%. So, it's tricky. Anupam Bapu Jena: So, what are your thoughts? I couldn't have summarized it any better. The core question is: Intuitively, I like the third person because I want to believe that when they deviate from the clinical guidelines, they're doing it--they're clearly doing it, or we hope they're doing it--with some intention. Right? And, the question is whether or not that intention is misguided or not. Are they making systematically good decisions? Are they making systematically bad decisions? Or, is it a wash? And, I think that ultimately is an empirical question. But, the reason I raise that as a thought experiment is because, even though it might seem obvious that doctor three is the type of doctor who we want to think about--one is who knows the information, but they sometimes depart from it because there's something else that doesn't sit right with them--there is still, I think, a large focus in medicine of saying, 'All right. Well, here are the guidelines. Why would you have not prescribed this medication to someone who met these guidelines?' And, an insurance company might, being a doctor for deviating from that, a regulatory agency that is measuring the quality of a doctor's care might say, 'Well, okay, you have 100 patients with high blood pressure, hypertension, but only 85% of them are being treated with a blood pressure medication. Why aren't the other 15% being treated?' Because we know that high blood pressure is correlated with kidney disease, heart disease, stroke, all those sorts of things. And, there is a push towards measuring the quality of doctors based on these guideposts, which I think are fine in general, but they don't allow for the possibility that sometimes deviations might be warranted. And, we should do a good job of figuring out whether or not those deviations are actually good for patients or bad for patients. So, another way to put this: We could look at data and figure out whether doctor one, doctor two, or doctor three, which one of them has better outcomes and sort of put this to rest, at least for certain conditions. |
| 52:43 | Russ Roberts: In terms of general advice for life, I think it's an interesting question. We think about entrepreneurs, the value of failure and the lessons that are learned from that. And, I think a lot about how my attitudes towards risk and reward have changed over the years as I've experienced things in life and read more about it. And, we recently had an episode with Adam Mastroianni talking about how hard it is to learn something from a lecture or a book. And, often, you have to go through it before you really absorb it and use it later. But, I would think in the area of medicine, trauma on the part of the doctor is a real mixed bag. If you've lost a patient when you were younger due to a mistake--you mentioned a mistake you made in the book where, when you were younger, you confused two patients. They were in the same room, they had similar conditions, and you flip-flopped the prescriptions by accident. I'm sure that you have treated prescriptions of two people in the same room really differently since then. That was a traumatic experience. You write about it. It's very moving, very powerful. It's really hard to absorb lessons like that thoughtfully. It's really easy to absorb them in an extreme way: either, 'Well, I'm never going to let that happen again.' But, sometimes you probably--you're going to make a different kind of mistake, if you're so focused on avoiding certain kinds. Or, you give a patient a treatment--we talked about earlier--and they don't make it. You talked about the one that survives. The one that doesn't make it, say--well, you're traumatized as the diagnostician and as the doctor. So, I wonder: It would be a very interesting book to interview doctors about how they've absorbed--not the literature, because the literature is a mixed bag. The literature is, by definition, averages; and it's a great place to start. But often, as we've talked about, it misses particular things, particular people. And, great doctors handle that distinctiveness of people well. That's the craft, the art that you were talking about. How do you get there from here? If you're thinking about--as a mentor to a younger doctor or a younger doctor aspiring to be great--absorbing your own personal data, of your own personal clinical experience, must be unbelievably hard to do well. Because I think about just something much less important, which is investing: I think most people really struggle. They have a bad loss; they say, 'Well, I'm never going to invest in the tech industry again,' because they've lost all their money. Or whatever it is. And, I think that's a huge part of being a successful actor in a world of uncertainty, is processing your own successes and mistakes in a thoughtful way. Really hard to do. And, it's a small sample. Anupam Bapu Jena: Yeah. 'I'm never going to invest in my kid's college education with crypto again.' I think--the way I would think about this is, first of all, it takes more than one person. I think it is probably too much to ask of any given person when the stakes are so high like this, to internalize all of what it takes to figure out what to do differently in the future. But, the way I would think about it is ask yourself, 'All right, what happened?' So, let's say something negative. Something negative happened. You had a young patient who developed colon cancer--on the younger age of things--and, you think to yourself, 'Should I now be screening young people for colon cancer?' And, there was a cost to that, which is interesting but I think not as important, but there's a risk perforating the colon. If you do a colonoscopy and you make those trade-offs based on the risk and the benefit, but you stop to ask yourself, 'All right, what happened? How bad was it?' That's the first thing. If it was bad, then you say to yourself, 'All right, how much do I think that this bad outcome was a result of something that I did and could do differently?' Because if the answer is not, 'Yes, it's something I did that caused this and I could do something differently,' then you're in a very different place than if, 'Yeah, there's nothing I could really have done.' Because there are things that you might have caused but you couldn't do differently. If the way that the medical system is structured is to work 24 hours in a shift, as it is in some hospitals, and a mistake happens at the 23rd or 24th hour, the system has to change. Or you have to figure out how to rejuvenate yourself in that last hour. That's a very sort of--error or a different type of error than something else that might have been potentially reversible. So, I'd be thinking about whether or not you can intentionally describe to yourself, with the help of others, 'Did this problem result from something I did, and could I do something differently to do so?' And then, the last piece of it is, 'If I were to do something differently, what does that impact? What impact does that have on practice in general? Does that mean I have to work 12 more hours a day to make it happen? Does it mean that I entail other patients to procedural risks that I might not have done so before?' So, example: if you decide to start recommending colonoscopies for younger people because of some bad outcome that you experienced, that would entail some risks. So, I think putting all those things together is probably the way to think about it, and it's hard to do on your own. |
| 58:21 | Russ Roberts: Let's close with a discussion of the Hawthorne effect in medicine, which in many ways brings together a number of the things we've been talking about. I gave you a bunch of different things about doctors and their characteristics. One of them was age--younger versus older doctors. So, talk about what you found when you looked at that, and then let's segue into the role of being observed and if those--because I think they interact, possibly. Anupam Bapu Jena: Yeah. Yeah. So, the first finding you alluded to is the age of the doctor. Who is the doctor that you went to walk into your room? Because I'll tell you, my intuition is--for better or for worse, the way I behave--is I look for a doctor who's got more experience. And, we recently had to make a decision like this, and I was looking for someone who was the most experienced. And, it is a reasonable question to ask though whether or not that greater experience confers any benefit. And, there's two ways this could go. One is you've got a doctor who's got lots of experience, they've seen lots of things, stands to reason that they would be able to get better outcomes, better diagnosis, better outcomes. But, it is also the case that doctors have a very prescribed training process. They spend an intense amount of time earlier in their career training, 80 hours or more in residency. And then, in the first few years after that, they're also really quite busy, and they are really familiar with what the most recent guidelines are: because that's what they've been doing for the last three to seven years perhaps, is learning what's the most up-to-date medical technology. And, to the extent that medical technology is a key driver of health outcomes, maybe they would do better. The challenge with studying this empirically is that if you look at doctors who treat--or older doctors, more experienced doctors--and you look at their patients, you don't know whether or not the outcomes that they achieve are because of their skills, or is it a function of who the patients are that they treat? Because, if a patient is sick, they're going to perhaps want a more experienced doctor. And if you looked at more experienced doctors, you would find then that their patients have worse outcomes. But, it's not because experience led to worse outcomes. It's that the sicker patients selected those types of doctors. The way we have gotten around that in our work is to try to find situations where we have a natural experiment, where patients are as good as randomly assigned to different doctors. Maybe in an emergency department or in the hospital--where you don't pick your doctor: you get the doctor who happens to be there that day. And, what we find is that in the context of general medical problems--like pneumonia, heart failure, infection--that younger doctors tend to have better outcomes as measured by mortality. So, a doctor who is, let's say, five years out of residency, the mortality rate for their hospitalized patients will be lower than the doctor who's 10 years out, 15 years out, and then 20 years out. So, in that particular domain, I think there is something to be said about the recency of medical training and familiarity with what's out there. The important caveat, though, is that if you look at high-volume doctors, that age or experience profile seems to go away. So, if you have an older doctor, let's say, who's 20 to 30 years out of residency, but they see a lot of patients, they seem to do just fine compared to a younger doctor. So, if you're selecting a doctor, maybe don't do it based on age, but think about, 'All right, what is the experience that they have with patients in general? Or maybe even people like me?' |
| 1:01:59 | Russ Roberts: So, now turn to the Hawthorne effect and how it applies to medicine. Anupam Bapu Jena: So, the Hawthorne effect is this idea that when people are being observed--when they're being monitored--that they change their behavior. So, if you're doing a survey of people's work ethic and you observe that they're working really hard, do you have an accurate representation of their work ethic when they're literally being monitored? Maybe not, right? It's not rocket science, but it is a pervasive problem across surveys in general. So, we had this paper a few years ago--and again, I have to give my wife credit for this offhand observation is that she was--and I can't remember if we actually wrote this in the book, but the story was, she's also a doctor. And one morning, it was a Monday morning, she gets an email from her hospital saying that this organization called the Joint Commission is visiting the hospital that week. And, this is an organization that visits hospitals every few years. It's a regulatory agency. They measure the safety and quality of hospital care. And it's an intense inspection period. And she looks at me, saying, 'There's this whole list of things that they're being asked to do, and we're going to be running around chickens with our heads cut off.' And, I said, 'Oh, interesting.' Right? I know these visits are very disruptive, and it might distract you from the way that you normally practice medicine, so maybe outcomes would get worse. And so, we looked at the data. We have several thousand visits to many hospitals in the United States, and we looked at what happens on the exact dates of those visits. And, the main finding is that mortality of people who, by chance, are admitted to the hospital during that week, their mortality falls compared to people who are admitted to the hospital in the surrounding weeks. So, there's something that happens during the dates of those visits that leads to lower mortality. And, I say that in the causal way because we basically go through a lot of rigamarole to show that the patients are similar. It's not like a person says, 'Oh, I'm having chest pain. Is the Joint Commission coming this week to the hospital that my ambulance might take me to?' No. It's totally random. And you can show that it's random in the data. So, it could be one of two things. One is that the people in the hospital are doing the things that the regulator wants them to do, and those things improve quality. That could be what's going on. And, we looked at that in various ways. And, we don't find any evidence that that's happening. Though that's not to say that it doesn't happen. It might be the case. But, the other is this idea that you just alluded to a few moments ago, which is the Hawthorne effect. If you know that you're being observed in the hospital, you're just going to behave differently. And it speaks to the idea of: All right, when you're doing a high-effort task, focus matters quite a bit. Distraction might be quite possible. So, I might be getting distracted a little bit less because I'm intensely focused on the task, because I know that someone is literally watching me do it. Now, the solution is not to then have the Joint Commission there every day of the year in every hospital, because that would be very stressful for a lot of people. But the question is, 'All right. Well, what can we learn from this? Are there things that we could do that maintain the focus that is required in certain periods without so much of the additional cost?' And, that's where we leave it in the chapter. And, there are places where we do this, like in the operating room, for certain things, we do, like, a timeout to make sure that we're doing the right patient--procedure on the right patient--right body part, just to make sure that there's focus in that very important decision. But, there's other decisions where you can certainly think that focus and attention for a short period of time might be really important. |
| 1:05:47 | Russ Roberts: I love this because, as an economist, I'll say things like 'Incentives matter.' And, people will say, 'Oh, yeah. But, oh, come on, you're telling me doctors aren't going to work as hard when the people aren't around. Don't you think they don't care about human beings?' Well, of course, they do; but they're human beings themselves and they're flawed, and it's hard to pay attention 24/7. And, it's painful. And it's easy to find reasons to not pay 99.99% attention. And, I think about it mostly in terms of teaching. K-12 [Kindergarten through 12th grade]--K-12 teaching is incredibly intense. And, to do it well, it requires--especially in high school--it requires, if you're serious about it, a very high-level of effort. It's not just whether you're a good teacher or whether you're smart. It's about paying attention and focus and execution, and devotion. And, it's that last thing: Devotion is painful. Devotion means you can't check your email. Devotion means you can't watch some of the sports. You can't shop on the Internet during your downtime. It means focus. And devotion. And that's hard. And, it's costly to an individual. So, you know, I think about it as a management problem for a hospital: You know, hospitals I think over the last--you tell me if I'm wrong--but my sense is that over the last 25 years, there's a lot of procedures that have been put in place. Rules of thumb to follow that are a nuisance. And don't matter every single time. But, probably better to do than not do. Obvious things are wash your hands. There's a whole bunch of--call out the name of the patient before you cut them open, find out--you make sure more than one person looked at the prescription. You talk about a number of them in the book. But the challenge is the head of a hospital or the head of any organization to get devotion to come forth from your staff once a year for a week? Yeah, you can do that when the Joint Commission is in town. The challenge is: How do you get something close to that the other 51 weeks? And how do you do that without driving them insane? Part of it is who you hire, obviously, and how you treat them. But, it is--to me, it's just a very deep and interesting question. Like many of the results in your book, it's not so much that this means we're now going to do this: We're going to have the commission here 52 weeks a year. But rather you illuminate important things that are relevant in the provision of healthcare that are not obvious. Anupam Bapu Jena: That's exactly right. Russ Roberts: Devotion is a good example of that, right? Anupam Bapu Jena: Yeah, I think-- Russ Roberts: 'Oh, come on, every doctor is devoted.' Well, this suggests not the same amount every day of the year. Anupam Bapu Jena: Yes, I think that's right. And I think one area where we can use data to try to help this is just figure out on where are these lapses occurring and for whom are they most impactful, right? Because if you take a really sick person, any deviation--small deviation--in optimal care could have a big outcome, negative outcome for them. So, that is true. But, it's also the case that for the really sick patients, everybody knows they're really sick, and they're really careful in those cases. So, it might be that where the blind spot is, so to speak, is people who are not really sick, but they're sick enough where if you were to make a somewhat glaring error--which you might make in a lapse of judgment because you didn't have your radar on at that time--that they might be affected. And, my intuition would be that that's where we'd want to look. But that, again, becomes a data question, empirical question, is trying to figure out where are the places, who are the types of patients where these lapses are happening? Because maybe that's where we--again, devotion is hard to build generally, but maybe there are places where we say, 'All right. These are the types of patients, or these are the types of settings where we want to be a little bit more focused.' Russ Roberts: And, of course, it varies by doctor. I believe, and I think it's true, that principals--good headmasters--of K-12 [kindergarten through 12th grade] schools know who the best teachers are. They can't always be measured. You can't always find out by some arbitrary measure of, say, test score improvement or other things, but they usually know who they are. And, I suspect that the people who run hospitals or chair departments know who the best doctors are, and they know that there's some that are not as devoted, not as focused, not as careful, whatever it is. And, I would think that's where the biggest bang for the buck is. I suspect there's both cultural and legal reasons that those folks keep doing their thing when they maybe should be doing something else. Anupam Bapu Jena: Yeah. I may offer--one other thought is that there's a very interesting insight there, which is: when we measure quality right now, we measure it in a data-driven way where we say, 'All right, let's look at the outcomes of these doctors based on the patients they treat.' But, there's more of a revealed preference way to measure quality, which is to say, 'Let's look at doctors who have medical problems--as all medical professionals will--and let's look at who they go to for care. Repeatedly. And: 'Are those doctors the ones that the doctors are themselves choosing for their own care or their family members' care? What do those doctors look like? Do they line up well according to standard quality measures? Yes, or no.' If they do, maybe that's interesting. If they don't, then what are we missing? Are we not measuring quality correctly? Are we not accounting for certain factors? Or are these doctors who are seeking medical care out to lunch? Any number of things could be true, but I think it would be an interesting investigation to figure out what it is that doctors do when they need medical care for themselves. Russ Roberts: My guest today has been Anupam Bapu Jena. His book is Random Acts of Medicine: The Hidden Forces that Sway Doctors, Impact Patients, and Shape Our Health, written with Christopher Worsham. Bapu, thanks for being part of EconTalk. Anupam Bapu Jena: Thank you. |
READER COMMENTS
Shalom Freedman
Sep 13 2023 at 4:24am
The example of the ninety-year old women whose stent procedure is a mistake sounds familiar. In some decisions for the elderly the risk-benefit decision is not only the doctors but also the patients. However, many decisions involve unanticipated risks, and this is especially true for elderly patients. In some emergency cases the patient is not capable of responding but in many the patient’s consent is required. It would be wonderful if there were a formula for making the right decision, but unfortunately in many cases whatever is decided by either doctors or patient or both for the elderly there is likelihood of some kind of negative side-effect.
Erik
Sep 13 2023 at 9:09am
The episode download link appears to be broken. Can it be fixed?
[It’s fixed now. Sorry about that.–Econlib Ed.]
John Notgrass
Sep 13 2023 at 5:05pm
A crossover of two wonderful podcast hosts. Yay!
Sabine Schnittger
Sep 14 2023 at 2:49pm
Anupam Bapu Jena made the point that clinical studies are done on younger, otherwise healthy(ier) people, and that the results cannot therefore simply be extrapolated to older people with multiple co-morbidities. I guess that must be right, but it should also be acknowledged the choice of sample for these types of studies is also intended to minimise statistical noise.
Lori Thein Brody
Sep 14 2023 at 9:10pm
I loved this episode as it checked many boxes for me. I have been a medical provider for over 35 years and an educator of students and new clinicians. I particularly enjoyed the thought experiment and was not surprised by my preference for #3. It is consistent with a normal distribution, where 67%-70% will fall within one SD of the mean…. so the guidelines are generally written for those patients who are pretty much in the middle of the pack. Doctor #3 seems to recognize the ‘zebras’ or those who might fall outside the guidelines or are more than one SD outside the average. Most guidelines I am familiar with are the outcome of accumulated evidence, much of which is washed and sanitized via inclusion and exclusion criteria, leaving a certain percentage of patients outside the 70% range. Here is where the other two legs of EBP come into play. Provider experience and patient preference. We can’t forget the patient in this conversation and their willingness and ability to abide by the ‘guidelines’ some institution has declared best practice; additionally, the provider experience who can see, feel, and hear that patient has insight that no algorithm can account for. Let’s not forget that EBP is a three-legged stool and remember the limitations of the evidence, which Russ has discussed frequently on this podcast.
Jay Stannard
Sep 24 2023 at 6:48pm
This conversation is interesting but it neglects two of the major drivers that prevent a doctor’s “intuition” from being used.
First, if a certain procedure is not standard operating procedure and a doctor prescribes it, the insurance will refuse to cover it.
Second, if a doctor would prescribe something that made biological sense, but was not part of the standard procedure he would leave himself open to a lawsuit. For instance there is a drug Diamox that is used for elevation changes because it increases respiration. There is experimental evidence that it may also be useful for Sleep Apnea, but it’s not indicated for that. Even though the drug has a low risk profile a patient could never get this drug for an Apnea indication.
This has a chilling effect on independent thought in medicine and essentially turns docs into CPT Code clerks. It’s medicine by flow chart and most doctors have convinced themselves that it’s the best this way.
Doug Iliff, MD
Sep 25 2023 at 9:23pm
This episode was interesting enough for me to buy the book. I’m not finished, but the chapter on cardiologists, conventions, and interventions piqued my interest. Dr. Jena found that less procedures (e.g., stenting) produced better outcomes. Was he aware of the massive study replicating results a decade earlier which found that high intensity statin drugs produced better outcomes than procedures in many emergency settings? The book was published in 2023, but may have been written and edited beforehand. I’d be interest to know if he considered this finding to be germane to the discussion.
Comments are closed.