John Ioannidis on Statistical Significance, Economics, and Replication
John Ioannidis of Stanford University talks with EconTalk host Russ Roberts about his research on the reliability of published research findings. They discuss Ioannidis's recent study on bias in economics research, meta-analysis, the challenge of small sample analysis, and the...
Taubes on Why We Get Fat
Gary Taubes, author of Why We Get Fat, talks with EconTalk host Russ Roberts about why we get fat and the nature of evidence in a complex system. The current mainstream view is that we get fat because we eat...
Explore audio highlights, further reading that will help you delve deeper into this week’s episode, and vigorous conversations in the form of our comments section below.


Nov 13 2018 at 8:52am

Russ makes frequent reference to Gary Taubes in this episode but I wish he would have directed listener attention to Stephan Guyenet’s work which seems much more consistent with the metabolic ward evidence that Belluz discusses. His perspective, detailed in The Hungry Brain, posits that the manipulation of the body’s food reward system by highly palatable, calorically dense foods is the relevant dimension for obesity, as opposed to macronutrient composition. It seems to resolve the confusion Russ and Belluz reference over how both a low carb approach and a low fat approach can work for different people. The success comes from restricting the availability of highly palatable foods which typically combine fat and carbs to trigger our food reward pathways.

Nov 13 2018 at 4:19pm

There’s theory and then there’s empirical data. Have you tried a no carb diet? I have _never_ seen a case of someone not losing fat (note, not weight, fat specifically) when ingesting no carbs with no macronutrient restriction otherwise.

Jim Kennedy
Nov 13 2018 at 5:35pm

I agree that there is huge variability in individuals and responses to diet. I agree that there is SOME effect of caloric overload and restriction.   Certainly, if you only eat 500 calories a day (NOT recommended!) you will lose weight. On the flip side if you eat 8,000 calories a day (NOT recommended!) you will gain weight almost regardless of what macro nutrient you eat. (macro nutrient-> carbs, protein, fat)

That said with people who with metabolic issues (such as type 2 diabetics) often do very well on a ketogenic diet. Actual studies (about 60%+ reversed type 2, others got better which is a huge sucess)  Part of the keytogenic diet is the fat intake makes you feel full. You can still eat too many calories, but it may be easier to NOT over eat. (vs low fat high carb; which won’t work for type 2 diabetics)

Iliff MD
Nov 13 2018 at 8:53pm

Around minute 27 Julia observes that we have made this subject too complicated, and she is certainly right.  I think she is struggling to remember the American Heart Association’s “Simple 7”, to wit: manage blood pressure, control cholesterol, stay active, don’t smoke, reduce blood sugar, eat better, and lose weight.  She adds maintaining a social network, and that pretty much sums up what we know about a healthy lifestyle.

Russ and Julia have previously agreed that there is no “silver bullet,” but when she adds that “exercise is the closest thing we have to a magic pill,” she is right on target.  As a family physician following the literature for more than four decades, the beneficial effect of regular exercise at all ages has been substantiated by an enormous body of research, with no “medical reversals” that I am aware of.  And much of it is laboratory or experimental, not just observational, research.

Here’s a quick summary of what we’ve learned.  The most important number reflecting all-cause mortality is your metabolic equivalent of task (MET), which reflects the amount of oxygen you can use per kilogram of body weight per minute.

On a plot of relative risk vs. MET, your risk approaches the asymptote of minimal risk at 10 METs for males, and 9 METs for females.  Beyond that, you may be training for competition, but you’re not reducing your risk of death or disability.

Determining your MET is done precisely in a laboratory on a treadmill, while wearing a mask hooked to instruments measuring oxygen consumption.  From a practical standpoint, you can get a good approximation by walking a measured mile on a track.  For males, 14 minutes is about 10 METs; for females, 15 minutes is about 9 METS.

You get to that point of fitness by regular aerobic exercise— 150 minutes per week of moderate exercise like brisk walking, or 75 minutes of strenuous exercise.  For a delightful summary of research, look up “23 1/2 minutes” on YouTube.

Personal experience from a 69 year old long past his Boston marathon days, but still able to run a mile in 7 minutes: don’t join a gym.  Your sidewalk is all you need outdoors, but given the vagaries of weather in many locales, you need equipment indoors in front of a HD TV.  The most time-effective devices exercise both upper and lower body— rowers, ellipticals with arms, bicycles with arms, and ski exercisers.  Then you just have to resolve to get off your ass every day and do something, in which case you might succeed four or five times a week.

Then walk a timed mile every so often to check your progress.

And thanks, Russ, for getting me off the hook with my wife about my alcohol consumption.




Earl Rodd
Nov 13 2018 at 9:35pm

It was refreshing and edifying to hear a fairly careful discussion of metabolism from someone who is not selling something.

Two additional thoughts:

There is another effect of eating fewer carbs which does not require changes in metabolism. As noted in the interview, different people will find the effect more or less important. The effect is that carbs tend to produce a desire for more carbs whereas fat makes one full. Thus for some of us, eating few carbs effectively leads to fewer calories even though we eat more fat. I’m not talking about being in ketosis, just significantly restricting carbs (e.g. vegetables rather than pasta with sauce).
While the podcast was not about weight, the discussion touched the topic. One aspect of modern American weight management which is seldom mentioned is how much of the statistical overweight population is due to psychiatric medications. Many psych meds have a strong side effect of weight gain. According to NAMI literature, the average weight gain on the most highly prescribed anti-psychotic, Zyprexa, is 35 pounds. Incredibly, when patients are prescribed these drugs, there is no protocol to refer them to a nutritionist to work to avoid the weight gain. Reversing it is nearly impossible. The mechanism is believed to be that the drug suppresses the feeling of being full so people eat more.

Nov 14 2018 at 12:08pm

That is great insight. I was unaware of it.  (I am not a Dr.) Thanks, Jim

Todd Kreider
Nov 15 2018 at 10:19am

I thought this was a good discussion, and I’d like to hear Russ interview a scientist who is familiar with the first wave of health pills of which two are related to metabolism, NR and its cousin NMN, both vitamin B3 derivatives and on the market now. NR has been shown to boost NAD+ levels in healthy adults by 40% at 250 mg a day ($1.00) and 55% at 500 mg a day ($2.00) and with the latter dose a group of health, non-obese 60 to 79 year olds walked 8% faster and performed 8% better on a balance test after eight weeks. Many more human trials are being conducted or are awaiting publication including one testing NR for improving heart failure, another that is testing NR on people with mild dementia and others for kidney failure and general health.

Rapamycin was on the front page of the NY Times in the summer of 2016 where Matt Kaeberlein was interviewed about the drug that humans have taken for 20 years and metformin, taken for decades, also shows health benefits in people.  These four pills could make a notable impact on health care costs in the 2020s.  Anti-aging researcher Brian Kennedy, who has given a couple of TED talks, may be a good person to interview since he doesn’t seem to be betting on a specific drug or supplement.


Nov 15 2018 at 3:34pm

As an epidemiologist there was a lot to like about this episode and unfortunately some bad.

“Correlation is not causation” is not a sufficient criticism of epidemiology.  All causation is correlated.  Throughout the start of the episode the guest referenced articles and maintained they were preliminary and that to resolve them randomized controlled trials would need to be done.  Bradford Hill addressed this idea all the way back in 1965 when epidemiologist theorists were trying to rationalize the fact that without a single randomized controlled trial they had proven one of etiological epidemiologies most important contributions to the world: smoking is bad for you.

Randomized controlled trials are indeed useful, but they typically cannot be used to verify a rapidly moving disease through a population has a certain cause, or that a new factor is causing an increase in population diseases.  The sheer number of epidemiological questions that arise that are not amenable to randomized controlled trials makes randomized trials an impossible expectation for human knowledge to move forward.  Rare or toxic exposures will never be used in randomized controlled trials – usually only post-hoc statistical and epidemiological analysis can determine their harm.  

Further, a meta-analysis of hundreds of studies is not hypothesis generating – it is designed to answer a question.  Now good and bad science (and scientists) are done to answer questions, but the method employed is designed to answer the question not simply generate more hypothesis.  Organizations such as Cochrane conduct reviews and meta-analysis to use the data to answer questions about causality and give health direction.

Some other criticisms such as comparison groups being different are solvable problems.  A randomized controlled trial accounts for population variation in the up-front setting, but theoretically all the tools exist to account for the variation after the study via regression, stratification, etc.  Even the dreaded third thing is only a problem if it is correlated with the other 2 factors AND its distribution differs between the two populations being studied.  Of course, because of the increased likelihood of randomness balancing these out we favor randomized studies when they can be done, but when they cannot we must evaluate the information with the tools we have just like most social sciences.

Many other criticisms were made that I think are legitimate.  I don’t believe anything about nutrition that comes from one study and epidemiologists are going on 3-4 decades of lowering the respect the world has for our field with the eagerness to get the news to recognize some new silly health scare.  We have learned little, if anything, about specific macronutrients or micronutrients and their contribution to health despite millions of dollars and thousands of publications.   Almost all the health data discovered by epidemiologists could be summed up as eat a moderate diet (even the rich in fruits and vegetables is a little dubious).  To illustrate how silly most published ecological risk factors are, I give a lecture to doctors where I trace the origins of a bizarre risk for cancer reported in a common textbook and how the risk factor was first humbly published as merely exploratory and never replicated.   But 30 years later is still appearing in textbooks as something people should know.   The Womans Health Initiative certainly blew a justified hole in the confidence of epidemiologists and an important reminder that for population interventions we must be willing to take the step of a randomized controlled study if possible.  

The discussion also addressed calories and exercise.  I think the tendency to over value the calories lost from exercise is s solvable problem because it is simply ignorance about the number of calories lost with exercise.  A brisk walk that burns 50 calories cannot justify a piece of cheesecake (500 calories) but I think as more people track calories and more devices that give approximations of calories burned this will be less of an issue.  Self control is always the real problem.  I remember in grad school the professors were excited about the public health intervention of moving the parking lot further away, making people walk up the stairs instead of elevator.  Interventions that would burn trivial calories that could be overcome with a tiny piece of candy or a few extra fries in order to cost thousands of people 15-30 minutes of time per day – a huge net impact on their productivity.  Sometimes the grandiose machinations of epidemiologist are pretty ignorant of the costs that would be imposed.  

The final part about metabolism was great.  We are redoing studies we did in the 1960s because we basically showed then that macronutrient composition has no impact on weight gain or loss (100% carb vs 100% fat vs 100% protein). What we may be adding now is that some foods satiate some people better.  Unfortunately I think the best look at the data has found that the biggest problem is we are swimming in calories and we must be disciplined to get out of the pool or find ways with our personal body type to satiate ourselves.  I think eventually a common and important modern skill will simply be calories tracking in and out and then controlling what you eat.  We might get some drugs that provide satiation in the next 5-20 years so this problem still might be solved pharmacologically.  

Nov 17 2018 at 12:30pm

I think both of you missed a softball question.  A ketogenic diet is one that will be, for most humans, consumption of less than 50 grams/day of carbohydrate.

And you both misunderstood the Paleo Diet© (as most do).  It is actually a pretty “balanced” diet, quite heavily weighted to vegetables, not meat.





required, not displayed
required, not displayed

This site uses Akismet to reduce spam. Learn how your comment data is processed.


This week's guest:

This week's focus:

Additional ideas and people mentioned in this podcast episode:

A few more readings and background resources:

A few more EconTalk podcast episodes:

TimePodcast Episode Highlights

Intro. [Recording date: October 16, 2018.]

Russ Roberts: My guest is Julia Belluz.... Our topic for today is epidemiology, and if we get to it I hope we can talk also about metabolism and diet--two concepts and issues you've written a lot about, especially recently. According to the U.S. Department of Veteran Affairs, epidemiology is "the study of health in populations to understand the causes and patterns of health and illness." We're going to be looking at a number of articles you've written on epidemiology that challenges of actually figuring out the truth about these issues, which is quite hard--as listeners know from our past conversations. I want to start with an article about a recent study in Lancet that argued that the safest level of drinking--and this is talking about alcohol--is none. Zero. Do you agree with that conclusion?

Julia Belluz: I don't. And, interestingly, this study didn't even show that, so it's kind of an easy one to pick apart. Yes, so these researchers were looking at many, many studies--I think it was something like 700 or 800 research papers on health effects of alcohol. And they found--they had like a really nice graph where they collated, I guess, the findings of all these studies, to show you where your risk of death starts to increase based on the number of drinks you are consuming each day. And, that risk looks like it only goes up after one drink a day. Or even one and a half drinks a day. And yet they came to this really striking conclusion which was that the only safe level of alcohol consumption is no alcohol at all. So, as you can imagine, that upset a lot of people who are reading about that. And others were upset because we know that there are really strong and terrible health effects from drinking too much alcohol. And they kind of missed the opportunity to have a more nuanced message about that and went for this 0-drinks--yeah, went for the guttural[?], like no drinks at all.

Russ Roberts: So, I looked at that chart, which you reproduce in your article--and we should mention for our listeners that Lancet is one of the premier medical journals in the world. Top three, probably. That chart--it's flat at zero in terms of extra risk between 0 and 1. And then, at 1, it doesn't like jump up. It just slowly starts to climb. Which of course means--who knows what. Because it's referring to all kinds of different kinds of risk that are probably very complicated: Could depend on the kind of alcohol you consume. I assume the number of drinks is some constant, some metric like 'an ounce,' or something, because--you know, I have a friend whose joke is that he only had one beer. I say, 'How big was it?' 'Well, it was 48 ounces.' So, you know: What's the--I assume it meant an ounce of alcohol, some standardized measure.

Julia Belluz: Yes. A drink. Yeah, that's right. I think it's a little more than an ounce.

Russ Roberts: Yeah. It's not much of a drink, I guess. But the idea is that it does move up slowly. It doesn't jump dramatically at any point. Right?

Julia Belluz: Absolutely. Yep. That's right. And I think one thing we are talking about, epidemiology, it's really important to know about these types of observational studies, is that they are just looking at phenomena in a population. So, you are looking at, you know: How do people who already drink alcohol and who have been drinking alcohol look different from people who don't drink at all? Or, how do heavy drinkers look different from light drinkers? And, what we know is that these groups of people are so different, right? So, people who drink no alcohol at all might be less likely to smoke, or might have other really healthy lifestyle behaviors, or might live in different communities from people who are heavier drinkers. So, yeah--you have [?] you know all about confounding factors and how these other attributes make it really hard to tease out what the actual effect of the drinking was, in and of itself. And so, these studies are only supposed to be hypothesis-generating. So, they're supposed to kind of be the start of a conversation about what--the start of a line of research into what might be, like, what does healthy, moderate drinking actually look like? And you'd want to run experimental studies to really tease out cause and effect there. But, of course, yeah: When it comes to anything that has to do with what we drink and what we eat, it's just--it takes a long time for these nutrition-related deficiencies and diseases to show up. And, there's all kinds of challenges with them. But, I guess the main point is that in the media, we often talk about these observational studies like they do have causal conclusions. And in this case, even the researchers did in their own conclusion. But this is an example where, you know, they can talk about associations between things but not that one thing causes the other. If that makes sense.

Russ Roberts: Right. Yeah, no. Exactly. I think that's the most important point for people to remember. We talk about it a lot here, but I think it's easy to forget that what we're really interested in is: What would happen if you took one more drink? And: What would happen to the average? The correlation between, in a population of people who drink a little bit more than everyone else does not necessarily, and may be grossly inaccurate for telling you if you had one more drink. Or if everyone chose to have one more drink. It's just--it's misleading.

Julia Belluz: Yeah. And it's probably important to be clear: Like, that, again, I think no one quibbles about the fact that heavy drinking is associated with all kinds of terrible health outcomes--disease, increased risk of accidents. Yeah, and normally death. So, no one argues about that. But there is a lot of debate about what does healthy, moderate drinking look like? Like, how much is safe? And, is a little bit of alcohol protective in some way? And that's kind of this hugely heated area of research. And one for which--yes, getting these really nuanced--getting the kind of studies we'd need to equivocally[?] answer that is almost impossible.

Russ Roberts: Right. You'd need a really, some kind of randomized control trial where you also had a lot of control over consumption, as opposed to just asking people in a survey after the fact. Which is what this kind of observational study typically does.


Russ Roberts: I just want to reference an early EconTalk guest, a friend of mine down at Cox who when at a meeting in a discussion of causation someone had the courage to suggest there might be another factor involved, besides in this case, say, alcohol--and someone else at the meeting says, 'You mean the dreaded third thing?' And in the case of these kind of studies, it's third, fourth, fifth, sixth, seventh. There's many, many factors, as you point out, that might be the underlying causal variable. And which would mislead us in drawing an accurate conclusion about the impact of a little bit more alcohol in our consumption.

Julia Belluz: Totally. And then it's also, like, what kind of alcohol are we talking about? So, there was another study that came out, also in The Lancet this year, and they were looking at the health effects of alcohol; and then in the supplementary materials they broke down--and it was also a meta-analysis so they were basing their conclusions on many, many studies. And in the supplemental materials they looked at the health effects by alcohol type. And you could very quickly see that beer drinkers had much worse health outcomes than wine drinkers. But they also collated the socioeconomic characteristics of the beer drinkers versus wine drinkers--

Russ Roberts: Yeah. Not the same.

Julia Belluz: Yeah. So, unsurprisingly they look completely different. Different levels of education, different types of work. Yeah.

Russ Roberts: Weight.

Julia Belluz: Exactly. And poverty.

Russ Roberts: Age. Everything. Gender.

Julia Belluz: Yeah. So, basically everything was different. Yeah. So it's a really--I really feel for nutrition researchers; and there are so many people who are doing great work. But I think, yeah: Sometimes they themselves go too far in the conclusions of these studies, and then we in the media often completely just misrepresent what the research actually shows.


Russ Roberts: But not you, Julia, because I've read a lot of your work, and you're actually--I'm half-teasing you; but I think it's true that you are more careful than the average--certainly than the average person in the media, who often has the natural temptation to get on the front page and to dramatize a finding. And of course, that's what these scholars did who did the study. I'm curious if you have any thoughts on why they said 'None,' given that their results suggested 'One' is where the danger kicks in--to the extent it kicks in. And it's still, again, extremely low risk, at low levels.

Julia Belluz: Right. I think--yeah. I don't know what was going through their minds, but I can guess that they might have been trying to draw attention to the very-well-known health risks associated with drinking. And, this was, if I'm remembering correctly it was a Global Burden of Disease study. So, this is where they are very carefully trying to get a measure on how--like, what kind of health impact is alcohol having on populations around the world. And, so, you could imagine that they were trying to draw attention to the very serious and awful health risks of alcohol and of drinking too much. And so maybe, yeah, they were just tempted to take it a little bit further and kind of exaggerate the finding, to draw attention to those risks.

Russ Roberts: Yeah. And you could argue it was well-intentioned: they were just better safe than sorry. But that's a bad sign. It might be good parenting--

Julia Belluz: Yeah--

Russ Roberts: Maybe. Maybe. But it's certainly not good science. Which is a shame.

Julia Belluz: And I think, yeah, some people felt like they were moralizing as well.

Russ Roberts: Yeah.

Julia Belluz: And, yeah.

Russ Roberts: Well, I always misquote--I don't get this quote exactly right, but it's close. H. L. Mencken said, 'Puritanism is the haunting fear that someone, somewhere is having a good time.' So, there is an aspect of that, I think in these--sometimes these kinds of preaching aspects of a study.

Julia Belluz: Right. Yeah. Absolutely. But I think one other interesting nuance to this alcohol story is the New York Times has done this incredible reporting on the influence the industry has had on shaping alcohol research, even within the National Institutes of Health. And, this is all about, you know, the industry wanting to say, moderate, a couple of glasses of wine can actually make you live longer and protect your heart, or whatever it is. And funding studies. And, you know, having influence over--as we know, the way you design a study, the questions you choose to ask, the people you include, how you interpret those findings shape what the conclusions end up being. And the Times investigations have shown that the alcohol industry has had a hand in that.

Russ Roberts: Not surprised.

Julia Belluz: Not surprised, at all, yeah. I guess when you look at how much--

Russ Roberts: A lot of money at stake.

Julia Belluz: Totally. And how much industry shapes many different areas of research. But, so, yeah: It is this like very contentious question and one of ongoing debate. But one, I'm afraid we're not going to have any satisfying answers to any time soon.

Russ Roberts: Well, this study could have been funded by the cannabis--the growing-cannabis association. But, looking for, to make a competitor less attractive. More expensive.

Julia Belluz: Right. Big Cannabis. Yeah.

Russ Roberts: Yeah. Big Cannabis. It's comin'. It's comin'.

Julia Belluz: Right.


Russ Roberts: Do we learn anything from this study? By the way, I just want to mention--it's important. There is a temptation--we've talked about this before, I think, with John Ioannidis, that when you do a meta-analysis that, 'Oh, it had 800 studies! It must be really reliable.' But, of course, if the 800 studies all used the same flawed methodology, you don't get any more light shed than if you only had one bad study. They are all kind of the same. Do we learn anything from this? Did you learn anything from this meta-analysis that was helpful?

Julia Belluz: I think one place it was helpful was in estimating that Global Burden of Disease aspect. So, I actually, I called John Ioannidis about this, because it, when the paper came out, he had just come out with this fantastic critique of nutrition science, and [?] find out what he says, in this paper he talks about how like the absurdity of all the health claims that are made based on these really shaky studies, and how like eating 12 hazelnuts a day can prolong your life by 12 years, according to some papers. And, 1 egg reduces life expectancy by 6 years. And, he pulls out all these really spurious findings that are based on observational studies. But, anyway, so those are side notes. So, I was talking with him about the value of the paper, and he was saying--and I agree with him--that it wasn't estimating alcohol, the disease burden of alcohol. You know, that it was a reminder about how it increases liver disease, and the risk of high blood pressure, injury, memory, many other health problems. So, I think, yeah, that was--yeah, kind of a big value to this, to have those Global Burden of Disease studies. But, yeah--

Russ Roberts: You mean to quantify, in some way, what the full impact might be--

Julia Belluz: Yes--

Russ Roberts: at the current levels of drinking, say.

Julia Belluz: Right. Yeah. And they were looking at 195--they were looking at data from 195 countries. And, yeah. So that was the main purpose of this study. So, this study was really valuable in that it estimated the burden of alcohol-related disease in 195 countries. But then, that one striking conclusion that zero drinking is the only safe level is what everyone grabbed onto and what everyone reported about. Instead of what the actual purpose of the study was, which was estimate alcohol's disease burdens.

Russ Roberts: Well, I think when John Ioannidis was on this program, I think he said that we should never take seriously any observational study about the impact of one food or drink on human health.

Julia Belluz: Right.

Russ Roberts: And I assume, when you said that, I assume he--I'm sure he believes that excessive drinking is a bad idea. As is, say, eating 10 or 12 pounds of broccoli every day. But, the sort of normal range of human experience is not reliably measured by these observational studies. And yet they continue to be done constantly.

Julia Belluz: Right. Yeah. And there's so many questions, like when you look at studies, for example, on the impact of eating meat, there's a lot of question about the preparation that you're using. So, is there something about the way we cook the meat that increases its toxicity and the way it's processed in the body. Or, with any type of food--if you are adding one thing to your diet you are subtracting another, so even if you randomize people to fish- and meat-eating, that means that the people who are eating the fish are eating less meat, and the people who are eating the meat are eating less fish. So, is it the effect of the new food you are introducing or taking out the other food that's having that impact; and how is eating more fish changing the rest of your diet? And obviously, we're eating foods in all these different kinds of combinations and they are prepared in many different ways. And when we add more of one thing, we are taking away another thing, so teasing out, as John suggests, the effect on one health outcome of one particular food is almost impossible, through observational studies. And even otherwise: if you want to run a big randomized control trial on, say, like blueberries--really--

Russ Roberts: It's fabulous. Fabulous--

Julia Belluz: Yeah; are you going to randomize people from birth to eating more blueberries? And then if they are eating more blueberries, what aren't they eating, otherwise. So it goes on and on, and it's a really tricky area.

Russ Roberts: I just want to make clear that, of course, here on EconTalk we do not give health advice. So, anything you hear today, you must take with at least a grain of salt. Some people think salt is bad for you. I don't particularly, but that's just my personal opinion, and I would not encourage you to rely on it. And the same goes for drinking, say, or anything else we're discussing. So, I do want to make that disclaimer. Consult your local nutritionist, doctor, or statistician before acting on anything you've learned in today's episode.


Russ Roberts: But before we move on, I just want to mention my favorite alcohol-related study, which was about, maybe 12 or so years ago. It made the front page of the New York Times; it made the front page of many papers and magazines and websites which I think were existing at that point. And it showed, purportedly, that alcohol consumption by women, particularly I think wine, led to an increase in certain types of cancers. And, I looked at the study fairly carefully--I may have mentioned this before, but it's always--to me this is such a horrible study it's worth repeating. They threw out from their study non-drinkers. And the justification was--I'm going to tell you the justification in a second, but in the raw data, people who drank small amounts had better health and less cancer than the zero-drinkers. And that, of course, is an example potentially, not for sure but potentially, of what's called hormesis--the idea that the dose makes the poison, and many things that could kill you in large amounts are actually not just benign but beneficial in small amounts. Alcohol has been suggested as being one of those; we don't know for sure, as we've been talking about. But, there is some evidence that that might be true. So here they have this awkward conclusion: They want to tell you that drinking causes cancer, but it appears that at very low levels, drinking reduces your risk of cancer. So, they were out the people who didn't drink at all. So, the only people in their population were drinkers. And they found that among drinkers, higher levels of alcohol were, of course, very bad for you--that's not, again, very surprising. The question is what happens at low-ish levels: If you have one and a half or two drinks a day, does that raise your risk? By how much? How do you offset that against, perhaps, the reductions in stress, which are also beneficial to your health that alcohol might provide? Etc., etc. But, what I loved is their justification for why they eliminated the non-drinkers. And, what they said was: We have to eliminate the non-drinkers because some of them could be non-drinkers today, but in the past they could have been heavy drinkers, and that will contaminate the data. Of course, the problem with that is that, in the past, the people who were drinking 5 glasses of wine a day could have been teetotalers, drinking zero. So, once you've opened that Pandora's box, that past--data based on past memory is suspect--the whole study is suspect.

Julia Belluz: Right. Wow. I didn't see this. So, you said that was like a dozen years ago?

Russ Roberts: Yeah; I'll try to dig it up for you. It drove me crazy. They didn't control for cancer in the family. They had those data; I don't know why they didn't control--I don't know.

Julia Belluz: Do you think it's the--so this wasn't just a case of the media going, over-interpreting the results?

Russ Roberts: No. The researchers--

Julia Belluz: It was just really bad--

Russ Roberts: Yeah. Because the incentives for the media are not so healthy right now; the incentives for scholars are not healthy. You want to get tenure; you want to get attention; you want to get love from funders, from your university, from your hospital--your natural incentive is to exaggerate, dramatize. That's where we are.

Julia Belluz: Yeah. A little while back we did this chart showing, like, from the moment an idea in science is generated to the time it's published in the media--all the places it goes off the rails. And there's so many places.

Russ Roberts: Give some examples.

Julia Belluz: Yes. So, I guess, from the beginning, from when you're conceiving a study--so, when you're thinking about what population are you going to use, the really representative population, the one that really makes sense studying, and then how many people are you going to put in this study? Will it be enough to show a real result, or is it going to be too small? And, so yeah, from the study design; and then interpreting your data; and if you have a multisite study, making sure did everyone at these different sites follow the protocols for the study properly and in the same way to make sure we are all doing the same, making sure we all have the same intervention and what we're suggesting is what we're looking at here. And then from that, so then that's--running the study and then interpreting the results, so we have all these incentives and reasons, including our own biases and things we're not even aware of when we're looking at data; and that might shape how we interpret the results. And then you are publishing the study, and you might have co-authors who are tweaking things, or the journal might be tweaking things. And then a press release comes out about this study, and that might water down or shift the findings in some way. And then the media report on the study, and we might mis-interpret or over-interpret or misrepresent. So, there are all these places where research can go off the rails. And, I think--have you ever talked to Sheila Jasanoff?

Russ Roberts: No.

Julia Belluz: So, she pioneered this science technology study, this program at Harvard in the Kennedy School there, and she's really interested in this idea of science as a human process. And, you know, it's obviously, I think, the best way we have at getting at the truth, but it is done by humans; it is conceptionalized [sic] by humans; and these all these places where it can go off the rails. And we need to build in protections for that and make sure that we account for that.

Russ Roberts: Yeah. We've had Brian Nosek on talking about the replication crisis in psychology, and of course now beyond psychology: The natural incentives to hide results that aren't interesting, that don't come out well, means that the studies we do observe probably are not as reliable; they tend to come from the right-hand tail or the left-hand tail of the possible results. There's a lot of error. And that's just the reality.


Russ Roberts: You wrote an essay for Vox--I think it was in the last 5 years--but it's one of my favorite pieces of journalism, partly because of the visual representation that you give in there. You looked at--I think it was your chart; I don't know, maybe you created it, but maybe you got it from somewhere else, but it's basically--you took a whole bunch of pieces of our diet--caffeine, broccoli, meat, carbs, etc., and you showed which studies, how many studies showed that they increased the risk of cancer and which studies showed that the decreased the risk. And basically for every single item in the list--and there were probably 15 or 20--there were lots of studies that showed, 'Oh, my gosh: this causes cancer,' and then there were also lots of studies that showed that they reduced cancer. And I think that's a very sobering and thoughtful finding that should create skepticism among thoughtful readers of the media, in terms of what you think about one's own diet.

Julia Belluz: Totally. And I wish we could take credit for that, but that was also John Ioannidis. We did reproduce it in that story, but it was another John Ioannidis came up with this idea of, I think it's the one where he flips through the Boston cooking school cookbook and randomly selects foods and then looked at what's been published showing that that food causes or protects cancer; and then they had this amazing chart in that study. Really, really fun idea.

Russ Roberts: We'll put up a link to your version of it, which is extremely accessible, which is probably less accessible; but he's pretty accessible.


Russ Roberts: Before we move on--we're going to move on in a minute to metabolism and diet, which you've got some really interesting personal and I think scientific insights into, but I want to ask before we do that: What's the single biggest fallacy about health, or nutrition? If you could just clear this up--you think this is--everybody should know this, but they don't. Is there anything that jumps out at you from your reading of the nutrition space or the health space more generally?

Julia Belluz: I think one--this maybe isn't--I don't know if this is what you're thinking of, but I think we make things way too complicated, especially in this country--like, there are so many vested interests, and people who are trying to profit off making certain diet claims or exercise claims, and what we actually know and research to be like, what does it mean to live a healthy life? It's, like, don't smoke, don't drink too much; eat a diet that's pretty rich in fruits and vegetables; make sure you get an adequate amount of sleep; make sure you have a strong social network, and people that love you and that you can depend on. And, am I forgetting one? So, smoking--

Russ Roberts: That's pretty good--

Julia Belluz: drinking--

Russ Roberts: Stay active: be a little active. You don't have to be a marathoner.

Julia Belluz: Yeah. And I guess that's another one, actually, that--I think this idea that if you want to lose weight you have to join a gym and start to exercise like a fiend--

Russ Roberts: Well, that's a belief that's held by people who have never visited a gym. When I started going to a gym a couple of years ago and I saw the people there, I thought, 'I hope they're new like me.'

Julia Belluz: Right. Okay. There we go. But, what we know--so, exercise is the closest thing we have to a magic pill, I think. Like, it does so many incredible things for us. And it helps people maintain their body weight. But it's actually not that effective for weight loss. And that's because people--researchers have found that when people ramp up the amount of exercise they do, they tend to think, 'Now I can go home and have a big slice of pizza,' when really you don't actually burn off, unless you are doing hours and hours of exercise, you don't burn that much, as much as you think you do, as much as exercise machines say. And, it's really easy to undo that calorie deficit you've created with a single slice of pizza or a Frappuccino at Starbucks or whatever it is that people are eating these days. So, I think that makes it like over-complication and then relying on this one silver bullet for health, be it exercise or a certain food or a certain diet I think is where we're often going wrong.

Russ Roberts: Yeah, we have such a--you said, I think, it doesn't burn as many calories as we think it does. I would just put a slight amendment to that: I'd say it doesn't burn as many calories as we'd like to think it does.

Julia Belluz: Totally. Yeah.

Russ Roberts: So, the ability to self-deceive there, honestly, just thinking you've actually earned a lot when you just really want that Frappuccino, is a huge problem. As is the quest for the magic solution. My favorite of this example is the 7-minute workout, you know: I don't really have to exercise. I just work for 7 minutes. And there's an app on my phone I can get. We're always looking for the shortcut, the magic food that will solve our problems. And, like most things in life it's mainly hard work and sacrifice which leads to lower weight and health; and that's not as much fun as believing that this new book with this new diet is going to take care of me.

Julia Belluz: Totally. And that's sustained work. It's like something you kind of have to think about, even if--like, I am someone who did lose weight over--I was chubbier as a teenager, and slowly lost weight. And it's been a sustained effort, and something that I still think about every day. And I think that's true for a lot of people who lose weight and keep it off. It's not like suddenly you discover this magic solution and you never need to think about it again. It's something that lingers there. And I think on the flip side, it's like, if you have one day or even two days in a week where maybe you eat a little bit too much or, yeah, an extra slice of cake that you shouldn't have, or whatever it is, it's not the end of the world. Like, what really matters is how you are eating over many months in a year, and how does that look on balance, not, you know, like obsessing about every little micro-decision that you make about what you are putting in your body.


Russ Roberts: Yeah. Mark Twain said, 'It's easy to stop smoking. I've done it a score of times.' I feel that way about losing weight. It's easy to lose weight: I've done it--I'm really good at it. And I forget that I'm really good at putting it back on, too. And, they say that's not good for you either. I've always been attracted to the Kingsley Amos line, 'Inside every fat person is a fatter person trying to get out.' I sometimes think of myself that way. It's like, 'Let's just go to my natural weight.' You know--that's probably about 20 or 30 pounds above what I'm already way above right now. So, it does take some discipline. It takes long-term discipline. And I think the behavioral insight, which I've talked about before on the program, is, sometimes you need very black-and-white rules that are very different from, 'I just won't have too many potato chips.'

Julia Belluz: Right.

Russ Roberts: I can't keep that rule, unfortunately: 'I'm just going to only have a little bit of the ice cream.'

Julia Belluz: I guess, I should say, and I hope I'm not contradicting what I said earlier, but I do think that, after writing about this for so many years, like where I am really coming out is, our food environment here is a nightmare. And it does make it really hard for people to make, you know, the correct, maybe choices that are the best choice for themselves. So, I think--yeah. I guess, yeah, it is discipline; it is sustained effort. On the other side, we are living in a food environment where obesity does have a chance to express itself on a much more frequent level than it did a hundred years ago, say. And, you know, our genes--there's a researcher on this at NIH [National Institutes of Health] called Kevin Hall, and he always talks about how our genes haven't changed in the last, you know, half century; but what has changed is the Frappuccinos and the pizzas and the sheer ubiquity of these things. And the fact that you almost have to--I personally feel like I have to think, be quite judicious and think very deliberately about where I buy my food, and, yeah, exposing myself to these things, you have to make a really conscious effort not to. And so, it is on the individual. But I think it's also very much a massive environmental problem.

Russ Roberts: Well, we're really good at food. We're really good at making inexpensive, really yummy food. It's a triumph of human creativity and ingenuity; but it's not necessarily what we want in the long run. We very much want it in the short run. I have a thing with my wife--I think this is an example of what you're talking about--where she will sometimes say to me after a meal, 'Would you like a little ice cream for dessert?' And the answer is, 'Well, of course, I do! What kind of a question is that?' And, the second point being, 'Don't ask. Please don't ask.' Because it's so hard to say no. Even though I don't want it--I mean, even though I don't think I should eat it, it's very hard to say no to that. And part of it, by the way, I think is just human interaction. My wife offers it with a full heart. She doesn't offer it because, 'Oh, you want some ice cream?' Correctly, it's a festive moment. It's dessert. It's part of the joy of being human is to--

Julia Belluz: Human ritual--

Russ Roberts: eat good things. Yeah, and human ritual. And, I want to, we want to do this together: 'Let's have some ice cream. 'So. Anyway.

Julia Belluz: Totally. We could talk about Brian Wansink, too, who had, who just his old research program--I don't know if you followed this--

Russ Roberts: Oh, yeah. You can mention that. Go ahead.

Julia Belluz: Brian Wansink is an eating behavior researcher at Cornell. And he published these really interesting studies showing how these little nudges and environment cues cause us--they shape our food choices and they cause us to eat less or a little bit more in certain circumstances. And he had just really creative studies to demonstrate that. And, of course, when some other researchers looked into the data and the methods that he was using, they found a lot of problems. And now he's stepped down from his position; and he's being investigated, and all of that. But I think--yeah, he's not the only researcher to show that, yeah, that the things that were offered and the things that we have in our environment shape the choices we make. And, like, you know, if you have ice cream at home, if it's there it's on you, then, to say no to the ice cream. And the ice cream is designed, meant to be over-eaten--

Russ Roberts: It's the Alice-in-Wonderland dessert. 'Eat Me,' it's yelling.

Julia Belluz: Exactly. And it's so hard to say no. And I think, yeah, as much as Brian Wansink's research might be hugely problematic statistically and methodologically, but--

Russ Roberts: He's onto something--

Julia Belluz: Yeah. The ideas--he's not the only one who is talking about these subtle environmental cues, and like how important it is to think about them, even in our own homes and in our workplaces. And yeah.

Russ Roberts: So, we keep our ice cream with a neighbor. Who we don't like. And so, when my wife says, 'Do you want some ice cream?' and I say, 'Sure,' she's, 'Well, we've got to go over to the Johnsons, and they don't, they're not really happy to see us, so maybe we'll just skip it tonight?' No. I'm kidding. But it is important to not buy something that should not be bought. Because, they've got, like a--I remember when I was learning time management--which I couldn't implement, but it was fun to study it anyway. I couldn't keep any of the rules. But I loved learning about them. I even taught them for a while until I realized I couldn't keep them: I thought, 'This is probably inappropriate.' But someone classic insight in the Franklin Time management--the teacher would say, 'Do you wish you read more books?' And everyone's hand goes up. 'Yes, of course, I wish I read more books.' And the teacher would say, 'Well, why don't you?' And probably we all looked kind of down at our shoes. And his answer was, 'Books don't ring.' When the phone rings, it grabs your attention, and you answer it even though it's not--of course we've gotten better at not answering our phones in America--but, yeah, it's an old seminar. But, the things that grab our attention because they stimulate our appetites or our juices or whatever it is, they grab us. And sometimes the healthier things are just sitting there quietly, [?] ignored, 'What about me?'

Julia Belluz: Absolutely. Yeah. And people--I always think about this how--and maybe it's helpful to other people to think about this--like, the bag of chips and the ice cream: It was designed by incredible scientists at these big food companies to like--they tested it on different people to see which formulation would cause people to not be able to stop themselves, and to want to--what was the most pleasing to our brains and reward systems. It's a pretty tough thing, individuals to be up against.

Russ Roberts: 'You can do it, out there! Come on.' But on the other hand, we don't want to be saying you shouldn't have any fun in your life.

Julia Belluz: Right. Of course.


Russ Roberts: Let's shift gears. I want to talk about something you wrote about that really fascinated me. It's related to what we're talking about now, so it's an easy segue. Which is: You spent 23 hours in a metabolic chamber. So, explain what a metabolic chamber is, and what that was like, and what you learned from it.

Julia Belluz: My husband thought I was crazy. So, that's the first thing. But, no--so, the National Institutes of Health, they have these rooms here on the outskirts of Washington, D.C., and they are designed to measure these chemical processes we call 'metabolism' second by second. So, essentially, the food we eat and the amount of work we are doing, the amount of physical activity we are doing, corresponds to how quickly our body is breaking down the food that we're eating to use for energy to power ourselves. And, the way that's measured is by, in these air-tight rooms, researchers track how quickly you are respiring CO2. So, how much oxygen you are consuming and how much carbon dioxide you are letting out. It gives a measure for how quickly you are burning energy. So, they have an ongoing study where they take people who have a normal body weight and people who are overweight and obese, and they put them through different types of measurements to better understand the characteristics of their condition, their body size. And one of those is this, what's called the metabolic chamber. And that's where--yeah, it's this airtight room that measures your gas exchange, second-to-second. And so, I went in thinking one of the reasons, you know, again, like I struggled with my weight, especially when I was a teenager. And I really thought I could forgive my brothers--they were both skinny and they seemed to be able to eat whatever they wanted, and same with my friends. And I thought, 'There's something going on in me.' And I thought it was a slow metabolism. I thought, like, there was this sluggish--like I wasn't converting the food I was eating or using it very quickly, and it was being stored as fat, and therefore I could gain weight more easily than they could. And so, I went into the chamber to both better understand this hugely important scientific tool that has led to some of the discoveries we've been talking about today, like the relative importance of physical activity for weight loss; and many, many other things. And also to get a better read on my own body and whether I had this slow metabolism I thought I was cursed with.

Russ Roberts: And? And, by the way--you only were in--'only'--for 23 hours. Did other people go for longer?

Julia Belluz: I think they don't keep people longer than 2 days, if I'm remembering correctly, in the chamber.

Russ Roberts: Because, they don't know what you ate before you got in there, or how much exercise you did. It's like the guy who told me that--I think it was a doctor who told me--that somebody came in for cholesterol measurement, and it was a horrible, frighteningly, like a record; and then he found out the guy had been eating French fries beforehand, and [?]. So it was a bad read. But you'd think there would be things you could do before that test that would kind of mess it up.

Julia Belluz: Before the metabolic chamber?

Russ Roberts: Yeah.

Julia Belluz: So, one of the big, fascinating takeaways of spending this time researching the metabolism and with these researchers at NIH, was that many of the things we do that we think--you know, there's this whole, like, popular mythology, discourse--yeah, it is mythology around metabolism boosters and doing certain things to speed up your metabolic processes. And you know, you burn off food faster and make sure you are, you know, like I think the underlying thread is that we can exert some degree of control over our metabolism. But what they see in the chamber, and what they've shown through these different fascinating studies, is that we actually--there's very little we can do to control it. And these metabolic processes are adapting second to second to our environment in ways over which we have little control. So, for example, they put people inside the chamber in very cold temperatures. And they found that--and then they also--sorry--and then they tracked them, and then they put them back in the chamber in normal temperatures. And I think they--so they slept in the chamber in cold temperatures for a significant period of time: I can't remember how long it was. But it was enough time to show that our bodies build up brown fat in response to that cold. So, brown fat is the type of fat that helps us stay warm. And, so, you know, this is a metabolic process. Your body is taking the food that you're eating and converting it into brown fat to keep you warm because you are exposed to cold temperatures. And so this is something you are not thinking about, and you have no, like absolutely no control over it. And then in the study--

Russ Roberts: You can think about it as much as you want, but you're not going to change it.

Julia Belluz: Right. And then you're, then in this study they were allowing people, the same people, to sleep in normal, like, warmer temperatures and the brown fat--their brown fat reserves--went down again. And so there's all these things happening in our bodies, at any given moment. Sorry, just to be clear now: Don't go and sleep in very cold temperatures and think you are going to lose a lot of weight. It was a very, like, insignificant amount of change in their energy burn. But it is something now they are trying to see if they can try to create a drug that has a more dramatic effect to similar pathways. But, sorry. The main message was that there really wasn't much I could do before going into the chamber that would have, like, a big effect, a significant effect, on my metabolism--you know, that I could deliberately do--

Russ Roberts: And what--

Julia Belluz: So, they've shown that, like, eating dark chocolate or caffeine or chili peppers, these don't speed up people's metabolisms in ways that lead to weight loss. It's just like what Dr. Oz says every day.

Russ Roberts: I like--I just try to read about thin people. That's what works for me. And my body thinks, 'Oh. I could do that. Let me try.' So all of this stuff is, again, more hope than reality. But, what did you find out about yourself?

Julia Belluz: So I found out I'm a very boring research subject. I had a very, like, perfectly normal--I did the exact--my metabolic rate was exactly what they'd have predicted for someone my height, my size, and my age and sex. So, I was boring. And, I was surprised by this. I really thought--like, I definitely, you know, exercise, and I'm careful about my diet and all these things. I don't just play my metabolism for when I gain weight. But I did think it was this contributor--that I really thought I had this sluggish metabolism and that explained, like, why it's a little bit harder for me. And so, I debunked that. And, yeah, it caused me to think about where we get these narratives from and this messaging. And, how much how they affect how we think about ourselves, and how sometimes I can be completely wrong.


Russ Roberts: So, you've lost this crutch. Are you less happy now?

Julia Belluz: Am I less happy? Um, I would say I'm happier. I think there's some liberty in knowing the truth, right?

Russ Roberts: Sure. I like what you wrote. You said--going back to our gym discussion. And of course, in the gym, you do have people slowly peddling an exercise bike with not much resistance. And I'm always thinking, 'I wonder what they think they are doing here.' But, you write--because there was an exercise bike in the little room, because they want you to exercise. So, you say,

What's more, the 405 calories I burned during 90 minutes on the exercise bike was both less than is advertised in spinning classes and just 17 percent of the total calories I had used, validating once again that workouts typically account for a relatively minor part of total energy expenditure.

Well, that's true. But you could--one could up in theory, one could up one's exercise and over a long period of time, make a big dent. But--it's just hard to do.

Julia Belluz: Yep. And it--I think you can absolutely do that. I think the question is, is that a sustainable method of weight loss relative to thinking a little more about the food you put in your body, which we know has much more of an impact on how--yeah--how much weight you gain or lose. So, prefer--like, I have a friend here who, he had a period of kind of between jobs and had a lot of free time, and he was exercising like crazy. And he lost a pretty good amount of weight. But now he started to work again, and he doesn't have time to do, you know, one hour of running and then an hour of weightlifting every day. And I think most people don't. So, like using that as your exclusive way to control your weight isn't really a reasonable approach for people who have normal lives, jobs, and families, etc.

Russ Roberts: We've had Gary Taubes on this program. And he's--I'd say, known for two things, among others. But the two that come to mind that he's talked about on the program: He's not a big fan on the program: He's not a big fan of sugar. Which you probably could add, join your list of things we know: probably excessive amounts of sugar are not particularly good for you. Maybe no amount is particularly good for you. And he's also known for advocating a low-carb diet as potentially at least a way to reduce weight. You wrote, I think in this article on the metabolic chamber,

When it comes to diets, the researchers have also debunked the notion that bodies burn more body fat while on a high-fat and low-carb ketogenic diet, compared to a higher-carb diet, despite all the hype. [boldface per original--Econlib Ed.]

Do you think that's a pretty iron-clad conclusion?

Julia Belluz: So, I think that's--I don't know if it's ironclad. Like anything in science, it's a measurative[?] process, and maybe I know these researchers are also working on comparative studies where they look at low-carb--they look at more head-to-head outcomes. Sorry. People following different diets. Compare them and look at their different outcomes. I think where I diverge from Gary--I think he's absolutely right that when we talk about the changing food environment, the big thing that has changed is this--we're inundated with processed, cheap calories. And many of them are taking the form of sugary, you know, carbohydrate--

Russ Roberts: Carbs--

Julia Belluz: Yeah. However, at the same time, people have been eating a lot more of other things, too. So, our fat consumption has gone up. We are eating more calories generally. I think it's simplistic to say it's only, like sugar is the only problem; and I think it's reductionist. I think there are many other things that have changed about our diets. And, people are consuming more of--and we're also consuming more protein, for example. So, meat consumption continues to go up. Cheese--we eat a ton of cheese. I don't think it's just the sugar. I would love if there was just one simple thing that we could say is the problem, but I don't think it's just not.

Russ Roberts: But that's a stronger claim, that the low--many listeners who have written me after the Gary Taubes episodes said they cut down carbs. Many, I'm sure, listening right now. And they write me and say, 'I lost 40 pounds and I've kept it off for 3 years.' Which blows me away. Because, for me, when I reduce my carb consumption, it does--I do tend to lose weight. I just, my body, it gets so unhappy with me that it says, 'Give me back that first potato chip.' It goes crazy for it. And responds, of course, very strongly by adding a lot of fat. And I--I'm curious whether you think that that's open and shut about just low carbs in general.

Julia Belluz: Okay. So, yeah. Where you are asking about that specific study, as well. So, what they were testing was this claim that some people who, yeah, proponents of the ketogenic diet make, which is that once you go keto, you can kind of eat whatever you--sorry--whatever you want of certain food types. And your fat burn will accelerate. You don't need to worry too much about the calories. There was a recent diet book by a doctor at Harvard who, I think he said, 'You can retrain your fat cells and stop worrying about calories forever, if you just follow the ketogenic diet.'

Russ Roberts: And what is--explain what the ketogenic diet is?

Julia Belluz: It's a very--so, you are eating, oh God, I'm going to mess this up. But it's a very low-carb, high-fat diet. So, you are eating, you are staying away from, definitely, like processed carbs, processed breads, sugary foods. And even fruits, by the way. And you are subsisting mainly on protein-rich foods and fats.

Russ Roberts: It's a paleo-like diet.

Julia Belluz: Exactly. Absolutely. And so, what they are testing in the study at NIH was fat burn and whether your calorie and fat burn really accelerate when you are following a ketogenic diet. And they found that wasn't the case. And so, I think--I think, for some people it's absolutely a wonderful way to live. Like, some people do need these--and Gary is one of them--like, he's talked to me about needing these strict parameters. You know, like he can't[?] have just a little bit of chocolate. He has to just like--I don't know if the chocolate completely--and so for some people I think it can absolutely work. And it sounds like it's worked for some of your listeners. I think for the majority of people, or many other people, it's very hard to live that way. And it's very hard to live that way in our current food environment--

Russ Roberts: Of course, I have friends who believe the China study--which I don't; I think it's really bad science. But they tell me that they believe the China study. They've cut out blah, blah, blah from their diet--they would say they are vegetarian now. And they've lost all this weight, and they've kept it off. So--

Julia Belluz: All the more power to them.

Russ Roberts: Yeah. One of my theories is: If you believe the diet, it might work. But otherwise, the average person, it doesn't. I don't know.

Julia Belluz: Yeah. And that is the difference. So, in every diet study, they find this massive individual variation. So, like every time--there was a recent study at Stanford called the DietFits Study, and they compared people following a low carb versus like just a regular, like a higher-carb diet. And the results were virtually superimposable. And what they looked like was that on average people didn't lose that much weight, but in both groups there were people who lost a lot, and gained some. And so there was this variation. And I think that's true for every diet. And I think for many people, it's like a matter, it's experimenting with different things, finding out what works for you. But I think the bottom line is that you need to eat fewer calories. Like, you can just eat what, you know, like on a ketogenic diet, you can eat, as the NIH studies showed, as much as you want and be able to magically accelerate your fat burn, and lose weight. So, that's where I come out. But I think the science is still evolving, and maybe we'll find out that ketogenic diets are what we should all be eating. I don't know. But for now I'm not convinced.


Russ Roberts: It's not surprising that your response to carbs might be different from mine; your response to fat might be different from mine. And one of the lessons, I think of nutrition and diet and metabolism--very similar to the lessons in economics--you have complex systems. Policy intervention that might work in one town might not work in another. It might work for one group but not for another. It might not work at all: It's an illusion based on reverse causation and the problem of confounding variables that you talked about. And it's a complex system. Which means, also, that the body--which means that changing one thing can lead to changes in the other thing that you don't control. And you have a beautiful example of this in your summary of what we know about metabolism--another piece you wrote--where you say, "Dieting can slow down your metabolism." It's like--Whaat?

Julia Belluz: Yeah. That one we have to be careful with, because that's also another study that came out of this group at NIH; but they found that--so, your diet, your metabolism does slow down a little bit. It recalibrates when you are losing weight. But it normalizes again. So, if you don't have as many fat cells in your body, you don't have--if you are shrinking, you don't require as much energy to sustain, to, you know, require as much energy to live. And your metabolism will slow down a little bit. But, where that claim came out of: There was a big study of The Biggest Loser, TV [television] show participants at NIH, of all places. And they found that these people who--they went on crash diets. So, they vastly cut their calorie intake and ramped up like to hours' of exercise a day. And they destroyed their metabolisms in the process. So, they saw what looked like a permanent decrease in their metabolic rate. And so their body was like fighting to hang on to every calorie they were eating. And it made their weight--it made maintaining their weight loss almost impossible. And over the long term, when these TV show contestants were followed up by the NIH, researchers, many of them had regained their weight, and some of them even gained more weight than they went in to the TV show with. So, and I think the big message there was: That might have been more of a response to the extreme intervention that they put themselves through, than what everyone might experience by just trying to eat a little bit less each day. You know what I mean?

Russ Roberts: Sure. But it wouldn't be surprising if you when you were losing weight, your body were thinking, 'Uh oh. Famine. Uh oh. Food supply not available'--

Julia Belluz: Right.

Russ Roberts: 'Conserve. Take it easy. Take care.' So, my point is that, you know, there are these unintended consequences in our bodies that are not that different from the unintended consequences of public policy that don't turn out the way we expect because things change that we don't expect to change. It's a complex system.

Julia Belluz: Absolutely. Yeah.


Russ Roberts: So, we're almost out of time. I just--I want to close with--you can talk a little bit more about exercise and what you learned from--you know, I think we do have a romance about the opportunity of exercise. And, in your time in the chamber you learned some things about that. When you were riding that exercise bike, I'm curious how hard you were pedaling. That's number one. But, more generally: Is there anything we've talked--we've gone all over the place; it's all been interesting. But, are there any lessons for losing weight other than eating a little bit less every day?

Julia Belluz: From--okay. Hmmm. I think the one other thing I did in the study was I had to estimate my calorie consumption. And, I thought I was being so thorough and generous; and, again, I'm quite careful about what I am eating or aware of what I am putting in my body, I think. And yet, I had massively underestimated--I think by the percentages in the story, but it was like 30% or something? Like, I really underestimated my calorie consumption. Because, based on, basically what I reported, it would be impossible for me to maintain my current body size if that's really what I was eating.

Russ Roberts: That's fascinating.

Julia Belluz: And so I think the lesson there is like--yeah. Exercise: Fantastic for health. Exercise as much as your time allows. And you'll reap so many rewards. Including, potentially, even a longer life. Like, it basically doesn't get any better than that. But I think sometimes in our quest for this, like, magic solution or this one simple answer, we overlook, you know, the salad dressing we put on our salad at lunch. And, you know, all these little things that add up in a day to many, many calories that, yeah, we're not even aware we're eating that might be making losing weight or maintaining our weight much harder than we'd like.

Russ Roberts: Because, when you were in that chamber, you were able to find out basically how many calories you were burning through just sitting, resting, your normal sleep patterns, working on a computer, etc. Which we all have a fantasy about, I think inaccurately as you say--and similarly we have a fantasy about how many calories we take in to offset all those changes.

Julia Belluz: Right. Exactly. And yeah. So, they were able to measure that. And it looked like--yeah, I got this very precise reading on how much I'm burning. And they also feed you three meals while you are in the chamber, and they know exactly--

Russ Roberts: Darn--

Julia Belluz: Yeah. They measure down to the gram. And then you have to send through the chamber wall anything you don't eat. Which they then record, to get this precise measurement of your calorie consumption. So, I had a pretty good sense; and I was surprised by, yeah, how much we sometimes, we might underestimate or we are putting in our bodies.

Russ Roberts: Did you smuggle anything in? Just in case?

Julia Belluz: You know, I was surprised I could have. I realized they don't bring any food in. And actually I should say they also watch you through--so there's like a Plexiglas window in the chamber. And they watch you through that and check in on you. And there's also a video camera in a corner of the chamber. So, they are watching you through that, as well. And I'm sure if I was, like, eating chocolate bars, they would have discounted me as research--

Russ Roberts: Enng--enng--enng.

Julia Belluz: I got in trouble when I wasn't resting enough. Like, you have these rest periods where they want to see what's happening with your metabolism at rest; and I definitely--sometimes I was too jittery. I wasn't resting enough. And I got scolded.

Russ Roberts: Last question: How did that experience--and, like you discovered you were eating more, probably, than you thought you were. Did it change you in any way? In terms of your behavior? Did you find yourself--how long ago was it, and what's happened since then? Besides you got a very nice article about it. Which we'll link to.

Julia Belluz: Yeah. I don't know if I changed my behavior. I think like that now my health routines are pretty set in. But again, it did make me think about how--there was a big gap between the reports I was getting about my health and sometimes how I feel about, you know, my health. And it did make me think about all this messaging in our society from like, yeah, the Dr. Ozes and the Gwyneth Paltrows, and the gym conglomerants, and the healthy food purveyors--

Russ Roberts: and the water industry--

Julia Belluz: The water--yes. Big Water. And Big Blueberry. And all these--yeah--we are constantly bombarded with messages about how we could be a little bit healthier. And how we should be doing x or y. And I guess, yeah--it made me a little more aware of how those messages might affect you. And even if you are a critical thinking, you know, informed person about these issues, somehow, yeah--they can affect you and shape how you think about yourself. And so, I guess that's the one thing that--that and thinking a little more about maybe some times that I'm underestimating my calorie consumption. So, what I'm eating, a little bit more that sometimes I'm underestimating it. And it also gave me a whole new appreciation for the research they are doing at NIH and how difficult it is--going back to our first conversation--how difficult it is to do this kind of research.

More EconTalk Episodes

Search Econlib