Betsey Stevenson and Justin Wolfers on Happiness, Growth, and the Reinhart-Rogoff Controversy
Jun 24 2013

Betsey Stevenson and Justin Wolfers, of the University of Michigan talk with EconTalk host Russ Roberts about their work on the relationship between income and happiness. They argue that there is a positive relationship over time and across countries between income and self-reported measures of happiness. The second part of the conversation looks at the recent controversy surrounding work by Reinhart and Rogoff on the relationship between debt and growth. Stevenson and Wolfers give their take on the controversy and the lessons for economists and policy-makers. This conversation was recorded shortly before Betsey Stevenson was nominated to the President's Council of Economic Advisers.

Richard Epstein on Happiness, Inequality, and Envy
Richard Epstein of the University of Chicago talks with EconTalk host Russ Roberts about the relationship between happiness and wealth, the effects of inequality on happiness, and the economics of envy and altruism. He also applies the theory of evolution...
Carmen Reinhart on Financial Crises
Carmen Reinhart of the University of Maryland talks with EconTalk host Russ Roberts about the ideas in her book This Time is Different: Eight Centuries of Financial Folly (co-authored with Kenneth Rogoff). They discuss the role of capital inflows in...
Explore audio transcript, further reading that will help you delve deeper into this week’s episode, and vigorous conversations in the form of our comments section below.


Jun 24 2013 at 11:33am

Excellent podcast Russ! This conversation highlighted many of the reasons I decided not to pursue an academic career. Scientific publishing seems designed to communicate sensational, cherry-picked results that mesh with current fads. Space is wasted on introductions, literature reviews, and references that are often identical from paper to paper. Little space is given to methods. Large data tables are discouraged, while flashy graphics are applauded.

Perhaps even worse, the journal system creates an indigestible pile of paper that requires enormous work just to to review, comprehend, and cite accurately. The system is horribly outdated and needs to be revamped.

Scientific fields need to organize scientific publishing around an online database tool along the lines of Wikipedia, but of course highly filtered and peer-reviewed. All available data and analyses on a given topic, including replications and null results, should be linked and organized around a single topic entry that contains a single continuously updated introduction and literature review. As Russ suggests, the analytic sections should include ALL analyses that were conducted along with basic results. Discussions and opinions by specific authors should also be linked to these analyses and the main article, along with rebuttals, etc. Meta-analytic tools should be built right into the system. The whole thing should be funded by the same agencies funding the research in the first place, the NSF, NIH, etc and made publicly available.

The current system is just absurd. In some fields (psychology, education) authors publish articles each year that are little more than carbon copies of what they wrote last year, with a new introduction, maybe a new dataset and a few other little updates. This is how most professors rack up 100+ page long CVs. This safe repetition by star academics is given vastly more journal space than replication attempts, null findings, or novel investigations, particular if they come from junior researchers.

Emory Smith
Jun 24 2013 at 12:16pm

Does every dollar have the same impact on happiness?

Does a dollar I earn,
a dollar I inherit,
a dollar I receive as a gift,
a dollar I get from a state transfer,
a dollar I steal,
a dollar I find on the street,

do they each influence my happiness the same? I don’t think so. There is more than quantity involved in the relationship of income to happiness. The source of the income counts. At least it does to me.

Jun 24 2013 at 1:26pm

The double-guest format was largely successful, although both guests were in agreement almost 100%. Now, if they were engaged in a heated debate, that would be fun…

Jun 24 2013 at 2:30pm

If you know what you love AND have the opportunity to do it for a living, then it might make sense to turn down a job where you’ll be making more money. However, if you haven’t discovered that, or aren’t able to do it because of different circumstances and you are primarily looking for a way to make a living, then the more you make the better, because the higher income compensates for you doing something you don’t love. This is my case, and I am doing everything I can to increase my income, working overtime, training, looking for pathways to higher income, and obviously the reason is not that I want pieces of paper, but that money allows me to enjoy the MANY things life has to offer, provide a good life for my son, nice house, nice car, good pre-school for my son, gym membership to stay fit and healthy, entertainment, vacations, too many things to mention. I would be curious to see how many people actually work their “dream jobs” and how many are simply trying to make a living, my guess is that the majority of people are simply trying to make a living, if this group is predominant, then income level is ALL important to our sense of wellbeing.

Patrick R. Sullivan
Jun 24 2013 at 3:23pm

I hope she talks this candidly to Barack Obama.

Martin Brock
Jun 24 2013 at 3:31pm

When Easterlin proposed his paradox in the 1970s, the distinction between “within country” and “between country” was more meaningful. Since then, globalization has increased people’s sense of inclusion in a global culture. Someone in Sub-Saharan Africa, with much greater access to a much broader variety of media, may perceive living standards in New York or London as much “closer” today than in the seventies. He may spend hours a day watching television. He may even know people living in the more developed world personally and text them on his cell phone while still experiencing a much lower, material living standard. He may wander through their streets with Google maps and even view the neighborhood in real time through web cams.

A person living almost anywhere today may judge his condition more relative to a global “norm” (perceived through electronic media) and less relative to the standards of his immediate surroundings, compared with a person in the 1970s. In the 1970s, living in the United States in a relatively affluent family, I could already watch television frequently and experience this dissonance to a considerable extent, but even I had no internet access and no cell phone. Today, satellite television and cell phones exist in areas without paved roads and running water. If happiness is a function of a person’s perception of his condition relative to his “neighbors”, this change must be very significant.

Jun 24 2013 at 4:13pm

I really like the discussion on academic papers. That is one topic that is very intriguing. How do we create a system that avoids group think and the internal politics of the various communities? Can that even be done?

One site that is making a small dent is

It is mostly focused on medical journals, but other topics are listed on the site.

Separately, as to the publishing of null results, I’m currently reading Nassim Taleb’s Anti-Fragility. In that he says that research that leads to disproving theories is much more robust than confirming research. So, perhaps rather than replication studies, the focus really should be on studies to disprove prior work. Using his example of “there are no black swans”, testing and retesting might repeatedly show only white swans. This infers we are correct. However, it only takes one black swans observation to discard all of those tests.

I strongly support the publishing of data and code along with the papers. Much of my skepticism of the global warming research stems from documented cases where the data is lost or altered and the exact methods of statistical analysis are never published. Papers that have known mathematical flaws are still published on the Internet without retraction. Why do the journal do this?

Similarly, epidemiologists (see the Gary Taubes podcasts) have the same issues. There is a cookbook algorithm to deconstructing their work.

1. Is the recommendation based only an observational study (which can only generate hypotheses, not prove causation)
2. Do the researchers ignore confounding findings in their own data? (such as sendentary, meat eating smokers having lower cholesterol – that’s good, right?)
3. Was the data collection valid? (surveys are notoriously unreliable)
4. Are there any errors in Excel or their statistical software?

It would seem like getting past those 4 points should be attainable by anyone publishing a research paper, but I’d suspect the error rate is much higher than one would hope. (>10% if I had to guess) Also note that the scope of peer review often does not address the 4 points above. It is exceedingly rare for peer reviewers to check the validity of statistical analysis, for example.

While the blogosphere seems to do a good job of finding some of these issues and bringing them to light, it seems like setting a standard of always publishing the papers along with the supporting details would help strengthen both the conscience of researchers and the scientific findings. Even if you assume researchers are perfect and have no bias or ill intent, there is still a very good chance they are wrong. Having more eyes on the problem would benefit all of us.

Patrick R. Sullivan
Jun 24 2013 at 6:32pm

Nicholas Oulton also did a number on Easterlin last year;

I find the results of these happiness surveys puzzling because they are inconsistent with other facts about people’s behaviour. First, if people care mainly about their relative position, why has there been so much fuss about the financial crisis? After all, for most people in the UK, the drop in income has been (on this view) trivially small, no more than 8% – and at least initially, it fell disproportionately on the rich.

Second, if people care about their relative position, why does this have to be expressed in terms of annual income? After all, most workers today can work part-time if they want. So why can’t A boast that his daily rate of pay is higher than B’s even if B’s annual earnings are higher and this is because smart A works only three days a week while poor dumb B, a slave to the rat race, works five?

Yet surveys of part-time workers regularly show that many would like to work longer hours if only they could. And while it is true that some leisure activities like skiing require a lot of complementary expenditure on stuff, many other activities – watching TV, surfing the internet, chatting with friends in pubs or cafés or avoiding Betjeman’s regret – do not.

In fact, people’s leisure choices provide powerful evidence against the view that only relative position matters. The classical economists argued that the amount of time people were prepared to work depended on the range of goods and services available for consumption.

Jun 24 2013 at 9:23pm

Here is a good critique of some of the happiness research. It may not apply directly to these guests, but it does point out some of the main issues.

Thomas DiLorenzo

The Murray Rothbard quote is insightful.

“One of the most absurd procedures based on a constancy assumption [i.e., the false assumption that people never alter their preferences] has been the attempt to arrive at a consumer’s preference scale not through observed real action, but through quizzing him by questionnaires. In vacuo, a few consumers are questioned at length on which abstract bundle of commodities they would prefer to another abstract bundle, and so on. Not only does this suffer from the constancy error, no assurance can be attached to the mere questioning of people when they are not confronted with the choices in actual practice. Not only will a person’s valuation differ when talking about them from when he is actually choosing, but there is also no guarantee that he is telling the truth.”

Jim Feehely
Jun 25 2013 at 12:30am

Hi Russ,

What this discussion said to me was that it is essentially nonsense to study society empirically because of what we don’t know we don’t know. And then to draw causal conclusions from the data is monumentally arrogant and stupid.

This stuff about ‘happiness’ gets studied largely because of economics’ obsession with money and wealth as an analogue for everything in society.

In my view, the only possible analogue for contentment is the range of options an individual has, economically, socially, politically, geographically etc. Certainly wealth gives an individual more options and the opposite is true of poverty.

It is utterly unreliable to ask a poor person how ‘happy’ he or she is. The answer will almost always be biased toward ‘happiness’ purely as a stoic coping strategy.

And I don’t see too many wealthy people who I, subjectively, would call ‘happy’. Rather, they are relatively powerful but often stressed, grumpy, impatient and critical.

If the ‘data’ is of any use, why not look at the data on reported and suppressed depression, suicide, intra-community violence etc?

But I think the substance of the matter is options, not wealth. And that means neo-classic economics is chasing the wrong societal goal – growth in wealth.

In relation to Justin Wolfer’s (a fellow Australia I see) speculation about the possibility of society reaching beyond mere happiness, I suggest he look at Taleb’s work on convexity. Clearly happiness v wealth is non-linear and it is clearly concave. So Justin does need to worry about an outbreak of ecstasy any time in the future. Humans are not meant to be ecstatic. Nature will ensure it never happens. And the non-linearity effects also explain why the super-wealthy are far from super-happy.

The mindless pursuit of economic growth has clearly not made wealthy societies ‘happier’. If it did we would not have the rates of depression, suicide and social alienation that are obvious to anyone who takes an objective look. And it is certain that the destruction of environment, ecology and bio-diversity on which economic growth is fundamentally based, will make the human race desperately sad in the the not-too-distant future. Nothing in nature assumes limitlessness. So why does economics?

If economics is to actually make a contribution, then it ought to be pursuing the tantalising, but difficult, objective of prosperity without growth. A cue to have a discussion with Tim Jackson?

Jim Feehely.

Patrick McCabe
Jun 25 2013 at 10:00am

[Comment removed pending confirmation of email address and for policy violation. Email the to request restoring your comment privileges. A valid email address is required to post comments on EconLog and EconTalk.–Econlib Ed.]

Mads Llindstrøm
Jun 25 2013 at 3:14pm

Emory Smith – excellent point.

In my experience, money earned is also spent more carefully than non-earned money. When people starts working and experience the effort they have to put into earning money, they naturally become careful about spending there hard-earned money.

Regarding the Easterlin hypothesis and log-income being related to happiness, is validating another hypothesis. Namely that 99% of research is either wrong or confirms common sense. Nobody should be surprised that money helps. And nobody should be surprised that a multimillionaire earning an extra $100 matters less, than a waitress getting a $100 tip.

Jun 25 2013 at 7:25pm

Recessions kill people?

Jim Thorson
Jun 25 2013 at 9:03pm

I have been a listener for quite a while on my long commute. I really enjoy your blog. Its the second blog I listen to every week (Car Talk is Number 1- who wants to think at 5:00 am on a Monday morning)

I was very interested in youi blog on happiness and income. Usiually you are careful to point out that correlation is not causation- for some reason, no- one brought up this point in your discussion.

An alterate hypothesis is that the causation is reversed- happiness results in higher incomes. Alternatively , perhaps they are both correlated to a third variable- whch I will call a “Feeling of Usefulness” or perhaps by another name- productivity.

I know that I am happier after a productive day than a non- productive day. Over the long haul, lots of productive days do tend to result in higher incomes.

Ananlyzing ones own behavior is hazardous, but its the best data I have on motivation, even if it is biased. Interestingly, getting what I believe is an undeserved reward does not make me happy. However, I can be happy if I feel i have been productive, whether or not I am directly rewarded. (just so long as this does not happen all the time)

Russ Roberts
Jun 26 2013 at 10:51am

Jim Thorson,

I don’t think simply having more money makes you very happy and I don’t think either of the guests do, either. If you take a look at the 21:45 mark in the highlights, I think you’ll see a discussion of the causation issue though we don’t say it explicitly. It would have been better to point it out explicitly.

Jun 26 2013 at 5:13pm

Regarding the mismeasurement of GDP etc. I would suggest to look at the literature of spurious regressions

Jun 27 2013 at 10:59am

Good podcast. Its been a good 2013 sp far.

Keep up the good work!

Jun 27 2013 at 1:25pm

If you take a dollar from a rich person and you give it to a poor person, the gain in wellbeing to the poor person is larger than the loss in wellbeing to the rich person. But there is a loss, and I think what’s important is to know that redistribution does take from one person to give to another, but it’s clear that you can increase aggregate wellbeing in your society by doing some of that redistribution. -Stevenson

I’ve always been skeptical of this claim as it seems to ignore the real-world motivations that caused the rich person to seek that extra dollar and the poor person not to. Yet, earlier in the podcast, some of those trade-offs were discussed when it came to not always accepting a higher paying job.

Edward Smith
Jun 27 2013 at 7:02pm

I learned about EconTalk through Ricochet, and it has revived a nascent interest in statistics that started with a lecture at Johns Hopkins University and in economics that was very nearly destroyed at the University of York (in the UK).

I am not always able to follow every point in the interviews, but I am getting better at it.

Keep this up, and I will listen to every interview.

Charles Roberts
Jun 27 2013 at 10:17pm

When I saw the title of this podcast I was excited because I thought I would learn something about happiness. Yet, surprisingly, there wasn’t much talk about what really makes us happy. Here’s my personal thesis about happiness based in part on a very scientific survey I took at the dinner table this evening with my family. Happiness comes from a good relationship with one’s spouse, family, and friends, pursuing challenges in life (gaining an education, pursuing a meaningful career, etc.), following a moral code (religion), being engaged in community service, and not being poor (a point made enthusiastically by my daughter). You can probably think of a few others.

I submit that people who do these things are generally smart and have good judgment. Smart people who have good judgment also tend to make more money than people who are not smart and have poor judgment. In my view, considering whether money makes one happy is like considering whether getting a tattoo on your face makes you unhappy. Yes, there probably is a correlation between unhappy people and people with tattoos on their face, but I would argue that the causation isn’t there. People who are not smart and have poor judgment tend to make decisions that do not bring them happiness.

Having listened to every Econtalk podcast, I feel like I know Russ Roberts as well as my good friends, although I’ve never met him. I would say Russ is a very happy guy and, I think we can all agree that Russ is very smart and has great judgment. And, for those of us who have gotten to know him through his podcasts, we all know that he has a great marriage and family, is devout to his religion, is not poor, and does this spectacular weekly service that we all get to enjoy.
There was some discussion by Russ and Betsey justifying their decisions to take a lower paying job. This is just another example of using good judgment to achieve happiness. We can all think of people who pursue money while sacrificing family relationships and, as a consequence, are miserable. Also, think of the lottery winners who are presented with lots of money and report that their quality of life goes down substantially (Google Sharon Tirabassi).

What would be really interesting is some research on whether it is possible to take an unhappy person and make them happy. If it’s a simple function of brains and good judgment, I don’t see a huge ability to change that in people, regardless of what the GDP numbers say.

Keep up the good work Russ.

Jun 28 2013 at 6:36am

How reliable are social science findings over time?

Stuff like various physics constants seem to remain the same over time(speed of light in vacuum etc) but would repeated experiments in the social sciences have any constants? Yes, the Milgram experiment was famously decently well replicated, but that seems to reflect more ignorance of the experiment among most people.

Human society is a self-modifying system. When behavioral economists identify a cognitive bias, eventually, that bias should disappear as everyone becomes aware of it. Indeed you’d expect some biases to be virtually wiped out in some sub-groups of society.

Getting around to happiness: how is local society defined? What does a random South African, Bulgarian, Guatemalan, or Canadian perceive as their potential level of material wealth, nowadays versus 40 years ago? What do people in various countries aspire to achieve?

I think it simply is not the case that people 200 years ago experienced nothing or even predominantly nothing but gut wrenching suffering. It’s a lot easier to imagine that a subsistence farmer in Eastern Europe today is far less happy with life, than a poorer but more average(more resigned?) subsistence farmer(or rather land-slaved serf) 200 years ago.

Finally, happiness, in all its myriad versions, is a neurological process/state. Surveys and the like are cute, but these are 19th century level techniques being used. Are we getting close to being able to use brain imaging/blood tests in order to get more accurate measures on these issues?

Russ Roberts
Jun 28 2013 at 6:15pm

Charles Roberts,

Just for the record, you and I are not related, as far as I know. Thanks for the kind words. They may even be true but I am not the best judge–confirmation bias and all that. I do think I am happy and if asked on a survey, I would respond with a high number. And I do think you are right about the fundamentals that lead to happiness.

Working on a book that will talk about some of the issues beyond survey results. I’ll keep EconTalk listeners posted as that project moves forward.

Jun 30 2013 at 9:53am

Justin Wolfers in the transcript: “Forget the Freud stuff, because we all know Freud is bad.”

I imagine Karl Popper would have agreed with that!

Jul 1 2013 at 1:27pm


Rothbard’s approach is even more misguided (or self-serving). What do you find out about people by observing what they do? That poor people buy cheap stuff, rich people buy expensive stuff. Does that tell you what they want? No, definitely not in the cast of poor people. It only tells you what they can afford. A poor person might really want decent schooling, healthy food and medical treatment for their children but since they can’t afford these things they won’t buy them. How can you conclude from that that they prefer bad or don’t care about the quality of schools, food, health care etc? (as an example)

Jul 1 2013 at 5:02pm

1) Justin Wolfers said, “When you look at happiness you get pretty much the same thing as when you look at GDP; what economists have been doing all along isn’t so bad.”

God Bless you! I have spoken with so many people over the years who consider the topic of income–especially increasing it–as taboo, sinful, greedy, selfish, uncaring, unchristian, and even hurtful to others and to society that I begin to wonder if anyone thinks differently. Perhaps this research is a bridge that can begin to span the gap in language and understanding between economists and everyone else.

2) Russ Said “Happiness…is kind of a rich concept to reduce to a single digit. Isn’t it?” Justin Wolfers said, ” I think just about any interesting social indicator is. Think about the rich experience of our lives that is summarized by Gross Domestic Product (GDP), or the incredible richness of the labor market we try and summarize by the unemployment rate.”

I am personally very familiar with the MANY limits of the unemployment rate, but had never thought of the GDP per capita as having similar limitations. I learned something there. That was nice.

3) Betsey Stevenson said, “But to give you just a really stark example of the Easterlin outlook,…if you take a village in Africa that doesn’t have running water, and you bring running water and you give it equally to everybody,…Easterlin’s view is that you didn’t make anyone in that village better off.”

That is an absurd statement. I really don’t think Easterlin–or anyone else–would agree with it. A more logical application of Easterlin’s conclusions is that by giving running water to everyone you would not make anyone “happier.” His research, at least as discussed in this podcast, does not address whether they would feel “better off.” “Better off” and “Happy” are not the same question.

4) Justin Wolfers said, “You are suggesting both GDP and happiness are terribly mismeasured. And the worse the measurement is the more that biases the estimated correlation towards zero. So it’s amazing that the estimated correlation is as high as 0.8, given that I’m finding that’s a correlation between two noisy measures. So, following your logic in fact we are understating how close the relationship is between GDP and happiness.”

Russ replied, “Well, that would be true if the errors in the two variables were random…It could be that it’s not just that, oh, people sometimes make mistakes in coding that could be higher or lower–it’s that there’s a systematic bias in how the data are collected, transcribed.”

THAT is an amazing back and forth. Justin made a big, reasonable argument seemingly out of thin air. Russ handled it, as I have heard him do on several occasions in the past, convincingly and without breaking stride. I love it when he does that. It is so cool to watch–or rather listen–to professionals in their field deal with problems reflexively that take laymen such as myself weeks to work through.

5) Betsey Stevenson said, “There is something else that comes out of our work that’s important, which is that greater inequality does reduce aggregate wellbeing, and that’s because of the fact that this relationship is between wellbeing and the log of income. If you take a dollar from a rich person and you give it to a poor person, the gain in wellbeing to the poor person is larger than the loss in wellbeing to the rich person. But there is a loss, and I think what’s important is to know that redistribution does take from one person to give to another, but it’s clear that you can increase aggregate wellbeing in your society by doing some of that redistribution.”

Justin Wolfers–not one sentence before this statement–said that their work showed correlation, not causation. Yet here she is treating the two variables as if they are causally related; such that changing one would change the other. Russ, this is the exact same thing that happened with your previous guest, James Galbraith. When he spoke about his research, he was brilliant and reasonable. But as soon has he started talking about policy, he behaved as though completely unhindered by HIS OWN FINDINGS. Here Betsey is doing that same thing. Contrary to her statement, it is not at all clear that, “If you take a dollar from a rich person and you give it to a poor person, the gain in wellbeing to the poor person is larger than the loss in wellbeing to the rich person” or that “you can increase aggregate wellbeing in your society by doing some of that redistribution” or that “greater inequality does reduce aggregate well being.” Her research does not support those claims.

Additionally, her research was about subjective happiness. She substituted the word “Wellbeing” for “Happiness” in all these arguments. Her research doesn’t support that substitution.

6) Betsey Stevenson said, “It is the case that the more money you get, the lower the incremental gain in happiness will be…So, doubling your income provides similar increases in wellbeing regardless of where you are in the income spectrum.”

Now that’s interesting information, but–again–I would advise Dr. Stevenson to use language that reflects correlation. The words, “incremental gain” or the phrase “doubling your income…increases wellbeing” strongly imply a causal change in happiness when income is moved up or down that her research does not support.

Jul 3 2013 at 3:04pm

If behavioral economics shows that loss aversion means that having something taken away from you is more painful that having an equal thing given to you is pleasureable then how can it be “clear” that redistributing income from higher incomes to lower incomes makes everyone better off? Even with diminishing marginal returns to wealth loss aversion might cancel out some of the gains from reditribution.

Eric Falkenstein
Jul 4 2013 at 12:11pm

The happiness question is not nearly as settled as Wolfers and Stephenson imply. They examine “the data” and claim they’re simply interested in truth, but the data is complicated, not like a simple test of logic or a singular statistical test. If he acknowledged the data were somewhat ambiguous, I would give him the benefit of the doubt, but as he presented Easterlin as a flat-earther, which suggests he really doesn’t understand the issue. In the words of AJ Kane, ‘Only the simplest mind can believe that in a great controversy one side was mere folly.’

It would be nice if he would say what specifically, caused Easterlin and his supporters to misinterpret or bias his data.

I think even S&W said this trend doesn’t apply in the time series of the US or China over the past 35 years. So, excluding the largest population and economy in the world, the fact is true? He mentioned a 162 country cross-section, but that would obviously include many really poor countries that even the Easterlin proponents would admit really appreciate money at the margin.

There is a lot of data from fMRI’s on processing social information, studies on neighbor and colleague incomes and happiness and health, and studies on glucocorticoid levels in social units, all that corroborate the relative rank hypothesis. It is truly a puzzle, why there are papers and books on the time on the prosperity paradox, because GDP/capita has doubled over the past 50 years in the US, and it sure doesn’t seem happier here, though clearly its not obvious because surveys and their samples are never apples to apples.

In any case, they did not definitively refute anything, other than doubt in their own mind.

Comments are closed.


About this week's guest:

About ideas and people mentioned in this podcast:Articles:

Web Pages:

Podcasts, Videos, and Blogs:



Podcast Episode Highlights
0:33Intro. [Recording date: May 28, 2013.] Russ: Now, you are my first pair of guests on EconTalk. It's going to be exciting to see how that goes. You folks are going to have to share the mike. We're going to talk about a number of issues today. We are going to start with research that you've done together on the relationship between happiness and income. Then we're going to talk about a column you wrote on the work of Reinhart and Rogoff. And if we have time we'll close with some unrelated issues. So, let's start with your contribution to what is sometimes called the 'happiness' literature. And let's begin with that literature, the origins of that literature, the work of Richard Easterlin. What did he find? GuestW: So, the Easterlin Paradox is 3 empirical claims and then the implication that follows. So, the first claim was when you look within a society, like the United States, at a point in time, you find that rich people are happier than poor people. The second was that if you look instead across countries, you fail to find any evidence that rich countries were happier than poor countries. And third, when you looked at countries through time, you fail to find any evidence that as they got richer, they got happier. So that seems paradoxical. Hence the title. And his preferred explanation was that what must be going on is what matters is relativities. What matters is your income relative to the Joneses, and that would explain why rich people in a country are happier than poor people but would also explain why rich countries weren't happier than poor countries--because when you are in a rich country your neighbors are rich, too. And that's something that will offset the impact of your own richness. GuestS: So, Easterlin's findings and his interpretation of the data have led him to a very, very clear policy recommendation. And he stated that a focus on economic growth is simply not in the best interest of society. And that's because he believes that the core relation you see, when you look within a country where wealthier people seem like they are happier or more satisfied with their lives than poorer people, simply reflects status. And that there is nothing that is actually gained as we develop because we--maybe because we acclimate or get used to things. But to give you just a really stark example of the Easterlin outlook, or the Easterlin hypothesis, now if you take a village in Africa that doesn't have running water, and you bring running water and you give it equally to everybody in that village, Easterlin's view is that you didn't make anyone in that village better off. If you bring it only to the chief's house, you have made the chief better off relative to everybody else in the village, because you have changed his relative position by increasing the difference between what the chief has and what everyone else has. Russ: And how did he measure--and this of course is always going to be an issue in this literature, it's one of the questions I have about it--how did he measure happiness? What was that measure based on? GuestW: So the standard in this, and this is what all economists do if you are analyzing happiness, is you analyze large cross-sectional surveys, where you go out and you ask people either how happy they are or how satisfied they are with their lives. GuestS: So, I have to admit that I came to studying these kinds of questions with a great deal of skepticism. And what's been really stunning to me is the work psychologists have done to really validate, not only how useful these questions can be, but also how universal they are. That people--that if I tell you how satisfied I am with my life, if you took a picture of me and you showed it to strangers, they would be able to give a reasonable guess that would be correlated with my life satisfaction. In fact, psychologists have done that kind of study where they ask people to rate their satisfaction with their lives; they've taken pictures of them; and they've shown those pictures to strangers; and they've found that what the strangers assign to people is correlated with the actual numbers. They've also done things like ask friends and family and found that friends and family tell us things that are similar to what the individuals say. So it turns out that simply asking people, overall, taking things all together, how happy would you say your life is, or how satisfied are you with your life, people give answers that are consistent and interpretable. Russ: Later we're going to talk about replication, I hope. I'm very suspicious of those psychology studies of showing a picture of a person, a snapshot I assume, not a video of their day. But, it's possible. I'll remain open-minded because I don't know that literature very well. But I would wonder about that. That would seem extremely difficult. Part of the problem I have is that when I think about my own happiness or my own satisfaction, yeah, there's a general overall number on a scale of 1 to 10 I might have from time to time. It might change when you ask me. But it's kind of a rich concept to reduce to a single digit, even with a decimal point. Isn't it? GuestW: Well, I think just about any interesting social indicator is. Think about the rich experience of our lives that is summarized by Gross Domestic Product (GDP), or the incredible richness of the labor market we try and summarize by the unemployment rate. Russ: I have problems with those, too. GuestW: I'm with you on that. As long as we're willing to admit that all statistics are imperfect, I'm on board. Russ: Okay, that's fair. The other thought I have is Henry David Thoreau, who said: The mass of men lead lives of quiet desperation. A bit of a pessimist. It might have been time-centric, when he said that; it wasn't the best of times when he was writing. But there is a certain illusion that we present to the world, a certain mask, a certain number that we might list in our 1-to-10 scale that might mask what we really feel, like deep down inside lots of times. So we all understand that this is a somewhat mushy concept. And yet, Betsey, you point out that it seems to hold up with some interest in some larger applications. GuestS: You know, it really does. They've also shown things that it's correlated in the way you'd expect it to be with life events. So, people who are recently divorced or are in the process of divorcing are less happy or less satisfied with their lives on average than people who are newly married. People who have experienced a death show declines in their happiness and life satisfaction. And have as a group average life satisfaction that is lower than people who haven't recently experienced that[?]. So it is surprisingly consistent and useful.
8:11Russ: So now, let's go after Easterlin. Talk about what work you have done, the two of you together, in this area and where does it fit in with other findings that people have found in trying to understand the relationship. GuestW: Yes. This is where we prefer to relabel the Easterlin Paradox, Easterlin's Hypothesis. And the reason is we think the data just don't agree with him. The usual ways social science debates go is we all agree on the facts and then we argue like hell about what they mean. In fact, our view has been the opposite, which has been trying to figure out what the facts are. So, let's go back to the three key facts. The first is we agree with Easterlin that within a society at a point in time, rich people are happier than poor people. The second claim that Easterlin made is he failed to find any evidence that rich countries are on average happier than poor countries. So that in a very particular way--absence of evidence is not evidence of absence. And what he was doing was looking at very small samples. So when he failed to find an effect, a statistically significant effect, he assumed it was 0. Today we have much more data. We've looked at data from 160 countries from around the world. And it turns out that levels of GDP per capita are incredibly robustly related to average incomes[probably misspoken? Should be happiness?] across countries. In fact, it's one of the tightest cross-country relationships I've ever seen. The correlation between GDP per capita and measured life satisfaction is 0.8. So it's crystal clear today rich countries are happier than poor countries. GuestS: The problem with Easterlin's earlier work is where he goes a bit beyond that, which is that he made a claim about a hypothesis he didn't test. So, the hypothesis he tested was: When you look across countries, do you see that the average happiness of a population is correlated with the average GDP per capita. And he was unable to find that there was any positive relationship there. He was unable to reject 0. This is what Justin said--he failed to find evidence of a positive relationship. But the claim he made was that the relationship within countries, the fact that rich people within a country, like the United States, are happier than poorer people, was a stronger relationship than what you see across countries. So, this is really delving into the nerd talk. Russ: That's what we're here for. GuestS: If that's the claim you want to make, you don't test the hypothesis when you look within countries do we see a statistically significant effect when we look across countries is there a statistically significant effect that's different from zero. But instead the hypothesis you want to test is: Is the relationship we see within countries statistically significantly steeper than the relationship we see across countries? That's the hypothesis he was claiming with words, but it wasn't the hypothesis he was ever testing. So one of the first things we did was just went and tested his hypothesis with his data. And it turns out we couldn't find any evidence that the within-country relationship was indeed steeper. And the reason was there was so much imprecision in the cross-country data that while he couldn't reject 0, he also couldn't reject that it was just as steep as what you saw within countries. GuestW: And so our conclusion is finding that small samples don't deliver clear results is not a paradox of any type whatsoever. Russ: So, let me just pause for a second. We've talked about some of the imprecision of just trying to measure happiness with a single number in a survey, right? These are all individuals that are asked, I assume. And then when we go to the cross-country, averaged, a bunch of individuals in, say, America, and we are comparing them to a bunch of individuals, averaged, in Costa Rica, and the United States has higher per capita income that Costa Rica, so the question is whether the United States has higher happiness on average than Costa Rica. Correct? GuestW: Yep. Russ: So, in the United States those surveys are done by, say, the Gallup organization, some of them; there are others done, I assume, by other people. When you say we have 160 something countries, whatever the number was, I wonder how reliable the data are in some of these countries? Just because we had a podcast, Morten Jerven a while back, pointed out that the GDP numbers--the per capita income numbers--aren't reliable. So, how do I know that these relationships are statistically meaningful, given that the data are probably a little bit sloppy? GuestS: That is a great question and I'm so glad you started by telling us how bad the GDP numbers are and how they are not very reliable, because I don't think we want to argue that these numbers are completely reliable. But I think what we want to do is compare them to other cross-country data that we have. And in fact, they may be better than the GDP numbers. One of the things we are showing is that this happiness is highly correlated with GDP. That may be important because it may say that the happiness data may actually give us a better snapshot of GDP at a point in time than the actual GDP measurements, because it is a much simpler question, it's something that has been tested, and it is robust. And so if it's something that is actually highly correlated with GDP, it may provide its own use in dealing with an international context, where it's incredibly difficult to measure all sorts of things. GuestW: Russ, let me add two things. The first is the best wellbeing data today come from Gallup. Gallup are actually in every one of these 160 countries. So the Gallup World Poll is a methodology that is as far as they can do something that is exactly the same in each of these countries; there are very careful translations and so on. And I should add parenthetically I work sometimes as a consultant to Gallup, just in the interest of full disclosure. The second though is to think about the implications of what you said, which is the Easterlin paradox proposes that the correlation between GDP and happiness is zero--we find it's extremely high, and it's 0.8. And you are suggesting both GDP and happiness are terribly mismeasured. And the worse the measurement is the more that biases the estimated correlation towards zero. So it's amazing that the estimated correlation is as high as 0.8, given that I'm finding that's a correlation between two noisy measures. So, following your logic in fact we are understating how close the relationship is between GDP and happiness. Russ: Well, that would be true if the errors in the two variables were random, right? GuestS: Right. Russ: It could be that it's not just that, oh, people sometimes make mistakes in coding that could be higher or lower--it's that there's a systematic bias in how the data are collected, transcribed. Which I think is true sometimes in the GDP data. But I take your point. I think one of the things that's fascinating about thinking about this, which I had not thought of until we had this conversation, is that I really had been focusing mostly on the sloppiness of the happiness measures. Then I realized later--whoa, whoa, whoa, whoa, wait a minute. They both have some problems. But let's-- GuestS: Oh, I'm really glad you point that out. Because when people think about our study, the relationship between happiness and income, they focus a lot on how sloppy these happiness numbers are. Whereas I think we worry even more about how sloppy the income numbers are. Russ: But you see the income numbers have more decimal points, of course. More digits. So they must be more accurate. Don't we know that?
16:17Russ: Moving along. So, what kind of reaction--there are other people working in this area. Did they confirm this finding of yours? Do they contest them? What's the state of the literature right now? GuestW: So, my reading is that most economists have our view, that there is no Easterlin paradox and there probably never was. And the leading psychologists in the field, people like Danny Kahneman, have also adopted that view as well. That's not to say that we've convinced everyone. There was a first generation of happiness scholars who thought that this was going to be the path to enlightenment, among whom we've made less impact. It's a difficult situation. There are people who have sort of made it their careers to sell the idea of happiness as being a better metric. And of course that was all the more compelling when that metric also gave them exciting, new, and different policy implications. Our policy implications turn out to be: When you look at happiness you get pretty much the same thing as when you look at GDP; what economists have been doing all along isn't so bad. It's kind of a boring implication for some of them. Russ: Betsey, do you want to add anything to that? GuestS: Well, one of the things that happened is you take Easterlin's view and, you know, I started by giving the example of bringing running water to a village doesn't make people better off. That idea has always been a little bit hard for a lot of economists to stomach, particularly development economists. So, some folks came up with this idea that, well-- Russ: Threshold-- GuestS: Yeah, maybe Easterlin is right after a threshold. And that idea, as far as I can tell, came out of desperation to hang on to the Easterlin hypothesis without having to believe in the absurdity of not being able to make people who are at very low levels of survival better off by giving them more access to food and clean water and things like that. But in fact, no data set has ever shown that there is such a threshold where further increases in income no longer make you happier. Justin and I finally documented this in a paper that just came out in the AER Papers and Proceedings, just going through and showing that there is no such thing as a threshold. It is the case that the more money you get, the lower the incremental gain in happiness will be, the richer you are. In other words, the relationship is with the log (logarithm) of income, not the level of income. So, doubling your income provides similar increases in wellbeing regardless of where you are in the income spectrum. But obviously doubling the income of a millionaire takes a lot more dollars than doubling the income of somebody operating on $1000 a year, in income. Russ: Well, so, you so said--and the technical way to describe this is there is no satiation. People increasingly--it's actually not the exact same thing. No satiation means you prefer more. You might not be happier once you got it, would be a possibility. But you are saying that when people do get more, they do get happier. When I looked at one of your charts--I don't remember which one it was--but in one of your charts, happiness had three different measures. I think they were: Very Happy, Fairly Happy, and Not Very Happy. Something along those lines. And as you looked at a point in time in the United States, what struck me was: so there are three categories. And I'm going to give them numbers, 1, 2, 3. Three is very happy; 2 is somewhat happy, and 1 is not very happy. What struck me is as you look across people with higher and higher income, there are more and more people in that group--in the very happy group. What also struck me is that once you get past 20,000, there are very few people in the not-very-happy group. So that most Americans--and this was 2007, which was a particularly happy time--may not be very representative today, but I think it was 2007, but what it suggested was that most Americans are either pretty happy or very happy. But that metric is not going to really allow me to make the claim you just made, that you get happier and happier. Because we are all going to run up against that very-happy 3. And we are not going to be able to tease out subtle differences in how much happier we are if we have twice as much material comfort in 30 or 40 years as we do now. GuestW: So, Russ, this is one of those questions that I think is very important in theory and turns not out to matter in practice. It's one we worried a lot about until we confronted the data. A lot of the other questions instead say, how happy are you on a scale of 1 to 10? Or how satisfied are you with your life on a scale of 0 to 10? And what we see is that Americans are up around 7 or 8 out of 10. And people are rightly saying: at some point we're all going to be at Nirvana. We've done some calculations. And we're not going to go off the charts. We're not going to have to turn it up to 11, spinal tap[?] for a couple more millennia. Russ: Yeah, exactly. GuestW: So, the issue is worth worrying about but it's not for my children, and it's not for their children, either.
21:45Russ: Well, I can't leave this topic without mentioning Adam Smith. We did a great deal of--I did a 6-part series with my friend Dan Klein on The Theory of Moral Sentiments. And in the The Theory of Moral Sentiments Adam Smith says, many times, numerous times, many three, in detail, that the pursuit of money in the hopes of being happy is a delusion. Do you think he was wrong, then, wrong, or maybe right then but that now maybe things are different? Do you think the numbers you are looking at now are telling us something else about the human condition? GuestS: So, I think what this literature is missing or needs to start focusing on is what is it exactly that's driving the relationship between wellbeing and income. Because I don't actually think it's the income per se. I think it's the opportunities that income provides. Now, this is my view. There's a tiny bit of evidence on it. When we look at data across countries from Gallup where they provide some pretty rich other questions, you see things like when people have more income they are more likely to say they get treated with respect during their day. That they have choice over how they want to spend their day. That they have good tasting food to eat. All of this describes a life where have you have more income you make choices to put yourself in a better situation, in a situation where people around you are more respectful. In a situation where you have more control over your day. In a situation where you like what you are eating more. So, that way we can construct our lives when we have more opportunities, more capabilities, is I think a big important part of this. And it's correlated with having more income. But that doesn't mean anyone should look at our studies and think that if you are offered two different jobs, the higher-paying job is going to lead to greater wellbeing according to what Betsey and Justin find. I don't think that our studies can lead you to make that conclusion at all. It could be that the lower paying job comes with all kinds of nonpecuniary benefits that are going to be terrific for you. But what we find on average is that people with more income have more capabilities for making the kinds of choices in their lives that allow them to have more income and have other things. And those people on average are better off. Russ: I'm pretty confident that the three of us--that none of us have chosen the highest-paying job. I remember being an assistant professor--I think I'm older than both of you--I'm 58. I was an assistant professor in 1980; my job paid, I think it was a little under $19,000. I was quite happy. I'm happier now. And I certainly would have been miserable if I'd taken a--been the chief economist, worked on the economic staff of, say, an automobile company or worked as a government economist making--well, maybe about the same. But the Wall Street and corporate opportunities were certainly higher for me and I wasn't interested in them at all. And I was very happy making $18,600 or whatever it was. I feel like I'm happier now, but it really isn't because I have more money. It's because my job is a lot more interesting than when I was an assistant professor and I like what I'm doing more. My guess is that much of that correlation, to the extent that it's true, between happiness and income, is that third thing that you were talking about. It might not be opportunities; it might be the meaningfulness of your job; it might be the freedom to do certain things you couldn't do otherwise. So, I do think you are right; I think it's something else. But maybe not. I have a huge emotional reaction--it's funny, I'm sort of torn between more is preferred to less, versus we know money doesn't buy you happiness. So, I'm a little schizophrenic on this. GuestS: I have two sort of personal things on that. The first was when I graduated from college. I had an offer from at an investment bank; and I had an offer from the Federal Reserve Board. And the investment bank was going to pay I think about double what the Federal Reserve Board paid. But I took the job at the Federal Reserve Board because I knew I wanted to go to graduate school and I thought the Board of Governors was going to set me up better for graduate school. And I also knew that it wasn't going to require the kinds of hours that an investment bank was going to require. And I remember my mom saying: How can you turn down that kind of money? And I said: You know, Mom, I can always get a job waiting tables in the evening if I want to make more money, and I'd be happier having the diversity of activities in my day. So, you are right to say we don't all choose the thing that's going to pay us the most. And instead of waiting tables to make more money I actually did 20 hours a week of volunteer work with that spare time. It was the ability to make that choice, to be able to choose that because I wanted to, that I think contributed to that wellbeing. GuestW: Let me address--there's a distinction here. Two things can be true at the same time. It can be true that income or whatever, it's a [?] matters for happiness more than the Easterlin hypothesis suggests. It can also be true that income matters less than most people think it does. So, Kahneman and [?] talk about the focusing illusion. If I ask how much happier you'll be with another $1000, you think only about that $1000 and all the great things you could do with it, and you think it will be terrific. And then when you give people $1000 it's not actually so amazing. So, by no means is this saying that money is the be all and end all. Russ: Do you want to talk about any of the policy implications? You said it seems to suggest that growth is a good thing. GuestW: I'd state that in the negative. There are people who use the false Easterlin Hypothesis to claim that growth is a bad thing. There's no evidence in support of that claim. Now, there's enough double or triple negatives in that sentence you'll probably end up where you ended up. But I just want to be a little more careful. There's a correlation versus causation issue here, that's what I'm trying to signal there. GuestS: There is something else that comes out of our work that's important, which is that greater inequality does reduce aggregate wellbeing, and that's because of the fact that this relationship is between wellbeing and the log of income. If you take a dollar from a rich person and you give it to a poor person, the gain in wellbeing to the poor person is larger than the loss in wellbeing to the rich person. But there is a loss, and I think what's important is to know that redistribution does take from one person to give to another, but it's clear that you can increase aggregate wellbeing in your society by doing some of that redistribution. Russ: Yeah, I think inequality is very difficult. Tricky. Because obviously it depends what the source of it is in the story you just told. GuestS: Absolutely. What happens--if you can take without there being a leaky bucket, then everything I said goes through. Depending on how leaky you think the bucket is as you do that transfer can change everything. Russ: Yeah, I think it's striking how little we know reliably about the leakiness of that bucket. We're talking in a metaphor that you and I understand, but to make it a little clearer I think would be: there's two issues. There's the disincentive effects, the magnitude of how much or less people will work when you give them money or take money away from them; and there's the administrative costs. I think the administrative costs are pretty small. But I don't think we know as much about the incentive effects as we think we do. I find them very ideologically driven. I think people who tend to be in my camp of smaller government tend to think incentive effects of taxation are large. Whereas people on the other side who are more interventionist tend to think they are close to zero. I think those are close to religious beliefs. I don't think we have a very good, reliable estimate of that. GuestS: This is definitely not something that we would [?] but I take that point. GuestW: We can stay away from religion and politics if you like, Russ. Russ: Good. Excellent.
30:43Russ: Well, it's not going to be easy. I want to switch gears. I want to turn to Reinhart and Rogoff, which I think is fraught with ideology and religion. GuestW: I think it shouldn't be. Russ: Yeah, I agree with you. So let's talk about that. First, for people who have forgotten about it already or who never heard about it, if you are in the blogosphere, the punditsphere, it's a raging inferno. But I think for a lot of Americans, they've never heard of Reinhart and Rogoff. Even some who listen to this podcast--although we did talk with Carmen Reinhart about the book--this time it's a different book. The controversy is over some work they've done trying to measure the relationship between debt and growth. So, talk about what they found originally and what's been the source of the firestorm, at least for those who are in the kitchen. GuestW: So, they put together several data sets looking at the growth of public debt through time. The most prominent of which was a data set in the post-War period of 20 advanced economies. But separately, and this is separate from the whole firestorm, they put together data on the same countries over the past 200 years and also a set of developing economies as well, over the past 50 years. What they did was they looked at the level of GDP growth for each country on average when its public debt was high, medium, or low. What they found was GDP growth tends to be lower, the higher is government debt. That's just a fact. And that's what they threw out there. The policy claims are more contested and more complicated. The one claim would be: high public debt therefore causes low GDP growth, and therefore we need to stay away from high levels of public debt. That's a causal statement; it's one that I think the initial analysis doesn't support. They are simply looking at correlations. Where the firestorm broke was a very smart graduate student at U. Mass. Amherst, wrote to [?] and they sent graduate student their data; and he found literally an Excel error--that instead of averaging growth over 20 countries they averaged it over 15 countries. They literally just added the wrong numbers on the spreadsheet. There were two or three other issues that came up subsequently, beyond that Excel error. The Excel error is a clear error, to at the other end there are when you are running empirical analyses judgment calls that one must make. And reasonable people can disagree about them. And it turns out roughly speaking that the more egregious the error, like the Excel error, the less it actually matters for the bottom line. And the more we delve into these judgment calls--and both sides, I think, make reasonable judgments--the more material the different judgments become in terms of changing the result. At the end of the day, the folks from U. Mass. Amherst want to claim that they found terrible errors and that Reinhart and Rogoff's negative correlation between public debt and growth is wrong. That's another mis-statement. That even once you accept all of the corrections and amendments of the Amherst folks, it's still the case that countries during periods of high debt tend to be growing slower than during periods of low public debt. Now, there's a second issue, which is Reinhart and Rogoff talked a little bit about what happens when public debt is high. And they defined high as being above 90% of GDP. And somehow that became a threshold that politicians really like--we've got to keep public debt below 90% of GDP. The claim that there's some magic number like 90% beyond which economic growth starts to fall off precipitously does turn out to have been an incredibly fragile and almost certainly false claim. So, it's true that higher debt is correlated with lower growth. We also don't have any evidence that that's causal. And the initial claim by some that there's a magic number beyond which debt shouldn't go looks like it needs to be revised. GuestS: So, let me jump in and add a couple of clarifying points. So, first of all I think one of the things that has gotten lost in this debate is the unbelievably tremendous contribution that Reinhart and Rogoff has made to gather this data that no one had gathered. When we wrote our column on this, I can't tell you the number of people who said things like: Oh, an Excel error would get you a failing grade as an undergraduate. Russ: Get you fired as a contractor. GuestS: As if they had just downloaded some data off the Internet and then had accidentally added it up incorrectly. One afternoon's work gone wrong. Their contribution, it really has been putting together an amazing data set. It's been disappointing to see that completely overlooked. And then making it available for people to have this debate about what it means. The other thing is that, Justin says--I want to jump in when he says there's a correlation but we don't know how to interpret it. Sometimes that sounds really dismissive, like, of course you should interpret it as something that's causal; it's an important correlation. But in this case, there is sort of no real reason to think--there's lots of reasons to think why it wouldn't be causal or why there would be some third factor that's causal. Russ: Or why causation could run the other direction. And that wasn't a secret. Everybody--I [?] the word 'lately'--but everybody understood that this could have been caused by: Gee, when you are not growing very fast, you run up a lot of debt. GuestS: Exactly. And one of the important things they were looking at is growth and debt in the same year. So, this isn't like a pattern--as you have debt, what does your future growth trajectory look like. This was just: here's the raw data and here's the pattern we see. That there was, as Justin said, a mistake means that how slow the growth is during periods of high debt is not as low as they thought. Substantively the point remains. I don't think that there's something different to take away from that. GuestW: I think there's a deep issue here about how we communicate economics. We write very careful papers that we publish in obscure journals in the hope that almost no one will ever read them. If they are interesting enough then someone will pick them up and maybe write a 100-word op-ed about them. If that's interesting enough, then maybe someone like Carmen will be invited to give Congressional testimony. If that's interesting enough, that 5 minutes of testimony, then 15 seconds of that will be excerpted on the news that evening. And if that's interesting enough, another politician in Europe will stand up and say something for about two sentences without acknowledging Reinhart and Rogoff as the source. And through all of that process we end up losing a lot of the nuance along the way. So, there are people who certainly made claims that were unwarranted, that was the debt causing growth. I think it's also clear to say that serious Ph.D. economists generally stayed away from making that sort of error. GuestS: I will say that one lesson that all economists and Ph.D. students should take is that, it's very rare that your data gives you the precision to make a threshold claim. We just investigated thresholds in income and happiness data--it really takes quite precise data. The idea that they were going to have this 90% threshold seemed kind of silly on its face. And now that the data has been corrected, that threshold didn't stand up. But I think that the people who consumed the research took that threshold more seriously than the authors intended.
39:20Russ: The part I find remarkable about it--I think the point that, Justin you gave a narrative of how your work can get increasingly interesting to people. I think the problem is that we as economists like the idea that people are reading our work, so we do tend, I think--I think it's very difficult to avoid what one of you called, Betsey called, a rookie mistake. It's easy to make rookie mistakes because it's fun to be on the front page of the NY Times or inside the op-ed page. It's fun when a politician pays attention to you. Maybe it shouldn't be, but I think a lot of people have trouble avoiding the feeling of--it increases happiness. It gets you up there, because people are paying attention to you and sometimes that's a big change of pace. The part that I find hard about this is that: they made an error; as you said, the error didn't--it changed the quantitative nature of their finding, not the qualitative nature. We had lots of qualitative and quantitative evidence that it wasn't decisive anyway. Which is the thing I find so strange about this. Japan, I think their debt-to-GDP ratio is I think over 230%. They are not doing great; but they are not doing horribly. They can still borrow. They don't face the interest rates that other nations face, they have much lower interest rates. So, there's obviously an enormous amount of imprecision. And yet people are going around calling Reinhart and Rogoff murderers, because they engendered austerity--which kills people. Now, my claim is we haven't seen that much austerity. It's true, maybe they gave some cover to politicians to talk about austerity. But other than Greece, there aren't a lot of countries that are cutting spending dramatically in the face of this crisis. GuestW: I think I have two reactions to that. One is: we should assume in the absence of any other evidence that people are acting in good faith. People on both sides of the aisle. On the flip side, what we do matters. It's the reason I became an economist. Recessions do kill people. That's a fact of life. They harm people. So, we have a tremendous opportunity as economists to help shape the debate and that does come with huge responsibilities. I'm not sure any side of this debate has covered themselves in glory. In fact, I think quite the opposite. It's been a bad episode for economics; we've only looked worse in the public eye. But I do think it's important we take what we do extremely seriously. GuestS: The thing that I find the most silly about these claims that they are responsible for austerity and that they wouldn't have been if they hadn't had the mistake that they had in their paper is that those two things don't go together. As we said, you correct the mistake and you still find the claim; and therefore if the claim was being used, it still would have been used. So I don't think that it was made not much stronger. People used the research that they want to use to make the points they already had. You earlier mentioned this idea, politics and religion; people have these religious views. What they do is they go out and find research that backs them up. And there were people that had religious views who used Reinhart and Rogoff to back up those religious views. And I think if the Excel error hadn't been there, if they had done aggregation in a different way, I think that their paper would have been used in the exact same way. Russ: That's an interesting point. I don't really care about--I think the whole hullaballoo or brouhaha, kerfuffle, whatever you want to call it, tragedy, depending on your perspective--so I have not carefully read their policy work on this beyond their academic work, beyond the book. I know they have written a sentence that said 90%--I've seen this recently--isn't a "real threshold" such that at 90.1 you start to struggle and that at 89.9 everything is hunky dory, everything's fine. But some people probably did, and maybe they did--I don't really care; I'm not here to judge them or decide; the three of us are not a court of law here, a court of anything to figure out how culpable they are in this. But I think the general lesson is: Anybody who said that was a fool, based on the nature of the imprecision of the empirical work. But that doesn't mean that you shouldn't be wary about large levels of debt, that may come along and bite you in unexpected ways as a nation; and you find yourself unable to proceed. Which is what we just went through--the start of this financial crisis, a lot of it came from overly leveraged firms that thought everything's fine; and then all of a sudden--suddenly, unpredictably, it wasn't fine. So that's a good lesson to me. That's something you should be wary of. It doesn't mean you should never borrow, as an individual, as a firm, as a nation. But to say that, we now know, 90% is where it gets rocky, seems really silly. GuestS: Well, what's silly is the idea that there's one thing called 'debt.' If we borrow a bunch of money to send everybody in the country off to a fancy beach vacation, we should expect something different than if we borrow a bunch of money to lay Internet throughout the country, improve the highways, and fix our failing infrastructure in our school system. We should expect different outcomes from different types of spending and different types of spending are more justifiable when it comes to accumulating debt. Russ: I agree with that. Justin, you want to weigh in on that? GuestW: I always agree with Betsey, Russ. Russ: You want to add any nuance? GuestW: Now you want me to look her in the eye and tell her she was insufficiently nuanced? Russ: I want you to enhance her argument, perhaps. I don't know. GuestW: I think there's a subtle and important issue here about how we communicate economics. We can talk all day about the technical issues and how to run a regression right and what different forms of debt we could measure and what control variables should be there. But at the end of the day, many of the issues you were just talking about are fundamentally issues of economic communication, or sometimes miscommunication. How our ideas move from dusty journals into the policy debate. If that's your perspective, and it's certainly mine, then it actually makes folks doing like what you're doing, economic education through the podcast, actually really important and critical place. And it's something that in our universities we don't pay a lot of attention how to carefully communicate these ideas. But actually the few times we have influence it's only because we have been careful about how we communicate. Russ: I certainly agree with the last part.
46:40Russ: Let's turn to this issue we referred to earlier, which is replication. A lot of people are using this--I think Betsey said--when they see an axe, they are going to grind it. They just need to grind their own axes. So, one of the axes that's getting ground in the aftermath of this modest scandal is economics is garbage. Which, I have to admit, I have a certain sympathy for the idea that empirical work is often unreliable. But I think it's gone way too far in response to this Reinhart and Rogoff thing. People are now saying: it's all trash, it's full of mistakes. What do you think the lessons we ought to learn for just the error part, not this debt-growth issue, but just the fact that a prominent paper was found to have an error? GuestW: All right, so I'm going to be dealing with defense of economists. People will say we're bad, people will say we make mistakes, people say empirics are unreliable. I think I can admit all of those things and still admit we're better than the alternative. So, you know, the alternative is bloviating ideologues and the falls[?] we see on TV. And even mistake-ridden papers, I think do a better job than that. Now, that's a defense. I don't want to be defensive. I think there's a lot we can do. Our attempts to--there have been several large-scale experiments, people attempting to replicate economics papers, and the results have been unbelievably disappointing, ranging from people being unwilling to share data--which is not true of Reinhart and Rogoff--to results simply not replicating, to in fact an experience I had. I tried to replicate someone's results, found the opposite of what they found, and that person was in front of Congress giving testimony based on their original findings. Their wrong findings, a couple of weeks later. So, replication is critical to having people trust and believe our results. And we don't have a strong enough replication culture. GuestS: And I think the problem really lies with the journals, that don't want to publish replications. What I'd like to see is actually a complete overhaul of how we do it. Part of the problem is that you get a really, a nice long paper published in the American Economic Review (AER), and it'll turn out that there is some error. Then maybe it's important but it doesn't throw the whole paper out. So the journal editor thinks: I don't want to publish 10 pages on the mistake that was in this. Maybe we need to have short, some sort of new, short version of two-page replications that get published, whether it replicates or not. And every paper in the journal gets a shot at having a replication study published, even when it can replicate. So you can say, no, this paper was replicated by someone who submitted their two-page replication. It's a problem because we don't have an outlet. And there's lots of people who have stories of replicating papers, finding a mistake, and then having a hard time figuring out where they are going to place that paper. Because the original journal has decided that they don't want to take it. And not--if you are a young scholar, you do not want to do replications. It's not going to build your reputation. It's going to annoy people; and you are going to have a hard time figuring out where to get it published. GuestW: And the result of that is bad results stay. Wrong results stay published and people keep thinking wrong results are right. Russ: By the way, for listeners out there, the AER is the American Economic Review, perhaps the most prestigious mainstream journal in the field.
50:31Russ: Now, perhaps we can learn something from psychology. I interviewed Brian Nosek, psychologist, a while back. And psychology is going through a real crisis of confidence in itself--it's a horrible way to phrase it, but psychologists, I'll say wonder whether a lot of the results in their field that they've trusted for a long time may not be true. And they have two issues. They have a serious issue of replicability. They also have fraud. In recent, in the last year there have been some fraudulent--clearly fraudulent, not just errors of omission but fraudulent findings. And they are doing something about it. And it's relative--excuse me, 'they.' People. Certain people are trying to encourage replication, change the culture in the journals, create online places. It's absurd that the AER would say, you know, we don't want to give up 10 pages--in today's world, that's bizarre. They should have a website devoted to replication, perhaps, if they could find candidates for doing it. If people are interested. I think that's the big problem. GuestW: Yeah. I think what's going on in psychology is fascinating and it maybe hints at deeper problems in economics. Forget the Freud stuff, because we all know Freud is bad. That's simple. The real problem that psychologists have had is that many of them run pilot surveys or pilot experiments, and when they don't work out, they just throw them away. And when the do work out, they publish them. Russ: Yep. GuestW: You run 20 studies, 1 in 20 is going to be significant at the 5% level. And they just have no idea how much stuff got thrown out. Well, think about that problem applied to economics. We don't actually have to get undergraduates in the lab to run a pilot study. All we have to do is download the data and run a regression. It's very cheap for me to do that, and then if it doesn't work out, throw it away. Never report it. So, their replication problem is essentially an under-reporting problem; and our under-reporting problem is maybe even worse, because it's maybe cheaper and easier for us to be looking for different ways and sifting through the variables to try and find something that is significant. And so this is a different rationale from needing replication. That it could be that the first authors just got lucky and something that works in U.S. data, well a good way of checking it would be to see if it works in British data as well. And so I think there are two things here. One, we need--when I write papers now I try to write papers based not on one data set but on 5 or 6 or 10 or in some cases 20 different data sets, to see if something is true 20 different ways. And the other is a different view of what replication is. It's not just taking someone else's data; it's actually going out and taking someone else's idea and testing it in a different way. Russ: Correct. GuestW: And we need more of that as well in economics. GuestS: I want to add two things. One is that we talk about replication, and too many people miss it. It's a very hard word to understand. Because most of the time what I mean when I say 'replication' is the broader sense of replication. If you take the same question and you try to answer it in a different way with different data, are you getting roughly the same results or is everything very sensitive to the initial conditions the author chose, the data set they chose, specifications they used. But I also wanted to come back to another one of Justin's points. He said, Let's throw fraud out; everyone knows that's bad. But actually, I think it's not a coincidence that fraud is coming out when they are trying to think about validating each other's work. If you have a culture where no one is checking anyone, then you don't have--there is no incentive to not engage in fraud because you are very unlikely to get caught. Russ: Just your conscience. GuestS: Just your conscience. Russ: Which, I don't want to minimize that, but you said no incentive--it's a little strong. But I take your point. Even economists have a conscience. GuestW: Actually you can make a narrower claim, Russ. You can say even some economists have a conscience. GuestS: But when you--I think it's useful to have some incentives besides people's conscience. To know that we check each other's work. So, it doesn't matter if it's a purposeful mistake or if it's an accidental mistake, that if something isn't done right, if something is wrong, it's going to be found out. And I think that that's the right way to run a field and to have research move forward. And I think that Psychology is learning that, yes, it's shaking things up; but they'll be better off for it. GuestW: So, there is the Stiglerian response to this, which is we don't need regulation, that people will self-regulate here. And this is something Betsey and I do, which is every time we finish a research paper we publish both the data and the current[?] on our home page. Hopefully that makes us as authors somewhat more credible. But more to the point, it creates, it provides the material for would-be replicators to go out and have a look through the data and see what they really say. And maybe that's even a useful discipline on us when we are doing the research.
55:49Russ: Yeah. It seems some of this is going to be solved, to the extent it is solvable, through emergent norms that I hope propagate through our profession. It seems--one of the issues that has come up in the Reinhart-Rogoff thing--I hate to come back to it but it's relevant--is how much did they share their data? Did they--eventually they did provide a spreadsheet. But evidently, initially--some people have said, again I'm just following the balls and strikes of this so I don't want to be unfair to either side--but some people have claimed they only would share their data source. Which is not the same as the actual data. Because there are decisions that get made, numbers get thrown out; you have to decide what line in the chart to use when you go to the government data set. So, some people have claimed that they made it hard for people to replicate their work. And some people have defended them, saying, yeah, because they spent an enormous amount of work pulling this together, and just to give it away seems difficult. But it seems to me they ought to. I don't know whether they did or didn't; again, I'm not commenting on that. It seems to me you ought to give away your work, meaning you ought to provide the data in the form that you used it; you ought to show the decisions you made. And I think the biggest thing you need to do--we talked about this in the podcast with Nosek in the psychology literature--you really want to provide a video, a running tally of what you did. Not just a little something like, 'we tried different specifications that didn't change our results.' Give me a list of how many regressions you ran, the 700, with every combination you tried. Because I want to be in the kitchen to see just how the dish got made. And if you do that--of course you can't promise people do it honestly--but if you do it honestly then there's some real credibility. And to do it honestly means that you would reveal how many times you went picking those cherries. And it's really awkward how many times people run those regressions, it seems to me. GuestW: So, we're starting to see that in economics. But we see it in a slightly different way, which is that people increase specifying what they are going to do, particularly with the large-scale experiments. So, the Oregon Medicaid Experiment, run by-- Russ: Right, we just did two podcasts recently on those issues, very interesting-- GuestW: Before they got a single data source back they said: here's the regressions we are going to run, here are the tables we are going to present. And that seems to be the clearest way forward here. I think it's easier with experiments. With those of us who are looking at observational data that have been around for decades, building the trust that 'I only run regressions I told you I was going to run, I'm not sure anyone believes--I'm not quite sure how to do that. GuestS: So, I don't know what--I don't know anything about Reinhart and Rogoff's willingness to share the data. But I will say that I think it is a harder call, when you spend years building a data set and this is part of the contribution you are making, to give it away and have the paper you were going to write next, because you handed them all your work, is a bit frustrating. I'm certainly seeing that happening in the profession. I think we are wrestling with how do we both have transparency, but also let scholars, who, part of their contribution is building data, get the chance to reap the rewards of doing the work that they want to do with that data without someone scooping them on it.
59:18Russ: Well, just to come full circle on our equating of economics, ideology, religion, and politics: The Dead Sea Scrolls were kept close by the people who found them for a long time. And of course, listeners will know the Bootlegger and Baptist argument, which is that sometimes you do things for self-interested reasons that you cloak in high-minded reasons. So, I'm sure they justified it by saying these are fragile, precious artifacts that we can't let many people look at or they'll be destroyed. But as a result, a very small group of people were able to do the research. And so, though I sympathize with your point, Betsey, it seems to me that most of what we are doing here in economics isn't the Dead Sea Scrolls. I understand your point. There is a temptation to keep the data close because you want to get the next paper out of it. But I'd like to see our culture--which the rest of the world is moving to, in every other area of our life, that information is relatively accessible. It seems like a good thing. GuestS: I agree. But I do want to say that I worry about our not having the incentives in our profession for people doing original data work. Because not only--you are not only asking people to share it right away, but also I think we don't have sufficient appreciation. It's one of the reasons I led off, when we started the conversation on Reinhart and Rogoff, by saying: Don't forget their contribution was an amazing contribution of data collection. Because it is important to recognize when people do that or otherwise we are not going to have people do it. Russ: Yeah. That's a good point. We need a lot more of that in economics. We're pretty lazy. Because there's plenty of stuff we can do already. GuestS: Exactly. Russ: I agree with that. If there's a tradeoff there, that's the bottom line.
1:01:02Russ: I'd like to hear from each of you briefly--maybe you could go on for at least an hour on your own, but briefly--how the recent macroeconomic events of the last 5 years have changed your view of economics, or our profession. If at all. GuestW: What a good question. Russ: And let Betsey go first if you need to think about it. GuestW: All right, Bets. GuestS: I was really glad you were going first. Russ: Yeah, I'm just kidding. Toss a coin. GuestW: I would like to think that we've learned an appropriate humility. I'm not convinced that's true. The other thing is I made a prediction two or three years ago that's turned out to be very false. At the height of the crisis I saw all economists becoming, no matter what field they were, theorists, labor economists, everyone becoming much more interested in the macroeconomy and the ongoing policy debates of the day. And we had seen over the previous decade macroeconomics go off and become its own tribe, that stopped talking to the rest of economics. And so I thought welcoming macro back into the field would be healthy for all of us. And I'm not sure that's actually happened, although I remain hopeful it may. GuestS: So, I would actually come back, summarize my views on the whole Reinhart-Rogoff debate where that the most important thing when you deal with empirical results is that you keep an open mind and you revise your views as new evidence comes to light. And I think these last 5 years of looking at the economy have really--as Justin talks about--introduced the need for humility among economists. Thinking about what we do know and what we really don't know. But also the importance of being open-minded, revising our views, looking for evidence and being willing to say, this thing I thought was going to work isn't working. Or, this think I thought wouldn't work is working. And I wish that we could all be more open-minded to thinking about what we could do the economy for the United States, because I just look with horror at the number of people who are outside our labor market right now. Or I should say outside the labor market or inside the labor market but unemployed. There are just so many people that aren't able to make a positive contribution despite them having the capabilities of doing that. And I just think it's a terrible thing for society; it's a terrible thing for the individuals. And we should be considering all the different things, all the different possibilities for how we could make sure that everyone was able to make as robust a contribution as they would like to.