Adam Ozimek on the Power of Econometrics and Data
Feb 8 2016

Adam Ozimek of Moody's Analytics and blogger at Forbes talks with EconTalk host Russ Roberts about why economists change their minds or don't. Ozimek argues that economists make erratic but steady progress using econometrics and other forms of evidence to understand the impact of public policies such as the minimum wage or government stimulus. Roberts pushes back and discusses the role of ideology, the complexity of where our views come from and the potential for confirmation bias.

Noah Smith on Whether Economics is a Science
Noah Smith of Stony Brook University and writer at Bloomberg View talks with EconTalk host Russ Roberts about whether economics is a science in some sense of that word. How reliable are experiments in economics? What about the statistical analysis...
Robert Burton on Being Certain
Neurologist and author Robert Burton talks about his book, On Being Certain, with EconTalk host Russ Roberts. Burton explores our need for certainty and the challenge of being skeptical about what our brain tells us must be true. Where does...
Explore audio transcript, further reading that will help you delve deeper into this week’s episode, and vigorous conversations in the form of our comments section below.


Walter Clark
Feb 8 2016 at 12:21pm

There’s something missing in the Adam Ozimek interview (about halfway through) on minimum wage. In fact I never hear it discussed. It is the idea of a “natural” minimum wage. The most useful thing about recognizing a natural minimum wage is that it explains Card and Krueger’s success in moving economists toward government enforced minimums. Understanding the natural minimum might even be useful in arguing with Card and Krueger supporters.
I have never heard that the dollars per hour of this floor has ever been calculated. But that doesn’t mean its affect is absent. This natural minimum is a “feeling” that gives the left their target of $15/hr and even keeps them from proposing any higher dollars per hour rates. Ever wonder why they don’t propose even $20 an hour, let alone a hundred?

Here’s a suggestion for how to calculate the natural minimum:
The pain of being put out of work is tempered by the estimated dollars an hour equivalent of the current basket of benefits the worker can expect from entitlements. There is also the benefit of being home to take care of errands, the kids and time for video games. The dollars per hour equivalent for that would be more difficult to calculate than the basket of welfare benefits, but there is a third benefit that is even harder to calculate: it is the avoidance of drudgery of being employed in work at the bottom. These latter two benefits could be a huge factor which doesn’t just add to the effective minimum wage of entitlements but may be multiplying factors.
For example. I’m on a kind of welfare in the form of a pension plus social security. The dollars per hour is less than when I was working, yet to go back to work at the old company they would have to pay me more than double, before I would consider it. You hear numbers even higher from most retirees. And that’s for work that is far more rewarding than what the minimum workers experience.
If nothing else, recognizing there is “natural minimum” will explain why progressives do not consider “unemployment due to minimum wage hike” to be a negative consequence. It also means you cannot argue eliminating the minimum wage unless you also argue for moving relief over to the informal institutions. The profound difference between charity and welfare is the idea of “entitled.” When relief is from a monopoly provider, it is entitled. There is nowhere else they can go. If support is not a monopoly, a charity can use discretion in who and how they provide relief (the one relieved can always choose to go to another charity). Society provides relief but it is in the absence of the feeling of entitlement. That lowers the natural minimum wage. And only then do all the arguments Don Boudreaux and others have gathered about the tragedy of being out of work, make sense.

Jim Lear
Feb 8 2016 at 8:08pm

Great conversation. Russ, I think you’ve unfairly compared economics to science in the past. The end of the conversation hints that science is also a dismal science. What causes obesity? What causes mental illness? What causes any number of diseases, such as MS? Remember when settled science showed that trans fats were healthier than saturated fats? Settled science has probably killed more people than economics!

In engineering, we often have the luxury of analyzing quiescent systems, but sometimes we can measure system responses in the presence of noise. A simple example is the radio. There is all sorts of electro-magnetic noise in the atmosphere, yet we can transmit low power signals that can be received hundreds or thousands of miles away. The requirement is that the receiver be highly tuned for the transmission frequency.

Perhaps macro economics could use a frequency analysis approach that is used in engineering. E.g. inject a small signal at one frequency into monetary policy, and do a frequency analysis of the economic data. With enough data, that frequency may be detectable in the economic outputs.

Feb 8 2016 at 8:12pm

Great podcast. In my 20’s and 30’s I was a full on free trader and fairly neutral on immigration. Now that I’m in my 50’s, and have watched long-time friends and neighbors go through lay-offs, downsizing, “retraining” and outsourcing, I’m 100% ready to sign on to significant limitations on illegal and legal immigration, and big-time reduction of trade.

We have vast numbers of Americans dropping out of the labor force to go on disability and other programs. Many are veterans. They will spend the rest of their lives on the dole, without dignity, hoping for a 2% COLA next year from the SSA, and using their time trying to prove that they are sicker and more disabled than their doctor reports. And for what, flat panels that cost $600 instead of $900? Plastic junk from China that costs $5 instead of $7? It’s not worth it.

Look at what we have now, virtually no wage growth and a huge amount of shadow unemployment. This, even after eight years of ZIRP and $19trillion in debt. And huge trade imbalances.

Of course some have greatly benefitted. Larry Ellison owns half the homes on the beach in Malibu. Warren Buffet (but pity his poor secretary), Bill Gates, the Koch guys, and the facebook guy, are all rolling in dough. George Soros is licking his vile chops for the next nation-wide meltdown, prodded on by him. The big Wall Street boys are buying their third homes. But everyone else is unsettled and worried. College grads are living in basements. STEM professionals are training their Indian replacements and losing their jobs in their 40’s — after doing everything right and taking all the ‘hard classes.’

The funny thing is, the billionaire boys continue nearly unified in their view that MORE trade and immigration is good, no matter their politics. Gee, I wonder why.

Nope, I say lets try something “new” and return to pre-1965 immigration levels, and pare back NAFTA, TPP and the rest, and give it a go. Heck, with fracking and new green technologies, we can even make our own energy now. If nothing else, all the economists on college campuses and in government buildings in D.C. will actually get some “empirical” evidence of how the modern American economy performs when the USA goes it on its own for a while — all 320,000,000 of us.

Just imagine, the kid next door will mow your lawn and serve you a burger instead of illegal labor, the veteran can do construction and support his family, the corporation has to train the 40 year old STEM guy how to code in the new language instead of bringing in the 25 year old from India (and his wife, and parents and siblings and and and), and companies may have to compete for hard (but less than perfect) american workers and perhaps even offer apprenticeships (oh the horror) to meet internal demand.

Wow, sounds terrible . . . well, maybe for Larry, Warren, George and the boys.

And the talking heads can’t figure out this Trump appeal. Ha.

Jim Gatti
Feb 8 2016 at 10:29pm

Casey Mulligan’s work on the impact of shifts in the supply of public assistance on labor force participation might provide an alternative explanation for the failure of the labor market to adjust to trade related dislocations. It would be interesting to bring him back to discuss this issue.

Miguel Garces
Feb 9 2016 at 10:29am

Thanks Russ, as usual a very thought-provoking and interesting podcast. I do wonder if there is a deep contradiction between your views of knowledge in economics as science and in practice. On the one hand, you believe that knowledge often does not accumulate in economics, and yet the theories you use when you do want to make a claim in economic practice presume large amounts of knowledge-accumulation in markets. Why would economics be so influenced by ideology and “politics” whereas economic practice somehow avoids those same problems? Perhaps I should be looking to Hayek for the answer.

Feb 9 2016 at 10:35am

The Cash for Clunkers program would have been hilarious had it not cost us billions. To this day, I cannot believe that anyone seriously thought that it would have worked (sorry Mr Blinder).

As for evaluating macroeconomic “experiments” like stimulus and QE, one has to remember that the experiment is not over until the variables are completely accounted for.

QE will not be over when the Fed stops adding to QE, it will be over when its balance sheet is returned to pre-QE levels. The various stimuli and deficit spending cannot be evaluated until the costs of paying off the resultant debts are known, which is when they are retired.

I fear that the final analyses of these experiments will be far less generous than anyone currently can imagine.

Mark Passarella
Feb 9 2016 at 1:44pm

As usual, excellent wide ranging discussion
However I need to nitpick a little
You reference PSA testing as an example of changing ones mind in evidence but I think it’s more nuanced. A randomized trial was attempted called the PLCO which purported to show difference in survival between screened and non screened men in U.S. However there was heavy contamination between the 2 groups meaning those who were “unscreened” could have their PSA done somewhere else and if it were abnormal they had the option to “bail out” of the “unscreened” group. Not surprisingly, the data suggested no difference in survival.
A separate trial in Europe called the ERSCP was performed in Europe and had more strict criteria about cross contamination. It came to the conclusion that screening did save lives. The U.S. Public health task force performed a meta analysis of the available data in 2012 and concluded PSA did not save lives. However the data from ERSCP was not included to the surprise of many. Needless to say it’s a complicated issue. You point is well taken though, and urologists like myself look at this issue very differently than 10 years ago as a result of this data.

Thanks and I do thoroughly enjoy your show.

Russ Roberts
Feb 10 2016 at 12:06am

Mark Passarella,

I hadn’t heard that about the PSA evidence. Will look into it. I’ll be very interested in your reaction to next week’s guest, Adam Cifu on ending medical reversal, the phenomenon of a medical finding being reversed by new, better evidence. Non-trivial problem.

Feb 10 2016 at 3:01am

Interesting discussion and I get it that we human machines get very attached to our priors. I was taken aback, however, by Ozimek’s “encouraging” example of an economist re-evaluating his prior assumptions: Krugman changed his mind about the effects of the minimum wage. Krugman has been a partisan liberal for around 15 years! He seems to reject half what he said and wrote in his books and textbooks pre-2000. A conspicuous case, it seems to me, of man whose priors flipped. The minimum wage slots right into the priors of Krugman 2.0.

Feb 10 2016 at 7:19am

Having just had my annual physical last week, I can confirm that my doctor told me that they are no longer requiring PSA tests and that it was up to me if I wanted it.

Just to demonstrate our perverse economic incentives, I said sure, insurance was paying for it so why not…

Michael Barry
Feb 10 2016 at 8:02am

Two comments:
1. Given that there have been multiple Econtalks smugly laying out the Ricardian argument, where was the Econtalk on this:
“Russ: And I, by the way, would not rely on Ricardo at all, even though I wrote a book that tried to make Ricardo’s intuition clear. … But in general if you ask me do I believe that freer trade is good for a country: My answer now is–actually has changed. It’s a little bit like your change. It’s subtle. I still like trade. But the reason I like it isn’t quite why I liked it when I was 30 years old, say. And talking about how efficient trade was. I think that’s a really bad argument now. … So, I’ve changed my mind on that. Why? Because I did confront the fact–partly what you are talking about–that there are people who are hurt by it, and they are obvious. But, I still have a belief in trade. But I really can’t argue that that trade belief is scientific. Whereas, based on, say, detailed statistical analysis. It’s not.”
2. Mr. Ozimek seems like a reasonable and serious guy. But his boss is Mark Zandi who produced the talking point for pro-stimulus advocates — the one they cited in all their hearings. Where is the accountability for that.

Feb 10 2016 at 3:19pm

When can econometric analysis change someone’s mind?

I think Russ is correct that in many situations it is unlikely to be persuasive. But here are two clasees of situations where I think it can persuade:

1. With respect to the magnitude of an effect. You are extremely unlikely to convince me that an increase in the minimum wage has no effect on low-skill workers, but I am open to the possibility that the effect on employment may be small. Likewise I have found it interesting that work cited by pro-educational choice groups shows even positive effects to be smaller than I would have guessed, but I still think educational choice is a good thing.

2. To identify cases where there may be other effects worth trying to understand. In my mind the main effect of the Autor piece on effects of Trade with China is to indicate that there are probably elements of the US labor market that we don’t understand as well as we thought.

What do others think?

Joshua Woods
Feb 11 2016 at 8:55am

Great episode again. I thought the minimum wage discussion was a very interesting aspect of this discussion but it reminded me strongly of a column Bryan Caplan wrote over at Econlog, “The Myopic Empiricism of the Minimum wage”

Where he highlights that people of a liberal persuasion touting the benign nature of the minimum wage are proposing a highly inelastic labour demand curve. The fact that this contradicts all the research showing the low effect of mass immigration on native wages where a highly elastic labour demand curve is needed doesn’t seem to get much consideration. Even though probably the most famous work in both areas was done by the same man – David Card.

Maybe a good guest idea?

Feb 11 2016 at 9:55am

Mark Passarella already said what I wished to say on PSA. There is great data it says lives, if you are conspiracy minded you might say the government just didn’t like the data. A second related issue – primary care doctors, God bless them, don’t review data and studies about things like PSA. Those docs have gotten so beat down by government regulations, paperwork, and insurance companies that I would be shocked if 1 in 1000 has reviewed PSA data. They just don’t have the time as they are unfairly squeezed in every direction. They rely on secondary sources and summaries. Also, last time I checked many professional organizations and cancer organizations still recommend PSA for men.

Isn’t it possible the impact of free trade on highly regulated workers differs from the impact on completely unregulated workers (illegal immigrants) just as a function of the barriers to hiring. If the workers displaced from trade all had unregulated hiring might be much easier. While the unregulated workers might not disturb the local workers because they operate in a grey economy.

Finally, I continue to find the equivalence claim between economics and other sciences dubious. Physics went from Maxwell’s equations to an iphone in 150 years. In medicine we went from blood letting to immunotherapy for cancer. Type 1 diabetes was a 100% death sentence and now people can live near normal life expectancy with it. And on and on. In economics we still have people debating very basic macro questions and the reason is probably because economics is inextricably connected to politics and so it too often is serving those masters. As Rush pointed out we did Cash for Clunkers in 2009! Thats like having a debate in electrical engineering about whether we could build a radio and concluding we could not do it. All the evidence about people dying and advancing science is smoke and mirrors. Real sciences make clear substantial unquestioned progress, fake sciences not so much (and I am a half-fake scientist in epidemiology). We can discuss why that might be, buts its incredibly clear which camp macro-economics is in.

Mark Packard
Feb 11 2016 at 11:54am


I’ve enjoyed your discussions on this topic and, since my research (judgment in uncertainty) has led me to cross paths with a lot of cognitive research, I thought I’d share some thoughts that might help.

The ease of changing one’s mind depends upon the nature and magnitude of the neurocognitive structures that are built on top of the belief in question. That is, how much of your worldview depends on the belief in question?

The evolutionary epistemology of Karl Popper and Donald Campbell, who coincidentally were building from Hayek’s lesser-known The Sensory Order, offers a natural selection story of worldview construction. In short, we build our cognitive maps of the world from our own experience. As we experience various outcomes, we interpret those outcomes in light of all of our past experiences and our interpretation of those experiences. If an experience fits well with our existing cognitive representation of reality, the experience simply confirms and strengthens that interpretation.

If some new experience or information does not clearly conform to our expectations given our cognitive “small world representation” (SWR) of reality, we have three options:

(1) We may force-fit the new data to conform to our worldview. This is far and away the easiest option, cognitively speaking. By seeking out a way to interpret the new data that fits our SWR, we can go on our merry way with full confidence that we understand how the world works.

(2) We can ignore the data, treating it as an outlier. While this is a rather simple solution, it is actually rather difficult, cognitively speaking, as the mind is forced to acknowledge that the current SWR is flawed. If enough “outliers” occur the mind must accept that its current SWR must be altered.

(3) We can alter our worldview to conform to the new data. This is clearly the most painful option, its relative ease or difficulty depending on the magnitude of reconstruction required to make the data fit. Where a new SWR that conforms to the data simply requires a small tweak here or there of relatively inconsequential opinions, leaving the bulk of the cognitive map intact, reforming that opinion is relatively painless. However, where changing your mind on something on which much of your SWR is dependent must be altered, the cognitive map must undergo severe reconstruction. Such reconstruction would entail forfeiting the relative certainty of how you believe the world works, moving to a position of strong uncertainty, being very unsure of how or why things are. Such uncertainty is an uncomfortable state to be in, and is strictly avoided by most. It takes a brave person who is especially tolerant of uncertainty to be willing to reform their SWR (we often call these types “open-minded”).

For most, changing their mind on PSA, for example, is probably rather easy, simply because its truth one way or the other has little real impact on the overall SWR. Your worldview doesn’t depend on whether you should really get a prostate exam. However, for certain doctors that have spent much of their careers studying prostate cancer, such evidence must come at a far more severe impact on their SWR, and therefore their task of incorporating that data must be much more difficult. It would be understandable if some of them stuck to their guns in spite of the evidence.

For economists, I would suspect that certain peripheral theories are much more easy to change one’s mind about than evidence suggesting, for example, that Keyes’ general theory (or, in contrast, Mises’ theory) is completely wrong. Especially if the economists’ career work is built on supporting that theory. Those that are able and willing to do that ought to be applauded for their commitment to understanding reality, but it is hard to fault those that struggle to do so.

Mark Packard
PhD Candidate
University of Missouri

Feb 11 2016 at 5:51pm


One possible reason international trade is harder for the market to adjust to then you think it should be may be because a large portion of it is only efficient due to non-market factors: minimum-wage laws, environmental laws, subsidies, general business regulations, etc. As compared to the normal reason trade occurs: specialization, natural-resource distribution, and idea/technological sharing.

A local market with a different set rules then a foreign market cannot change as fast because it has to adapt to not only the new circumstance (which occurs in all disruptions) but with a market with a completely different set of rules. It may no be able to effectively adapt if the rules governing the two markets are different enough.

Feb 12 2016 at 12:24am

~ 00:41 Russ Roberts said, “I’m just a little bit skeptical—well actually a lot skeptical—about the power of sophisticated econometric techniques to cut through complexity and to hold everything else constant.”

Dr. Robert’s statement reminded me of a quote from one of his pervious guest’s books, which helped me come to terms with the ‘hold everything else constant’ problem.

“The principal challenge facing masters of ‘metrics is elimination of selection bias that arises from unobserved differences. Experimental random assignment eliminates selection bias.” (Pg 11-12, Mastering ‘Metrics, Angrist & Pischke).

Angrist is referring to perfect randomization and says, without reservation, that it is the gold standard for statistics. After saying that, he writes a whole book on how to structure experiments that are not perfectly randomized. Intentional irony aside, his point about perfect randomization, I think, is that holding everything else constant is not necessary so long as everything else can be made equal. Apparently perfect randomization makes everything else equal.

On the other hand, if Dr. Roberts is talking about holding everything constant for comparison between studies, then I think he correctly identified the nearly universal flaw in our current understanding of statistics and the reason macroeconomics fails as a science. Statistics produce averages. Averages are ratios. Ratios are fractions. Fractions have numerators and denominators. It is not appropriate to compare averages [which are fractions] unless we know for sure that their denominators are the same. The fact that we write averages as a single number, rather than a fraction, is the primary cause of this error. As Daniel Kahneman is fond of saying, “what you see is all there is.”

Coincidentally, prices are ratios. Prices are generally written as a single number rather than a fraction. I personally suspect this failure to recognize and treat prices like fractions is a major—if not THE major—cause of business cycle fluctuations and the reason monetary policy explains the boom bust cycle better than any other theory. Inflation and deflation are changes in the denominator.

Robert Swan
Feb 12 2016 at 5:39pm

Adam Ozimek rightly describes himself as an optimist when he puts economics on a footing comparable to other sciences. As I said in a comment a while back, economics should have one foot in the mathematics camp (game theory, etc.) and the other foot in the engineering camp — trying to apply/refine theories as they are tested in the real world. There is no such division today, and it carries a prop which would be better jettisoned — something akin to astrology.

It seemed fair enough, though, to compare it with medicine, which has its own pure, applied and mystical strands.

Mark Passarella:
On PSA screening you mention a study which concluded … “that screening did save lives.” In itself, that doesn’t really say that a well informed health system will promote screening. While the PSA test itself is benign, if you act on a positive, the follow-up is much more likely to harm you than save you. Even on true positives, the standard line is that far more men die with prostate cancer than die of it.

Looking forward to the upcoming podast on (the “cesspool” of) epidemiology.

Daniel Barkalow
Feb 12 2016 at 7:37pm

It seems to me that, if you compare economics to physics, the phenomena in economics are more complicated, while the models are simpler. Asking “Does the minimum wage hurt employment?” is a bit like asking “Do objects float?” Surely any answer that doesn’t take into account at least three factors is going to be wrong in some situations for the physics question, and I don’t imagine that the correct answer to the economics one is any simpler.

This suggests that we should expect to see many empirical studies with contradictory results, and the process of examining what’s different each time is actually necessary to eventually build a theory that isn’t clearly inadequate, and that theory will still not be able to answer questions that don’t specify the situation in enough detail, but will at least say what you need to ask and plug in to the theory in order to get an answer.

Robert Swan
Feb 15 2016 at 5:12pm

Listened again and one or two more thoughts (knowing just how eager everyone is to hear them!).

I was reminded of a late ’80s cartoon of Paul Keating (then Australia’s Treasurer). At the time he was keen on talking didactically about the ‘J’ curve — things would go backwards for a while, but then you could expect wonderful growth. The cartoon was drawn after some unexpectedly bad figures came out. It was three panels, all with Keating in academic gown and mortarboard. The first had him pointing to the blackboard with the ‘J’ curve. The second had him looking at the latest treasury figures. The third had him back in front of the blackboard telling us about the ‘W’ curve.

So “cash for clunkers” was a bad idea — we won’t go there again, will we? Well, as Russ pointed out, it was tried before with the wasteful slaughtering of pigs. I have my doubts that a comparable scheme would have been entertained during (say) the Reagan administration. Likewise, Keynsian economic theory seems more to ebb and flow than to actually advance. Rather than progress “one death at a time”, it seems plausible that economics simply loses its memories one death at a time.

Is Ozimek’s ‘J’ curve of progress actually just a ‘W’ curve, always moving, but getting nowhere?

Michael McEvoy
Feb 22 2016 at 12:53pm

to Kevin and Mark Pasarella – Thanks for the shout out to us primary care docs; I am one of them. While I do rely on summaries, bullet points , etc. , I do believe that the best primary docs make it their business to dig a bit below the surface on reviews of screening methods . I am , like Russ, a bit of a skeptic by nature . Thus all through the 1990s I was growing suspicious of our aggressive approach to prostate cancer. My suspicions grew until I found them crystallized by Mark Scholz in his book, The Invasion of the Prostate Snatchers. Mark does a great job explaining many of the issues surrounding screening , diagnosis and therapy. Yet curiously enough, he does not come down completely against PSA screening use. His position is nuanced and a bit too much to elaborate here but I do encourage those interested to look at his very accessible blog ( same title – Prostate Snatchers) BTW, the choice of the Prostate Snatchers term is likely a publishing and promotion gimmick, i.e. a little over the top.

Comments are closed.


EconTalk Extra, conversation starters for this podcast episode:

This week's guest:

This week's focus:

Additional ideas and people mentioned in this podcast episode:

A Few More Readings and Background Resources:

A Few More EconTalk Podcast Episodes:



Podcast Episode Highlights
0:33Intro. [Recording date: January 26, 2016.] Russ: Our conversation today is something of a followup to the recent EconTalk episode with Noah Smith. And in the aftermath of that conversation you wrote a piece; and the title was, "Can Economics Change Your Mind?" which was one of the topics that Noah and I had talked about. And I found your take to be very provocative; and it helped me think more clearly about some of the issues that were raised in the Noah Smith episode that I was either confused about or confused you, the listeners, about. So I wanted to follow up and go a little deeper. So, summarize the argument that you made in your piece. And you asked for some responses from readers; and you got some interesting responses. So talk about the piece itself and what kind of response you got. Guest: Sure. So, to answer the question of how empirical economics is or how ideological it is, I thought it would be useful to think about: is evidence actually changing our minds? Because if it wasn't, that would be a really good sign that it was ideology that was operating. And I sort of started with reflecting on myself in areas where evidence, empirical, especially regression analysis and the more complex statistical stuff--that you tend to be skeptical of, and other skeptics tend to downplay[?]--where that's changed my mind. And one big example that came up for me, and this is, you know, I wrote about in the article, is the work by David Autor and David Dorn and Gordon Hanson, and others have done work on this, on the effects of trade with China on the United States and trade more broadly. This is an area where, I think with a lot of economists, I thought that the costs of more trade exposure were going to be relatively temporary, relatively quick. I thought the economy was going to reallocate; people were going to find new jobs; and there would be winners and losers from trade but the costs would be dissipated relatively quickly in our economy. Especially given how flexible and dynamic the U.S. labor markets are. But Dorn's work--and Autor--Dorn and Autor and Hanson, their work looked at really the micro evidence and what's happened to areas that were more exposed to trade with China. And what they found was that adjustments have taken a long time. And they've been much bigger than you might have expected, especially compared to what economists were thinking in the 1990s. And the evidence was good. I mean, they looked at supply shocks from China and how this--their increase in trade has been relatively exogenous to U.S. demand-side factors. And so it was really sort of in the natural experiment vein in that it wasn't driven by U.S.-specific factors. And they did very careful measurement and econometric techniques that really tried to isolate that. And they found that what's happened is that people that live in these areas exposed to trade have dropped down in the labor market. They've gone onto government assistance programs. And not just temporary ones but more permanent ones like disability. And this has really made me think that the costs are actually much larger than what we would have expected, given the flexible U.S. labor markets. And, you know, that was a pretty big thing that changed my mind about--and I didn't do it lightly. And didn't discard the theory lightly. But the empirical work was really, it was really well done, and there was other evidence that followed it. It made it sort of hard to, hard to ignore. Russ: So, can I comment on that for a minute? Guest: Yes. Russ: And I apologize. I did not read that paper; and I will look at it after this episode; [?] have something about it on EconTalk on our website. But, it's an interesting point. It's a subtle point you are making. And I, like you, I'm an advocate for free trade. I don't know if you are an advocate, but I also feel that in general, trade--the response to trade in labor markets tends to be fairly quick in the United States; maybe not so much elsewhere. But our labor markets are more flexible. One way to deal with that evidence is to imagine that perhaps you've overestimated the flexibility of U.S. labor markets. It's not so much that trade has--the nature of trade has changed--or that you learned about the nature of trade. But did you wonder about the possibility that maybe labor markets in general--not just in response to trade, but other types of domestic economic changes--are not as responsive as maybe you thought? Or that policies have changed that have made them less responsive? Guest: Yeah. I think it does say something about U.S. labor markets in general. And that sort of raised the question. But I don't think it's a call for pessimism that all changes in the economy are going to be more costly. And a great issue to contrast this with is immigration. Which is another issue where economists have been relatively positive about the U.S. labor market's ability to adjust to. And the best micro evidence over time has sort of confirmed this. And if you look at similar approaches that focus on what are the local labor market effects of immigration, they haven't been costly; they haven't taken a long time to adjust. And native workers haven't been harmed by it. Russ: Not everybody agrees with that. Guest: Well, yeah. I do think that there's broad agreement that the local labor effects have been relatively positive. I think that you find other people who say, 'Well, you have to sort of take the general equilibrium into consideration' and that some native workers are going to move and relocate to other areas. And they more so disagree that the total effect in the United States, you can't look at local estimates; you have to look at U.S. national estimates. And, you know, that's [?] have; but I do think that the relatively strong consensus that the local labor market impacts haven't been very negative.
6:59Russ: Well, the reason I was thinking about it is that, I was thinking about housing workers, construction workers, either in housing or just general types of construction. If you look at the post-Great Recession housing numbers, and manufacturing, the manufacturing numbers are really--they were hit--they've been being hit in the long term by trade with China and productivity and technological innovation that's made workers--lowered the demand for workers. So, manufacturing has been shrinking very steadily over time; and then the Recession comes. That really hits it; that reinforces that secular trends. And at the same time, in the housing/construction market--of course, we built a lot of houses in the first part of the 2000s, and then that just died. And so we had a huge number of workers who were out of work. And if you'd gone back, say, you know, to the early 1990s, I think my crude estimate of the analysis in the post-Cold War situation where a lot of defense workers had to find new jobs, they did pretty--the labor market worked pretty well. It didn't seem to work very well this last time. A lot of people dropped out. We are not, I think, fully understanding of why. So, I raised the point partly to preserve my own belief in trade--of course--my own bias, my own prior. But I'm curious; I was kind of pushing you on that. Is that--do you think about that at all? Guest: Yeah. Absolutely. I think there's a really big question about the deeper parameters at work here and what is it about this trade? Was it just that it was large? That it was geographically concentrated? Because, like you said, the economy has reallocated in the past to different structural shocks. And something about this, it's not adjusting very well. It might be the case that the effect is bigger here. And I certainly think that the effect to trade is bigger than the effect to manufacturing and work from all tours[?] also suggested that it's bigger than the effect of automation. So there's something about the effect of trade that's been bigger. And it may just be because it's larger and it's more concentrated. Or it may be because it happened in such a distinct sort of exogenous shock that it's more easily identified than the others. But I tend to think that there's something unique about the trade exposure to low--trade exposure especially to low-skilled countries that's had a bigger impact than other labor market shocks than we've seen. But, to make you feel better--which I think you should still feel good about trade--I remain very strongly pro-trade. I think the benefits still outweigh the costs. And importantly I think the economic consensus is still really strong. If you look at the U. of Chicago--they do a survey of economists, and they still agree, resounding consensus, that free trade can benefit the average person in the United States. I don't think that that consensus is weakened. I just think we have to worry a little bit more about the adjustment costs. And also think about what kind of policies we might do to respond. Because--you know, doing nothing is more costly than we thought. Which lowers the cost of doing something.
10:24Russ: So, let's stick with trade for a minute, even though this isn't where I expected to go--because I think it's such an interesting issue. You haven't dented my love of trade or belief that trade is good for people. But I love the way you summarized that U. of Chicago Survey Response. You said, 'There's a consensus that trade is still good for the average person.' Why do we believe that? I believe that. But why? And there is some empirical evidence for it. There are studies that look across countries that have freer trade versus less free trade. But I would argue that most of the economists who believe in trade as a positive economic force for that average person have a theoretical construct in their mind--that, it's confirmed, not by sophisticated evidence, but by what we might call casual evidence. What we might call--it's not anecdotal. It's that the things we observe in the world tend to confirm our overall world view about the power of incentives, the power of competition, say. The role of low prices, and just an arithmetic that it frees up income for people to buy more of other things. But we believe, most of us do, who are free traders, that competition drives prices lower; and that helps consumers; and therefore the average person benefits, even though some workers are going to be harmed by it. And I think that belief is very widespread among economists. But it's an interesting thing to think about why that is. It's not that somebody did an experiment. Although we could do one. Or we could imagine ones that would have a more--a country would get cut off from trade, as they are in an embargo or war. But of course there's other stuff going on there. But I think most of us who like trade, who think it's good for America or good for the average person, have some theoretical construct in mind. And it is supported not by real--what we would call empirical or scientific evidence--but by, what I would call casual evidence. Do you agree with that? Guest: Yeah, I do agree with that broadly. There's a lot of general, slow accumulation of historical experience. There's--I think you can't dismiss the power of Ricardo's basic model of comparative advantage. I think those remain very influential. But I think there is a role for empirical evidence here, and it's a role that's often neglected. Evidence doesn't just show you what is true; but lack of evidence shows you what hasn't been disproved. Russ: Explain. Guest: So, you could easily imagine a case where the benefits to trade--there was detailed microeconomic evidence that it wasn't showing up: that producers weren't actually lowering their prices and the prices weren't being passed on to consumers. You know: that the set of goods available to consumers--one of the big benefits of trade is variety, variety of things that we can consume--you can imagine evidence that shows that wasn't increasing. And so one of the reasons that we still tell the story, and we still believe that trade is beneficial is that absence of evidence disproving it. Russ: Right; but people who are skeptical about trade, and there are quite a few of them, some of them running for President these days--we're not going to get into that--but a lot of people in the public sphere, not just economists--there are economists who are skeptical about trade. But most economists, as you say, there's pretty much a consensus in favor of it. But there are a lot of smart people who are not part of that consensus who aren't[?] economists. And they'll point out things like: 'Well, in the 1880s, the United States was growing very quickly and we were very protectionist and that was really necessary; and that's probably good today for smaller countries.' I disagree, but that's what they'll argue. They claim to have evidence. They'll try to show you correlations between lost jobs and trade deficits; they'll do micro-analysis that each dollar of the trade deficit costs so many jobs and puts so many people out of work. And I believe good arguments against those positions. But I think it's fascinating how hard it is to really make the case. And I, by the way, would not rely on Ricardo at all, even though I wrote a book that tried to make Ricardo's intuition clear. And, I mean, the truth is, Ricardo typically--not typically, actually himself, was talking about a 2 by 2 case: two countries, two goods. We now live in an n-country, n-good world. Lots of countries, lots of goods. You could argue, a lot of people argue those assumptions don't hold any more; didn't really hold then. I think about how hard it is to actually get real evidence on it. And I think it's very difficult. I mean, I remember a reporter once asking me--and I may have told this story before on EconTalk--a reporter once asked me: How did many jobs did NAFTA (North American Free Trade Agreement ) create or save? And I said, 'I have no idea.' And he said, 'Well, what do you mean, you have no idea?' And I said, 'I don't think that's measureable.' And he said, 'Don't we know when factories move to Mexico?' I said, 'Sure. But we don't know how many factories get started because Americans get lower prices because of trade with Mexico that allows them to free up income to spend on other things. Those other things, those factories expand; those factories hire more people.' And he said, 'Well, what's the answer?' I said, 'Well, I told you: I don't know.' He said, 'You're ducking the question.' I said, 'I'm not ducking the question. You just don't like my answer to the question.' He said, 'But you're a professional economist.' And I said, 'You expect something of a professional economist that I don't think we can provide.' So, I said: I'm still comfortable arguing for free trade, even though--and let's put to the side whether NAFTA is really a free-trade agreement and all the complications of actual legislation. But in general if you ask me do I believe that freer trade is good for a country: My answer now is--actually has changed. It's a little bit like your change. It's subtle. I still like trade. But the reason I like it isn't quite why I liked it when I was 30 years old, say. And talking about how efficient trade was. I think that's a really bad argument now. I think many economists use that argument. And I concede now that trade hurts a lot of people who were competing with foreign products. But for me, the real proof of trade's benefits is the dynamism that it introduces into an economy and how it helps, even over generations, to transform people's lives in creative and flourishing ways. Not just that we are richer or we make more money. So, I've changed my mind on that. Why? Because I did confront the fact--partly what you are talking about--that there are people who are hurt by it, and they are obvious. But, I still have a belief in trade. But I really can't argue that that trade belief is scientific. Whereas, based on, say, detailed statistical analysis. It's not. Guest: Well, again, I would say-- Russ: I don't think I'm alone, by the way. That's all. I want to make that clear. Guest: I agree with you for the role of, the important role of history and what you call casual evidence; and stories. And I think that it's important to remain pluralistic with regard to how we understand the economy and how we understand the world. But I do think that if there were micro evidence disputing some of those benefits, it would erode the consensus and support of trade. And part of the reason that there isn't micro evidence is because that the stories are true. And I think that that absence of evidence is supporting of the consensus.
18:09Russ: Well, let me take a domestic example, again. Which is--I'll pick something that I think is very contentious in the public debate. Not so much among economists, but generally. And that is: Are big box stores, discounters such as Wal-Mart and Home Depot, are those good for America? And a lot of people believe--I think they are wrong--but a lot of people believe that the micro evidence clearly shows they are bad. Because they are low-paying jobs, the people who work in those stores. And that's their micro evidence. And we have to argue, those of us on the other side--and I'll let you state your side when I'm done--but the other side have to argue, that's true; but those people have jobs now because of big-box stores. The demand for their skill level. It's good that there's a demand for their skill level. And then you get all the benefits, the people who get lower prices, because these improvements, retailing, have made it less costly. And those costs have been passed on, in the form of lower prices to consumers. Their standard of living is therefore higher. But the people on the other side, they have lots of micro evidence. I just don't find it convincing. Guest: Yeah. I mean, I think that the micro of it in some of these issues is kind of interesting. You have--a relatively persuasive instrumental variable in the distance of stores from WalMart. And I remember when that came out, from Wal-Mart's headquarters in Bentonville, Arkansas, I remember when that came out, I think Arin Dube and David Neumark ironically were on the same side of this issue in independent studies. But then there were people who came back to that and said, 'Well, look, if you take this instrument and you look at the change in manufacturing employment it implies some sort of effect there.' And they sort of--they--it wasn't a very robust instrument. And so, I'm happy to call that, I think, fairly, empirically a draw. Or not very persuasive in either direction. I don't think that that was resolved much by the empirical literature. I think what we have there is an absence of strong evidence of job losses. And so what we are left with is the impacts on productivity. And I think that one important lesson from data and from evidence is that a lot of the growth in productivity in the United States, aggregate productivity throughout the 1990s, came from the retail sector. And it came from logistics, and it came from companies like Wal-Mart who made retail more productive. And so I think that's another really big, important body of evidence that we can point to and say: Look, on WalMart you've got arguments for and against the job effect and I can agree that there are cases made on both sides and that the data and methods proposed so far haven't yet resolved this. But I think data elsewhere suggest that there are benefits that are an order of magnitude more important--of greater importance--that suggest that productivity in retail is what matters. And of course I mean that's common sense to an economist; but we have the data there to show it.
21:12Russ: So, I want to shift gears a little bit. Because you raised something that I think is really important, that Arnold Kling has written about, which is: Sometimes you can change your mind and you can come to a different opinion or you can accept a study; but you don't always think about what else is implied by the acceptance of that. You just sort of say, 'Well, that result makes sense to me.' Or, 'I like this study,' or 'I like the way they did that.' And yet people, I think, struggle to cope with the wider implication. So, I want to use the minimum wage, again, which we've talked about on the program recently. A lot of studies were done that showed that the minimum wage doesn't much of an effect, or any, on employment. And Paul Krugman, in response to your column, used that as an example of one of the things he's changed on: used to believe that labor market effects were important from the minimum wage increasing, but now he's, based on the empirical evidence, he's come to change his mind. Right? So, here's the question I have. And you can take a stab at this. I wish Paul were on the show. He's not. But there are a lot of people who have had that view. They've "changed their mind.' But the question is: What else does that imply? If you think that increases in the minimum wage don't affect labor market action and employment, doesn't it force you to accept some other things? Other changes? Wouldn't you look for other kinds of evidence then of how firms would respond to incentives, if you are going to believe the minimum wage doesn't have much of an effect? Guest: Yeah. I mean, absolutely. I think that that's a big missing part of the debate right now. I'll talk about that in a second, but just to make a big, sort of general point about the minimum wage, I agree with Paul Krugman's read that the evidence has moved him and other economists. And even though I disagree with them about the evidence, I find that heartening. I really do. I find that to be a positive sign for the field that they can be moved by evidence. Even if I think the evidence doesn't suggest exactly the same thing they do, I have to admit that the evidence is good. The work that's been done on the minimum wage showing no effects--it's not bad. It's not bad research. It's not like, deeply and deeply flawed. Russ: No, no. It's creative and interesting. Guest: Exactly. And there's an element of persuasiveness to it. Now, I think that there's lots of other evidence that suggests that maybe not--that what they are looking at might not be quite right. You have individual level data coming out that tracks individuals over time; and it suggests that while these groups we are looking at that we think of as being highly impacted by the minimum wage--teenagers and fast food workers--it might not be targeted well enough. And that if you track individuals over time it shows a bigger impact. And you have other research suggesting that the problem is that the research focuses on levels instead of growth rates. And if you focus on growth rates in employment, you find a much stronger obvious effect. And then you've got research on prices. You've got research from [?] and Ross[?] showing the effect of population shifts. And it's a really vibrant--especially in the last 5 years--growing field where we are learning a lot. And I don't think the research stops at the cross-border, natural experiment estimates. But I understand why people who see that study have changed their mind. I do think that if we fast forward 5 years down the road that the field is going to be moving back towards the minimum wage having a negative effect. And that's because of this new persuasive evidence that continues to flow. So, it's sort of--it's bad news for the workers who are going to lose their jobs because minimum wages are going up. It's bad news for right now, the state of the field I think is a little bit not supported by I think the best evidence. But I think it's good news in terms of economists aren't, you know, extremely so wedded to their ideological priors that they can't be moved on the data. And I understand why they feel less[?] moved. It's a nuanced-enough question. It's the research on the other side is good enough. I understand even though I disagree why the field has moved somewhat on this. Russ: Yeah; of course it's a very tough empirical challenge, even if you believe in empirical techniques. Because minimum wage workers are a very small portion of the labor force--something around 5% or less. And so, trying to tease out the impact on them from this one change, when there are a lot of other things going on, is very hard to do. Which means it always is a possibility that you haven't measured it correctly. And the challenges of empirical work that are out there become serious. Guest: But I would be a little more optimistic than that. Because I don't think it's a fundamentally an unsolvable problem where one study says this, one study says the complete opposite; and we haven't moved forward. Instead, I think the research that comes out looks at slightly different angles, slightly different adjustment mechanisms. And we know so much more about the minimum wage than we did 10 years ago. And we know about--if you look in this place you see it, if you look in that place you don't. And what sort of confounding variables you want to control for. Looking at the literature closely, it doesn't look like a draw where two sides just lob evidence back and forth. It looks like progress to me, where we are getting a better understanding of what happens, and we are using better data and better approaches all the time. So I am optimistic that that's one area where in 5-10 years we are going to have a stronger consensus; we are going to have a better consensus. And it looks to me like the consensus is going to be that the minimum wage causes job loss. Russ: Well, we'll talk again in 5 or 10 years, to see if that's true. I'd like to think that that's true. And maybe it will be. So, I guess we'll find out.
27:32Russ: But I do think, even that level of agreement, if it does happen, is going to ignore some of the subtler impacts of the minimum wage--which inevitably are going to be missed. I think one of the tragedies of the literature in this field is the focus on unemployment. Which is--it's really important, no doubt about it. But there are a lot of other things that happen on the job that are affected by the minimum wage, other margins of choice that employers and employees make, such as on the job training, civility, kindness. And getting started. That go beyond the job, whether you are employed or not. And so those things have sort of been lost in the discussion. And I think they are not unimportant. They may be extremely important. So, even, if the job loss was small, it could be that there are other losses occurring because people are not getting on-the-job training that they used to get. That they would get in a competitive market if they weren't being--if the price weren't being distorted by the minimum wage. But those are going to be very hard to tease out. Guest: Well, you do--it's not completely ignored. I mean, Barry Hirsch has a really interesting paper on the Fast Food industry; I think Georgia where he did a survey and he looked at--yes--the employers, what their margins of adjustment were. And so we have, we have, what they did, whether they raised prices, whether they increased training, decreased training kinds of things. I don't remember all the specific categories but it[?he?] is something that is being looked at. And I also think better data can help us answer these kinds of questions. And it hasn't been done yet. But we have--one of the reasons to be an optimist about empirical evidence is just the growing size and, you know, quality of the data that we have available. And you can conceivably look at data from the Social Security Administration, data from the IRS [Internal Revenue Service], and track workers over a long periods of time and see what the long run impacts of being exposed to minimum wage when you are young are. And--it's not something that I think is, like, fundamentally unanswerable and will never be resolved by data. I think it's more about--just about enough time till we get there. Just a matter of time till economists are looking there, more.
29:43Russ: Okay. Well, let's move from micro to macro. Let's talk about the stimulus. You gave a great example to me--you didn't mean to, but you gave a great example of the argument over stimulus and fiscal policy, when you said, 'It's not like just one of these cases where people cases where people just lob bombs back and forth'--saying, you know, different data, different studies. And I want to use as a case study--I've used different pieces of it before but I want to use this one a little bit different than I have in the past. Which is, that in the 1970s, in the aftermath of stagflation, and in the 1980s, a lot of economists got very skeptical about demand-side policies such as fiscal stimulus as a way to get economies out of recession. And I've read this--I think it's true. I lived through it, sort of. I'm not a macroeconomist formally. But I kept an eye on it. I think a lot of economists in the field of macro said, 'We've learned that that's not the way to fight recessions.' Then, we got the Great Recession of 2008. And all of a sudden there is a "consensus"--I don't think it's quite a consensus, but all of a sudden a lot of economists thought that the Stimulus was a good idea. And think so now, in retrospect. But, a lot of them don't. A lot of them think it didn't help at all. Or was even negative. They both have empirical evidence on their side. They sometimes write as if it's open and shut: that their case is obviously right. I don't think that's true on either side. I don't think we've made a lot of progress there. What are your thoughts? Guest: Macro is definitely harder. There is no doubt about that. There is less data. It's harder to isolate partial equilibrium. You have less degrees of freedom, and time series is tougher to prove things with. So, I definitely agree with you there. But I think that if you look at the way the field looks at the Stimulus Act, one big problem with just looking at it and saying, 'Well, everybody just thinks what they want and data doesn't matter here,' is that the Stimulus Act isn't something like the minimum wage. It's not a discrete policy where you turn the fiscal level up and it goes from a 0 to a 1 to a 2 to a 3. The Stimulus Act was like a dozen different things. And so to say that research hasn't told us whether the Stimulus Act was good or not or increased jobs--well, I mean, you could write a 100 page paper on just what was in it. There were things like direct money to households; there were spending on green infrastructure; tax cuts; Cash for Clunkers. So it's not surprising that there wasn't one good study that said, 'Oh, we have the multiplier from the Stimulus. It's good; we're all good; benefits are greater than costs.' That's a problem of looking at a broad question and looking for a very specific answer. But I do think that within the Stimulus Package, there are things we can learn; there are things we have learned. To take an example where I don't think you are going to get a lot of disagreement from people, the Cash for Clunkers Program gave people incentives to trade in their old, poorly fuel-efficient cars for new cars. I don't think many people think that that was successful. And I think that that's because of really good micro evidence that took an instrumental variables approach, and showed that what happened was that the demand for new cars was shifted forward in time from a very short period. So you are looking at 1, or 2, or 3, or 4 quarters in the future--all that demand is where it came from. And it was a very temporary burst. And I would be very surprised if the next time a recession comes around and we are talking about fiscal stimulus, it's going to include a program like Cash for Clunkers. And I don't think that that's an area where there's going to be much disagreement. Russ: Yeah. I agree with you. Although I wonder how many people defend that, at the time--maybe I'll put something up about this when we post this episode. It was a tragic episode for me--it reminded me of the Great Depression when we slaughtered pigs to help farmers--we slaughtered pigs and then didn't serve them as food to anybody. We just killed pigs as a way to raise the price of pork and help farmers. This struck me as kind of the same intellectual mistake. Just a total failure to understand where prosperity comes from. And it doesn't come from destroying things. It comes from creating things. But I agree with you: I think most economists at the time or a lot of economists at the time thought it probably wouldn't work. And there have been some pretty respectable studies that convinced people that indeed it did not work. Guest: And I mean, that's complicated, not-obvious econometric stuff. It wasn't just looking at the aggregate data. It was using instrumental variable approaches and it was specifically identifying the shift forward in time of demand and where it came from. So I count that one as a victory for empiricism.
35:09Russ: Yeah. I hope you are right. Again, I think a lot of these ideas come back. They have a political appeal, right? And I think one of the points I want to make here that I think is hard to remember--I have trouble remembering it--is that people do say things publicly sometimes for various reasons other than just to determine what they think is true. So, when economists--your point about the complexity of the Stimulus is an excellent point. A lot of people just said, 'Well, it's $780 billion dollars, and that's all you need to know.' But of course that shouldn't be--you need to know something about what it's spent on and how it's time period is pushed out. And a lot of people--and some of the very prominent Nobel Laureate types--or would-be Nobel Laureates, people with strong academic reputations--said at the time, 'This is great. This is the right size.' Now, I'm willing to concede that that's a remark that they made that is not a thoughtful academic remark but rather an intended-for-the-public remark that they would not maybe have made that remark in a seminar. So I think you do have to distinguish between those two. And maybe it's a little bit of a cheap shot on my part to point out that people said things that turned out to be "wrong," because they were talking for a different audience. I don't think that's the best idea, to talk to different audiences, but I understand it. Guest: Yeah. I think that's right. I think that people are going to say it a different way and they would probably be a little bit more cautious in a different venue. And I think that--you know, one of the things with the multiplier is that it wasn't a very--like you said, it was sort of settled in the 1970s. And so, this wasn't something like--when the minimum wage comes up, people can look at very recent new evidence that's been highly active and talk about what research applies, what research is the best. But the Stimulus is something that is sort of, I think Valerie Ramey called it a backwater, until the 2008, until the Recession. I think one of the reasons that you had such a wide range of multipliers is that, 1. People weren't looking at it very closely for a while; and 2. You have different multipliers: you have full employment multipliers and you have recession multipliers. And just focusing on the time series evidence, it's entirely possible that there are not enough degrees of freedom there to identify that. So it's an area where I think there's good work going on; and if you look at papers that are written about fiscal stimulus, about the multipliers, there has been an explosion of interest here. I think there's some new interesting approaches that are enlightening and that we're learning a lot about it. And I'm hoping that we'll see that multiplier range narrow. But I think part of the problem is that it just wasn't that popular of a topic for a long time. Russ: Yeah; and to come back to the underlying issue here about ideology versus science: maybe I haven't stressed enough, and I'm relying on people to go back to past episodes. I apologize. But there is always an out for one's own ideology, or even just prior beliefs, right? The world is complicated. So, 'Okay, the Stimulus didn't work this time. That's because it was a financial crisis.' Or: 'That's because it was a supply-shock crisis.' Or: 'That's because you didn't spend it the right way.' And there's always--it's very difficult to accumulate evidence. There's lots of evidence; but it's hard for it to become cumulative--as I heard Robert Skidelsky say once--Keynes's biographer. He said, 'Economics is not a progressive science.' And what he meant by that is our knowledge doesn't build on a finding and add to it the way it does in, say, physics. Now, he maybe is being too pessimistic there. But in macro, it feels that way a lot. Guest: I think there's obviously truth that macro is harder. And that you have ideas that become popular and generate intense research projects that are then tossed away, largely. For example, in macro, the idea that you could explain recessions without money was a big research project in the 1980s. And a lot of people focused on it, and a literature built up; and it was revealed to not be a very valid way to describe most recessions. And so we don't still build upon that now. Although it's important to know; and I think that the sort of legacy of that research, even though the main theory was proven false, was it's a useful legacy; because now when someone who is not an economist says, 'Well, the real cause of recessions are productivity shocks'--they word it in an non-economist way, of course--but we can point to that literature. And so even from failed projects I think we learn stuff, and we do accumulate knowledge in macro in terms of ideas that we rule out. And in terms of theories that are not very credible. Russ: Yeah. Maybe. I'm not sure. I think a lot of ideas do come back, for a variety of reasons, either because circumstances have changed or because it's convenient to believe them. I think that's the biggest challenge we have, as economists.
40:23Russ: An unspoken issue here is that, as economists we like things that make us important. And so it's very tempting--one thing I found interesting in response to your piece--let's get back to that--is that a lot of people said, 'Well of course we change our minds. All the time.' So, that's comforting. Like you said: it makes us feel good that we have a real science. People change their mind in history, too. There are trends in history about what caused what--what caused WWI. Sometimes new evidence. A telegram will be uncovered in the archives or some correspondence between diplomats that was hidden, that was sealed, that was lost--and that causes people to change their minds. So, evidence--I want to make it very clear here: When I argue against empirical work, I'm not arguing against facts. I'm not arguing against evidence. Facts matter. Evidence matters. I'm just a little bit skeptical--actually a lot skeptical--about the power of sophisticated econometric techniques to cut through complexity and to hold everything else constant. And I think too often as economists we consume and agree with things that conform to the way we want the world to work rather than the way it actually does work, because it's hard to figure out how it actually does work. Guest: And there's certainly truth to that. I think that the lesson--I don't think we probably disagree that much about how much we could learn from empiricism. But I think that lesson has to be extended to ideology, to facts, to stories. And in those areas we have even less of an ability to be confident that our biases aren't at work. Because at least when it comes down to empiricism we have it all there on the table. I can point to the study; you and I can talk about the assumptions behind the study; we can debate about where it fails, where it succeeds. But if what you have is a story, that's much harder. If what you have is just, 'Well, I have an accumulated lifetime of facts and experience,' that's even harder to argue about. And so I think in those areas we are going to see our biases operating more strongly, because that's fundamental human nature. So, I agree with you that economics needs humility. I think probably economics needs more humility. And economists need more humility. But stories and simple facts and our own personal narratives about the world need even more humility. Russ: Well, that's a fantastic point. I'd like you to expand on that. Because I do think, in preparing for this episode, when I thought about things that I've changed my mind on in economics--and I have--I'll just pick one: How well do markets work? I think they are a lot messier than I thought, say, 25, 30 years ago when I was younger. And part of it is the examples we talked about earlier. I think the people who get left behind, who don't benefit from economic change, I don't think markets work as smoothly as I thought they did. They don't work as well as they do on my blackboard. So I've become what you might call a little more Austrian about how markets work. They are messier. I don't like the way I talked about markets 25 years ago. And so, why did that happen? Why did I change? Why did I become more willing to admit that there are inefficiencies? I still don't think that necessarily government can fix them. So, I didn't change that ideology. But the way I think about them and the way I apply them in the real world has definitely changed. So, why did that happen? The answer is: I don't really know. Right? Part of it was hanging out at George Mason U. for 9 years--that was, you know, a very influential set of people there who affected me. And plus, that was where I worked: maybe that had an effect on me. But it's hard to point to those things. So, now I've got a nice, firm set of new views about how markets work. And you're right--you can't really argue with me, can you? This is a fascinating point. Guest: Yeah. And I think that the task for economists is to sort of let go of that and step back and assemble our thoughts and the evidence: Why do we actually think this? And for myself personally, I agree with your basic characterization--I'm moderately libertarian leaning. But I can step back and I can point to sort of the studies and the veins of literature that have sort of influenced me in this regard. And I think one really interesting area, just to take one of, you know, a lot of pieces of evidence you can do about labor markets, is the research on compensating variation, where, if a job is less pleasant in some regard or more dangerous in some regard, that wages tend to make up for that. And I think that there's a long, broad literature on this. And I think that that's the kind of thing where you see it; it sinks into your mind; it helps create a picture of markets as operating relatively effectively. But you sort of forget that that evidence is driving that image you have of markets. And I think that there are lots of studies along the way. And I think the way our ideologies form as economists is we see studies and they confirm what we think; we see studies and they reject what we think; we sort of forget those studies but we retain the images of markets. But I think it's really important to go back. I think it's really important to remind ourselves: Why do we think markets work as well as they do? Russ: Or not. Guest: Why do--yeah; and why do we think that they don't, when they don't? Do we think we need the-- Russ: I'm just making the point that people on the other side of the fence from me, who think they don't work that well--why do they and I disagree so much? What's the source of that difference? Guest: Exactly. So it's important for us to be able to talk about the evidence. And I think that's really how you change people's minds. And unfortunately the economic thought industry, the content industry, is driven far too much about persuading our own side. And so we really don't need to go back to those fundamental questions. We don't really need to dig through our memories and through our minds and look at our assumptions about how well markets work, and pinpoint the evidence and the data that's convinced us of this. Because we're talking to people who already agree with those fundamentals. So we just argue and try to convince our allies on the margins. But I think if you really want to reach across the aisle and you really want to convince people who don't agree with you, you've got to be better at digging deep into the evidence that motivates this image we have of markets. And those things need to be closer to mind.
47:03Russ: I just want to make clear: When I say 'ideology', it's not really the best word. Or 'bias.' But I really think talking about it is--some of it is ideology, a philosophy of how the world--of the right role for government, say, or the private sector or the civil society. But I'm really--writ large, I'm really talking about what we would call a world view: the lens that we use to organize our thinking about the complexity of reality. And when something comes along that doesn't fit in to that, I think what's important to remember is how easy it is to say, 'Oh, well, that's--' just to make a noise. Not see it, not look at it. An example I like to use, against my view, and I've talked about this before, is Wikipedia. When I was younger I would never have believed Wikipedia was possible. A bunch of volunteers giving up their time for no real reward--it's a waste of time for them. Why would they do that? And I think many economists--I wouldn't have been alone--would have said, 'Wikipedia, if it happens at all, will be awful; and it won't be useful. It will be shoddy. Of course: there's no money involved. There's no reward. There's no incentive. It's not even fame. The authors are often anonymous unless you start to peel back the layers; and even then, sometimes they have nicknames. So how could this--it just won't happen.' But it did happen. So I think a thoughtful person, you have to realize the world's a little more complicated than you thought. So I think those are very, very powerful and useful things to realize. And I think they accumulate over time. If you see enough of them you start to realize that maybe your lens needs adjustment. Or, maybe you need a different lens. You can actually be sufficiently empathetic to the other side that you realize that their lens is pretty good. Maybe you need to change your lens. And that's no fun. Part of what I'm arguing for here is how hard it is to do that. For any human being. And certainly for a professional economist or an academic who has got a reputation and a track record of past pronouncements, I think it's very hard for us to do that. And so we are a little bit rigid. A lot rigid. Guest: Yeah. And I agree with that. And I think--I don't think excessive skepticism of empiricism helps us very much in that. I think it's more--we need more skepticism of our basic lenses than we need skepticism of empiricism per se. I would say that for most people that's the direction they need more work on. Russ: My claim is, is that we use that empirical sophistication as a way to dress up our lens--to the outside world, to make us look a little less naked. Right? As you said, it's very hard to argue about this long history of personal experience and narrative--, so, say, 'Well that's not the real reason I believe these things. The real reason is I have evidence. I have science behind me.' And so what we have is dueling econometric studies. But that's not really what's going on. That's really, I think gets at what I'm trying to trying to suggest: which is that a lot of the disagreement[?] in economics over public policy issues and understanding what causes recessions, what causes recoveries, is that we have these sort of vague ideas that come from our personal experience. They are not made up. They are not biases--it's just sort of convenient for us. They are just an accumulation of lots of stuff. Some of it evidence, some of it fact, some of it comforting because of the way we want the world to work. And then we dress it up with these studies. That's my real, I think the critique I want to stand behind. By the way--some people accuse me of moving the goal posts. And I'm happy to concede I've moved the goal posts on what I really believe. Because I haven't figured it out. I think it's an incredibly complicated issue of how our world views come to be what they are. So, I'm not--I'm totally happy admitting that. I may have overstated a position in the past. But I think that kind of sums it up. What do you think of that? Guest: Yeah, that's great. I think your changing your mind--as you see evidence. Welcome to the list of data points, Russ. You are going to be the next data point, next time I write about this. But I think in general we've got macro evidence and micro evidence about this. And I think the macro evidence is that the economics field evolves, and our understanding of issues evolves, and it doesn't just go back and forth, but it gets better. And closer to the truth. And we continually rule out bad ideas; and we have a massive, massive stock of bad policy ideas that are extremely popular but most economists rule out from that evidence. I think it's too easy to focus on the things that economists disagree about. But if you think about the things that economists agree about, the things that we generally rule out that, you know, non-economists love, it's a pretty big stock of policies and ideas about the way that the world works. So I think that's the macro evidence--that empirical evidence matters and should matter. And the micro evidence I think is the illustrations I have and the examples I've gotten about people responding about people who do change their mind on an issue that they really cared about, based on empirical evidence. And I think that we probably-- Russ: Give some examples of that. Guest: Sure. So, I already talked about myself and I can talk about my boss: Mark Zandi is the chief economist at Moody's Analytics has been persuaded by recent evidence that shows that recessions might tend to have long-run impacts. And that's something that was, you know, not a very popular idea among economists; and it started to gain some steam just based on, you know, better modeling and better data sets and nothing really super-complicated. You can read that paper; it's pretty accessible. We looked at a bunch of countries over a long period of time and we have a better measure of pre- and post-trends. And it looks like something might be happening there. So, that's one example, I think. In response to the Great Recession there have been some high-profile people who have changed their minds. Narayana Kocherlakota, who is the President of the Federal Reserve Bank of Minneapolis--he, at the beginning of the Recession, was very skeptical of demand-side explanations. He thought demand-side factors were very short-lived and temporary. And as a result he is what you would call a monetary policy hawk. He didn't think that we needed to continue easing and to continue with Quantitative Easing programs and continue being accommodative in monetary policy, because he thought inflation was right around the corner--because demand-side policies can't matter that much. So this must be supply side. In which case, we're going to see inflation very soon, from all this easing we are doing. And it didn't show up. It just didn't show up. And that fact, combined with, you know, research he cites from people that worked for him at Minneapolis that were doing micro studies of housing lock and other structural factors--and they just weren't finding anything. He just sort of changed his mind. And he realized this is a demand-side thing, and we don't have to worry about inflation that much. And I mean he's not the only one. It's easy to look and find people who haven't changed their mind about the Recession. But even Arthur Laffer, of the Laffer Curve fame, supply-side economist--he thought inflation was right around the corner. And he thought it was going to be big; and he was very loud and prominent about this. And when it didn't show up, he didn't just say, 'I was wrong, but inflation is coming next,' or 'I was wrong, but'--yadda, yadda, yadda, I'm still right. He said, 'I was wrong. That means my model is wrong. And I have rethinking to do.' And so I think we need to look at these people. And, you know, they prove that it can be done. Obviously we have deep, inherent biases that make this sort of thing hard for people. But it can be done. And the lesson isn't to distrust empirical evidence because people are often not using it properly. But the lesson is to be more skeptical of our ideological priors and try to use empirical evidence better, because it really can be persuasive. And it really can change your mind. Russ: Yeah, and I've talked about that inflation/monetary issue. Although that's a little bit--that's a really crude, natural experiment, right? Massive increase in the Fed's balance sheet; no inflation. People who predicted it will still stand by it, saying, 'Oh, we paid interest on reserves; the money didn't get into the economy.' But I concede it's more complicated than that. But again, that's from experience, right? I think we do learn from experience. And there are cases where I think people fail to learn from experience. Even though the evidence to me seemed pretty clear on one side or the other, that something did or didn't work. I've used the example, the post-WWII draw-down in U.S. government spending, that that didn't--a lot of Keynesians afterwards found a way to reconcile that with their--that the Recession didn't happen. They found a way to reconcile it.
56:19But I want to go back to your column for a minute. Because you have some examples in there where people found interesting things that caused them to change their mind. And I don't want to suggest that people are so rigid that they never learn anything from empirical work. Even sophisticated empirical work. Of course--you can come along and you find a study that shows something interesting and you change your mind. The question is: Why doesn't everybody change their mind? Let me give a medical example I'm going to be talking about soon in an upcoming episode. And we've talked about a little bit in a past episode about the challenge of epidemiology, and of, say, measuring the effects of screening. So, for a long time it was encouraged for men to get a PSA (Prostate-specific antigen) exam to find out if they have prostate cancer. Well, they finally did a serious clinical trial; and they found out that probably it doesn't help you at all. And a lot of doctors can use your own--this is not, I'm not giving out medical advice here; that's not what that this program is--but a lot of doctors have clearly decided--and talk to your own doctor--but a lot of doctors have clearly decided that's not a good policy: you shouldn't get a PSA exam. And they've based it on the evidence of the clinical trial. They have a real--it's overwhelming to so many doctors. Again, there's some that still do it. I'm sure. But a lot of doctors have said, 'Welp, the evidence doesn't hold up.' I think that's the kind of thing that does not happen so much in economics. Yes, an interesting study will come along about some impact, and you'll go, 'Oh, that's interesting. That seems right.' And maybe it changes your mind. But things that fundamentally change your mind about something dramatic that's close to your heart--that's a lot harder. Obviously it's hard in every field. It's hard in physics. I don't mean to suggest that economics is the only field that has this problem. But I think in economics it's particularly challenging because we don't--the best we can get usually is a natural experiment. And natural experiments--by 'natural' I mean that it's not a lab; not everything is actually held constant. Control isn't working as well as well as it would in an actually lab experiment. And so, you can always find ways to decide that maybe it isn't representative of what would have happened in a real experiment. And I think that's our problem. That inherent complexity. Guest: Well, I don't think that the other sciences are that much better off than economics, actually. I think that there's--and there's evidence of this, too. It goes beyond just the prostate screening example that you gave. David Cutler had a recent study--it's a well-known fact, a well-known fact because of very careful multivariate regression empirical studies that health care spending varies greatly across the United States even after you control for demographic and other factors. And what David Cutler's work showed is that one of the driving forces behind these differences in costs, which also don't--they are costs that don't lead to better health conditions. So it's waste. It's money being spent on conditions that don't matter. For example, he said that screenings suggest that they shouldn't be done, but some doctors are still doing it. And in aggregate those kind of things really end up to a huge economically important driver of cost differences. Cutler's work suggests that if physicians followed professional guidelines, in end of life care, the end of life expenditures would be something like a third less. So a third of the excess cost in end-of-life care is driven by physicians who aren't following simple best-practice guidelines. And so I think that that's one example where medical care is showing the same sorts of things that you say are problematic in economics. And the life sciences in general--there's a recent NBER (National Bureau of Economic Research) study that looked at what happens when a prominent researcher in the life sciences dies. And they looked at the various subfields within the life sciences and what they found that when a prominent researcher in a subfield dies you have what looks like an improvement in research. So, what happens is his co-authors--his or her co-authors have less publications, less citations. And it brings new entrants into that field. And their work tends to cite the influential author's work less. And it basically just confirms that the well-known, quote, that science advances one funeral at a time. And this is the life sciences. They have actually experiments there. Their empirical credibility is 100 years ahead of economics, and it's a still a problem there. So I don't think that the bias of scientists and the culture of science is only a problem for economics. But I think, importantly, that all these fields make progress nonetheless, because that's the nature of science: that we do manage to, you know, manage to overcome these things and still learn and still progress. Russ: My guest today has been Adam Ozimek. And Adam I want to thank you for letting me talk a little bit more than I would have liked to, in this episode. This is my therapeutic couch to work through these ideas, you and other guests on these topics. I'm still figuring out what I think about it. And I appreciate your helping me get there. Guest: Sure, Russ. Thanks for having me.