"You no doubt wish to see the garden?"
By Amy Willis
EconTalk host Russ Roberts has made no secret of his skepticism about statistical analysis and econometric modeling. Yet in introducing this episodes guest, Columbia University statistician Andrew Gelman, Roberts wonders aloud if he’s gone too far. Does Gelman convince him to retract some of his skepticism?
1. Throughout the conversation, Gelman reiterates that relying on statistical significance in thinking about policy issues is problematic. What are the two major problems he sees with the use of statistical significance? How might these problems be mitigated, and how much statistical literacy does the public really need in order to think about issues such as early childhood interventions?
2. What is “p-hacking?” How does Gelman use the analogy of the “garden of forked paths” to explain its effect? To what extent do you think p-hacking constitutes “cheating,” and why?
3. What does Gelman mean when he says a particular statistical study’s conclusions may be time-bound? How does this influence the way such a study’s results might be interpreted?
4. How much do you look to data to inform your own views on policy? What does Roberts mean when he says he’s, “tempted to rely on my gut feeling and be honest about it?” To what extent should statistical analysis be a part of the political process?