Bent Flyvbjerg on Megaprojects
May 25 2015

Bent Flyvbjerg of Oxford University speaks with EconTalk host Russ Roberts about the political economy of megaprojects--massive investments of a billion dollars or more in infrastructure or technology. Flyvbjerg argues that such projects consistently end up costing more with smaller benefits than projected and almost always end up with costs that exceed the benefits. Flyvbjerg explores the reasons for the poor predictions and poor performance of giant investment projects and what might be done to improve their effectiveness.

Jason Barr on Building the Skyline and the Economics of Skyscrapers
Why does the Manhattan skyline look like it does with incredible skyscrapers south of City Hall then almost no tall buildings until midtown? Jason Barr of Rutgers University-Newark and author of Building the Skyline talks with EconTalk host Russ Roberts...
Peter Boettke on Public Administration, Liberty, and the Proper Role of Government
Peter Boettke of George Mason University talks with EconTalk host Russ Roberts about the proper role of the state in the economy. This is a wide-ranging conversation on political economy. Topics include Adam Smith's view of the state, the tension...
Explore audio transcript, further reading that will help you delve deeper into this week’s episode, and vigorous conversations in the form of our comments section below.


Evan Downie
May 25 2015 at 10:48am


I am, as many I’m sure, an avid listener.
This was a great guest for the show. I started to laugh when he was explaining the de-bias process. Knowing where Russ was going to land on that concept.

I have one correction, that is probably only relevant to my fellow countrymen. Frank Gehry is 100% a Canadian architect. Not an American as stated.

Trevor C
May 25 2015 at 11:29am

We have a light rail project going on here in Ottawa, Canada. Right from the start it has seemed like a political exercise and not as well planned as it should have been. Getting permission to use the land you want to build on would be the first thing to do, as opposed to begging and pitching a fit later.

We need transit solutions but I’m skeptical that this multibillion dollar project will get us there.

And it’s all built on new debt.

May 25 2015 at 3:43pm

Interesting episode. The mass-transit ridership issue is complicated, and it’s debatable how useful it is to look at these projects over a short time horizon or in isolation from each other.

The most significant megaproject in the twentieth century U.S. was certainly the Interstate Highway System, which fueled the growth of suburbia and driving culture in general. This wasn’t the kind of modest, incremental approach that Flyvbjerg recommends: it was a gigantic top-down system designed by professional planners and implemented by the Federal Goverment.

It would be interesting to see the “ridership” numbers for the highway system over time. I don’t have that data, but it’s certainly true that the number of miles driven per person is dramatically higher than it was when the system was built. I remember talking to one older guy who’d lived adjacent to a highway since it was built in the sixties; he said that when it was new you could toss a ball back and forth in the middle of the pavement, and that you might wait as much as five minutes before a car came by. Anecdotal, but you hear a lot of stories like that. Now of course the road is congested.

The funny thing about the highway project is that the country was much more optimized for mass transit development in 1950 than it is now. It seems pretty clear that the powers that be (many of whom came from Detroit) decided that cars were the future, and that their role was to make the world safe for car commuters. It wasn’t so much a response to conditions on the ground as an “if you build it they will come” approach. And it took a massive concerted effort, one that involved not only infrastructure spending but also financial incentives for home ownership, like relaxed mortgage standards and tax deductions.

I suspect that there’s some kind of nonlinear aspect to the public’s adoption of different transit options. One new rail line or road is nice, but people really need the entire ecosystem of their everyday lives- work, school, social circle, activities, church- to be accessible via the same system. So there’s a lot of momentum, and it takes a major shock to change things, whether that comes in form of a technological breakthrough or government’s heavy thumb on the scales.

May 25 2015 at 3:49pm

Proud to live in Denver, the first city that turned down the Olympics in 1976.

May 25 2015 at 7:57pm

The Federal Interstate Highway System was initiated by President Eisenhower in 1956 for the purpose of moving troops and military vehicles around and across our country. It was a military transit solution that was proposed.

At that time it took a week or more to drive across the country. At the borders of some states highways were discontinuous; after paying at a tollgate, the highway became a narrow road, sometimes one lane. You then had to turn and twist roads to find your way to another 2-lane highway.

At that time the hated zoning laws were almost non-existent and the knowledge that houses, towns, and cities would follow the development path of highways was also non-existent. Though it fueled suburbia and the driving culture, no “ridership” numbers were made or considered.

“One of Eisenhower’s enduring achievements was championing and signing the bill that authorized the Interstate Highway System in 1956. He justified the project through the Federal Aid Highway Act of 1956 as essential to American security during the Cold War. It was believed that large cities would be targets in a possible war, hence the highways were designed to facilitate their evacuation and ease military maneuvers.

Eisenhower’s goal to create improved highways was influenced by difficulties encountered during his involvement in the U.S. Army’s 1919 Transcontinental Motor Convoy. He was assigned as an observer for the mission, which involved sending a convoy of U.S. Army vehicles coast to coast. His subsequent experience with German autobahns during World War II convinced him of the benefits of an Interstate Highway System. Noticing the improved ability to move logistics throughout the country, he thought an Interstate Highway System in the U.S. would not only be beneficial for military operations, but provide a measure of continued economic growth. The legislation initially stalled in the Congress over the issuance of bonds to finance the project, but the legislative effort was renewed and the law was signed by Eisenhower in June 1956.”

May 25 2015 at 8:29pm


It’s true that the Interstate Highways System was promoted on the basis of national defense, but I certainly don’t think knowledge that it would shape development patterns was “non-existent”. European architects and planners, most notably Le Corbusier, had been promoting a highway and car-oriented suburban development model for decades, and were very influential in American academic and professional circles, although the Americans had to wait for the end of the depression and the war before they could put the new ideas into effect. The pattern was already being realized by the late 1940s, and the highway system put it into high gear.

My point was that the system was built 1. without depending on initial usage rates for its justification, as you say, and 2. on an enormous scale, so that it could almost totally replace the passenger train system that preceded it. I suspect that doing it halfway would have been much less than half as effective.

Julien Couvreur
May 26 2015 at 1:03pm

There is one question which wasn’t answered clearly in my mind (although Russ did raise it):
Is it the case that the biases apply equally to good and bad megaprojects (with regards to costs and benefits), or that they disproportionately distort the perception of bad ones?

All the “sublimes” or biases which were discussed in this interview seemed universal (making both good and bad megaprojects look rosier than they are), yet M. Flyvbjerg seems to think bad projects receive comparatively more bias in selection.

Mort Dubois
May 26 2015 at 2:37pm

I’m sympathetic to everything that was said, but the devil’s advocate within me kept asking these questions:

1) Is immediate Return on Investment the best way to evaluate projects which do expand our frontiers of knowledge?

2) It seems that mega projects are on the edge of our ability to handle complexity. How do we expand our abilities without some failures?

Once we build institutions that can process the lessons of our current low success rate, it seems likely that the success rate will improve. Also, it seems that a burst of optimism and a raft of failed projects is pretty much the way that humans advance their technology – see the railroads in the late 19th century, or the construction of the worldwide fiber optic network in the late 20th century. After the first wave of investors lick their wounds, it often turns out that the new infrastructure is quite useful, and leads to further development.

Last thought: a dollar wasted on ill-considered infrastructure spending doesn’t disappear from the economy. It may take a temporary detour from the world of profitable investments, but there’s nothing preventing it from being redeployed after it’s spent some time supporting various unions, real estate moguls, crooked politicians, and whoever else profits from mega-projects. Would we prefer a world where nobody every undertook projects that didn’t have a proven ROI from the git-go?

Patrick R. Sullivan
May 26 2015 at 2:59pm

In honor of the Belmont Stakes to be run next week (be patient, Lauren) you should note that one megaproject that succeeded was the original New York City subway.

Financier, real estate developer and horse breeder August Belmont built the first line–where the city’s politicians had failed for decades–in 1904. As a profit-seeking venture. Which it did, and anyone with access to the NY Times backfile (which most libraries do) can read the details.

Belmont’s scheme was to have the city settle the property rights issues, so he could tunnel under others’ properties, and float a $30 million bond issue (the city would pay a lower interest rate than a private issue of the same bonds) to raise the construction costs. Belmont and his investors put up a $5 million bond, to be forfeit if they didn’t pay off those city bonds. Which he did with farebox revenues.

Unfortunately, Belmont succeeded only too well. When the city’s politicians saw the fat returns Belmont and his investors were making, they wanted in on the deal. Which they got by not duplicating the first arrangement, when it came time to build the second line. In the latter the city became a partner…which planted the seeds of the destruction of the system. I.e., it became a political football, which is why NYC is today stuck with a money losing operation.

Ironically, one hundred years later some French communists saw the need to build a bridge over the Tarn river in southern France, and they did the same thing; they found their own August Belmont to build it as a business venture. Speaking of aesthetically sublime; the Millau Viaduct.

Finally, there’s the Indiana Toll Road to consider. About which, I’ve written extensively.

Also, since most businesses fail rather than succeed, it isn’t too important that only a minority of megaprojects make it financially. The key is to make sure the people who built them are on the hook for both the losses and the profits. As Mitch Daniels did in Indiana.

Robert Wiblin
May 26 2015 at 6:44pm

This was just an outstanding episode, one of my favourite EconTalks ever.

I am more hopeful about debiasing methods than Russ, but one concern I have is this: if Flyvbjerg starts halving all the estimates for some kinds of things like passenger numbers, won’t the people using traditional methods to get these projects off the ground just get even more outrageous in their approach, and find a way to double their estimates again?

It seems like unless the methods provide some absolute estimates to go with, these debiasing methods will just set off an arms race.

May 26 2015 at 7:07pm

I wonder if even the moderate leftist economists are aware of the folly behind megaprojects. Forget Paul Krugman, guests such as Stiglitz, Sachs, and Robert Frank have all extolled the virtues of public infrastructure spending. And yet, even if they happen to be worthwhile, they never take into account the political economy realities.

I live in the Bay Area and we have BART – a high speed transit that connects the various parts of the Bay Area. Ostensibly, Bart sounded like a good idea, but it was quickly perverted by the political mechanisms at work. Many more stops were added to the rail than were economically justified. Unions ensured glorified bus drivers were paid engineer salaries with generous hours and benefits. The additional stops benefited the nearby neighborhoods, but didn’t charge them a dime it. And the end result is a governmental product that operates on a loss; to say nothing of loss of overall investment.

Its in light of these failures that I keep harping back to liberals who want more and more for “stimulative purposes.” Its economically dishonest I think for mainstream progressives to pretend like these issues don’t exist.

Michael Byrnes
May 26 2015 at 7:26pm

Russ Roberts said:

“One is that, well, really large projects generally aren’t worth doing. They are very few opportunities to build a great dam or a great bridge. Most of them have already been built: the ones that are productive. Anything like that that comes along, we should just be extremely skeptical about it because it turns out most of them fail, in terms of cost-benefit. The second way to think about it is: Well, actually there’s lots of great megaprojects that might get done. It’s just that the political process doesn’t choose those.”

Is there a third possibility?

There are megaprojects that are worth doing and the political process does choose them, but it bungles them. Maybe because of patronage, maybe because gaining political support requires adding a bunch of nonsense or taking a suboptimal approach to the project to keep certain key players happy or whatever.

On a different note, the guest mentioned the Big Dig but not another Boston area megaproject that has seemingly been very successful – the harbor cleanup. I recently read that the harbor cleanup came in on time and under budget. I wonder if that is on Flyvbjerg’s list of successful megaprojects – or whether it was not as much of a success as it gets credit for.

Daniel Barkalow
May 26 2015 at 8:57pm

I think it’s a bit misleading to consider a project to be a failure if it doesn’t meet the schedule, doesn’t meet the budget, or doesn’t meet the expected benefit. If it actually gets done and is economical at the actual costs and benefits, it should be counted a success even if the estimates were that it would be even more successful.

In my field (computer programming), there’s a standard method for coming up with a good estimate: ask the developers how long it’ll take, and then multiply it by something between 2 and 10. It’s actually a good technique, because you come up with a reasonablly accurate estimate, and you aren’t asking people for line items on the 50-90% of the project which is unforseeable. Rather than trying to get people to estimate accurately, it’s probably actually better to use his technique and say that the raw estimate is of the expected budget (which should be explained), and the “de-biasing” is to account for things that can’t be predicted in advance and certainly can’t be itemized.

It’s probably best to stick with those original estimates, but only do a project if you’d still be happy with it if it took X% longer, cost Y% more, and only brought in Z% of the benefit. And 90% of megaprojects would still be disappointments in each way, but you’d hopefully have stuck to only those projects that were actually good ideas anyway.

May 26 2015 at 10:28pm

Yes, Eisenhower had probably heard of Le Corbusier, but what really turned him on was seeing the German army cross Germany by marching down the Autobahn. At the time our news programs were pushing the great advantages of quickly moving armies and the great disadvantages of not being able to.

I agree with both your points.

The DMC looks like a great project to me, but I wouldn’t call it a magaprogect. How does one differentiate between big, really big, and mega? DMC seems like a big ‘new city’ type project. Whereas the Interstate Highway System might use less concrete than the DMC (or more), but that’s not the criterion. A Mega project, to me, would be something, or some size, that had never been done before. In the Highway System, we’d built highways before but they were state or city roads. Now here were highways that went everywhere and through all the states. Does that qualify and Mega? It did replace the railroads which I find to be a bad effect.

The DMC will probably be very successful because it’s being privately managed and financed.

May 27 2015 at 8:12am

Great episode!

Toward the end of the show, Bent cites Frank Gehry who sees hiring local contractors as key to successfully developing and executing very large architectural projects.

In addition to the good point about reputational incentives that Russ makes, there are two obvious knowledge-related reasons to hire local companies with a proven track record.

First, they are more familiar than outsiders with the regulations that apply to a particular site and with the agencies (and individuals) that enforce them, and that is a big leg-up that can save a lot of time or money.

Second, they tend to know what kinds of construction materials can be sourced reliably and at reasonable cost from within the region, and they will have developed the techniques to best take advantage of these.

I was lucky enough to visit the new modern-art museum Gehry’s team built in Paris for the Vuitton Foundation shortly after it opened. They had a big room dedicated to the architectural project itself, from design to construction.

The exhibit judiciously featured beautiful videos honoring the work of the companies that manufactured the glass, steel, and wood components, including some storied, centuries-old names like St Gobain and Eiffel who have managed to stay at the forefront of their fields.

In addition to being able to translate complex specs into organized flows of thousands of unique elements ready to be matched to their counterparts from other suppliers and mounted into place by crane-borne construction crews, these companies are also well integrated into local freight networks and conscientious about training new generations of specialists.

It was eye-opening and moving.

Patrick R. Sullivan
May 27 2015 at 9:53am

One big advantage that local construction companies have over non-local, is ‘travel time’ expense. Something like dead-weight loss is in play when you have to bring in furriners.

Oddly enough, econ phd Phil Gramm didn’t understand that–or at least, professed to be surprised by it–when he brought in Texans to D.C to build a house for him and Wendy.

May 27 2015 at 1:25pm

Dr. Roberts:

Thank you for having Dr. Flyvbjerg on. He is clearly a leader in this field and has changed some minds. The comments by Daniel Barkalow were awesome, even though the contrary point is also true, if a project meets all its goals, schedule and budget, but the goals, schedule and budget are not more valuable than the costs then the project would also be a failure.

I am much less optimistic about change. The arguments for mega-projectgs that I see focus on the elements such as Dr. Flyvbjerg brought up about the Chunnel. That firm currently has 2.8 billion lbs. in equity and is paying off a portion of the original debt. So this project has possibly a value of 4 to 5 billion lbs. That is a lot of value that many supporter point to as evidence of the need for the project. But the benefits are rarely compared to the 9 billion lbs. of capital poured into the building of the Chunnel. Contrary to the comment above that stated the contractors would be able to re-invest their profits, wasted resources do not reappear and the profits that are present for the contractor would be re-invested whether the project is a failure or a sucess to the owner. Waste is waste no matter if the contractor profitted.

As for cost benefit, these can be so easily rigged even by unconcious priors, you rarely see any hired examiner really state good cost benefit estimates. The incentives to tell your client this is a good project is simply a bar too high to expect the typical examiner to push back.

As for the risk, I will share just one example. A contractor who was working on a large (not mega project) had a incentive to finish early and in return was allowed to work 24 hours a day. Toward the end of the contract the contractor started to do night work next a highly dense residential development and the client withdrew authority to work at night and extended the time for the contractor to complete the work and possibly earn bonuses. It is a possibility that the contractor found a way to work the contract in his advantage?

It is a highly analytic tasks of assigning the incentives of risk. Costs can be just as devastatingly bad if the terms assign risk that is controlled by the owner to be paid for by the contractor with inflated bid prices the likely result.

May 27 2015 at 4:46pm

Hi Russ, I think you missed something in dispensing with the idea of building a bridge in small increments.

The “small increment” of a bridge is a private boat owner offering passenger rides across the body of water. The next increment after that is a 2nd boat or a 2nd operator or a larger baot. At some point in that progress, some private operator could try offering a vehicle ferry at higher cost. All these small increments will either prove or disprove the business case for a bridge, long before you need to pour any concrete.

Of course exercise this assumes you aren’t subsidizing (or advocating the subsidization) of vehicle traffic by providing free infrastructure.

May 27 2015 at 6:49pm

Great podcast. These issues aren’t limited to megaprojects. I tout and cover these same topics daily in my professional civil engineering career. I focus on developing and installing infrastructure projects ranging from a few thousand dollars to a few million dollars. It is amazing how much pressure one experiences from regulators, state funding coordinators, and others to install the wrong but quick, expensive, and profitable solution.

While the state and federal funding application process requires the public owner to estimate the operation, maintenance, and replacement (OM&R) to ensure the local taxpayers can self fund the projects once installed, the analysis usually indicates they can’t so then everyone looks the other way and continues marching forward. Falsifying OM&R costs are just as pervasive as as falsifying forecasts.

I am seeing more projects with higher and higher OM&R cost in smaller and smaller communities, creating in my opinion, a new definition of a megaproject in my opinion. A local example in my area is the installation of a $4M mechanical wastewater treatment plant in a community of 440 people. While the community was given a $4M grant by the federal government, the community can’t self-fund the OM&R so naturally the state funding coordinator adjusted it to a level the community can afford. Of course, nobody told the mechanical treatment plant so after start-up the local ratepayers are going to be in for a shock. However, this community won’t have access to several thousand taxpayers and powerful politicians to help bridge the economic fallacy the project was based upon.

Thanks again for a great podcast. Dr. Flyvbjerg information, coupled with Mr. Marohn’s information from Strong Towns will help us combat the successful installation of the wrong solution.

May 28 2015 at 11:47am

Enjoyable podcast.

I agree with Patrick Sullivan. The failure rate of megaprojects seems to be about the same as anything else. As Taleb said (I think) most things fail. That is much under appreciated. But, also as Sullivan points out, putting the people who build something on the hook for its success adds prudence.

Additionally, I’m amazed at the number of cities (re)building streetcar lines. I predict they’ll learn what their predecessors learned 60 years ago: people prefer cars.

I’m equally amazed that cities don’t seem interested in innovative approaches to mass transit like equipping buses with traffic light changing technology, Uber, or adding driverless trams or pods to the mix.

Brian B
May 28 2015 at 2:26pm

This podcast was quite interesting, and reminded me of a project in Connecticut that isn’t quite a megaproject, but pretty close.

This project I’m referring to is a transit bus roadway between the cities of New Britain and Hartford. The 7 mile roadway cost about $600 million to build, only buses can access the road, and here is the best part- if every bus that runs is filled to 100% capacity 100% of the running time, the bus system will still lose money. But like the guest said, projections are oftentimes much much lower……A few weeks ago, this project finally was completed and bus service started. For the first week it was free. During rush hour, the buses ran at something like 10% of capacity, and the rest of the time they were empty. Keep in mind, New Britain and Hartford are two cities with declining urban populations. The reason the politicians cited for doing this project? One was to save on a hectic commute on the highway between Hartford and New Britain. At rush hour times, it could be as much as 20 or 25 minutes (about a 7 mile stretch). The other reason? 75% of it was FEDERAL dollars, so if they didn’t build the busway, it would be like losing $450 million!

Brendan Riske
May 28 2015 at 3:10pm

great episode! I loved it.

Here is on area which wasn’t discussed but i think is very important, environmental impact. Many megaprojects have huge environmental costs which are never internalized. Damns in particular are terrible for the environment, and the building of roads a a side effect on wildlife from car kills. All this great infrastructure in china and asia in general has come at the expense of their environment and ecosystem. How many projects are worth doing when you consider their impacts in that way? Look at how the decisions to build the Nicaraguan canal or this potential trans south american railroad. The effects will be catastrophic for those ecosystems if built.

Michael Byrnes
May 28 2015 at 7:37pm

Stuart wrote:

“The “small increment” of a bridge is a private boat owner offering passenger rides across the body of water. The next increment after that is a 2nd boat or a 2nd operator or a larger baot. At some point in that progress, some private operator could try offering a vehicle ferry at higher cost. All these small increments will either prove or disprove the business case for a bridge, long before you need to pour any concrete.”

I don’t think that’s true. Obviously, transit via a bridge is much, much faster and cheaper (ignoring costs of the bridge) than tranist via a boat. The cost, in money and time and limited availability, of boat transit is so high relative to driving over a bridge that it really doesn’t compare. Only the subset of people who put the highest value on crossing are going to take the boat, whereas the success of the bridge depends on people who put a much lower value on crossing – even if 100% of the high-value crossers use the bridge it’s probably not enough to assure the success of the bridge.

I think the problem arises when one assumes that building the bridge will, in and of itself, generate the traffic on which the building of the bridge was justified.

Jun 1 2015 at 11:04am

A very interesting interview that had a lot of depth behind it. A few thoughts:

* I’m actually not surprised that the failure/success rates of public vs. private megaproject appear to be similar. I just don’t think that “private megaprojects” are inherently 100% private. With projects of this size and scope, my assumption is that private firms go into this with some thought that they would be bailed out (analogous to the Big Banks/loss of prudence issue that Russ has made), or that their banking partners would save them (Trump has written about this), or that they could use the court system to their benefit. Will be very interested to read Prof. Flyvbjerg’s paper.

* I appreciated the discussion on the effect of lower interest rates on megaprojects as I’ve heard similar arguments to try to justify more government spending. Yes, there is a marginal effect – those projects whose benefits are greater than the newly reduced cost may indeed be considere, but lower interest rates is not an invitation to start spending on anything/everything politicians desire.

* Both from the podcast and the comments, it seems that the megaprojects that are considered success stories have pseudo Black Swan postive effects – huge benefits derived from them that analysts didn’t see coming when trying to project ROI. Something like the Internet or the Interstate road system. Per Prof. Flyvbjerg’s discussion about decision points, I don’t see how you can try to quantify the unseen when you’re at one.

* Aside from the stadium issue that was discussed, I find the biggest shell game going on these days among local governments are convention centers. Chicago has increased theirs, so Milwaukee will increase theirs. And if Milwaukee does it, then Indianapolis has to (local goverment: to be able to compete, of course). And if Indianapolis does it, then Pittsburgh has to. And so on and so on. It’s to the point where suburbs have them (Rosemont and Schaumburg to compete with Chicago), and even smaller towns have them (Blooimington, IN, about an hour outside of Indianapolis). As with stadiums, actual benefits never match the promised/predicted benefits, yet this convention center race keeps going on….by the way, even as we’re told by the technology sector that online technologies are replacing face-to-face meetings.

Stéphane Couvreur
Jun 1 2015 at 4:23pm

I wonder how reliable the figures are. It may be difficult to obtain the data and evaluate the profitability of a project. Often, a project is split between several actors, at least a constructor and an operator. The risk is allocated between them in complex ways: uncertainty about demand, about construction costs, operational risks, etc. Each actor has good reasons to hide his private information from his partners, in order to be able to renegociate the terms if needed. Even insiders don’t have a P&L for the entire project, not to mention outsiders.

Jun 1 2015 at 6:31pm

It was never clear what the metric for a successful mega project was. It sounds like timeliness and meeting the budget/revenue numbers stated. Those are fine metrics, but surely in retrospect a lot of the mega failures were mega successes right? For instance, we are in retrospect quite happy NYC has a subway.

Robert Swan
Jun 3 2015 at 8:36pm

I think IT mega-projects (and smaller ones) are the most fraught with failure and I was a little disappointed these only rated a brief mention. With physical things, at least you usually end up with something tangible. I first heard about the UK NHS IT system in 2008 and it was clearly a disaster. Checking on it today for this posting I see that it was abandoned in 2013 (Abandoned NHS IT system) after UKP10b had been sunk into it. Think about that. Nothing at all to show for 10 billion pounds. Staggering.

I like the idea of “sublimes”, and IT projects can certainly have them, but I think they have something more. Something insidious. Developing software consumes very little in the way of resources. Fred Brooks described software as “pure thought stuff”, and I think this makes it far too attractive to embark on a project without clear objectives; to make it up as you go along. Another thing which I have seen repeatedly in software is the urge to “integrate” things that were never designed to integrate. Package X does 60% of what you need, package Y does 30% and package Z covers the rest. A bit of glue and, voila, no need for that notoriously risky development process. How do you merge a Lamborghini and a Land Cruiser, and will the result necessarily be a fast safe vehicle that can tow a boat?

On the matter of politically sublime projects, in the late ’80s there was a spurt of building around Sydney — the Harbour Tunnel, the monorail (since dismantled), the Darling Harbour development — and a very appropriate term emerged to describe their underlying psychology — edifice complex.

Thanks for yet another interesting conversation.

Comments are closed.


EconTalk Extra, conversation starters for this podcast episode:

About this week's guest:

About ideas and people mentioned in this podcast episode:Books:


Web Pages and Resources:

Podcast Episodes, Videos, and Blog Entries:



Podcast Episode Highlights
0:33Intro. [Recording date: May 14, 2015.] Russ: Our topic for today is megaprojects--enormous projects involving infrastructure and other types of construction. And you've done a great deal of work on that area. Let's start with just the definition, which is of course somewhat subjective. What do you mean by a 'megaproject'? Guest: Well, we usually say any project that costs more than a billion dollars and that affects more than a million people. Russ: And what would be some examples? Guest: So, in the United States right now the largest civil megaproject is the California high speed rail line. That's an example. Before that, there was Boston's Big Dig, also called the Central Artery Tunnel. That's another example of another megaproject. Building a subway in New York City would be an example. Obviously in the private sector, oil and gas platforms like Deep Water Horizon that blew up in the Mexican Gulf is an example of a megaproject. So is the Keystone Pipeline. Russ: And they can be private or public, but many of them of course are public. They are often the ones that get the most attention: they get written about. One of the things that you observe that fascinated me is that over time, megaprojects have gotten more mega. And we're not just talking about dollar value either due to inflation or higher costs. The bridges are longer and the skyscrapers are taller. Is that correct? Guest: That is correct. And that's a very clear historical trend, even going back 100, 150, 200 years we find that, and tunnels are getting longer. More recently if you look at many IT (information technology) systems--we actually also consider large IT systems megaprojects--they are also growing bigger, if you count them by how many lines of code is written into a program or how many function points, which is a way to measure the size of IT programs--how many go into a program--it's also escalating very quickly. Russ: And what's remarkable, at least it was to me, is that there is an unfortunate pattern that is--it persists over time and over space, which is that megaprojects typically don't cost what they are promised to cost. And the benefits are not what they are promised. Talk about some of the empirical regularities that you've discovered. Guest: Yeah. It's so regular that we have a name for it. We call it the 'megaproject paradox.' And it's the paradox that we have all these megaprojects being built, more and more, and larger and larger. Spending more and more money on it as a percentage of global GDP (Gross Domestic Product). At the same time, it's very clear that the performance of the product is dismal. That they are just not doing very well. They do not deliver the promised benefits. They cost much more than was originally estimated. They take much longer. There are many more unhappy people in the wake of projects like that than was said would be the case, and so on. So, it's a very clear pattern. And it is a paradox. And it's actually what originally got me interested in this field: I was wondering--could that, could this really be? Is that really true that we are building something that has a dismal track record, and we do more and more of it? Because it doesn't sound rational. When you think about it. Russ: Yeah. In a way there are two paradoxes. One is that it gets built at all. The second is that we keep doing it. Guest: Yeah. Exactly. Russ: Any one of them looks like a mistake. And mistakes happen. But that you can continually make them is somewhat surprising. [4:49] Now, we had Phil Rosenzweig on recently on the program talking about decision-making. And we talked about some of the intense preparation and also the anxiety that comes along when you are bidding on a very large project as a contractor. And whether you--we talked about the Winner's Curse: that sometimes people will bid on a project, overbid, win the bid but then lose money on the bid. In these projects, that's not what's happening. The people--if I understand it correctly--when something like the Big Dig, which is the Boston Tunnel project, happens, and there's an enormous cost overrun due to delays and other factors, who ends up paying for that? Guest: Well, it varies. If it's a public project, typically it's the taxpayer. But let me just say that the Winner's Curse actually does apply to megaprojects. That is one of the mechanisms that do happen during the tendering process. But the problems actually start much earlier than that. So I just wanted to say that the Winner's Curse is there and it's a contributing factor. But other things are happening as well. So if a public project, it's the taxpayer. Who pays for private projects, it would be the shareholders. If it's a publicly owned project or whoever owns the company that is doing this.
6:10Russ: So, let me start with that puzzle. There are many puzzles here. But one of them is: If I tell you that I am going to build this tunnel for you, in a certain length of time, and it's going to cost a certain amount of money, and it turns out to be wrong, why is the taxpayer on the hook? Why aren't I on the hook? I made a bad estimate. Let's say it a different way: when a contractor comes to my house and he says, I'm going to paint your house, the inside of your house, for $2000, and it ends up taking much longer than the contractor expected, and he comes to me at the end of the project and says, 'You know, this took so much longer; I'm going to have to charge you $4000.' And I say, 'But you told me $2000.' Sometimes there's a reason that the delay happened. It might be my fault; I changed my mind; and then you negotiate. But if I'm a bidder on a tunnel and I say it's going to cost a billion and it ends up costing two, why is that the taxpayers' problem? Guest: Well, because often--even if you tried to do the painting of your house example, you will know that contractors are very good at coming back with reasons for why it's not their fault and why it's something else; the fault [?] or actually your fault. And they will have good reasons to increase the [?]. Russ: Sometimes. Guest: If you are very good at writing contracts, if you are very good at forcing what's going to happen; if you are very experienced, you might be able to write fairly water-tight contracts that would the risk on the contractor and therefore it would not be the taxpayer's problem. But that's typically not what we see. So, very often public decision-makers, policymakers, politicians and so on change their minds, or regulations change during the construction period. So, typically megaprojects like what we're talking about take between 5 and 15 years to do. And a lot of things change during that period. It's impossible to put all that into a contract because it's unpredictable. So, let's take the Channel [zone?] between the United Kingdom and France. The politicians decided to change the safety standards during construction. So, the trains had to be delivered to a higher standard because of certain train accidents that had happened, and so on. And obviously that's not the contractor's fault; and therefore the increased cost will go straight to whoever is going to pay for the project. Right? Of course, if it had been the contractor's fault, if the contract was written in a watertight fashion, then the contractor would pick up this risk and would pay for it. That also happens. Russ: And you'd think that that would become standard practice. There's always disputes. But you'd think you'd want to have some kind of clause in there for delays or inefficiency or whatever it is. And of course sometimes a contract will reward speed of achieving the goal sooner than possible. There will be a bonus. And of course that's risky; it can encourage people to cut corners. We understand that.
9:09Russ: Let's talk about the Channel Tunnel as an example. How did that turn out? How did that turn out for the taxpayer and for the traveller? Guest: So, the Channel Tunnel is 100% private project, so there was no additional cost to taxpayers on that, at least not in any direct way. There was an 80% cost overrun, in real terms, not including inflation, on the construction costs. And there was 140% cost overrun on the financing cost. The project was debt financed. And therefore, obviously, it turned out more than twice as expensive than estimated. At the same time, only a fraction of the passengers actually turned up on the service. Originally only 10% of the predicted Eurostock[?] passengers turned up. Eventually growing to about 50%. So this combination of much higher costs, or double cost, and half the customers, is obviously bad business. And the Tunnel indeed went insolvent and could pay its debts, and had to be financially restructured. And the people who lost money here were the shareholders. So, anybody who had bought shares in Eurotunnel, the organization that is running the Channel Tunnel, and the 257 banks that had lent money to the projects--those were the people who lost money here. Not the taxpayers. If it had been a public project, then it would have been the taxpayers. Russ: And, of course, if a private, a set of private decision-makers take their risk and they lose their money, that doesn't bother me. I wonder, do you have any feeling for the differences between private and public megaprojects in terms of reliability and cost versus benefits? Obviously not every private project is--you don't have data on that like you might have for the public. You do have, for example, a systematic study that you did on the Olympics, which you know, is public. And their costs are always, in the time period that you looked at, greater than estimated. Do you think there's a difference between the private and the public? Guest: That's a very interesting question. We actually have a major study on that, that is, that we are doing right now. And it's interesting because the general assumption is that the private sector is more effective in delivering megaprojects than the public sector. And we just wanted to test that. And the results are not final. But so far, the indication is that there is not a difference. That people are [?]: that it looks like both sectors are struggling with delivering megaprojects effectively. Russ: That disturbs my biases. But it's a comfort. Because I might start to think there's a possibility that there's my depressing, that the depressing track record of public is just par for the course. But I look forward to when that work is completed; maybe we can talk about it.
12:21Russ: Let's talk about the reasons that these problems consistently underperform in terms of costs and benefits. And, I think the ratio is 9 out of 10, is what you found, right? Guest: Yes. Russ: So, 9 out of 10, what, have benefits that are underestimated? costs that are overestimated? And, delays? Guest: No, it's actually worse than that. Nine out of ten have costs that are underestimated. Nine out of 10 have benefits that are overestimated, and 9 out of 10 have schedules that are underestimated. So, when you combine those, it's actually a very small fraction of projects that both are done to budgets, to schedule, and deliver the promised benefits. Russ: And you give--I like your categorizing of the different problems. You call them 'sublimes'. What are the 4 sublimes that make macroprojects so attractive? So, the first sublime we call the political sublime, and it's the fact that politicians love to start projects like this and they love to cut the ribbons of projects like this. You will find lots of pictures on the Internet of a politician cutting ribbons with the Queen of England if you are talking about England, talking about the United Kingdom, or the President of France if you are talking about France, or top level politicians in the United States. So, there's something sublime about it being a focal event like that, that really gets attention. Obviously politicians need to get reelected, so it's a way to get in the news. Megaprojects really do get in the news for good and for bad. And a politician actually once told me, very honestly, 'What do you think, Bent, that I would like to point to when my political career is over, that I passed some law that is sitting on a shelf somewhere in a library or I can drive across a bridge or go into a tunnel that I can tell my grandchildren I did this?' I decided this. That's an example of the sublime in action. So, that's the political sublime. Then we have something we call the technology sublime. And that describes the fact that there's nothing engineers like better and technologists like better than to push the envelope, technologically speaking. So, to make something longer or taller or faster, whatever it is. And if you are an engineer, what do you want to work on--an average bridge or the longest bridge in the world? Every engineer is clear on that question: they want to work on the longest bridge in the world. I would do that, too, if I had the choice. I think any human being can understand that. And that's the sublime of doing something technologically unique and exciting and you get that opportunity a lot in megaprojects. They really push the technological envelope in all sorts of ways, all the time. So we have the longest bridge in the world every few years; we have the tallest building in the world every few decades; and so on. So there's lots of opportunities for doing this, and engineers love it. So that's a second drive. In addition to the politicians really liking projects like this, engineers really love them, too, and are so happy to propose them and to work on them. The third sublime is the economic sublime. So, the definition of a megaproject is anything north of a billion dollars, and very often they are multi-billion dollar projects--$10, $20, even hundreds of billions if you are talking about military projects, like some of the air fighters, defense in the United States and across the world, like the joint strike fighter plane. And this means that there's enough money for everybody, to go around. First of all the consultants are very well paid; there are very big consulting contracts. There are huge contracts obviously for construction. There's huge contracts for landowners. Very often these projects are very land-consuming, so they need to buy land and these could be huge opportunities for landowners. If you are talking about rail projects there are lots of opportunities around and above stations, and again that is real estate becoming more valuable, an economic thing. And even if you look at it from a labor union point of view: labor unions love megaprojects because they create jobs, again something economic. So you have, actually, groups that often adverse to each other like unions and employers sometimes are in the labor market, they actually join forces when it comes to megaprojects and say, yes, we like it. We want more, because there's so much money it, it's good for everybody from an economic point of view. At least if we are talking about the construction phase, that's the way it's perceived. So that's the economic sublime. And then finally there's the aesthetic sublime. Which is probably less powerful than the first three, but nevertheless actually plays a role sometimes. So, the aesthetic sublime is best illustrated by something like the Golden Gate Bridge. It's a beautiful bridge; it's world-famous for its aesthetics. And who does not like to walk across the Golden Gate or to drive across the Golden Gate? I certainly love it. And so do millions of other people around the world and in the United States. And so if people can build something that big and make it beautiful at the same time, you know, they would really like the do that. The aesthetic sublime played out with the Bay Bridge[?] rates[?] between San Francisco and Oakland, in the sense that the earth quake damaged the bridge and it had to be 'retrofitted,' as they called it. And at first they just wanted--you know, a bridge going from point A to point B that's a very functional thing, and that's what was proposed. Then somebody said, we can't build something that pedestrian right next to the Golden Gate. The two bridges in the bay area--we cannot build this ugly duckling next to the Golden Gate. We need something that is aesthetically as pleasing as the Golden Gate. Let's aim for that. And then they started to redevelop the product and it of course turned out that it became much more expensive, as you would expect. That's an example of the aesthetic sublime at work. And you see that in many projects that are highly visible, like bridges and skyscrapers and so on, that there's also a drive to make it beautiful. And many people see that as a benefit. Again, I certainly do. If I'm going to have a megaproject, I'd rather have a beautiful one than an ugly one. And that's your fourth sublime, and a driver for megaprojects. And if you take them all together, you actually find that these are strong drivers that all together have [?] more and more proposals for more and bigger megaprojects.
19:58Russ: So in a lot of these projects, okay, the costs are underestimated a little bit and they end up being a little bigger than people thought. Let's talk about the benefit side for a minute. A lot of times in these projects economists are brought in to tell you how many dollars are going to be earned from all the visitors to this new museum, or because of the Olympics, or because of the time saved--there will be all these travelers on these trolleys or light rails or whatever it is. And in all these cases they almost always tend to be wrong. Is that your experience as well? We've talked mostly about the cost side. But you also said the benefit side is frequently over-estimated. What's going on there? Guest: Yeah. The estimate is optimistic. And we can discuss the reasons for this later. But right now just as an empirical fact, I can tell you that estimates of benefits are optimistic. Just to give you an example, for rail projects around the world on average rail forecasters estimate twice as many passengers as actually show up. So, 100% forecasting error. And year in, year out, with no improvements. Russ: And we understand the political incentives for that--you want to hire, like you said, because you are a politician; you want to get the tunnel built or the rail line built or the commuter train built, you are going to naturally point to this. You are naturally going to hire firms that are going to be aggressive in how they estimate the people who travel. What's interesting to me is that it seems like--I'm going to emphasize 'seems like'--the taxpayers get fooled every time: 'Well, this is going to be great because the study showed such and such. The study showed that the benefits are going to be huge.' I do see a little bit of a backlash. Most cities--excuse me, not most cities--most citizens in cities seem to have caught on that the Olympics is maybe not such a great prize to win. For example in Boston; I think it's going to have to go up for a referendum and I have a feeling it's going to struggle to get there. And one of the reasons it's going to struggle is that the politicians are not going to be the only people waving around the numbers. There will be advocates on the other side saying, 'This is a waste of time.' And these numbers are almost wrong. Is there any improvement possibly going on there? Guest: I think that that is happening now, with the Olympics; and that's one of the things I'm very happy about, about the study we put out in 2012 about the Olympics that is being used to inform the debate. Now, before you could--there was no study showing systematically across the Olympics what they had cost, what they were said they would cost and what they actually cost. So [?] than actually cost. Until we did this study in a systematic fashion. And it's being used in Boston. I've been in contact with the media in Boston and being interviewed about that. I know the study is being used in Norway, same thing: they have skepticism regarding putting on the Winter Olympics in Norway again. It already happened in Switzerland. And Poland, same thing. So there are several countries around the world that are beginning to use the studies like that. And I do think that that's the only way to get improvement, is that actually citizens take these studies and they start questioning the decision makers, the politicians and other policymakers when they want to put on these big, expensive things. And of course if a city decides that it wants to do this, that's fine. If everybody agrees we are going to host the Olympics or whatever, and we are willing to pay the costs, that's fine. That's the way democracy is supposed to work. But if the decision is made because people are misinformed about the cost and it turns out to cost twice, three times what people were told, that's not an informed decision. That's a misinformed decision. And that's not the way democracy is supposed to work.
23:53Russ: So, a lot of people right now are arguing that because interest rates are low in the United States, this is a good time for megaprojects. Let me ask this question in maybe a little bit of a strange way. So, if you ask those people, 'What should we be doing?' So it's true that the finance charges are low; that doesn't mean that the benefits outweigh the costs on any particular project. So just because the interest rate is low doesn't mean we should put a human being on Mars. It doesn't mean we should try to, say, build a superfast rail system in California. It's true that for any cost, the borrowing part of it will be lower than it otherwise would be. But it doesn't make the projects valuable in and of themselves. And I think the response would be, to that point, is: 'Yeah, that's true; so you should only do projects that are good. So let's make a list. We'll make a list of the top 10 projects that the United States should be doing right now.' What you are suggesting in a way is that at least 9 of those 10 will turn out to be not good. Aren't there surely some megaprojects that are out there that--how could it be? Surely there are some big things that could be done publicly--again, I'm talking about public projects here--that would be good. And they are not being done for political reasons or because people have been afraid of the interest costs. Now that interest rates are low, it's inexpensive to borrow, shouldn't we be doing these now? Guest: Well, the question about borrowing is, like you said, only if you are financing the megaprojects by borrowed money and therefore incur debts then obviously you will save money by doing it when the interest rate is low. So, yes, that would be an argument for doing projects like that when interest rates are low or other things being equal. And actually the benefit-cost ratio would be better for projects like that. If you are not using borrowed money, it doesn't matter what the interest rate is, in terms of financing costs. So that's a different question. If you are the public sector and you are taking money out of the public budget, then it's not borrowed money. If you are the public sector and you borrow the money then again it would be an argument for doing it when interest rates are low. Your second question whether there are projects that are worth doing--certainly there are projects that are worth doing. And it would only be if the business case is for the 10 projects you are talking about were made in the conventional manner, then 9 out of 10 would be problematic, and 1 out of 10 would be okay in terms of cost over-run, for instance. But you don't have to do it in the conventional manner, that's my first answer. You could actually go out and evaluate projects in a much more realistic way instead of this completely unrealistic way that is generally used where you overestimate the benefits and underestimate the costs. You don't have to do that. We have developed methods here at Oxford University, and other colleagues around the world have developed methods that actually makes it possible to develop much more realistic business cases. We have developed--even developed de-biasing tools, as we call them, where you can take a conventional business case and then you can put it through our de-biasing methodology. It comes out realistic at the other then, you know; which is a pretty amazing thing and a lot of fun to see the difference-- Russ: Magic. Guest: between the original business case and the de-biased business case. But even with the conventional way of doing it, there is this 1 project out of 10 that is okay. And we find them around the world. So we know lots of examples of projects that were worth doing. So, you have, some of the first high-speed rail lines in France were actually fine in this area, pretty much built to project and delivered the promised benefits. There's urban rail projects in Germany that way have delivered much better than expected. In Denmark we have what was the longest suspension bridge in the world when it was completed. It's doing much better--it's generating more than twice the revenues than were expected. So, the benefits are much higher. And it's like a cash cow sitting right in the middle of the country generating huge sums of money. My favorite example is probably Bilbao in Spain--the way that the city government of Bilbao decided to regenerate this old industrial rust-built town with a steel industry and shipyards and so on, completely going down the drain economically speaking. And it then decides to build the Guggenheim Museum designed by Frank Gehry, American architect. Which is a miracle, aesthetically, in many people's views, including myself. And supporting that with like, 10, 20 other projects including a subway, a new airport terminal, pedestrian bridges, cultural districts, and all sorts of other stuff. And pretty much doing it on budget and on time. And generating revenues that are 2 to 3 times higher than what they expected. And really putting Bilbao on the world map regarding arts and tourist destinations where people have to go to see interesting things and experience an interesting urban environment. So, I can list lots of projects like this. They do exist; there are lots of projects that are worth doing. Which to me makes it even worse that all these bad projects are done, that they are done, because there are so many worthy causes out there that could use the money in a much more productive way for everybody, except maybe the builders and landowners and consultants and so on.
29:49Russ: So, let me try to ask my question a different way. That was really interesting. But it didn't get at what I'm trying to think about. Guest: Okay. Russ: Given the proportion of failures, which is much greater than 9 out of 10--it's 9 out of 10 as you point out, in any one dimension--cost, benefit, timing--so, given the proportion of failures, there are kind of two things that you might think of. Two explanations. One is that, well, really large projects generally aren't worth doing. They are very few opportunities to build a great dam or a great bridge. Most of them have already been built: the ones that are productive. Anything like that that comes along, we should just be extremely skeptical about it because it turns out most of them fail, in terms of cost-benefit. The second way to think about it is: Well, actually there's lots of great megaprojects that might get done. It's just that the political process doesn't choose those. It chooses these other ones. You want to try to think about which of these two hypotheses might be more compelling? Guest: I think they are both compelling. So, I do think that the political process is incentivized in a manner that ends up choosing the wrong projects. I also believe that for some projects, like dams is an example that you mentioned, and maybe energy projects in general--so often dams are often hydroelectric dams, and therefore energy projects, in addition to being water projects, water management projects. If you can deliver things in smaller units, our data indicate that you are much better off. That your risks of losing money and your risk of not delivering the promised benefits are much smaller if you do things in smaller units. You can go small or smaller--it doesn't necessarily have to be totally small. But if you can go smaller, that's probably better than going larger. So, Norway is an example of a country that has taken that to heart. They basically stopped building large dams and now is building lots of smaller dams, and they find that that is better for the environment and they actually produce 98% of their electricity by water. So, they are completely sustainable--98% for the whole nation, of all their power is produced by water, by old large dam and newer smaller ones. So, that's an example of that. But obviously there's limits to that. You can't build a bridge in small increments. Russ: The Marx Brothers can. But that's a Night at the Opera reference. But yeah, go ahead. Guest: Yeah, right. So there is an example of that. But if you are like the responsible politician for building a bridge across the Bosporus in Istanbul, you know if you have an advisor who says to do it in small increments you might want to get another advisor. I mean, you can build it in small increments away from the construction sites, industrial production and so on, and maybe save money by doing that. But in the end you have to put it all together, and the bridge is as long as it is. You can't substitute it by 10 smaller bridges. So, for some projects there's no way around big. But for other projects there is. Whether there is a way around generally that's preferable. Then there's the additional thing, like what if a nation or city decides, 'We don't care whether it costs more. We don't care whether we are going to lose money on it. We do want the world's largest building, the world's tallest building.' Or, 'We do want the world's longest bridge.' Should they be allowed to do it? Of course. There's nobody who can say they shouldn't if democratically or whatever the decision may consist--they decide that this is what they want and it's generally supported, then of course it's completely legitimate to do it. But it's not what we see. What we see generally is the first thing you said, is that the political process is rigged in a way where ineffective projects actually get decided and built. There's a whole incentive structure that things end up wrong. Russ: Well, it's a classic example of what we call, a term we often use on this program, the bootlegger and baptist problem. So, the bootleggers are the people--they like banning liquor sales on Sunday because then people will buy from them. The illegal sellers. The baptists like it because they think it's wrong--God doesn't want you to drink on Sunday. So you have an alliance between people who have a self-interested motive and the people who have a more high-minded motive. My favorite example in the United States is some city is going to bid on bringing a sports team. Well, the main beneficiaries of the sports team are the owner and the fans. It's a small group. But they don't sell it that way, of course. They sell it by saying, 'It's really important that the city be considered a Major League city.' So, it's important--and I'm thinking, it's a nice idea but to make the poor people pay for it in the form of higher taxes doesn't seem like necessarily such a good idea. Guest: And that has been studied also, and it is not a good idea. Russ: Well, it's going to be for some people, and they seem to think it's a great idea. But yeah, overall, they are not very good.
35:22Russ: Let's talk about de-biasing. I was talking to some friends from St. Louis recently; I lived there for 14 years, and I lived in the part of the city near the Loop, and it's called the Loop because it used to have a trolley there. And I was sad to hear that St. Louis is going to rebuild the trolley there. And my friends knew I wouldn't like that and they were telling me about it, and I said, 'Well, I'm sure it will be crowded all the time,' [?] it probably won't be--as you say, it's an aesthetic sublime there; there's a certain romance about bringing the trolley back. And there's another factor, which is people who use the trolley are not going to be the ones paying for it. It's probably going to be fairly subsidized by either state money or federal money. That's usually how these projects--and that's another factor that affects these decisions to build these projects. But what I'm curious about is your debiasing trick. So, it sounds like magic: You put the cloak over the project and you rustle under it a little bit and you say some magic words and then suddenly it's shrunk to half its size when you pull the cloak away. Do you just simply--in this case, I would just say, 'Okay, whatever you are predicting for the ridership of the trolley to be, cut it in half and you'll probably be in the ballpark.' What other methods do you use in your debiasing techniques? Guest: I think that the way you describe it is very charming, but also far from the truth. It's not a trick. It's based on Nobel Prize-winning theories developed by Daniel Kahneman, professor at Princeton University. And he's sort of the godfather of what's called behavioral economics, which is revolution, is in economic thinking. Which is now also revolutionizing management thinking. And those are the theories that we built our debiasing methods on. So, it's very, very solid theoretically. Nevertheless, you know, it is quite simple when you think about it. What we do is that we just look at previous projects of the same kind as the one that is being planned now, and we just look, what was the actual performance of those. And you are right if the actual performance was as in effect, you know, that only half the passengers for a given rail line on average show up on that rail line. Meaning, you lost the other half. They were just like a fiction of the forecast. Then we actually adjust the forecast down by 50%. So, that's as simple as it is on the passenger side. We do the same on the cost side. We do the same on the schedule side. Which, every variable needs to be debiased. We look at it and we look at simple track record. We don't look at what people project into the future. Because we know from experience that's highly unreliable. And the theories say that you get a much more accurate forecast if you simply look at the track record for that particular type of project, and you assume that the next project is going to perform like the average of the previous projects. Unless you have a very specific and very strong reason to think that it's not going to perform like previous projects. Like if you are bringing in the world leaders to manage it, or somebody who had a proven track record that they actually were right on target with every forecast--they made, you know, the last 10, 20 forecasts--and therefore you have a reason to believe that they are accurate. Then you would not de-bias them like that, because they've already done it themselves and you would be double debiasing. It would be an error. But otherwise, it's that simple. You just use past experience and you adjust the forecast by that. Russ: Well, yeah, I'm a little bit skeptical about experts. So I'm not going to go without naming names. I know that some real and transportation forecasts were done by Nobel Prize winners in economics, using Nobel Prize winning techniques, that were cutting edge, state of the art, best, etc. And they were of course off by large amounts. I would assume that in every one of these projects, when you were pointing out these issues, they would say, 'Oh but this,' they'd always say, 'but this one's different.' Don't they? Guest: Yeah. They do. I mean, they do, and they used to get away with that. Because there were no statistics to prove that the pervasiveness of the pattern that we are talking about. So project owners and project managers, project planners, would say, 'Sorry, we were just unlucky this time. Next time we'll get it right.' Now we know that's not true, that the likelihood that they will not get it right is so high that we just can't count on it. We have to do something different. But you are right. The truth of the matter is that anyone who wants to cheat on this, can cheat on this. I could cheat on this, if I wanted to. It's so complex that it's very easy to manipulate the forecast--a little here, adjust them a little there and you can get the number that you want. That's the truth of this kind of forecasting. And that's how it's done. If, however, you use these debiasing techniques that I am talking about, which are completely transparent--it's much easier to understand even by laypeople than by the complex forecasting methods that are used these days--you have a better chance of getting people to understand what's going on. So, if you are talking about the trolley in St. Louis--and I would just--in my database, I could find 25 or 50 similar projects from around the United States and around the world. And I would say, this is the way they performed, Russ; and on average this is what happens. And you see that this is what they think is going to happen in St. Louis, or if St. Louis is completely off the chart, you know, compared to usual behavior, you should smell a rat there, and you know something is fishy. That's the first thing these methods do. And then, second, of course, you can, when you know it's fishy you can adjust it. Because you actually have the data from all these other projects and you know exactly what happened. But as I said, if you are not paying for it you still might push politically to do it anyway if you can get someone else to do it for you. Guest: That's true. That's true.
41:32Russ: What is Hirschman's hiding hand argument? Guest: Hirschman first of all was a famous economist who happened to be working at Princeton like Kahneman, and these two [?] men studied also at Harvard before that. He originally came from Germany. He developed a theory called the 'hiding hand' theory, or the hiding hand principle, as he called it, which states that we'll all be okay, you know--that we should just go ahead and start projects, because there is a hiding hand that is hiding our own problem-solving ability from us. It's hiding the real cost of the project, so we think it's going to be cheaper than it actually is, and that's actually good, according to the hiding hand principle because it gets us tricked into starting projects that we wouldn't otherwise have started. But not to worry, says Hirschman and his hiding hand, because when the troubles begin, when you realize that you [?], we human beings are very good at solving unexpected problems, and we'll deal with this. That's the way we are, and we'll solve it and we'll make the project successful. That's the theory. Russ: It's an appalling argument. Actually, I find it kind of shocking that it was put forward at all. And you say there was some discomfort about it, or what did you call it, tongue in cheekness to it [?]--the idea that, it's a horrible idea, it's going to cost a ton of money, but when it's over you'll be glad you did it and you'll do the best you can. And otherwise you wouldn't do it. I call it the myopic argument: it's saying we're not going to look too carefully at the estimates, because then I'd be discouraged. Ignoring the possibility that maybe you should be discouraged most of the time on some of these. Right? Guest: Yeah; but you know, you have people stating this. It's rare that you get it very clearly and in public, but there's a recent example of that. The former Mayor of San Francisco, Willie Brown, former California State Assembly member, he says that we should consider the first estimate of the first project just a down payment. And he says 'Everybody knows that. We shouldn't be hung up about that, when the cost overruns start to come in. That's not a problem because the low budget is actually useful. The thing is we need to start digging. We need these projects and therefore we should have a reason to start digging; and the low budgets help us do that.' So, he's on record saying that for the Trans-Bay Terminal Project in San Francisco which is a multi-billion transit and real estate development project. And he said the same for the San Francisco open Bay Bridge. Same thing. Russ: I have the quote here. I'm going to read it. Because it is special. He was quoted--I've got this from your paper--he was quoted from the San Francisco Chronicle. Here's what he said:
"News that the Transbay Terminal is something like $300 million over budget should not come as a shock to anyone.

We always knew the initial estimate was way under the real cost. Just like we never had a real cost for the Central Subway or the Bay Bridge or any other massive construction project. So get off it.

In the world of civic projects, the first budget is really just a down payment. If people knew the real cost from the start, nothing would ever be approved.

The idea is to get going. Start digging a hole and make it so big, there's no alternative to coming up with the money to fill it in.
That's a really remarkable statement. Guest: That's the core of the hiding hand principle right there. Russ: So, let me, even though I made fun of it, let me defend the hiding hand for a minute. So, there are many, many projects that were built long ago that may have been disasters for cost-benefit, "mistakes" as a result, but I'm kind of glad they are here. So let's take some examples. I'm glad New York has a subway system. I'm glad--I love the Golden Gate Bridge. I love the Brooklyn Bridge. I suppose the Panama Canal is a good thing. And these are all examples I think you mention of--at least some of them--big cost overruns. The Sydney Opera House--it's a gorgeous extraordinary building. Which had huge cost overruns, unbelievable. Once they are done, aren't we glad? Isn't it for the best? Isn't it great that--I mean, if Bent Flyvbjerg was around at the time we would have said, 'You're making a big mistake.' And then, look, now we've got these things, says Willie Brown, and they're great. Guest: Can I just ask you to name me another building by the architect of the Sydney Opera House? Russ: Well, I've read your paper, so I know there isn't one. So, talk about that. Guest: Yeah. Russ: It's a fascinating example. Guest: So, of course the Australians are happy about the Sydney Opera House. And actually the general consensus around the world is that it is the greatest building of the 20th century. Certainly probably the most viewed. It's the youngest building that has got United Nations' heritage status, and so on and so forth. So, it's really spectacular. At the same time, the architect who did this building has not done any other major building in the world--because of the mismanagement of the Sydney Opera House, because it was done by conventional formula for developing projects, like low-balling the projects up front and then asking for forgiveness instead of permission--which is exactly what happened. The cost overrun was 1400%. And as always when cost overruns happen, the architect was blamed, and his work situation became so uncomfortable, he found that he actually left the project in the middle, with his family, and flew out of Australia, never to return. He never returned, he never saw the building finished. He died a couple of years ago. You know, the cost of Sydney Opera House, to me, is because he left the building, he got such a bad reputation, both in Australia and outside of Australia, that he never got to design another major building. For decades the tour guides on the Sydney Opera House were not allowed to mention his name. That's how much unfriendliness there was between the Australians and the architect on this project. And it destroyed his career, as I said. That means that we don't have any other buildings. It's equivalent to taking Frank Gehry, who is considered the world's most famous architect by now and who is considered to be in the same league as Jørn Utzon regarding doing magic aesthetics and magic buildings. So, look at his, you know, 50, 100 major buildings around the world. You can choose one; you have to erase the rest. That's the cost of the Sydney Opera House. It was not a success; it was a huge failure, in those terms. Russ: That's one example. But in general--again, trying to give the hiding hand argument or the myopia argument its best case: Willie Brown is on to something, in principle. I think in reality where he's deceptive is that he likes to dig holes where his friends are because he's a politician. He's not necessarily going to dig the best hole, or the hole that's the right size, or the hole in the right place. I think it's around $5 billion, the Transbay Terminal Project. It looks kind of nice on paper. It might be something of a success. It will certainly be a success for the people who pour the concrete. And so it will probably be a success in some dimension. But when it's over, let's say it's beautiful; and 'okay, so it cost more than it was worth. But aren't we glad we have it?' That would be the claim. Guest: Of course you are glad we have it. And I'm glad, for instance, that the Channel Tunnel is here. When I go to Brussels or Paris, I just get on the high speed train and I'm right there. I can work very comfortably. Much better than airplanes. And it's very hard to believe when you are on that surface that it's a huge failure financially. Something can be a technological success and a financial failure, of course. And of course if you are not the person or persons paying financially, you would be happy about the project, and you are glad that it is there. But you should ask the banks and the original shareholders of the Tunnel and these other projects that you are talking about. You should ask the people who actually got to pay for the projects and got to pay the extra sums, whether they are happy about it. Of course, later generations are happy about a project, because they get it for free ride. Why would you not be happy about something that you get for free? That's not the relevant question. In that way you can justify anything. You need to go back to the decision point moment and ask yourself whether it was a rational decision at that point or not, and ask yourself who gains, who loses. And is it a right decision from that point of view.
50:47Russ: So, what are the prospects for improvement? You are identified, I assume, based on what you have said--people understand that you are the skeptic. That you are the guy that is the realist. You are the--in a way you play the role of the economist, saying 'This isn't free. This isn't the bargain it looks like. Watch out.' That's nice. It's interesting for our listeners to hear. Give me some idea of how your work, if at all, has had an impact on some of these projects and how they are estimated and how decisions are made. Guest: Yep. So, fortunately there is actually a lot of demand and increasing demand for realists in this field, because it's now becoming so expensive, so much money is going into this field that people are beginning to realize that it's not a good idea to do it in the wasteful way that we have been doing it, when it goes to the scale that we are now at globally. We are talking about somewhere between 6 and 9% of global GDP going to these types of projects these days. So it's a huge business. And even if you could save 5% in efficiencies, it would be huge numbers around the world. And you can, easily, in my view. So governments are beginning to realize, private companies are beginning to realize that banks who have got burned on financing projects that turned out to be financially non-viable are very interesting in this. So they hire people like me to get a more realistic view. Right now we are training, here at Oxford U., all the top civil servants in the U.K. government who are responsible for the major projects in the United Kingdom. Several hundred top civil servants, from permanent secretaries and a couple of layers down. And that's an indication of demand from the whole government: 'We don't want to waste the taxpayer's money like this; we actually want to do it in a different way. Let's get some training in how we do that.' So, in the future, no U.K. government official will be allowed to lead and manage a major project without having gone through what we call the Major Projects Leadership Academy at Oxford U., which is a one-year training program. Denmark, the Netherlands, Switzerland, South Africa--other countries are looking to these improvements with great interest and have already implemented aspects of this. So, there really is demand. Also on the citizens' side we talked earlier about how people are using the new information about Olympic costs to discuss debate, whether they actually want to put them on, you know--whether they actually want to foot bill and the financial risks you take on when you host the Olympics. And this now has repercussions. The IOC (International Olympic Committee), I know that the IOC is now working internally to improve things so that the Olympics do not have to be as costly. So the overruns do not have to be as big. They really try, they really see that there is a problem that not enough countries and cities will actually want to be hosts. And therefore they need to improve things. So, mechanisms like that are starting to kick into place. And in addition there really is demand for the more realistic approach. That doesn't mean--I'm a realist. I'm not an optimist. So I don't see the whole field changing overnight into something different. There are very strong interests in the old business model. And it is a business model. It is a way of making huge amounts of money for certain groups of people. And of course when things are like that, these people are not going to give this up easily. Or voluntarily. So this is going to be a fight between old ways and new ways, like happens so often, you know, in many fields and many businesses. And that's the name of the game when you are innovating things. I think. Russ: Well, your observations about the Olympics really help illustrate the Hiding Hand's arguments' danger, because you might say, 'Well, if no one hosts the Olympics then we won't have the Olympics, and the Olympics are so entertaining, they are so inspiring. I suppose they are not financial viable. But they are so great.' And the answer is: 'Well, there's a way to stage them probably that isn't quite as grandiose as they've been staged in the past, and would still provide much of the pleasure that people get from them around the world. To the extent they do; and I don't want to overestimate that.
55:25Russ: I want to close with a little bit of my own romance and get your reaction to it. So, I understanding deeply the aesthetic sublime that you talked about and the human striving for something that has never been done before, the technological issue, the technological sublime. The project in the United States right now that captures my imagination is the Hudson Yards project. So, what they've done there, what they are trying to do there, is they are covering a massive rail yard with a platform, and then somehow building an enormous number of buildings on top of that--skyscrapers--on top of that platform. So, the rail yard will still be there, which spans many, many square blocks. And yet they'll have essentially added land to the island of Manhattan by covering this rail yard and building on top of it. And it's incredibly ambitious. It's, I think, a $20 billion project. And I'd like to say it's private--it's somewhat private, at least. You know, of course the city is intimately involved in real estate. So, that kind of project--it's a wonderful thing that human beings try to do that; it's inspiring; it's creative; it's part of the human spirit. And as long as somebody else is paying for it, I'm all for it. And I hope they make a lot of money; I hope it's a great project, as long as they are not getting too many subsidies from the City of New York. But there are a lot of private projects like that that I assume--I'll be interested to see how your study comes out, because I would think many more of those would do better, the public ones, because I just suspect that maybe there are private examples--server farms in the West of the United States--you just don't see: they work great and you don't get the data on those. Why don't we close by talking, if you know anything about any of that. Guest: So, I agree with you that the taxpayer should feel more relaxed when things are done in the private sector. Maybe the shareholders shouldn't feel as relaxed. They should then pay attention to what the companies are doing. I think that that would be my advice. Russ: Absolutely. Guest: That's what I would do. Taking the Channel Tunnel as an example again, one of my friends suggested that we buy shares in it, when it was put up for the IPO, the Initial Public Offering. And given my knowledge about megaprojects, I just said 'No.' I'm really happy that I did, because it went insolvent. And the shareholders lost their money, basically. So that's what you need to pay attention to. It's not like people can relax just because it's in the private sector. The taxpayer can relax, if the contracts are written right, because there actually is a very unfortunate tendency for things ending up with the taxpayer anyway, even if everybody thought it was private sector. Russ: Correct. Guest: There is some escape clause somewhere in the contract that said: Well, by the way, the public sector is picking up the risk if everything goes wrong. That happens too often for comfort. Russ: Now, you did mention, very early on in the conversation, the handful of projects that were public and that seem to have been effective: Bilbao is one example you gave. And you gave a few others. I forget in the paper whether you found this or not--but, you looked for things that they might have in common, those successes, right? You'd want to look for if there is anything we can learn from those successes, or were they just good luck? Were they just random? What did you find? Guest: That's another ongoing study that we are doing now. We are looking at the projects that were successes and we are trying to find out, were they just lucky or is there a method to their success? And we find both, but there's enough projects where there is a method to the success and then we try to learn from those. One of the keys to success is, first of all, to have a really strong owner. So, the people who own the project have to have to be really strong. And have to know what they want. They have to have clear objectives. They have to have power, and they have to have staying power. So they have to be in place for a long time. There is a problem that many of the projects we talk about here have a much longer life cycle than the political life cycle. So politicians are elected every 4-5 years, whichever country you are in. But the projects last 5-15 years. So that's not very well synchronized. So, somebody at the front of the project might think, 'Hey, I'm going to decide this because I'm not going to be around to see whether it's a good idea or not. But I will get all the kudos and the benefits and the public exposure of being the person who made the decision. And I have the benefit of not being around when we are going to see whether it was a good decision or not.' So, that's a recipe for disaster. You actually need decision-makers that are in place and have staying power. Then, they need to hire people who have done it before. If you are building a megaproject or several megaprojects, if you are doing the Atlantic Yards, you want somebody who has done an Atlantic Yard before. This sounds so self-evident you'll think that it's always adhered to. It's not the case. You'd be surprised to see how many project managers are doing projects that basically they haven't done before. So, you are paying, as the owner, for an experiment, where you having somebody trying to do a project that they haven't done before. So, those would be two main things. Then of course also you would want to use the estimating techniques that will give you a realistic estimate. Some of the tools that we have developed; or you would want to de-bias your business case if it's been done by conventional forecasts, so you actually have a realistic case. Like they did in Bilbao. The Bilbao Guggenheim, for instance, you won't believe it when you see the building. Because it looks like if anything were going to go over budget, it would be that building. If anything was going to go over schedule, it would be that building. Because it's so complex and innovative; and usually you say the two main causes for overruns and delays are innovation and complexity. But here it was built right on budget. Actually a little bit below budget. And right on time. So it is possible. If you set things up the right way. Russ: Was it built by a firm that had built a Gehry building before? Guest: Some of the contractors involved had built Gehry buildings before. But others were local. And this is actually one of the things--according to Gehry, one of the secrets to having success with his buildings is using local contractors. Really good local contractors, is a good idea, according to Gehry. Russ: Hmmm. I wonder if that's--I wonder why. Maybe they'd be embarrassed to have a big cost overrun and take too long. Their friends keep saying to them, 'Hey, where's the building? What's going on there?' I don't know. Your other observation, if I remember the paper, is that it's hard to systematically compare these successes to the failures because there are so few of them. Your database isn't big enough. Guest: That's true. And that's a kind of funny result--that we can't find enough successes to have a statistically valid database. We have a statistically valid database for projects on average for failures. But we can't find enough successes to have a large enough database to produce statistically valid answers. That's okay--there are methods you can use in order to establish whether it will be luck or not for the individual client or the individual builder. Russ: So, Bent, you are from Denmark originally. And for listeners, I am going to tell you that his last name is spelled F-l-y-v-b-j-e-r-g. I think I got that right. But it's pronounced "Flu'byeh". And we will of course have links to Bent's papers and his webpage. So you don't have to worry about spelling it correctly.