In the mid-1960s the architect Jørn Utzon left Australia due to the controversy of his massively over budget and delayed project. This project finally completed a decade behind schedule and at fifteen times the original budget, i.e. a 1,400% cost overrun. Today it is one of the world’s architectural masterpieces, but at the time Australians called it “copulating white turtles”. It is, of course, the Sydney Opera House [1].

With its massive schedule and budget overruns the Sydney Opera House exemplifies the planning fallacy. Millions of projects suffer the planning fallacy, but very few have iconic outcomes.

What is the planning fallacy?

Plans and forecasts that are unrealistically close to best-case scenarios and that could be improved by consulting the statistics of similar cases [4].

It was not an exaggeration to say ‘millions’ above. The planning fallacy affects projects of all kinds, e.g. home renovations, IT implementations, public construction projects, academic papers, etc. The effect of the planning fallacy is felt when projects take longer, cost more and/or deliver less than expected.

Some specific examples: IT projects and megaprojects.

IT projects

The Standish Group has published the CHAOS Report every year since 1994. These reports are a snapshot of the state of the software development industry. In 2015 the report studied 50,000 projects around the world, ranging from tiny enhancements to massive systems re-engineering implementations [5].

In 2015, less than one third of the projects were judged as “successful”, i.e. on time, on budget with a satisfactory result. More than half were “challenged” meaning that they were delivered late, over budget and/or with less than required features. And one fifth were “failures” meaning that they were cancelled or never

used.

The larger projects had higher failure rates than the smaller projects. This is consistent with Kahneman’s views: “There are many ways for any plan to fail, and although most of them are too improbable to be anticipated, the likelihood that something will go wrong in a big project is high” [4].

Except for smaller projects doing better, there has been no improvement in these statistics over time [5].

Megaprojects

Megaprojects are projects with budgets of $1 billion or more. There is no shortage of headline examples of massive cost overruns, e.g. Boston’s Big Dig, the Channel Tunnel, The Suez Canal, and of course the Sydney Opera House.

With reference to [6], let’s explore the hosting of the Summer and Winter Olympic Games, all of which are megaprojects as they range in actual costs from $3 billion to more than $20 billion (Sochi).

The Olympics have the highest average cost overrun (156%) of any type of megaproject. Moreover, cost overrun is found in all Games. And almost half (47%) of Games have cost overruns above 100%.

If, perversely, one would want to make it as difficult as possible to deliver a megaproject to budget, then one would (1) make sure that those responsible for delivering the project had never delivered this type of project before, (2) place the project in a location that had never seen such a project, or at least not for the past few decades so that any lessons learned earlier would have been forgotten, and (3) enforce a non-transparent and corrupt bidding process that would encourage overbidding and "winner's curse" and place zero responsibility for costs with the entity that would decide who wins the bid. This, unfortunately, is a fairly accurate description of the playbook for the Olympic Games, as they move from nation to nation and city to city, forcing hosts into a role of "eternal beginners." It is also a further explanation of why the Games hold the record for the highest cost overrun of any type of megaproject.

During the 1990s, the IOC began to see that more effective knowledge transfer between host cities might be a way to counter the "eternal beginner" syndrome. The Committee therefore initiated what has become known as the Olympic Games Knowledge Management Program, which is a knowledge transfer program aimed at increasing efficiency in delivering the Games by having new host cities and nations learn from earlier ones.

The Olympic Games Knowledge Management Program appears to be successful in reducing cost risk for the Games. The difference in cost overrun before (166%) and after (51%) the program began is statistically significant.

Why does it happen?

In [2] and [3] we see two main reasons for the planning fallacy and associated issues that gives rise to project failures: delusion and deception.

Delusions arise because of our innate susceptibility to cognitive biases like overconfidence bias (see Don't be so sure ...), loss aversion (see When losses loom ...) and confirmation bias (see The Mother of all Biases).

Deceptions (also called strategic misrepresentation) arise because the interests of the project sponsors and the project team are not always well aligned. For example, when a project team, whether external consultants or internal staff or a mix, are tasked with planning a project they have an incentive to provide information that pleases the project sponsors, typically the C-suite.

“The culture of many organizations suppresses uncertainty and rewards behaviour that ignores it. For instance, in most organizations, an executive who projects great confidence in a plan is more likely to get it approved than one who lays out all the risks and uncertainties surrounding it. [7]”

Similarly, external consultants are interested in winning a contract by offering the lowest possible price, since they know that re-contracting is often possible and delays generally tolerated [2].

The net result is that “deliberately or not, costs are systematically underestimated and benefits are overestimated during project preparation [3]”.

What should we do about it?

The primary recommendation to counteract the planning fallacy is reference class forecasting [4]. The effectiveness of this technique has been demonstrated in the Olympic Games Knowledge Management Program. Reference class forecasting involves:

· Identifying an appropriate reference class (i.e. projects that are suitably similar to the one being planned)

· Obtain the statistics of the reference class (obtain broad measures like budgets and schedules and normalize these with key parameters, e.g. cost per square meter, days per line of code, etc.) and use this data to generate a baseline prediction for your project.

· Consider the experience of the project team and if you still believe optimism might still be at play, adjust the baseline prediction.

Several other useful ideas in project planning come from [8].

· Collect and aggregate individual forecasts, instead of engaging in group discussion.

· Focus people’s attention, directly and explicitly, on potential obstacles.

· Reduce social and commercial pressures that might encourage optimistic predictions.

· Decompose the work effort into smaller units and plan for each unit.

· Finally, apply a stress test: do not proceed on a risky project unless it still looks worthwhile with a 400% cost overrun and only 25% of the project benefits realized.

We should take cognizance of these guidelines or else we risk proving that George Bernard Shaw was right when he said: “we learn from history that we learn nothing from history”.

References:

[1] “What You Should Know About Megaprojects, and Why: An Overview”, Bent Flyvbjerg

[2] “Distortions and deceptions in strategic decisions”, McKinsey & Co.

[3] “Better forecasting for large capital projects”, McKinsey & Co.

[4] “Thinking, Fast and Slow”, Daniel Kahneman

[5] “Standish Group 2015 Chaos Report”

[6] “The Oxford Olympics Study 2016: Cost and Cost Overrun at the Games”, Bent Flyvbjerg at al

[7] “The Case for Behavioural Strategy”, McKinsey & Co.

[8] “Advances in Experimental Social Psychology”, Vol. 43, Chapter 1, Buehler et al