Cost benefit analysis is probably the area of economics that I have the most experience and expertise. It is also the area I like to naturally go back to on a regular basis. I have a number of videos and posts explaining how cost benefit analysis can be and should be conducted. This post will look at how cost benefit analysis can be used to manipulate the readers of cost benefit analysis reports and summaries.
In short, cost benefit analysis is a technique economists use to determine if an initiative is worth investing in. Costs and benefits are identified, measured and converted into dollars. A process known as discounting is used to convert future costs and benefit into present day dollars. The theory behind discounting is that a dollar received today is worth more than a dollar received tomorrow. After both costs and benefits have been converted to dollars and discounted, they are compared with each other. If benefits are greater than costs the initiative is considered economically viable and the initiative is recommend to proceed.
Cost benefit analysis is intended to inform decision-makers regarding investment decisions. If cost benefit analysis is done correctly it can be a valuable aid to making decisions. Unfortunately, that is a surprisingly big ‘if’. There are numerous tricks that be used to misinform decision-making and lead decision-makers to make the wrong decisions. It can also be used to support bad decisions and be used to misinform the public once the cost benefit analysis reports are made publicly available.
This post will run through several areas where cost benefit analyses are most likely to be manipulated. These areas are as follows:
- Base Case
- Assumptions
- Exaggerate Future Growth
- Insufficient Sources Cited
- Long Evaluation Periods
- More effort to find benefits than costs
- Qualitative discussion around benefits and not costs
- Lack detailed options analysis
- Select modelling or models that produces best results
- Exclude Negative effects on the network
- Don’t discuss negative externalities
- Use Generic Sensitivity tests
- Exclude profits from construction from analysis
Base Case
The base case is often the easiest target for manipulation. As the base case is the yardstick for all measurements of benefits and costs. The base case is essential to the accuracy of the cost benefit analysis. If the base case is incorrect, the whole analysis is inaccurate.
What is typical done is that the costs in the base case are intentionally exaggerated to make the cost reductions and benefits of the project appear larger than they should be. Attention is often placed on the project rather than the scenario where the project does not exist. This makes it easier for analysts to exaggerate problems. If existing problems do not exist, future problems are often forecasted based on likely unrealistic projections around growth and user behaviour.
Assumptions
One of the biggest obstacles when conducting a cost benefit analysis is lack of information or incomplete information. This lack of information requires the analysts to make assumptions. Most cost benefit analyses have a reasonably long list of assumptions holding the analysis together. Good analysts are able to come up with realistic assumptions based on experience, historic trends, rules of thumb, interpolation or extrapolation of data, strategic plans, etc.
It is also reasonably easy for experienced analysts to manipulate the results of cost best analyses using assumptions. If for example, 20 assumptions have been made for a cost benefit analysis. It is possible that 15 of those 20 assumptions are close to inconsequential to the results of the cost benefit analysis. A good analysts will know that manipulating these assumptions are pointless. The analyst will therefore focus on the 5 assumptions that make a difference. These assumptions are likely to be slightly optimistic in favour of the project. That is often all it takes to give the results of the cost benefit analysis a big boost to achieve a benefit cost ratio (BCR) of greater than one. I did a review of one cost benefit analysis where an increase in the annual growth rate by just 0.4% more than doubled the benefits of the project.
For those of you not familiar with the BCR, it is simply the discounted benefits divided by the discounted costs. A BCR greater than one means that the benefits are greater than costs and therefore the project is a good investment.
Exaggerate Future Growth
This leads me to the next trick used by analysts, which involves exaggerating future growth and future demand. Future growth is close to impossible to accurately determine. It is also close to impossible to accurately disprove as well, unless the numbers are exorbitantly large.
There are several methods of predicting future growth. Such methods include the use of historical data, strategic plans for future development, surveys and interviews of key stakeholders, and future projections based on capacity and capacity constraints. All these methods are legitimate and have merit. A skilled analyst would also be able to tell which method would most likely produce the highest projected future growth based on the information available from the preliminary analysis or draft project proposals.
As mentioned in the assumptions section of the post, even a slightly higher growth can have a significant Impact on benefits. How is this possible? It is often the case that once demand reaches close to capacity, user costs increase substantially. As new projects generally add to capacity, the project will not experience these increased user costs but the base case will, hence the creation of higher costs savings. A higher growth will result in higher costs in the base case for a longer period of time.
Insufficient Sources Cited
A common trick used in cost benefit analysis is to not include all sources of information or calculations or derivations of outputs. This generally means that many elements of the analysis cannot be verified or even questioned. Most readers of cost benefit analysis reports are not experts in the field themselves but are interested in drawing information from these reports. The omission of information and sources could mislead these readers into thinking the analysis is accurate as they do not possess the knowledge to recognise what is absent.
Even experts will struggle to determine the accuracy of an analysis if pieces information are not available to question. A good reviewer would insist on all sources, models, calculations and assumptions are disclosed.
Long Evaluation Periods
Longer evaluation periods produce more benefits but very little change to costs. A large majority of costs are incurred in the first few years of the evaluation. Benefits generally occur across the life of a project and are often greater towards the end of the evaluation period as the existing assets (base case) are insufficient to handle increased capacity.
Evaluation periods are often based on asset life, which is not necessarily a good indication of the actual economic life of the project. For example, assets may have a physical life of 50 years but because of changes in technology, demand, or supporting infrastructure, the realistic operating life of the assets are likely to be considerably shorter. The remaining physical life of the asset should be treated as a residual value (the value the asset can be sold for or trade in for at the end of the evaluation period used in the analysis). To assume all asset acquired will reach their asset life could substantially overestimate the benefits of the project; hence, making the project more attractive for investment.
More effort to find benefits than costs
If an initial attempt at a cost benefit analysis does not produce sufficient benefits to cover the cost of the project, i.e. BCR is less than one, analysts often attempt to find more benefits in an attempt to bring the value of benefits above the value of costs. Trying to quantify as many benefits as possible is not necessarily a bad thing if those benefits can be substantially supported by evidence. If they cannot be supported by evidence they should not be included in the analysis.
The bigger problem lies with consistent treatment of benefits and costs. Additional research and investigation may legitimately identify and monetise benefits for inclusion in the cost benefit analysis but the same level of investigation does go into finding additional costs. I am not talking about initial investment costs but costs to users and third parties. These costs could relate to the environment, they could relate to reduction in alternative services or disruptions to other parts of the broader network or system.
In regard to initial investment costs, some may argue that optimism bias (overly optimistic cost estimates which do not adequately account for project risks) often results in understatement of investment costs; this is typically cited for Government projects. Optimism bias can make a project appear better than it is.
Qualitative discussion around benefits and not costs
At the start of a cost benefit analysis it is common practice to conduct an exercise of identifying as many impacts, costs and benefits that can possibly be associated with the proposed project or the proposed project’s referent group. These impacts, costs and benefits are typically categorised as material (significant) or immaterial (insignificant). Material impacts, costs and benefits should ideally be monetised for inclusion into the cost benefit analysis. Those material impacts, costs and benefits that cannot be monetised should be included in the cost benefit analysis report and described in detail with an explanation of why they cannot be monetised for inclusion in the analysis.
Most cost benefit analysis reports include a list of benefits that are included in the analysis but rarely even mention costs or negative impacts that could not be monetised for inclusion. This is another area of inconsistency in the treatment of costs and benefits which results in the project being portrayed in a more positive light than it should be.
This takes me to the end of Part 1 of this post. Part 2 of this post will look at the remaining areas mentioned at the start of this post. These areas are as follows:
- Lack detailed options analysis
- Select modelling or models that produces best results
- Exclude Negative effects on the network
- Don’t discuss negative externalities
- Use Generic Sensitivity tests
- Exclude profits from construction from analysis
Thank you for taking the time to read this post.
The full video, which also includes Part 2 is available in the link below:
