Cost Benefit Analysis Methodology Procedures Manual

5. Cost Benefit Analysis Procedure

5.1. Introduction
Approved by Head, Office of Airspace Regulation Version 1.0: November 2007

5. Cost Benefit Analysis Procedure

5.1 Introduction
This Chapter details, step-by-step, the activities outlined in the framework for conducting
economic evaluation using CBA.

5.2 Cost Benefit Analysis Steps

Conducting a cost-benefit study should include the following, as indicated in Figure 2.1:
● Defining the objectives and scope of the proposal/project
● Clarifying the proposal options
● Identifying the costs and benefits, both quantitative and qualitative
● Discounting the future costs and benefits
● Calculating the decision criteria
● Performing sensitivity analysis and addressing issues of risk and uncertainty
● Identifying the preferred option
● Preparing the report.

5.3 Step 1: Define Objectives and Project Scope

5.3.1 The Importance of Objectives
Project identification and specification should be linked to CASA strategic objectives—eg,
as indicated in ministerial policy statements, annual reports and other relevant documents.
Consistency with stated Government policy vis-à-vis airspace management is an important
consideration, as well as how this fits with Australia’s international obligations and agreed
standards.

5.3.2 Proposal/Project Specification

● How will it meet objectives?
● Will it involve new capital works/equipment acquisition?
● Will there be a need to replace existing facilities/assets?
● Will there be a need to upgrade or enhance existing facilities?
● What are the constraints?
● Who is likely to be effected and how might impacts manifest?

5-1
Cost Benefit Analysis Methodology Procedures Manual
5. Cost Benefit Analysis Procedure
5.4. Step 2: Identify Project Options
Approved by Head, Office of Airspace Regulation Version 1.0: November 2007

It will be necessary to determine the level of detail required for the CBA as well as to
identify the beneficiaries of the project and how they are related to the stakeholders.

It is important that the proposal/project defined is not so broad that it is actually a program
of discrete proposals/projects. Conversely, the project defined should not itself be a
component of a discrete project. It must constitute a stand alone investment. Poor
specification of the project can lead to inappropriate assumptions and incorrect results.

It is recommended that at this initial stage of the CBA process, consideration is given to
the development of a consultation plan for industry stakeholders. This plan should include
the consideration of the need for/benefits to the process of having one or more formal
(structured) VM workshops as part of the process particularly in terms of answering a
number of the questions posed above (and in the following section) as well as those
identified in Chapter 4 section 4.2 (Steps 1 and 2).

5.4 Step 2: Identify Project Options

5.4.1 Range of Options
Options are prepared to fulfil project objectives. “Are there other ways to achieve the
same outcome?” could be an important question to address. The range of feasible options
will vary with the nature of the problem. Tasks set at the strategic level may generate a
wide range of options. It will be necessary to determine the feasible options.

It will be important to determine whether there are there variations on one basic identified
option eg, variations in the design and operational concepts.

When describing the options, the analyst should include:

● A schedule for the project/proposal phase comprising a planning and development
schedule
● The expected operational date of the option plus an operational schedule (eg, airspace
organisation and structure, route structure, etc) and a replacement schedule (for
individual system components)
● Type of equipment required if there are different levels of service to be considered
● Economic life of the key assets involved
● A transition schedule (if appropriate)
● Identification of who will be investing in the project (as well as to whom the ‘case’
needs to be justified to).

The key question initially will be: What is the base case? Proposal options are evaluated
relative to a base case. CBA cannot be conducted without a base case. The base case
provides the benchmark against which the proposed project can be measured.

5-2
Cost Benefit Analysis Methodology Procedures Manual
5. Cost Benefit Analysis Procedure
5.5. Step 3: Identify Costs and Benefits
Approved by Head, Office of Airspace Regulation Version 1.0: November 2007

Agreeing and defining the base case can be problematic and may benefit from use of a
VM workshop (as well as some ‘internal’ discussion with technical experts within the
organisation proposing the change proposal/project).

5.5 Step 3: Identify Costs and Benefits

All relevant costs and benefits must be included in the evaluation. Quantitative costs and
benefits are to be included in the discounted cash flow analysis, whilst qualitative costs
and benefits must be described and discussed as appropriate. It needs to be recognised
that these can accrue to a wide range of parties and, given the particular situation, could
include some or all of the following: operators of commercial and GA aircraft, the military
(as an operator of aircraft and as a significant user of airspace for training purposes),
passengers on commercial and other aircraft, and operators of air traffic and related
services. Identification of the likely ‘impacted parties’ and the probable nature of such
impacts is a typical example of the sort of questions dealt with well in a VM workshop at an
early stage of the evaluation process.

5.5.1 Identify Quantitative Costs

The nature of the particular proposal being evaluated may be such that there may be a
number of project phases to consider.

The stages of development and the years in which costs are to be incurred needs to be
specified. There may be costs incurred during a planning phase (eg, R&D, testing of
various technologies and equipment applications, user community consultation) as well as
in the implementation and operating phases of the change proposal or other initiative.

There may be a number of capital and other cost components incurred over time that need
to be included in the evaluation such as:
● Capital (or investment) items (eg, equipment or software) – typically one-off
expenditures necessary for the project/proposal
● Land acquisition and land restitution costs (including demolition, land clearance, site
preparation, removal of redundant equipment/facilities, etc)
● Construction costs (incl. professional fees)
● Upgrade or refurbishment costs
● Project management costs
● Decommissioning costs
● Transition costs eg, parts of the existing/current system need to operated and
maintained during the transition period to a new system. While not part of the base
case, these are costs associated with a new investment/change proposal

5-3
Cost Benefit Analysis Methodology Procedures Manual
5. Cost Benefit Analysis Procedure
5.5. Step 3: Identify Costs and Benefits
Approved by Head, Office of Airspace Regulation Version 1.0: November 2007

● Recurrent costs:
❍ Operating, repairs and maintenance costs associated with the new system which
can accrue to civil and/or military users and can be ‘within’ CASA or external to
CASA
❍ Materials, supplies, utilities and other services
❍ Overheads – administration, personnel and training, for example
● User costs:
❍ Travel time
❍ Flying training time for military purposes
❍ Delays or increases in training sortie time for military operations
❍ Restrictions on access to facilities to one or more classes of user
❍ Accident costs
● New training and related implementation costs (eg, new documentation and
stakeholder consultation and communications)
● Safety and risk costs.

In quantifying user benefits, the analyst must determine who the users are and how many
users there are. There are likely to be multiple user groups involved, for example, one or
more classes of aircraft operators, passengers, air traffic control, military aircraft operators,
etc.

In projects where there are likely to be significant improvements to safety or reductions in

risk, then some attempt should be made to quantify these in monetary terms. There may
also be security implications that, where practical, also need to be quantified (at a
minimum, addressed in qualitative terms). Otherwise, the extent to which safety and risk
will be affected should be described qualitatively under the category of unquantified costs
and benefits and in a way that captures any analysis undertaken under the Common
Framework that can be used in the CBA either in a quantitative or qualitative way.

Quantification of safety and risk factors is not easy, but there are various approaches that
may be used which can be important considerations in project evaluation. In such a case,
it is better to quantify the intangibles in monetary terms so that assumptions can at least
be subjected to sensitivity tests. In any event, such items should be identified, a
qualitative description included and, where possible they should be given a subjective
weighting.

5-4
Cost Benefit Analysis Methodology Procedures Manual
5. Cost Benefit Analysis Procedure
5.5. Step 3: Identify Costs and Benefits
Approved by Head, Office of Airspace Regulation Version 1.0: November 2007

5.5.2 Identify Quantitative Benefits

There may be a range of benefits to be estimated and, where possible, quantified such as:
● Cost savings between options (eg, differences in operating and maintenance costs of
equipment and aircraft). Savings in operating costs such as reductions in fuel and oil
costs and flight time related operating costs (primarily crew and parts of maintenance
costs) are treated as benefits in CBA. These may also include cost savings for aircraft
operators due to a reduction in delays and reduced investment, operating and
maintenance costs associated with new technologies. Reductions in fuel and simpler
(more predictable) crew scheduling could result from a reduction in delays and these
will accrue benefits to aircraft operators.
● Asset disposal.
● Residual values (should be included in the DCF analysis as a ‘negative’ cost item).
● User benefits such as travel timesavings for passengers that could accrue due to
accommodating the optimum flight profile as desired by the operator, ie, optimum
routing, altitude and speed. Travel time benefits can also come from a reduction in
aircraft delays.
● Improvements to the ratio of transit time to training time for military operations (training
sorties).
● Incremental net revenue from changes in costs and charges.

5.5.3 External Costs and Benefits

As noted earlier, an externality is defined as any production or consumption process which
‘spills over’ such that other parties (apart from CASA and airspace users) receive a benefit
for which they do not have to pay or incur a cost for which they are not automatically
charged.

Examples to be considered in an ACP cost-benefit analysis could include:

● Environmental considerations (in particular air quality/pollution/aircraft emissions).
Reductions in fuel burn, for example, will have obvious positive implications for aircraft
emissions benefits
● Noise (in particular aircraft take off and landing effects)
● Aesthetic/visual considerations (towers or other physical structures)
● Regional impacts (eg, tourism, industry development, etc).

It is possible to attempt to quantify some of these costs in monetary terms (eg, the SEV
has data on costs associated with air pollution/emissions and other externalities). Where
the analyst cannot attach monetary values to such effects, they should seek to ascertain
who are the ’winners’ and ‘losers’ and the impact of the gain or loss on the economy.

5-5
Cost Benefit Analysis Methodology Procedures Manual
5. Cost Benefit Analysis Procedure
5.6. Step 4: Discount Future Costs and Benefits
Approved by Head, Office of Airspace Regulation Version 1.0: November 2007

5.5.4 Equity and Broader Distributional Considerations

The CBA procedure involves aggregating costs and benefits across individuals without
taking into account the distribution of those costs and benefits between individuals. Total
economic worth will increase so long as total benefits are in excess of the total costs for a
project. CBA is focused on determining whether a particular proposal delivers a net
economic gain (measured in monetary terms) to society as a whole, as opposed to what
parties or individuals receive benefits or pay the costs.

For any particular project/proposal, some individuals (or groups in the community) may be
made better off and others made worse off. For example, people who stand to lose their
peace and quiet or local view due to increased airport capacity and those who gain greater
accessibility and improved air travel frequency will rarely compensate infrastructure.

Whilst CBA cannot resolve equity issues, it is important to draw attention to them by
qualifying the impacts of proposed policy changes on different parties. Equity and
distributional concerns should be considered and should be included in the overall
evaluation if they are likely to be significant. Where information is available, the ‘winners’
and ‘losers’ and the magnitude of their gains and losses should be identified and
discussed in the CBA report. This is to ensure that decision makers are aware of who may
‘gain’ and who may ’lose’ as a result of the project. The size and/or the nature of the
project and/or the significance of likely equity issues will influence the level of effort made
to assess the magnitude of the distribution of gains and losses between different groups.
The recommendation is that, at a minimum, impacted parties and the nature of these
impacts are identified.

5.5.5 Presenting Incremental Costs and Benefits

Economic evaluations should be based on costs and benefits, with project options
incremental to the base case. The most effective way of evaluating a project is to include
all the absolute costs and benefits associated with the options, and then compare the
options to calculate the costs and benefits of the project option(s) incremental to the base
case. This method facilitates data checking, interpretation of results and any subsequent
modifications.

5.6 Step 4: Discount Future Costs and Benefits

5.6.1 DCF Analysis
Discounting – what is it and why do it? As noted earlier, discounting is the reverse of
adding (or compounding) interest. It reduces the monetary value of future costs and
benefits back to a common time dimension – the base year/date.

Discounting satisfies the view that people prefer immediate benefits over future benefits
(social time preference) and it also enables the opportunity cost to be reflected
(opportunity cost of capital).

5-6
Cost Benefit Analysis Methodology Procedures Manual
5. Cost Benefit Analysis Procedure
5.7. Step 5: Calculate the Decision Criteria
Approved by Head, Office of Airspace Regulation Version 1.0: November 2007

5.6.2 Discounting Parameters

The analyst needs to determine the following when preparing to undertake the DCF
aspects of CBA:
● The appropriate price year for cost estimates and the level of prevailing inflation
● Whether analysis of relative prices is necessary for some cost items (eg, labour costs)
● What the base year (or discount year) is to be
● What is to be the base/initial evaluation discount rate
● The evaluation period (or project period).

5.7 Step 5: Calculate the Decision Criteria

As noted previously, all costs and benefits over the evaluation period are discounted to a
present value to enable comparison between overall costs and benefits. This enables the
economic worth of the options to be determined relative to the base case.

It recommended that at a minimum the NPV is calculated and that this should be the key
decision criteria as this is considered to provide a better measure of society’s wealth
maximisation than, for example, the internal rate of return of benefit-cost ratio. In other
words, in an unconstrained market, the option with the highest NPV provides the best
economic return. Where there is a budget constraint however, the NPV/i ratio facilitates
capital rationing and indicates the highest return per dollar invested. It is therefore
possible that an option may well result in a lower NPV but a higher NPV/i ratio than
another option.

While the other measures can be readily calculated eg, IRR, BCR and payback period,
they should be utilised only as supplementary indicators.

5.8 Step 6: Sensitivity Analysis

Sensitivity analysis should be undertaken to test the robustness of results under different
scenarios, using different assumptions for various variables. It is a necessary part of any
investment appraisal as it can:
● Test the impact of using different discount rates (the agreed rate should be used with
sensitivity analysis two or three per cent points ‘above’ and ‘below’ the agreed rate)
● Assess the possible impact of uncertainty
● Illustrate what would happen if the assumptions made about some variables proved to
be wrong and show how changes in the values of various factors affect the overall
costs or benefit of a given project
● Indicate the critical elements on which the positive outcome of the project depends.

5-7
Cost Benefit Analysis Methodology Procedures Manual
5. Cost Benefit Analysis Procedure
5.9. Step 7: Identify Preferred Option
Approved by Head, Office of Airspace Regulation Version 1.0: November 2007

As noted earlier, the development of one or more scenarios should be undertaken, with a
particular focus on a credible ‘worse case’ scenario where a number of assumptions
/variables are assessed in a pessimistic (but realistic) light.

In general, if the proposal/project can be justified under very adverse scenarios, then the
analyst can be confident that the project is robust.

The incorporation of any specific relevant risk analysis should be incorporated at this
stage.

5.9 Step 7: Identify Preferred Option

All relevant results and issues must be considered when identifying the preferred option.

In summary, the identification of the preferred option requires:

1. The ranking of options by NPV and NPV/i and possibly BCR and IRR and other criteria
(eg, payback period) in the initial base evaluation.

2. The ranking of options by NPV and NPV/i in the subsequent sensitivity tests.

3. The weighting of costs and benefits which have only been quantified in physical units
or described in qualitative terms (‘intangibles’) between options, even though this is
inevitably subjective and somewhat arbitrary.

4. The overall ranking of options based on steps (1) to (3).

It is recommended to use NPV and NPVi for decision-making. Where a project is robust,
the ranking of options in the sensitivity tests will usually reflect the ranking of the initial
evaluation. However, the ranking of options in the sensitivity tests may vary in comparison
to the initial evaluation ranking in the case of less robust proposals/projects.

When the unquantified and external costs and benefits are broadly similar in nature,
ranking given by the cost-benefit criteria are usually sufficient and undisputed. However,
when there are significant differences in intangibles between options, a judgement
between competing options will need to be made. This may require an assessment of
whether the net intangible benefits of the second ranked options can be valued at the
difference in NPV between it and the first ranked option.

5-8
Cost Benefit Analysis Methodology Procedures Manual
5. Cost Benefit Analysis Procedure
5.10. Step 8: Prepare Report
Approved by Head, Office of Airspace Regulation Version 1.0: November 2007

5.10 Step 8: Prepare Report

5.10.1 Full Evaluation Report
The final step of the CBA process is producing the report with the appraisal findings and
recommendations. This detailed report should include:
● An Executive Summary of the evaluation, including the results and recommendations
● The background to the evaluation, including
❍ Reasons underpinning the proposal
❍ A statement of why the initiative needs to be implemented immediately
❍ Implications of deferring implementation of the proposal/project
❍ The proposal’s/project’s classification
● The objectives of the project/proposal
● Evaluation considerations
❍ Strategic issues particular to this proposal which will influence both the choice of
the options and the identification of appropriate costs and benefits
● A description of the options, including the base case
● The identification of all costs and benefits, including the key assumptions and inputs
sourced from technical analysis undertaken to inform the CBA
● The annual cost and benefit streams
● The results of the evaluation, including NPV, NVP/i and possibly BCR and IRR
● The results of any sensitivity tests including specific risk modelling/analyses
● A discussion of qualitative factors (costs and benefits)
● The identification of the preferred option and how it compares against the other
options.

5.10.2 Summary Reporting

Summary reporting should contain at least:
● The objectives of the proposal/project and the program goal
● A brief description of the options, in order of preference, including the base case
● The annual cost and benefit streams, including the key assumptions
● Details of any specific technical analysis undertaken to inform the CBA eg, risk
modelling or operations modelling
● The results of the evaluation, including NPV and NPVi
● A discussion of qualitative factors including risk and issues of uncertainty
● Details of any supplementary analysis.
■

5-9
Cost Benefit Analysis Methodology Procedures Manual
5. Cost Benefit Analysis Procedure
5.10. Step 8: Prepare Report
Approved by Head, Office of Airspace Regulation Version 1.0: November 2007

Blank page

5-10