Institute for Transport Studies

FACULTY OF ENVIRONMENT

Transport Appraisal in other

countries: lessons for the NATA
Refresh
Report - October 2007

A Report to the Department for Transport

Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

Transport Appraisal in other countries: lessons for the NATA Refresh

Version 1.1

Report - October 2007

Peter Mackie
Charlotte Kelly
Institute for Transport Studies
University of Leeds
Leeds
LS2 9JT

Tel +44 (0) 113 3435336

Fax: + 44 (0) 113 3435334

2
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

CONTENTS
1 Introduction ................................................................................................................................... 4
2 Assess whether the key HEATCO findings are correctly summarised .......................................... 5
2.1 Valuation of local and regional air pollution............................................................................ 8
2.2 Development of the rail appraisal guidance and the standard values for rail franchise
operating costs ................................................................................................................................. 8
2.3 Extension of the NATA framework to cover aviation and maritime appraisal ......................... 9
2.4 Use of monte Carlo techniques and sensitivity testing in assessing optimism bias ............... 9
2.5 Value of time – Meta Analysis.............................................................................................. 10
3 What did the HEATCO programme learn about how transport appraisal results are used to
inform decision making in different countries ..................................................................................... 10
4 Danish (2002) and (2007) study results ...................................................................................... 10
5 Recent Studies of transport appraisal practice in other countries ............................................... 13
5.1 PIARC .................................................................................................................................. 13
5.2 CfIT – High Speed rail: International comparisons............................................................... 14
5.3 OEI (NL)............................................................................................................................... 15
5.4 RAILPAG ............................................................................................................................. 16
6 Given the scope of the NATA refresh what lessons can be learned from all of these studies that
can be incorporated into the NATA Consultation document?............................................................. 18
7 References.................................................................................................................................. 19

3
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

1 Introduction
The Institute for Transport Studies was commissioned by the Department for Transport to review the
evidence base on transport appraisal in other countries to inform the NATA refresh process that is
currently taking place. We were asked to:

• Assess whether the key HEATCO findings are correctly summarised

• Consider what the HEATCO programme learned about how transport appraisal results
inform decision making in other countries

• Look at how the conclusions from COWI (2002) and Danish Transport Research Institute
(2007) compare with the findings of HEATCO

• Assess whether there are any other recent studies that exist of transport appraisal practice
in other countries and what are their conclusions?

• Consider what lessons can be learned from these studies that can be incorporated into the
NATA consultation document.

This report will look at each of these in turn.

4
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

2 Assess whether the key HEATCO findings are correctly summarised

The HEATCO project primary objective was the development of harmonised guidelines for project
assessment at an EU level. As part of this process the project undertook a survey of current practice
in project appraisal in Europe detailed in Odgaard et al (2005) and HEATCO (2005a). A summary of
this information is provided in table 1 and considers all 25 EU countries. This was an extension of
the work originally completed by Grant Muller et al (2001). It should be noted that since the time that
this has been produced appraisal practice has changed in some of the countries, for example, in the
UK with Noise values (TAG 3.3.2) and carbon values (TAG 3.3.5) both introduced in 2006. This
survey was followed by HEATCO (2005b), which considered the state of the art of in project
assessment for the key elements of appraisal and considered in detail the key differences between
how the countries in the EU conduct their appraisal practice. One of the contexts for the HEATCO
report was the appraisal of international projects and other projects with transboundary effects,
including the TEN projects.

Annex 1 is the section of the DfT note which describes the key differences in transport appraisal
practice between EU countries. This is an accurate reflection of the work that was conducted on the
initial surveys that looked at the 25 individual countries (presented in HEATCO, 2005a). However DfT
does not appear to have reviewed the later HEATCO deliverables. We strongly recommend that a
member of the NATA-refresh team reads HEATCO D5 (HEATCO 2006b) and Annex A to that
Deliverable (Shires and de Jong, 2006) since there are many tables of results which can help to
benchmark the UK against other EU member states. All the HEATCO deliverables can be
downloaded from http://heatco.ier.uni-stuttgart.de/

The findings summarised in the DfT note (taken from HEATCO) were:
• there are significant regional differences in the approach to, and tradition for using, transport
project appraisals;

• the appraisal frameworks used for roads are far more developed than those for rail, air, inland
waterways and sea transport;

• there is a lack of consensus about which elements to include within appraisals, especially
environmental effects;

• there is a lack of consensus on approaches to valuing impacts; and

• there is a significant range of values used, especially for safety.

We believe for the majority of countries the list above is still a good general summary. Point 3 could
be qualified with the statement that certain direct impacts such as construction and maintenance
costs and travel time and vehicle operating costs are routinely included.

5
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

Table 1 Coverage by country and element in case CBA is used1,2

operation and administration

Disruption from construction

User charges and revenues

Air pollution - local/regional

Passenger transport time

Indirect socio-economic
Benefits to goods traffic
Vehicle operating costs
Costs for maintenance,
Construction costs

Climate Change
savings

effects
Safety

Noise
Austria (*)
Belgium (*+)
Denmark (*)
Finland (*)
North/West

France (*)
Germany (*)
Ireland (*)
Netherlands (*)
Sweden (*)
Switzerland (*)
UK (*)
Czech Republic (*)
Estonia (+)
Hungary (*)
Latvia (+)
East

Lithuania (*)
Poland (+)
Slovak Republic (*)
Slovenia (*)
Cyprus (+)
Greece (+)
South

Italy (+)
Malta (*)
Portugal (+)
Spain (*)
Colour codes:
: Included with a money value
: Measured quantitatively, qualitatively or not included

source: HEATCO(2005a)

The full HEATCO recommendations are provided in Annex 2. Table 2 presents a comparison
between the UK appraisal conventions and the HEATCO guidelines.

1 6
It appears that all countries include some effects with a money value for inclusion in CBA. Caution should be applied when interpreting this
result, as CBA is not a requirement in all countries and as such is only used under some special circumstances. Furthermore, it should be noted that
the figure provides a summary for all modes, i.e. it might be that some effects are not included for some modes. (*) - reflects recommended/required
approach; (+) - reflects typical approach when CBA is used.

2
Since the production of this table some appraisal practice in countries have been updated.

6
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

Table 2 Overview of general assessment approaches (UK and HEATCO)

HEATCO UK
Appraisal Period Planning and construction + 40 years 60 years
Discount Rate National Values of risk premium – free 3.5% for the first 30 years, 3% for the
rate and sensitivity testing at 3% remaining 30 years
Unit of Account Factor costs Market prices
Currency Euro GBP
Marginal costs MCPF= 1 and use a cut off value of MCPF = 1 (all UK) (RNPSS = 1.5 in
of public funds RNPSS = 1.5 England only)
Scenario Reference scenario as “ do minimum” Reference scenario as “ do minimum”
Definition
Decision criteria NPV, BCR, RNPSS (Ratio of NPV and NPV, BCR
Public Sector Support)
Treatment of Sensitivity analysis at a minimum, UK provides less prescriptive advice
future risk and Monte Carlos simulations as a more and just recommends undertaking a risk
uncertainty sophisticated approach analysis
Equity issues Winners and losers tables at a Disaggregation of impacts between
(inter- minimum, distributional matrices as a stakeholders categories and mode to
generational) more sophisticated approach identify winners and losers
Treatment of Qualitative assessment at a minimum. Framework approach to appraisal
indirect socio- Use of spatially computable general based around the 5 core objectives:
economic equilibrium models where possible environment, safety, economy,
effects integration, accessibility,
Price base 2002, constant prices + PPP adjusted 2002
prices
Treatment of Adjustment on the basis of national GDP growth rates
values over time GDP growth rates (elasticity = 1) except The value of non-working time is
for global warming and VTTS (0.7) assumed to increase with income, with
an elasticity of 0.8. Working values of
time are assumed to grow in line with
income, with an elasticity of 1.

Turning to the individual elements of benefit, these are not easy to summarise succinctly, but among
the findings of HEATCO D5 were:
• at purchasing power parity, UK employers business values of time were around 10% above
the European average, UK non-work time 10% below and UK freight values close to average
• safety values were typical for N and W Europe; some countries such as Norway, Denmark
and Switzerland are higher. The rest of Europe is much lower even at PPP. The relativities
between fatal, serious and slight are typical.
• D5 gives a recommended approach and values for NOx, NMVOCs,SOx,PM2.5s,and noise
• D5 gives a recommended approach and values for carbon emissions but this predates the
Stern Report and is not recommended for use in the UK.

The DfT note identified the following areas from the HEATCO review and General Recommendations
for areas where the NATA framework might be developed. These are:

7
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

1. The valuation of local and regional air pollution

2. The development of the rail appraisal guidance and the standard values for rail franchise
operating costs
3. Extension of the NATA framework to cover aviation and maritime appraisal
4. Use of Monte Carlo techniques in assessing optimism bias.

In addition to these areas we would also suggest considering the results from the value of time
savings meta analysis results that were produced as a result of the project (see annex 3) and
summarised below in section 2.5. HEATCO (2005b) reviewed the state of the art in project
assessment and HEATCO (2006) produced a proposal on the harmonisation of the appraisal
practices in practice and these reports will be used in the next section to summarise the findings from
HEATCO for the areas identified by the DfT note.

2.1 Valuation of local and regional air pollution

HEATCO (2005a) found that the majority of EU countries have values for PM, NOx, SO2, HC and CO
for local/ regional pollution. The majority of these countries use the Impact Pathway approach to
base their monetary values. Other approaches include damage cost and avoidance cost
approaches. The HEATCO recommendation for the valuation of local/ regional pollution was
(HEATCO, 2006b):

To use country specific values taking into account local population density and regional climate. The
list of pollutants that should be covered are:
• Primary PM2.5 for transport emissions
• NOX as precursor of nitrate aerosols and ozone
• SO2 for direct effects and as precursor of sulphate aerosols
• NMVOC as precursor of ozone

The HEATCO report provides cost factors for the UK for road transport and the methodology to use
to calculate the monetary values.

HEATCO identified that in appraisal the main focus is on monetising impacts for noise, air pollution
and global warming. However other environmental impacts such as vibration, severance, visual
intrusion, loss of important sites, resource consumption, impairment of landscape, soil and water
pollution are included but rarely covered by assigning a monetary value. Countries such as the
Netherlands are looking into values for some of these aspects, but have found that the resulting
monetary values can be very site specific (see section 5.3). The study by PIARC (2004) (see section
5.1) also found that some countries are placing values on these other impacts.

2.2 Development of the rail appraisal guidance and the standard values for rail
franchise operating costs
The HEATCO work did not look at standard values for rail franchise costs.

8
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

The HEATCO work did look at the elements that are included in rail appraisal in the 25 EU countries.
They identified that the appraisal framework for rail was less standardised than for roads. The
development of rail appraisal guidance is something that has been recently considered in RAILPAG –
(EU and EIB, 2005) Railway Project Appraisal Guidelines (see section 5.4) in more detail.

2.3 Extension of the NATA framework to cover aviation and maritime appraisal
The HEATCO survey found that several countries have entirely separate frameworks for analysing
different modes and only around one third of countries had formulated standard frameworks for the
appraisal of air, inland waterway and sea transport projects. In the Nordic countries general
appraisal guidelines have been developed that are applied to all sectors (see section 4).

2.4 Use of Monte Carlo techniques and sensitivity testing in assessing

optimism bias
HEATCO (2005b) analysed the requirements in other countries for sensitivity analysis. This found
that in most cases a sensitivity test is required on traffic growth and the value of time. In some cases
a sensitivity test is also required for the discount rate, real GDP growth and investment and safety
costs. NATA currently has the following guidelines for sensitivity testing (TAG 3.9.2):

• the level of patronage at which the net benefits of the scheme are zero.
• the level of cost overruns at which the net benefits of the scheme are zero;
• higher operating costs;
• the level of time savings (where appropriate) at which the net benefits of the scheme are
zero; and
• the responses of incumbent operators, such as fare and frequency responses of existing
bus operator responses to new light rail systems or continuing service of on-street stops
despite the provision of new interchange facilities.

While the UK approach is in line with the majority of countries HEATCO made some
recommendations that take this further. The recommendations was that sensitivity analysis should
be the minimum used and Monte Carlo simulations used as a more sophisticated approach. With
regards to the specific impacts the following recommendations were made:
• Sensitivity testing for the discount rate at 3%
• For the National VTTS values it is recommended that an appraisal should be tested using
+/-20% of the national VTTS values. This is different to the recommendations in TAG 3.9.2
whereby it suggests testing the level of time savings (where appropriate) at which the net
benefits of the scheme are zero.
• For the safety values the recommended sensitivity tests are v/3 and v*3, where v = value of
safety per se.

9
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

2.5 Value of time – Meta Analysis

One part of the HEATCO work was to produce an international meta analysis on values of travel time
savings (VTTS). The results are presented in Shires and de Jong (2006)—see Annex 3. This work
reached a number of conclusions. Which are summarised as:

• A (largely cross-sectional) income elasticity of the VTTS of about 0.5 for business travel,
0.7 for other passenger transport and 0.3 for freight.
• Long distances are associated with higher VTTS for commuting and other purposes.The
current M6 toll study is expected to shed further light on this.
• SP and SP-RP studies give somewhat lower VTTS for employers business trips than the
cost savings approach.
• Significant effects for purpose (business, commuting) and mode (especially for air; less so
for bus relative to car and train).
• Higher values of time in Southern and Eastern European countries, all other things
(including GDP/capita) being equal.

3 What did the HEATCO programme learn about how transport appraisal
results are used to inform decision making in different countries

While HEATCO did not focus on how transport appraisal practice specifically informed decision-
making the PIARC report (section 5.1) asked the countries whether the evaluation framework was
accepted and how the results of the evaluation were used and the CfiT study (section 5.2) also
considered this issue. The road directorate in Norway analysed the extent that CBA results affected
the prioritisation of projects in 15 countries and found that in 5 out of 15 countries it was significant in
explaining the ranking (Danish Transport Research Institute, 2007:18)

Previous work for the EU in projects such as TRANSTALK has addressed this issue (Giorgi, 2002,
Pearman, Mackie and Nellthorp,2003).Our opinion is that the countries with the best developed CBA
tradition and evidence base are the Netherlands, Sweden and the UK. These countries also have
somewhat similar cultures regarding the role of CBA in decision support. There is therefore most to
be gained from a close comparison of decision support practices and the role of appraisal in those
countries.

4 Danish (2002) and (2007) study results

Two reports have recently been published by the Danish Transport Research Institute considering
the use of socio-economic assessments as the basis for decision-making in transport. The 2002
report considered the appraisal practice in 7 countries (Finland, France, Holland, Norway, UK,
Sweden and Germany). It then examined in more detail four of these countries (France, Norway, UK
and Sweden). The 2007 report updated the previous report for the four Nordic Countries (Sweden,
Norway, Finland and Denmark). Additionally the second report considered how the four countries

10
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

appraisal practice compared to the HEATCO recommendations3. Table 3 provides a summary of

some of the components used in economic evaluation in the countries reviewed and compares it to
the HEATCO recommendation.

Table 3: Summary of DANISH 2002 and 2007 reports

HEATCO DNK SWE NOR FIN DEU NLD FRA
(2007) (2007) (2007) (2007) (2002) (2002) (2002)
Discount Rate National 6% 4% 4.5% (risk 5% 3% 4% 8%(5%)
(real) Values of free 2%,
risk risk
premium – premium
free rate 2.5%)
and
sensitivity
testing at
3%
Unit of Account factor market market market factor factor factor market
Marginal costs - 20% 30% 20% - - - -
of public funds
Decision NPV, BCR, NPV, NPV, NPV, BCR, CBR NPV NPV,
criteria RNPSS BCR, BCR BCR,FYB, NBCR CBA,
IRR IRR FYB,
IRR
Local Air Nox Particles Particles Particles Particles Particles Particles PM10
pollution – So2 NOX NOX NOX NOX NOX NOX (only
Emissions NMVOC SO2 SO2 SO2 SO2 SO2 emission
included PM2.5/ HC HC CO CO CO with
PM10 VOCs VOCs HC HC HC monetary
Carcin- value)
ogens
Valuation DC DC DC SP, DC DC - - DC
method
Global air yes yes yes yes yes yes yes yes
pollution
(valued)
Valuation DC &AC AC AC AC DC AC AC Tax
method
Differentiation Purpose - Purpose, Mode of Mode, Mode, Purpose Mode, Mode,
of passenger Passenger type of transport purpose, purpose, purpose purpose,
VoT work and time, Purpose. type of time, type of type of
non work Type of distance time, time,
trips time, population
distance groups,
comfort,
distance
Method WR/ CSA WR, SP SP, WR SP, WR WR SP SP RP, SP,
for work IC, WR
trips
Last study 2006 1995 1997 1999/2000 1991 1998
DC = Damage cost, SP = Stated Preference, AC = avoidance costs, WR = wage rate, CSA = Cost
saving approach, IC = International Comparisons

3
The 2007 report will be used as the basis of comparison for the Nordic countries the 2002 report will be used for the other countries

11
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

All of the countries use a socio-economic method to some extent in the decision making process.
For the Nordic countries this very often provides an input into the national transport plans. The report
paints a similar picture to the results in the HEATCO study, which also found that the road sector is
the furthest developed in terms of socio economic methodology.

The 2007 review identified a number of differences in the way that dealing with uncertainty was
covered. In Finland sensitivity analysis is conducted on the central parameters, but there is currently
no formal requirement for doing so. In Sweden uncertainty analysis is dealt with on an ad hoc basis
for investment costs, traffic levels and the discount rate. This is different to Norway where it is a
minimum requirement to conduct an uncertainty assessment for investment costs and traffic levels.4

The reviews also looked in more detail at how non-monetised impacts were dealt with in the
appraisal. Currently there are no formal methods for dealing with these impacts in Denmark and
Finland. Sweden has recently been working on the development of an impact profile. In Norway the
excluded non monetised effects are split into the following themes; local environment, recreational
life, natural environment, cultural relics and cultural environment, the landscape image, agriculture
and fisheries, geographic and water resources, transport quality, local development patterns,
accessibility for cyclists. Each of these themes are assessed firstly by value and then by extent, then
these are combined to generate a society value. The results are then presented alongside the CBA.
From the 2002 review it was identified that with the exception of the UK for the other countries
“systematic methods for taking account of the non included effects did not appear to have been
developed” (2002, p41). It was reported in (2007) that it was felt that impacts that are not monetised
and included in the CBA do not get sufficient attention in the decision making process. The Nordic
countries were all working to improve this element of their evaluation. In presenting the results of the
appraisal the Norway and Swedish versions are very similar to the AST in the UK.

The review found differences between the countries in how the socio- economic evaluations are
applied in the different countries. For example in Sweden the socio – economic evaluation is used to
conduct an analysis of alternative potential plans for the next 10 year national transport plan (which is
updated every 4 years). This is used to provide evidence for the first of the three national transport
policy objectives, which are: welfare economics, traffic safety and the environment and regional
development, which is then given to the politicians to decide on the framework. At a National
decision making level the results of the CBA cannot be directly recognised in the decisions made, but
they are an important element of the evidence given to politicians for making a case. The Swedish
guidelines dictate that sector authorities should conduct “post”- analyses for major projects in excess
of SEK 50-100million 2-3 years after opening. The Norwegian process is very similar to Sweden.
One of the problems that was identified in Norway was a difficulty in comparing projects across
transport sub-sectors. A working group was set up to “investigate the extent to which the
methodologies used by the different sectors (road, rail, sea and air) were consistent”. This work has
apparently improved consistency in assumptions and economic principles.

All of the countries considered had evaluation guidelines and have produced models to aid in the
evaluation. A particular consideration to note is that in Norway all sectors have to follow the general

4
there is no change in the descriptions for the Nordic countries between 2002 and 2007 for uncertainty

12
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

guidelines (Norwegian Ministry of Finance, 2005). However because these guidelines were too
general this has led to the rail and road sectors developing their own guidelines. The rail guidelines
were based on the road guidelines, so there is a high degree of consistency. In Finland general
guidelines have been developed that should be followed in all sectors. One problem is that because
they are very general there is a degree of freedom over which people can use them. In Denmark the
guidelines for conducting socio – economic analysis set out the principles for the different sectors
have been developed in conjunction with a working group from the road, rail and Danish Transport
Research Institute.

5 Recent Studies of transport appraisal practice in other countries

5.1 PIARC
In 2001 PIARC conducted a survey to consider the methods of transport appraisal for roads used in
its member countries (PIARC, 2004). The survey found that like the HEATCO survey a wide range
of appraisal frameworks were used with almost all countries using CBA in some part of the
evaluation. The countries that returned the questionnaires in 2001 were; Australia, Canada, Czech
Republic, Denmark, France, Germany, Hungary, Japan, Mexico, Netherlands, New Zealand, Norway,
Portugal, Switzerland, Sweden, South Africa, UK and USA. A table was produced within this report
detailing in 2001 what impacts were included/ not included and which were included as a monetised
impact or non monetised impact. The general story is that almost all countries use CBA as a
component of the evaluation often linked with MCA or an environmental or socio- economic analysis.
All countries include road user time costs, Vehicle operating costs and most include accident costs in
their economic evaluations. Externalities are generally non monetised, with the exceptions of those
listed in Table 4.

Table 4 Modified from PIARC(2004) Table 2.2 (2001 survey)

Impacts
Comfort
Air pollution – local
Air pollution – regional
Air Pollution - Global
Dust Nuisance
Community Severance
Impact on employment
Mitigation/ clean up
Countries that monetise the impacts
France, New Zealand, South Africa
Czech Republic, Denmark, France, Germany, Japan, New
Zealand, Norway, Sweden
Denmark, France, Germany, Japan, Norway, Sweden
Australia, Denmark, France, Germany, Japan, New Zealand,
Norway, Sweden
Germany, New Zealand, Norway, Sweden
Germany,
Canada, Germany
Germany, Norway

PIARC (2004) considered what methods were used to compare projects and found like the other
studies that there is great variation among the countries. For example in Japan “CBA is used for
acceptance or rejection of the project. Prioritisation depends on an MCA allowing for social factors
and the importance of implementation” (p67). In New Zealand projects are initially prioritised based
on the BCR and then Government’s stated transport priorities are considered alongside the BCR and
may lead to a re ranking. Whereas in Portugal “decisions on which project proceeds are based more

13
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

on the plan to complete the main and complementary road networks than on the economic
evaluation.” (p69). In Sweden economic evaluation is the tool used for deciding on which projects
should be included in the 10 year plan and at the project feasibility stage. While South Africa uses
economic evaluation of road projects to “select project options, prioritise competing projects and for
global road network strategies”(p21).

PIARC also considered whether there was any acceptance for the evaluation framework used in
each country. They found that generally there was acceptance for economic evaluation. In New
Zealand prior to 2002 there was an almost perfect correlation between the BCR and implementation.
This has recently been modified with the inclusion of government transport priorities. In Sweden
there is a good correlation between implemented projects and projects that did well on the appraisal.
They found that exceptions were mainly for political reasons. This is a similar story to Denmark and
Mexico. In the Netherlands, high priority from evaluation does not necessarily guarantee
implementation : “public consultation for road projects in the Netherlands often leads to disagreement
of the weighting of impacts”(p71).

The key recommendations from this report are that PIARC member countries should:
• Continue to refine and develop CBA and MCA methodologies
• Cooperate in developing monetary values for environmental and social impacts and methods
for their inclusion in project evaluation
• Expand the project evaluation methodologies to cover all land transport modes
• Give more attention to use of quantitative risk analysis for projects with significant risks or
which are controversial

5.2 CfIT – High Speed rail: International comparisons

The Commission for Integrated Transport (CfiT) commissioned Steer Davies Gleave to consider what
changes could/ should be made to Bitain’s appraisal criteria and processes to better capture the
costs and benefits of high speed rail. In doing this CfIT(2004) conducted 7 case studies in the
countries of Britain, France, Italy, Spain, Germany, Japan and Australia to identify what elements of
the appraisal had hindered/ helped in the goal for high speed rail investment. The key differences
found between the appraisals include:
• All countries with the exception of Japan consistently use CBA for rail projects.
• Only Spain, Germany and Britain formally use MCA.
• Regional economic benefits were quantified in Spain and Germany
• Estimates of the employment generated by a project are included at least in part in Spain,
Germany and France.
• Britain does not count the benefits of high speed rail to non residents within the CBA.
Germany applies a multiplier to international projects producing the reverse effect.
• Spain and France count additional tax revenue as a benefit.
• European countries are less good at assessing the wider economic impact of major transport
projects – only Japan undertook this analysis in detail.

14
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

In terms of how the appraisal results inform the decision this review identified that in Germany the
appraisal process appears to directly determine whether projects are included in the national
transport plan. In Spain economic appraisal is required by law, but only determines the prioritisation
of projects (the “when” and “how”) not whether the project should be undertaken at all. In the case of
Japan the approach is a combination of financial and economic impact appraisal, but equally
important are issues of national prestige.

One of the conclusions of this review was that “in many respects British Appraisal Framework
represents best practice”. However a number of recommendations were proposed that relate
specifically to current appraisal practice and high speed rail, which were:

• Values of time vary significantly between region and type of journey. It is recommended to
use project specific values where projects are large and distinctive in their journey time
benefits
• The allowance for risk and optimism bias in the new green book for capital intensive projects is
excessive as applied to high speed rail.
• Britain should consider undertaking analysis of the wider national economic impact of major
projects such as high speed rail. Evidence from Japan and the Netherlands is that when this
assessment is undertaken the analysis of high speed rail economic impacts indicates them to
be higher than revealed by narrow CBA alone.

5.3 OEI (NL)

In 2000 the OEI – guide was published in the Netherlands. (CPB and NEI, 2000). This is the Dutch
standard appraisal guidance on CBA, which states that a CBA must be carried out for all large
governmental infrastructure projects and can be considered similar to NATA and its supporting
materials such as WebTAG. A comparison with neighbouring countries indicated that the
Netherlands “spent a lot of attention on determining the relevance of indirect impacts” (p. xi). The
Netherlands conducted an evaluation in 2002 of its appraisal practices based on the original
document produced in 2000. They found that there was a lot of support for the Overzicht Effecten
Infrastrcture (OEI) and that people felt that it made the information transparent and was factual.
However there were a number of areas where it was felt that improvements could be made to the
guidelines. From this evaluation a number of process and content recommendations were made to
improve this guidance and these were reported in Buck Consultants Ltd (2002). A summary of the
key recommendations for process changes were:
• An agreement should be made on the scope of a CBA
• Should determine at which point in the planning CBA should be applied
• Should Stimulate market widening for the exporting and testing of CBA
• Join up the OEI and the MER (environmental assessment) in the appraisal
The key content recommendations were:
• Share the knowledge of the guide with all stakeholders that are involved
• Broaden the guide for a number aspects:
o Expand the methodology for indirect effects
o Conduct monetisation studies for external effects

15
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

o Determine how to conduct the equity analysis (the distribution of effects between CBA
stakeholders)
• Develop an unambiguous and practically applicable definition for the business as usual or
reference case
• More Input on freight appraisal
• Set guidelines for a balanced presentation of the CBA.

One result of these recommendations was that the Dutch Ministry of Transport, Public Works and
water management together with the Dutch Ministry of economic Affairs started a research
programme to address some of these issues. The result of this was that a series of additional
guidelines (appendices to the guidance) were published. These included a study on the distribution
effects of CBA, risk, the monetisation of nature, water and soil and the value of reliability in transport.
The distribution effects report (MVW, 2004a) was a methodological report, which concluded that
more research was required before the guidelines could be updated. The risk report (MVW, 2004b)
concluded among other things that a discount rate of 4% (risk free) should be used and that the risk
premium should be 3% as the default value. The nature, water and soil report (MVW, 2004c)
provided an example of a case study where the external effects of nature, water and soil and
recreational changes had been monetised for a rail project. For example, in the CBA, wood, drinking
water, clean surface water, recreational possibilities, recreational disturbance, residential
Unemployment and inheritance nature were all given values through a combination of commercial
values (e.g. wood and drinking water) hedonic pricing for the recreational aspects and WTP for the
inherited nature. One example is that recreational possibilities lost as a result of the project were
valued at 0.5 euros per trip per household. The conclusion of this work was that the monetisation of
the impacts were very location specific and more work would be needed. The reliability report was
published in 2005 (Rand Europe and Transport Research Centre, 2005) and was commissioned to
research filling the gap in appraisal for including the effects of transport time reliability in the CBA.
The report has published provisional values of reliability for passenger transport by car and public
transport (0.8 = car reliability ratio, 1.4 = public transport reliability ratio). The report concluded that
there was no consensus on values of reliability for freight and that more work was needed in this
area.

5.4 RAILPAG
The development of rail appraisal guidance is something that has been recently considered in
RAILPAG (Railway Project Appraisal Guidelines) published by European Commission and European
Investment Bank (2005). The aim of this guidance was to provide recommendations for a common
framework for the appraisal of rail projects across the EU. It is based on the work carried out for
TINA (Transport Infrastructure Needs Assessment) in 1999. The guidelines recommend an extended
CBA approach (p42) coupled with a stakeholder effects matrix to present the results. For each
project the relevant effects and stakeholders have to be identified. A simplified stakeholder effects
matrix is presented in table 5. Typical effects for the rail industry refer to user service (e.g. fares,
travel time reliability etc...) operation (e.g. direct and indirect costs), assets (e.g. investment) and
external effects (e.g. environmental impacts). The matrix is completed by filling in each cell with the
NPV of the effect to the stakeholder. The NPV for the project can then be calculated by summing the
value in each cell. Non monetised effects are dealt with using a colour coding system. “A cell is
coloured green for mild effects, yellow for those that deserve some attention and red for those that

16
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

have more substantial weight in the decision” (p44). The presentation of the results in this manner
can be used to determine the economic weight that the non monetised impacts should have and
clarifies how costs and benefits are distributed.

Table 5 Simplified RAILPAG Stakeholder effect matrix.

SE STAKEHOLDERS
MATRIX Users Transport Insurance Contractors Infrastructure Non Government
Service Companies & suppliers Managers users
Operators
USER
SERVICE
EFFECTS

OPERATION
ASSETS
EXTERNAL
EFFECTS
Source: RAILPAG (p43)

17
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

6 Given the scope of the NATA refresh what lessons can be learned from all of
these studies that can be incorporated into the NATA Consultation
document?

The conclusions we draw are as follows:

1. The UK remains at the forefront of the use of CBA to inform decision making. While
there are certain technical points on which UK practice differs from the European norm
and the HEATCO recommendations, this is not really a problem.
2. There has been an element of opaqueness since the revised Green Book in the
treatment of benefits to foreign travellers within the UK. These are far more important
in European appraisal than in the UK and a degree of consistency with European
appraisal practice in this area is desirable.
3. There also needs to be consistency among Western European States in the valuation
of transboundary pollution and in particular carbon emissions.
4. If changes are envisaged to the treatment of financing costs in CBA, these might benefit
from discussion with counterparts in Sweden, the Netherlands and the EIB.
5. The UK seems to be lagging some other countries by not yet using money values for
local and strategic pollutants other than carbon. Addressing this properly will involve
significant modelling and valuation challenges (eg PM2.5s)
6. More dubiously, other countries seem to have made progress in areas such as
biodiversity and natural resources which have always been considered difficult in the
UK because of their context-specificity. It might be worth looking at how convincing the
Dutch work is.
7. There is scope for cross-fertilisation with the Dutch and Swedish on topics such as the
value of travel time with income,distance and purpose and reliability values.
8. Continuing to articulate the role of CBA in decision support and policy analysis is
essential. This is the strongest signal that appraisal is taken seriously, as an input to
decision making.
9. Outside the studies reviewed here, we recommend continued liaison with European
academics and policy makers in the approaches to estimating and valuing wider
economic benefits.

18
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

7 References

Buck Consultants Ltd (2002) Evaluatie OEEI-leidraad

http://www.verkeerenwaterstaat.nl/Images/Evaluatie%20OEI-leidraad_tcm195-160558.pdf

CfIT (2004) High Speed rail: International comparisons

http://www.cfit.gov.uk/docs/2004/hsr/research/

CPB and NEI (2000) Evaluation of Infrastructural projects; Guide for cost benefit analysis. Section 1:
Main report. Research programme on the economic effects of infrastructure

Eddington R (2006) The Eddington Transport Study. Main Report: Transport’s role in sustaining the
UK’s productivity and competitiveness, December 2006 Stationery Office

European Commission and European Investment Bank (2005) Railway Project Appraisal Guidelines -
RAILPAG
http://www.railpag.com/index.php

Danish Transport Research Institute (2002) Use of socio-economic methods in selected countries.

Danish Transport Research Institute (2007) Appraisal methods in the Nordic Countries. Transport
Infrastructure assessment. Report 3.

Giorgi L et al (eds) (2002), Policy and Project Evaluation in Transport, Ashgate, Aldershot see
especially chapter by W. Parsons

Grant- Muller S. M., P. Mackie, J. Nellthorp and A. Pearman (2001): Economic Appraisal of European
Transport projects: The state of the art revisited. Transport Reviews 21(2) pp237-261

HEATCO (2005a) Current practice in project appraisals in Europe. Analysis of Country Reports.
Deliverable 1.
http://heatco.ier.uni-stuttgart.de/

HEATCO (2005b) State of the Art in project assessment. Deliverable 2.

http://heatco.ier.uni-stuttgart.de/

HEATCO (2006a) Case Study Reports. Deliverable 6.

HEATCO (2006b) Proposal for Harmonised Guidelines. Deliverable 5

http://heatco.ier.uni-stuttgart.de/

Ministerie van verkeer en Waterstat and Ministerie van Economische Zaken (2004a) Verdeling van
Effecten Infrastructuurprojecten. (Dutch – distribution analysis)

19
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

Ministerie van verkeer en Waterstat and Ministerie van Economische Zaken (2004b)
Risicowaardering (Dutch – risk analysis)

Ministerie van verkeer en Waterstat and Ministerie van Economische Zaken (2004c) Waardering van
Natuur, water en Boden in Mattschappelijke Kosten- Batenanalyses (Dutch – nature, soil and water
report)

Norwegian Ministry of Finance(2005) veilder for samfunnsokonomiske analyser, finansdepartmentet,

2005, http://www.finans.dep.no

Odgaard T. , C. Kelly and J. Laird J (2005): Current practice in project appraisal in Europe. Analysis
of Country reports. HEATCO Deliverable 1.

Pearman A.D., Mackie P, and Nellthorp J (eds) (2003), Transport Projects,Programmes and
Policies—Evaluation Needs and Capabilities, Asngate, Aldershot.

PIARC (2003) Economic evaluation methods for road projects in PIARC member Countries
http://publications.piarc.org/ressources/publications_files/1/628-09-07-VCD.pdf

RAND Europe and Transport Research Centre (2005) The value of reliability in transport. Provisional
values for the Netherlands based on expert opinion.

Shires J and De Jong G C (2006) An international meta analysis of values of time. Annex A
HEATCO deliverable 5.

20
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

Annex 1: Differences in transport appraisal practice between EU countries

The EU’s HEATCO programme recently reviewed transport appraisal practice across EU states. It
concluded that:

• Whilst all countries use cost-benefit analysis in some form, the guidelines about its application
vary considerably across countries and between modes. Guidelines for Northern/Western
countries are much more comprehensive than those for Southern/Eastern countries;

• Appraisal frameworks for rail are less standardised than for road and only one third of
countries have guidelines for air, inland waterways and maritime projects;

• There are large differences between countries in terms of whether, and how, the main impacts
of transport proposals are appraised. For example:

- most countries place monetary values on; construction costs, system operating and
maintenance costs, passenger and freight time savings, user charges/revenues and
safety impacts;

- only half of EU countries (and no Southern countries) monetise noise impacts. Only half
of countries monetise local/regional air pollution impacts;

- most countries do not place monetary values on climate change effects and disruption
from construction impacts

- Eastern and Southern EU countries seldom place monetary values on environmental

impacts.

• there are differences between countries about which unit of account (market price or factor
cost) is used, discount rates and appraisal periods;

• only a few countries include distortionary impacts from tax financing and transboundary
effects. However where this is done, there is a consensus on the approaches used;

• differences exist about which elements of construction costs to include, residual lives and
asset lives. Most countries adjust costs for optimism bias, but only a few do so using advanced
methods;

• only half of the countries have standard figures for system operating and maintenance costs;

• all countries include travel time savings and most (particularly Northern/Western EU countries)
disaggregate by work/non-work and by mode. However there are differences in how countries
assess time savings. For work trips, the most popular approach is the cost saving approach,
whilst non-work trips tend to be based on willingness to pay (particularly in Northern/Western
countries) and a relationship to the wage rate;

21
Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

• all countries include vehicle operating costs associated with road transport and mainly to a
standard definition;

• there is a significant range of monetary values for safety is used and no consensus on whether
values should change over time. About one third of countries use a non-standard definition of
accident types;

• of those countries that place a monetary value on noise, nearly all are based on Hedonic
Pricing studies and all include noise annoyance in the valuation, but only a few include health
costs related to noise. Not all countries increase noise values over time;

• for local/regional air pollution there is no consensus on which elements to include for those
countries that include a monetary value. There is also no consensus on valuation techniques,
though the impact pathway approach is most commonly used;

• there are clear regional differences on the treatment of climate change effects. Most
North/West countries value climate change effects, but few Eastern and Southern countries do
so. There is no single approach for assessing the money value and a significant range of
values are used.

• very few countries include environmental effects other than noise, local/regional air pollution
and climate change

• very few countries include indirect socio-economic effects, such as agglomeration benefits,
community severance and access to the transport system.

22
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

Annex 2: HEATCO’s general recommendations for developing transport appraisal frameworks

1. Appraisal as a comparative tool. To estimate the costs and benefits of a project, one has to
compare costs and benefits between two scenarios: the ‘Do-Something’ scenario, where the project
under assessment is realised, and a ‘Do-Minimum’ scenario, which needs to be a realistic base case
describing the future development. If there are several project alternatives, one has to create a
scenario for each alternative and compare them with the ’Do minimum case’.

2. Decision criteria. We recommend the use of NPV (net present value) to determine, whether a
project is beneficial or not. In addition, depending on the decision-making context respectively the
question to be addressed, BCR (benefit cost ratio) and RNPSS (ratio of NPV and public sector
support) decision rules could be used.

3. The project appraisal evaluation period. We recommend the use of a 40 year appraisal Period,
with residual effects being included, as a default evaluation period. Projects with a shorter lifetime
should, however, use their actual length. For the comparison of potential future projects, a common
final year should be determined by adding 40 years to the opening year of the last project.

4. Treatment of future risk and uncertainty. For the assessment of (non-probabilistic) uncertainty, we
consider a sensitivity analysis or scenario technique as appropriate. If resources and data are
available for probabilistic analysis, Monte Carlo simulation analysis can be undertaken.

5. Discounting. It is recommended to adopt the risk premium-free rate or weighted average of the
rates currently used in national transport project appraisals in the countries in which the TEN-T
project is to be located. The rates should be weighted with the proportion of total project finance
contributed by the country concerned. In lower-bound sensitivity analyses, in order to reflect current
estimates of the social time preference rate, a common discount rate of 3% should be utilised. For
damage occurring beyond the 40 year appraisal period (intergenerational impacts), e.g. for climate
change impacts, a declining discount rate system is recommended.

6. Intra-generational equity issues. We recommend, at minimum, that a “winners and losers” table
should be developed, and presented alongside the results of the monetised CBA. Distributional
matrices for alternative projects might be created and compared amongst each other. Additionally
stakeholder analyses should be undertaken as well. It is recommended to use local values to assess
unit benefit and cost measures.

7. Non-market valuation techniques. If impacts in transport project appraisals cannot be expressed in

market prices, but are potentially significant in the overall appraisal, we recommend that – in the
absence of robust transfer values – non-market techniques to estimate monetary values should be
considered. We recommend that the choice of technique used to value individual impacts should be
dictated by the type of impact and the nature of the project. However, Willingness to Pay (WTP)
measures is preferable to cost-based measures. Values should be validated against existing
European estimates.

23
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

8. Value Transfer. Value transfer means the use of economic impact estimates from previous studies
to value similar impacts in the present appraisal context. Value transfers can be used when
insufficient resources for new primary studies are available. The decision as to S3 HEATCO D5
Summary whether to use unit transfers with income adjustments, value function transfer and/or meta-
analyses will depend on the availability of existing values and experience to date with value transfers
related to the impact in question.

9. Treatment of non-monetised impacts. We recommend, at a minimum, that if impacts cannot be

expressed in monetary terms, they should be presented in qualitative or quantitative terms in addition
to evidence on monetised impacts. If only a small number of non-monetised impacts can be
assessed, sensitivity analysis may be used to indicate their potential importance. Alternatively, non-
monetised impacts may also be included directly in the decision-making process by explicitly eliciting
decision maker’s weights for them visà-vis monetised impacts.

10. Treatment of indirect socio-economic effects. We recommend that if indirect effects are likely to
be significant, an economic model, preferably a Spatially Computable General Equilibrium (SCGE)
model, should be used. Qualitative assessment is recommended, if indirect effects cannot be
modelled due to limited resources (high costs for the use of advanced modelling), insufficient
availability of data, or lack of appropriate quantitative models or unreliable results.

11. Marginal Cost of Public Funds. Our recommendation is to assume a marginal cost of public funds
of 1, i.e. not to use any additional cost (shadow price) for public funds. Instead, a cut-off value for the
RNPSS of 1.5 should be used when relevant.

12. Producer Surplus of Transport Providers. We recommend to estimate (changes in) the producer
surplus generated by changed traffic volumes or by the introduction and adjustment of transport
pricing regimes.

13. Accounting procedures. a) Factor costs should be the adopted unit of account. This requires
measures expressed in market prices - which include indirect taxes and subsidies – to be converted
to factor costs. b) We recommend to convert all monetary values into € with a price level for a fixed
year. In this report, monetary values are given as €2002, i.e. with 2002 as base year. However, the
monetary values should be adjusted with the Purchasing Power Parity (PPP) as explained in Annex
B, which also contains a table with PPP adjustment factors. However, these factors are only available
for past years, whilst future PPP factors are likely to change as the economic growth rates differ
amongst countries. As we assume, that income and prices grow faster in Member States with
currently low income, PPP factors will tend to converge closer to 1 in the future. Therefore, we
recommend that two calculations are made – one with and one without PPP adjustment – assuming
that the true value will lie between the two results. c) Monetary values, i.e. preferences, for non-
market goods like reduced risk of getting ill or reduced damage to the environment will increase with
increasing income; thus we recommend increasing monetary values based on GDP growth – a table
with possible country-specific GDP growth is given in Annex B.

14. Up-dating of values. The unit values supplied in this report represent the state-of-the-art for the
individual impacts addressed. Nevertheless, all values will be subject to change as new empirical
evidence becomes available and methodological developments take place. As a consequence, we

24
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

recommend that values are reviewed and up-dated on a regular basis e.g. after three years at
maximum.

15. Presentation of results. As far as possible, impacts should be expressed in both physical and
monetary terms. The results of the sensitivity analysis and the non-monetised impacts should be
reported together with the central monetised results.

25
 Transport Appraisal in other countries: lessons for the NATA Refresh
Report - October 2007

Annex 3: Shires and de Jong (2006)

The European Union’s HEATCO research programme carried out a meta-analysis of value of time
(VoT) studies. Below is a summary of the report’s main conclusions.

The meta analysis estimated regression equations on almost 1,300 values of time from studies
around the world, mostly studies conducted after 1990. The researchers also estimated models on
more than 130 values of time for freight transport. For both passenger and freight transport, they
estimated double logarithmic models with ordinary least squares. They also used fixed effects and
random effects panel models that account for the fact that there were repeated observations for the
same country. The estimation results led to the following general conclusions:

ƒ A (largely cross-sectional) income elasticity of the VoT of about 0.5 for business travel, 0.7 for
other passenger transport and 0.3 for freight;

ƒ Long distances lead to higher Value of Travel Time Savings (VTTS) for commuting and other
purposes. Stated Preference and combined Stated Preference – Revealed Preference studies
give somewhat lower passenger VoTs than the cost savings approach. They found no
significant effect of study method for freight transport;

ƒ The researchers found significant effects for purpose (business, commuting) and mode
(especially for air; less so for bus relative to car and train);

ƒ They also obtained higher values of time in Southern and Eastern European countries, all
other things (including GDP/capita) being equal.

The estimation results have been applied to each of the 25 EU countries to get VTTS by purpose,
either for comparison against existing national values, or as a basis for our recommendations for
countries with missing VTTS.

26