POLICY

ANALYSIS SUMMARY

Roles .................................................................................................................................. 2 Conceptualization Phase .................................................................................................... 2 Problem Statement ................................................................................................................... 3 Goal Tree ................................................................................................................................... 5 Criteria .................................................................................................................................. 7 Unit ....................................................................................................................................... 7 Causal Diagram ......................................................................................................................... 8 Common mistakes ................................................................................................................. 8 Difference between causal diagram and problem diagram ............................................... 10 Problem Diagram .................................................................................................................... 11 Alternatives ......................................................................................................................... 11 Managing Uncertainty ........................................................................................................ 11 Rationalizing the Problem ................................................................................................ 14 Net Present Value ................................................................................................................... 14 Common Mistakes .............................................................................................................. 15 Impact Table ............................................................................................................................ 15 Estimations ......................................................................................................................... 15 Score Cards .............................................................................................................................. 16 SMART ..................................................................................................................................... 17 Robustness .......................................................................................................................... 17

ROLES
..of the analyst Policy analysis helps people make decisions, but does not make the decision for the client! Decision-making is difficult, that is why policy analysts help decision makers with their decisions by providing relevant information on a useful format. Gathering and representing relevant data in such a way that the role of decision maker becomes a little easier and can be carried out a bit better. Your main role is help people make decisions and not make the decision for them. of the advisor Advisors dont just leave the filled in impact table on the desk of the client. They interpret it also and construct arguments out of it to explain a client why one alternative is better than another of the advocate Where advisors makes an initial ranking based on logic and presents that to the client, advocates also present a ranking order, but dont give well-balanced argumentation. It is their job to defend a certain alternative over other alternatives. They will highlight information in it that supports their alterative and leave out information that is not in favor of their alternative. of the problem owner or stakeholders They are usually the decision-makers.

CONCEPTUALIZATION PHASE
You can identify who is involved in this problem Actors What all these different actors want their objectives You can make a model simplified representation of reality You can show which measures Alternatives the different actions actors can Problem conceptualization consists of 1. 2. 3. 4. Problem formulation Actor objectives analysis (goal tree) System description (causal diagram) Means-end diagram

The objective of the conceptualization phase is to ultimately identify the alternatives (tactics) the different actors can take.

PROBLEM STATEMENT
Your role is to make the problem statement theoretically correct, with a wish and a dilemma, focusing on tension. Problem statement is always made from the perspective of a certain actor actor specific. The dilemma might return multiple times among other actors, but every actor has different interest and thus will have different wishes. The wishes might even be dynamic in time Problem formulations should contain a clear wish (desired situation/objective/what the actor wants to accomplish) and a dilemma (reason why the objective is not reached so easily) o How can the hindrance of drug-using tourists be reduced, while keeping unrestricted access to soft drugs for Dutch people? Contradictory interests lie at the heart of the problem Wish should indicate a desire to elevate the current situation to a future situation Problems statements should always contain dilemmas (tension) The dilemma in the problem statement ideally is represented in the goal tree. Without a dilemma there is no problem The dilemmas represent the problem is terms of a tension between something desirable and the reason why that desirable thing is not easily reached. Define the problem statement such that multiple alternatives are possible. o Stray away from words like minimize or optimum in the wish because it cause you to focus on one alternative only, thereby limiting the solution space. It can be very practical when formulating a problem to think about the current way of handling things after you have identified the wish of the actor. This will make the dilemma easier to find. Doing research can be interpreted as an alternative (which should not be mentioned in a problem statement). If the objective is to do research itself (using a university as problem owner), it can be OK. Good problem statements are not about particular solutions to which you can react with many reasons why these might or might not work. The Yes, but pitfall can be removed from the conversation by focusing on the problem, rather than potential solutions. The group can use problem formulation to sharpen its vision on what the problem precisely is, and that the perception of the problem might be different for everyone in the group.

Common mistakes 1. No dilemma these can lead us to solve the problem with simple and strange alternatives a. We want to reduce the emission of harmful substances close off the chimney b. How can we reduce the traffic jams? prohibit driving c. How can we increase profit? Rob a bank 2. No wish or gap It usually involves people that lay the blame on many things with other parties, other people, other actors. These people see the cause of all problems as external. Both wish and dilemma should focus on the same thing. a. We want to generate environmentally friendly electricity without increasing CO2 emissions b. How can we produce more efficient without increasing the production costs? c. What can we do about the traffics jams without everything getting jammed? 3. Naming a specific alternative People happen to find it much easier to come up with an alternative with advice or with an idea than actually name the problem lying behind it all. A problem formulation with an alternative causes you to come up with much less broad and creative possibilities. a. How can we build the nuclear plant without exceeding the budget? b. We have to get the trucks off the motorway but we have to maintain the flexibility of transport system too c. That tunnel has to be built there without damaging the nature too much. 4. Limited solution space using words like optimize does not focus on comparing multiple alternatives. Optimizing focuses on one alternative, but also goals formulated like no less than 1000 passengers put up a certain constraint.

GOAL TREE
Usually you represent the dilemma of your problem formulation in the goal tree as two sub- goals of a main goal. This main goal then usually gets a rather abstract label with normative words like sustainable, better, etc. We approach complex problems by making them explicit and eventually rationalizing them. We are striving for a well-balanced set of criteria, representing all aspects (also implicit ones) of our problem owner and other actors involved in the problem field. Usually the wish of the problem statement is apparent in the normative upper goal and the dilemma is represented by the two sub-goals. For example, How to ensure a future proof sustainable electricity supply, without compromising the reliability of grid? The wish: future-proof sustainable electricity supply The dilemma: the fluctuating nature of sustainable electricity supply decreases the reliability of the grid. It is desirable to keep the grid reliable.

Normative upper goal Wish

Future-proof sustainable electricity supply

Problem Formulation Dilemma

More sustainable electricity production

Greater grid reliability

Operationalized Goals [Units]

Lower electricity consumption [Mw/y]

Larger share renew. elect.prod. [% energy mix]

Less grid downtime [min/cust/year]

Less mismatch between s/d [supply/ demand]

The main dilemma in the problem statement is often represented in the sub-goals and criteria in the goal tree. Do not use verbs in the goal tree! It could mean a risk of bias. Verbs might lead to potential alternatives in the operational goals this would cause bias in the analysis if used as a criterion. A verb is a word (part of speech) that in syntax conveys an action (bring, read, walk, run, learn, reduce), or a state of being (be, exist, stand). Upper goals use normative statements. They are independent of whatever problem, even without any problem bring perceived, its always there. Measurable factors with a clear direction (high/low, more/less)

Check that there are no alternatives among the sub-goals and there are also no causal relations. These are two of the most common mistakes when constructing goal trees in relation to a problem formulation. Goal trees are actor specific. Even if the upper goal was the same, other actors might define it (operationalize it) with different criteria using different units. o Only once all actors involved in criteria creation (ie all actors have their own goal tree) can the policy analyst create a well-balanced set of criteria. Ask the question why do you want this? to get to the normative, abstract upper goal. Ask the question what does this mean? to make these abstract notions into concrete, measureable criteria. o Objective trees only contain definition relations, not causal relations. \ One-to-one relations lead to an information loss.

Common Mistakes 1. Causal relations in the goal tree they consider lower goals to be a means to reach higher goals. a. What is cheap energy? i. Lower production costs - means ii. Less lower costs means 2. Goal trees contain alternatives If we introduce alternatives in the objective tree, we basically create criteria for specific alternatives. This alternative will always have a good score on its own criteria, and will therefore introduce substantial bias in your analysis a. Environmental energy i. More photovoltaic cells ii. More wind mills 3. Using verbs in a goal tree Verbs represent actions and thus cannot define goals. This again introduces substantial bias into your analysis. a. Increasing the number of thermal power plants is an action 4. Main goal too generic if you keep asking why then you end up with the general interest of that particular actor. These interests are always there, also if no problem is being perceived and thus no alternatives to solve it are considered. The interest turns into an objective the moment this interest is violated. a. Automobilists want good automobility b. Presidents want stay in power c. Entrepreneurs want continuity in their business Effect on groups Our natural way of acting almost looks like solving the problem is a goal in itself, rather than making a real change for the better. A huge desire is to as quickly as possible come to a solution (remove the dissosance). Challenging this strong mechanism, which seems to be hard-wired in the human brain, leads to stressful reactions.

Apparently, people dont want to be concrete and precise. Partly because they are afraid too much openness might lead to strategic actions from others involved, but partly also because they simply do not know. Strive for open, clear and explicit arguments.

CRITERIA
Criteria must have at least one ingoing arrow to be a criterion (else they are not determined by other factors in the causal diagram, which is the reason why you make one). Criteria are developed with the purpose to compare alternatives on. Criteria do not have direction (adjective) otherwise it would be an objective. More profit should be profit with an appropriate unit. Criteria can originate from all layers of the objective tree, as long as they are correctly operationalised.

UNIT
The unit should not be ambiguous. Do the relevancy test, cover the factor and the unit and see if it is understandable to the client and makes sense. Share of green energy [GWh green electricity]. Unit is wrong. Share resembles part of something larger, so you would expect a percentage like unit. (This percentage then would have to be defined in terms of what gets divided by what!). Operationalization mistakes: Scales from 0 till 10 actually hide that the analyst does not know how a certain factor must be operationalized. This is a classical mistake, since lots of speculation is added to the analysis.

CAUSAL DIAGRAM
The causal diagram is an attempt to describe the system state. A causal diagram causes us to starting asking what factors influence this criteria? and what factors are influenced by this criterion? A CRD should represent the system in such a way that the actors involved can recognize at least also their vision on how the system works. The causal diagram is connecting the simplified real-world system, represented by factors, to the criteria. o Factors are defined as characteristics of the system Causal diagrams are actor independent as they should represent the problem area in an, for all actors involved, acceptable way. The criteria from which you start making your causal diagram stem from the goal trees you have designed for the actors in the problem field. The CRD can be used to find new alternatives that you have never thought off, because it so clearly represents the problem field, about which usually only vague and very actor specific images where available. Not necessarily, all factors need to be the origin of a new alternative. Once alternatives are considered, the causal diagram becomes a problem diagram. Most operational goals in the goal tree can be easily transformed into criteria by removing the word representing a direction, higher/lower, more/less. Notice that the causal diagram does not contain o Verbs so no actions are present o Adjectives no words like more, less, higher, low o Normative statements like good, sustainable or better

COMMON M ISTAKES
1. Not operational factors in the diagram It is not always easy to translate these abstractions (motivation, sustainability, integral approach) into operationalized factors that represent the characteristics of the system. The diagram does not do more than reinforce the idea of the client that visitors or motivation is the key to the solution of the problem it does not figure out the mechanisms that influence the criteria, it merely comes up with an abstract concept. What tends to happen is the abstract notion becomes the center of a very unpractical star- shaped causal diagram:


Gem. Average profit per opbrengst visitor bezoeker Level water track Inwoners Inhabitants zoetermeer Zoetermeer + + + Bezoekers Visitors Price per ticket + + + + Length of the water track Omzet Turnover +

Costs of DWD

Competition new amusement parks

Surface DWD + Energy Usage DWD

GNP Zoetermeer

Line shaped diagrams are long causally related chains of factors that are not connected to any other factor in the diagram. This indicates that the analyst seemed to be on a certain line of reasoning that he was not able to leave. Clients love to see their thoughts being reinforced so they can tell others I told you I was right.

Clients often have a single actor perspective on the problem, which can be one of the reasons why the problem has persisted. Here is where you can be of added value!

DIFFERENCE BETWEEN CAUSAL DIAGRAM AND PROBLEM DIAGRAM

F IGURE 1 : C AUSAL D IAGRAM

F IGURE 2 : P ROBLEM D IAGRAM ( INCLUDES T HE A LTERNATIVES A ND E XTERNAL F ACTORS ) As all actors are now involved, the result can be called actor unspecific. Problem statement and goal tree are developed for a specific actor and thus representing his perspective on and perception of the problem.

PROBLEM DIAGRAM
If there is a need to also quantitatively conceptualize the problem, we transform the causal diagram in a set of equations with factors as variable and some model parameters. The problem diagram gives a summary of the elements of the conceptualization phase and is, as such, not part of the final results. Alternatives and external factors are part of the problem diagram, not the CRD. In the problem diagram it is shown what factors the alternatives influence. This also completes the qualitative conceptualization of the problem.

ALTERNATIVES
An alternative for a specific actor is doing something that solves my problem or doing something that influences the criteria. In other words, anything that influences the criteria can be treated as alternatives. o It makes it possible to estimate the effects of different alternatives on the set of criteria that is used. All alternatives are worth considering as a solution and should be able to change one or more factors in the causal diagram via the causal links they will then also influence the criteria.

MANAGING UNCERTAINTY
We cannot predict the future. Although the uncertainty introduced by external factors makes problem solving more difficult, it should not be ignored. It is possible to explore the space of possible futures, helping the process of finding robust alternatives that will fit multiple possible futures. The problem with external factors is that they do influence the criteria, but they are also out of control of the problem owner. Thus these external factors bring with them uncertainties you have to cope with.

External Factors
These are the factors that cannot be influenced by the actors involved. It is still an important factor to be considered. o These factors form a source of uncertainty for this actor, but these factors do influence the criteria that are important for the actor. External factor do influence the criteria, via the chain of causality through the causal diagram. That is why you spend the effort to take external factors into account and make scenarios, because they do have an impact on the criteria (and because they are uncertain). o Economic climate consumer behavior o Oil price expenditures of the private parties & building costs o Laws and Regulations i. ease of people getting primary care ii. tariffs; you can receive for primary care iii. interest on bank loans; you have to lend money to expand your business

Clients want to know the uncertainties most relevant to them. Relevance, in this case, consists of two things: i. ii. To what level is the external factor uncertain? To what extent does it influence the criteria?

External factors are placed in a quadrant along these two characteristics. One axis of low/high impact and on another axis with low/high uncertainty.

Immediately forget about factors that have a low impact actually reconsider why you have put these in your causal diagram in the first place. High impact is important, as they influence what our client and stakeholders find important.

Formulating Scenarios
It is important to know what the effect is of the alternatives the problem owner implements. The minimum number of values for a scenario is two. # = 2 # !"#!$%&' !"#$%&' Usually people use a low and high value for a scenario factor, this is also the minimum that you can use, as using only one value would make this factor certain, while it is not. So theoretically you have two values for two factors, making four scenarios. You could make more, for instance three values per factor and end up with six potential scenarios. However, you should balance the effort of having more scenarios and thus creating more scorecards versus the actual added value it has for the client. Scenarios are made with the most impactful and uncertain external factors.

Quantitatively define the boundaries of what is meant by low and high oil price. For instance, low oil price means less than 10 E per barrel and high oil price means more than 250 E per barrel. Effects o n a g roups thinking People (actors) have the natural tendency to predict the future. People either i. ii. Try to predict the future and then choose the best alternative fitting for that future. Have a certain alternative in their minds and then simply defend the future in which this alternative fits best.

No scenario is better than another, they are all just possibilities. By using a scenario analysis, it is possible for the client to see what are the robust alternatives. They are the alternatives that perform relatively well in a range of futures. Common m ade m istakes 1. Non-external factors people take factors that are not external factors at all. By using non-external factors in a scenario analysis, you will give the impression that you have made the context uncertainty explicit, while actually you did not. To check is to ask why the client cannot influence that particular factor. a. For instance, because the oil price is determined though world-wide trade and the client is only a small player in that field. 2. Only one value taken for a scenario factor Only one numerical value is used for this factor. I expect the oil price to be 120 E per barrel in 2015 They do two things wrong: a. They predict the future b. They treat the factor as certain, as they give its value for that future.

RATIONALIZING THE PROBLEM

Rationalizing is comparing alternatives and logically ranking them, a system state is described by a goal and a problem is defined by the problem statement.

NET PRESENT VALUE

NPV calculations give you an overview of the value of costs and income that are dynamic in time. NPC calculates the expected return on investment given the risk you take by investing in a certain project. Year Cash In Cash Out Cash Flow NPV NPV is a quantitative technique that corrects absolute numbers of money into the value these numbers represent today. Devaluation is needed, as money in your hands now gives you opportunities to invest, while not having money now gives you opportunity costs: opportunities you miss because you have invested your money elsewhere = (1 + )! o Cash Flow is merely Cash In - Cash Out Uncertainty about the implementation of a project is usually represented in a higher r-value in the NPV analysis Normally you take around 5% risk free interest rate and add some percentages if the project has high technological risk, acceptability risk, market risks, geographical risk, risks in financing. Typical r-values range between 5 - 15%. 20% is quite high, so the project is risky Very often many actors in your problem field might only be interested in the monetary issues, however, it is your role as an analyst to point them at the fact that for some other actors other issues are also at stake. For several reasons a now is more worth than a in the future, as money devaluates due to two factors: 1. Interest 2. Uncertainty about the future NPV does not always have to be a criterion (though often it is!). Sometimes the NPV is negative, especially with public works like railways or roads when the cash in is not so easily operational. It would be strange to just skip all 0 1 2 3 4 5 6 7 8 9

these projects, so in those cases the least negative NPV might be better (of course the decision making is depending also on other criteria).

COMMON MISTAKES
1. R-value too close for multiple projects project 1 (r-value of 11%), project 2 (r- value of 12%) The NPV calculation and comparison has not much added value to decision maker. 2. R-value too high r-values of higher than 20% is too much. Applying such large values has an effect that already after a few years of devaluating cash flows hardly influences the total NPV anymore. 3. Using only NPV to decide money-related criteria often get substantially more attention than criteria of another kind, also the NPV criterion receives lots of attention. Many times you will hear that a negative NPV automatically means this alternative should not be chosen. This is a single criteria decision making, which might be financially sound but does not bring the solution closer to the problem.

IMPACT TABLE
An impact table contains measured (raw, unprocessed) data, no relative data compared to one of the alternatives. The zero alternative A0 represents the reference situation. This can be doing nothing or not changing anything compared to the current situation of the system Some of the raw data might be qualitative, most of it we try have quantitative. Showing the effect of implementing several alternatives in a certain scenario in the system. The effect is the scores of these alternatives on the criteria, and the criteria represent what is important for the actors in the system. The criteria in the impact table represent what the most important issues are for the decision makers.

ESTIMATIONS
Missing information can easily lead to wild speculations. Your estimations will not be very accurate, but that isnt the goal, the idea is to get an estimation that is order of magnitude comparable to the real value (i.e. approximately within the range of 10 times smaller 10 times larger). The difference with measuring compared to estimation is that you dont have explicit measuring tools nor explicit information at your disposal to make your estimation.

Common Mistakes
Very often people break down the factor they would like to estimate, but not a sufficient extent. Avoid at all times that people say 100 trees, no I think it must be 1200 without having arguments to discuss. By discussing this, the confidence of the people in your estimation increases.

SCORE CARDS
Scorecards are tables of analysis results in which both qualitative (non-numerical) and quantitative (numerical) data can be present. o This offers the opportunity for the client to compare alternatives on both types of data, while usually the quantitative data is too quickly put aside and important, but hard to operationalize, concepts might easily be forgotten in the decision making. Define categories of (numerical) values that would make an actor very unhappy, neutral (indifferent) or happy. The result of an analysis can be presented in a Score Card. In this stage no normalization of the scores, nor any weighting is applied, so, the criteria can be both numerical and non-numerical Score cards cannot be used to rank alternatives and are actor specific

SMART
SMART models derive from impact tables by taking the quantitative criteria, normalizing their scores and adding weights that represent the actors ideas about what criteria is and how much more important than another one. Normalized value = (actual value worst score) / range Where, range = best score worst score SMART must be quantitative, else normalization isnt possible SMARTs are scenario specific, for each scenario you need one (just as you need one impact table or score card for each scenario). SMART is model that evolves from an Impact Table through normalization of the scores and (subjective and thus actor specific) weighting of the criteria. Detailed weights are not correct; decimals give unrealistically the impression of precision. Total scores per column can be determined only after normalizing scores, so can be done only in the SMART.

ROBUSTNESS
Robust alternatives are alternatives that in a SMART analysis end up relatively high in the rankings for most scenarios (preferably all) that are considered. They perform relatively well in a range of futures. What we are looking for therefore, are alternatives that do pretty well in most of the potential futures.