UNDP Georgia Methodology for Quick Capacity Assessment

Prepared by Ivane Shamugia Capacity Assessment Specialist

November, 2008

Table of Contents
INTRODUCTION 1. The Premise 2. Introduction 3. Overall Objective 4. Sub-objectives Step 1: SETTING A BACKGROUND 5. Initial discussion 6. Letter of Request 7. Desk Review Step 2: GATHERING OF CRITICAL DATA & INFROMATION 8. Interviews Step 3: INTERPRETATION AND ANALYSIS 9. Analysis 10. Assessing the Feasibility with the UNDP Governance Step 4: VALIDATION OF THE FINDINGS 11. A Validation Workshop 12. General Meeting with Senior Management 13. Adding Feasibility through the Level of Impact Chart Step 5: CAPACITY DEVELOPMENT RESPONSE & INDICATORS ANNEX 1 Interview Guide & Sample Questionnaire ANNEX 2 Format Quick Capacity Assessment Report ANNEX 3 Format Capacity Development Response Proposal 3

6 7 7

8 9 12 13

Acknowledgments Id like to acknowledge the contributions of many people who have helped bring this methodology to fruition: Thanks to Peter van Ruysseveldt, deputy resident representative, who has actually initiated the creation of the Quick Capacity Assessment, tailored to suit the local Georgian circumstances, needs and sensitivities, and provided valuable input on the Methodology. Thanks to Natia Natsvlishvili, democratic governance programme team leader, for reviewing the methodology and providing important insights. The work of the Methodology is possible due to the generous support of Sophia Svanadze, strengthening local and regional governance in the Kvemo Kartli region project manager. Sophia provided excellent, incisive advice and encouragement, from earliest stages through its test and beyond. A special thanks to Bikash Ranjan Dash, capacity development advisor, who provided fantastically thoughtful, thorough, and well-organized feedback throughout the development of the Methodology. Id also like to take this opportunity to acknowledge the valuable support generously offered by the UNDP Europe and the CIS Bratislava Regional Centre: Thanks to Dafina Gercheva, capacity development advisor and Joe Hooper, capacity development specialist, who helped to complete the methodology. And thanks to all of you who have emailed suggestions. I always welcome more!

The Premise Following the recent major shift from the stand-alone technical assistance activities towards the broader concept of the capacity development, the UNDP CO in Georgia has mainstreamed the capacity development concept in many of its activities. The champion was the Capacity Building Fund (Phase I) that was launched in 2004. The overall objective of the CBF has been assisting the Government of Georgia in implementation of its package of governance reforms. Phase I of the programme has already been completed and the assistance to the amount of about $ 1.5 million has been provided. Although, as acknowledged by all national and international stakeholders there is still much to accomplish in terms of capacity development of the institutions. Therefore, with the aim to further support capacity development in the Georgian context UNDP launched the Second Phase of the programme in 2007 (Capacity Building Fund Phase II). In this process UNDP intend was to apply the newly designed corporate capacity assessment and capacity development tools in identifying the capacity strengths and weaknesses and design respective capacity development response strategies (through the mechanism of sub-projects). The corporate framework, however, needed to be adopted to better match the local conditions and constraints. In particular, at the first place it was necessary to develop a simplified toolkit for QUICK CAPACITY ASSESSMENT. To this aim, the UNDP Georgia, based on the corporate capacity development framework, has elaborated a simple and applicable methodology for assessing capacities for central, regional or local institutions. Even though, the following tool is primarily designed for the CBF, it is envisaged to make extensive use of the elaborated methodology with the UNDP Georgia programming activities. Introduction UNDP defines capacity as the ability of individuals, institutions and societies to perform functions, solve problems, and set and achieve objectives in a sustainable manner. Capacity development is thereby the process through which the abilities to do so are obtained, strengthened, adapted and maintained over time1. The complexity of capacity development challenges resists the use of blueprints to support this process. What works well in one situation, may not work in another. Asking capacity for why?, capacity for whom? and capacity for what? will yield different responses depending on the local context, specific priorities and issues at stake2. UNDP suggests that the capacity development process should be aimed at increasing the knowledge, upgrading skills of staff and strengthening the supporting enabling and organizational structures and systems; it calls for number of action-oriented basic principles for capacity development that can serve as signposts and safeguards to help keep development efforts focused on capacity outcomes. It also distinguishes between technical (capacities associated with particular areas of expertise and practice in specific sectors or themes) and functional capacities. The UNDP Capacity Assessment Methodology does not include technical capacity assessment questions or indicators. The key conception underlying the UNDP approach to supporting capacity development has proven to be cross-cutting functional capacities - relevant within and across the enabling environment, organization and individuals. They are the management capacities required to formulate, implement and review policies, strategies, programmes and projects regardless of the application area, as they focus on getting things done.

1 2

Capacity Assessment Methodology Users Guide, May 2007 UNDP Practice Note on Capacity Development, September 2008 3

The UNDP Capacity Assessment Framework however, can be expanded to include various technical capacities as required. Which technical capacities to include depend on the situation and therefore cannot be pre-determined. Their selection has been left to the specificity of needs expressed by the stakeholders concerned. Conducted during the initial stages of development planning, a capacity assessment serves as input to formulating a capacity development response that addresses those areas where capacities could be strengthened or that optimizes existing capacities that are strong and well placed3. The following note presents the process guide for managing a Quick Capacity Assessment. It does not offer a blueprint, but is intended to promote common frame of reference for development practitioners and others looking to conduct a quick capacity assessment. It should also be used in conjunction with the UNDP capacity assessment framework as it provides explanations and terms and concepts referenced herein. The Quick Capacity Assessment has been designed to suit specific capacity development contexts and streamlined to suit the relevant time frames, activities and funding. It is conducted when a more targeted or scaled-down approach is desired. With an emphasis on best fit approach to organizations purpose and needs, quick capacity assessment methodology refuses to offer JUST a quick and easy solution for a common woe, but instead offers a shift in focus from just a quick to best fit to account for the difference in organizational contexts and needs - getting the right work done right. Throughout a capacity assessment process putative-beneficiaries are consulted heavily. The main tenet of this approach to assessment is that the putative-beneficiaries and the capacity assessment specialist are collaborators in a project at almost every stage of the process. It is important to understand that this approach is focused on process and is meant to generate a sense of ownership of decisions and actions. Overall Objective Capacity resides on different levels enabling environment, organizational and individual. Capacity therefore needs to be assessed and addressed across these levels, any of which can be taken as the point of entry for a quick capacity assessment. The overall objective of the capacity assessment is to analyze the existing capacity assets and gaps to build on the capacity assets and address the gaps by formulating a capacity development response, defined as to make the institution capable to effectively and efficiently perform functions, solve problems, set and achieve objectives, and deliver services to the community. Sub-objectives In order to achieve this general objective, five specific objectives were set: 1. To set a background; 2. To gather critical data & information on capacity strengths and weaknesses; 3. To interpret and analyze information; 4. To present and validate findings with relevant stakeholders; 5. To formulate a capacity development response proposal. The following note suggests five-step process to achieve those objectives. These steps are consecutive and need to be completed within a specified timeframe.

UNDP Practice Note on Capacity Assessment, September 2008 4

A set of sub-objectives to contemplate in subsequent capacity assessment phases, correspond to one of the following deliverables: Capacity Assessment Report (See Annex 2); Capacity Development Response Proposal (See Annex 3). Step 1: SETTING A BACKGROUND Initial Discussion A process with no real management commitment will be compromised from the start. As important as senior managements commitment is, it is important to have the full support of the individuals in middle management or front line supervision whose support, or lack thereof, can make or break the process, hence, prior to the assessment, a steering team (from both sides) comes together to agree the plan, clarify objectives and commit their organization, team or group to support the assessment process. The senior decision-makers of relevant organization then need to foster support and leverage resources within their own organization or team, e.g. dedicated time and availability from specific people. Decision on a focal point should not take a back seat during this discussion. The Letter of Request (LOR) The LOR can be a formal letter from a recognized official representative of the relevant organization printed on organization letterhead. The essence of the LOR is that it gives its commitment only for it to be demonstrated; as the commitment is there to ensure that the breakdown does not occur. This also involves ensuring the participation of those who are supposed to be the beneficiaries of the process. Furthermore, the purpose of the LOR is to secure the establishment of the stakeholder ownership, thus ensuring that the promised support functions of any partner offices will be able to take on the additional workload that is required and that the organizational processes and systems can support the assessment. If the recognized official representative of the relevant organization has attended meetings or held discussions with the assessment specialist prior to submission of the actual LOR it 5

would be helpful to include a reference to the meeting. The LOR can be relatively straightforward and simple and must provide sufficient detail to be understood clearly. Prior to interviewing phase, and after the LOR is submitted, one-hour meeting can be organized to brief/make a presentation, allowing beneficiaries gain common understanding on the quick capacity assessment methodology and process. Desk Review For the purpose of the review process, which in turn serves as an important supplement in the course of preparatory phase, relevant materials and legal documents (laws, decrees, statutes) will be obtained from various sources available to help achieve preliminary understanding of the organization. As outlined in the UNDP Practice Note on Capacity Assessment and in the Capacity Assessment Methodology Users Guide, UNDP Capacity Assessment Framework is made up of core issues (Leadership, Policy and Legal Framework, Mutual Accountability Mechanisms, Public Engagement, Human Resources, Financial Resources, Physical Resources, Environmental resources, Human Right Based Approach and Gender Equality - these issues have been selected because they represent the capacity issues that UNDP sees most commonly encountered within and across a variety of sectors). They provide a comprehensive set of issues from which a capacity assessment team can choose (depending on the needs and priorities of the client and the situation) as it defines the scope of an assessment. It may be useful to organize an information session in advance with the individuals in the middle management or front line supervision to explain the methodology of the assessment. If scheduled prior to interviews, the appointment may also help to reach a common understanding among those conducting the assessment as to how the questions should be formulated (see Annex 1). Importantly, there should be sufficient room for translating or formulating the assessment questionnaire in the primary language of target participants. Step 2: GATHERING OF CRITICAL DATA & INFROMATION Interviews Aimed at finding out existing capacity assets and gaps, capacity assessment can be conducted through a variety of means. Interviews are best defined as a method of data collection which involves an interaction between an interviewer and the interviewee for which the purpose is to obtain reliable information on capacity assets and gaps. Unlike the self-assessment questionnaire, the following quick capacity assessment process generates its basic assumptions based on face to face, one-on-one targeted interviews, where the data is collected by an interviewer rather than through a self-administered questionnaire. Moreover, instead of using a structured questionnaire framework, where interviewers read the questions exactly as they appear on the survey questionnaire and the choice of answers to the questions is often fixed in advance, special emphasis is placed on semi-structured questions because of being less intrusive to those being interviewed and conducted with a fairly open framework which allow for focused, conversational, two-way communication. A semi-structured interview will offer flexibility to probe for details, allowing new questions to be brought up during the interview as a result of what the interviewee says. It 6

is a conversation with a purpose4. Aside from depth interviews other qualitative method worthy of consideration are focus group discussions, which may be used by the capacity assessment specialist/team as a qualitative prelude to interviews. It is noteworthy, that sufficient room should be included in the assessment work plan to reschedule interviews if needed. The questions provided in ANNEX 1 are not intended to represent an exhaustive set of questions for any capacity assessment. Given the contextual demands of any assessment, it is expected that the questions will be adapted, added to and/or deleted, to address contextspecific needs5. Step 3: INTERPRETATION AND ANALYSIS Analysis Once the interviews have been completed the capacity assessment specialist will summarize and interpret its results, which start with an analysis of existing capacity. The analysis of existing capacity assets and gaps informs the level of effort required to build on the assets and address the gap. In some cases, the capacity assessment specialist may consider further exploration of the areas in question, i.e., they may consider additional follow-up questionnaire (e.g. selfassessment questionnaire) or informal discussions. Assessing the Feasibility with the UNDP Governance The purpose of feasibility is to help assess the viability of a development response proposition made by the assessment specialist by thinking through the initiatives from its logical beginning to its logical end to determine its practical viability potential, given the realities of the environment in which it is going to be implemented. The feasibility may involve three levels. The first level involves operational feasibility and the question that is asked at this level is Will it work? The second level involves technical feasibility and its associated question is Can it be built? The first and second levels can be addressed together and simply referred to as technical feasibility. The third and final level brings the operational and technical levels together into a common unit by asking Will it make development sense if it works and it is built for the organization concerned? Respectively, there can be three possible outcomes for assessing feasibility: (1) Feasible; (2) Feasible with changes; and (3) Infeasible. Step 4: VALIDATION OF THE FINDINGS A validation workshop Before finalizing the results of the assessment and preparing the assessment report it is important that findings are presented in a way that allows for the consideration of comments, validations and other forms of feedback. The capacity assessment specialist and the primary client decide to organize a validation workshop. Such a workshop will bring together all relevant stakeholders. It generally
4
5

Kahn, R.L., Cannell, C.F., Dynamics of Interviewing: Theory, Technique & Cases, 1957. For more information on questions and indicators please see the Capacity Assessment Methodology Users Guide pp. 23-69 (the questions provided there are intended to serve as thought-starters). 7

starts with a presentation of the assessment results, allowing key stakeholders gain common understanding on the findings incorporated in UNDP capacity assessment framework (core issues, sub core issues and functional capacity).This can be followed by a discussion of the results to share information, discuss the rationale for the scores assigned and build consensus on priorities in moving forward. What is important is that all stakeholders are given an opportunity to contribute their views and to do so in an open and honest dialogue. This will increase the level of trust among them and increase the credibility of the exercise, so that ownership over the results is shared by all and a strong foundation is built for the way forward. In the light of people's responses and after gaining a range of insights on specific issues, capacity assessment specialist may modify the assessment findings. General meeting with senior management During this step of the process, the role of senior management is to give formal approval to the findings of the assessment and contribute to the selection and prioritization of the proposed capacity development activities. Funding to the proposed development initiatives could also be influenced by other participating donors, who might be interested to direct their country-earmarked funds to contribute to these priorities. Other funding, such as core funding from UNDP will remain unchanged. Adding Feasibility through the Level of Impact Chart To ensure a feasibility of the development response proposal, and whether the proposed activities are viable, the level of impact chart incorporated in the development response proposal shows a degree of impact, activities may have on the respective institution. During this step of the process, the role of the putative beneficiary is to contribute to the selection and prioritization of the proposed capacity development activities. Step 5: CAPACITY DEVELOPMENT RESPONSE & INDICATORS The assessment of capacity and the resulting interpretation of its findings drive the creation of capacity development response proposal, which is an integrated set of deliberate and sequenced actions, attempting to build momentum for the capacity development process by designing a combination of high-priority short-term initiatives and immediate quick-impact activities corresponding to the targeted and scaled-down responses. These actions are built on existing capacity assets and the capacity gaps identified and covered by the quick capacity assessment. Any assessment would be difficult to translate into operational solutions without standardized indicators that break the capacity factors down into observable and measurable units. The indicators are broadly defined under the Capacity Development Results Framework (CDRF) to provide a comprehensive list for review during the capacity needs assessment and to facilitate the measurement of progress and final evaluation of results. In every case, specific capacity development indicators will be devised, based on the generic indicators but containing case-specific information about the development goal, involved stakeholders, and other particularities of the context of the capacity development effort. Methodology for Capacity Assessment suggests making an extensive use of the main technical features of the CDRF which include a standard set of indicators which can be enhanced through transformational learning interventions to favor the achievement of development efforts.

ANNEX 1 Interview Guide INTRODUCTION Introduction The purpose of this meeting is evaluation of the Organizations capacity by using qualitative research method (interview). The project will aim to provide relevant assistance to the Organization from the point of view of its role and fulfillment of its obligations, to strengthen the skills of the Organization employees and promote to better management by the Organization. Securing Confidentiality: The conversation will last for 30 minutes. The meeting will be recorded and further analyzed by the UNDP Georgia specialists. Your opinion will be entered into the final report not in the nominative, but in the generalized form i.e. as one of the respondents opinion. Therefore every opinion expressed by you will be kept in confidence. Basic Regulations of Interviewing Please, speak openly and freely. Be sure that your opinion will be kept in confidence. Express your opinion openly and without bias. There are no correct or incorrect answers; there are only different answers, which we welcome. Express your opinion freely even if you know that it doesnt coincide with other respondents opinion. Please specify your opinion as much as possible, for instance with examples and specific events. I expect your opinion will be honest as we are interested in both positive and negative opinions.

Sample Questionnaire Core Issues Cross-Cutting Functional Capacities Questions

Is there a clear written expression of organizations reason for existence, which describes an enduring reality and reflects its values and purpose? Is there a strategy that is both actionable and linked to overall mission and overarching goals and helps to drive day-to-day behavior at all levels? Does the organization have the capacity to frame, manage and interpret a comprehensive analysis of the policy and legal environment? Does the organization have the capacity to develop policy, legal and regulatory frameworks and mechanisms that support an integrated approach to budgeting and implementation? Does the organization have the capacity to: Monitor and evaluate the development and implementation of policies and legal/regulatory frameworks and mechanisms? Does the organization have the capacity to ensure transparency of the organization by publishing annual reports to keep the society informed, holding press conferences, public awareness campaigns, posting information on the Internet? Does the organization have the capacity to enable equitable and meaningful stakeholder participation throughout the decision-making process? Can you describe a mechanism in place to involve target groups (civil society, private sector, general public) in decision-making process; such as, annual planning and/or drafting the annual budget?

Answers

Formulate policies and strategies Policy and Legal Framework Analyze a situation and create a vision Plan, Budget, and Implement Monitor and evaluate Formulate policy and strategy Public Engagement Engage in Multistakeholder Dialogue Plan, Budget, and Implement

10

1. Do you think there is any mechanism used for

monitoring and evaluating the work fulfilled and the services delivered by the staff? If yes, how is it used to improve organization? 2. Could you describe a mechanism to collect solid data on activities and outputs; does a sophisticated electronic database and management reporting system Analyze a Situation and exist, widely used in increasing information sharing create a vision and efficiency on continual basis? 3. What do you think is mostly needed to improve the performance of the organization with this respect? Does the organization have the capacity to develop Formulate policy policies and strategies relating to human resource and strategy development? Does the organization have the capacity to frame, manage and interpret a comprehensive situation analysis of and to create a vision for the HR assets and needs? Analyze a situation and 4. What challenges/problems do you encounter most create a vision frequently in the course of implementing current projects and plans? Could you please elaborate on what you believe some specific skills within the organization likely require upgrading? 5. Are there either sporadic or regular performance Plan, Budget, appraisals, relevant trainings or coaching or a job and Implement rotation considering staff development? If no, then which circumstances do hamper this process? Does the organization have the capacity to monitor and Monitor and evaluate evaluate performance and trends in HR capacity and productivity enhancement? Is the departments physical and technical infrastructure Analyze a situation and set up so as to meet your urgent basic needs and how topical for the organization, do you think is to strengthen create a vision network (Internet) and computer skills?

Monitor and evaluate

Mutual Accountability Mechanisms

Human Resources

Physical Resources

11

ANNEX 2 FORMAT Quick Capacity Assessment Report The following note presents the results of a multi-phase assessment process and disseminates the findings from outcomes of conducted assessment. This information is shared with governance bodies within UNDP. The following overview provides a chronological listing of the report content: 1. The Premise 2. Introduction to the organization 3. Methodology and process 4. Timing and number of conducted interviews 5. Overall summary of the findings from the assessment 6. Annex: questionnaire

12

ANNEX 3 FORMAT Capacity Development Response Proposal The assessment of capacity and the resulting interpretation of its findings drive the creation of capacity development response proposal. The following document is an integrated set of deliberate and sequenced actions which are built on existing capacity assets and the capacity needs identified and prioritized by the quick capacity assessment. 1. Introduction 2. Summary of prioritized capacity gaps 3. Capacity Development response proposal 4. Proposed capacity development initiatives: description 5. Level of impact chart 6. Capacity Development Indicators

13