Procedures for the Conduct of Sustainability Evaluation Page 1
Department of Social Welfare and Development
Kalahi-CIDSS Project
Procedures for the Conduct of Sustainability Evaluation
(Note: As part of demonstration to the LGU, the KC AC shall lead in the conduct of the Sustainability Evaluation during the implementation of the Project in covered areas. Eventually, the SET shall become an institutional tool by the MDC led by the MPDC.)
1. Preparing for the Sustainability Evaluation
Preparation for sustainability starts during the Project Identification, Selection and Planning Stage, specifically during the Project Development Stage, wherein design considerations and technical specifications are observed to ensure functionality of subprojects. Compliance with the design and specifications are monitored/ensured during construction (Subproject Implementation Stage). These are complemented by the Operation and Maintenance stage wherein community stakeholders are capacitated on the proper operation and maintenance.
The main person responsible for the effective conduct of the Sustainability Evaluation is the KC Area Coordinator (AC), in close coordination with the Municipal Area Coordinator (MAC). In preparing for the sustainability evaluation, the AC together with the ACT and MCT should undertake the following tasks:
a. Gather documents and other reference materials. Meet with the RPMT to (i) generate a list of all completed subprojects in the municipality; and, (ii) gather and review documents on file about the subprojects (i.e., subproject completion report, previous SET results, etc.);
b. Form (or expand) the MSIT. Meet with the mayor or his/her designated representative, and the MCT to discuss plans for the formation of Multi- Stakeholders Inspectorate Team. Municipal representatives will be the permanent inspectorate members in all subproject evaluation in the Barangays.
The suggested composition of the MSIT is as follows:
MSIT Member No. MPDC and Municipal Engineer 2 (MSIT Team Leader/alternate) AC/DAC 1 (Co-Team leader) Representative from MIAC 1 (depending on subproject ) Mayors representative 1 SB Representative 1 Barangay Chairperson 1 Representative of Association/BSPMC 1 Representative from another/ 1 Procedures for the Conduct of Sustainability Evaluation Page 2
MSIT Member No. non-prioritized barangay PLGU Representative 1 CSO Representative 1 NPMO and RPMO representatives (if available) 2
Existing MSIT may be expanded to include other members mentioned above.
The MSIT shall designate a secretariat composed of two or more members for the conduct of subproject Sustainability Evaluation. Members of the MSIT Secretariat can come from the MIAC and MCT. The secretariat shall perform the following functions:
1. Prepares official communication to MSIT members, O&M groups and other stakeholders 2. Prepares logistics and coordinates activity 3. Ensure reports preparation and transmittal 4. Prepares or assists in the preparation of the process documentation 5. Other related secretariat functions
c. Orient the MSIT and Plan for Sustainability Evaluation. Meet with the MSIT to: (i) orient them on their roles and functions, (ii) discuss the objectives and procedures of the sustainability evaluation including the tools, (iii) plan for the sustainability evaluation, and (iv) finalize the schedule and other logistics and administrative preparations.
The AC, MAC, and ACT/MCT should ensure that during the planning meeting, agreements are reached on the following concerns: Schedule of actual conduct of sustainability evaluation. List of Participants and sending of invitations signed by the Mayor. Preparation of Program and other needed materials for the activity. Logistics and administrative preparations, including tasking. Other specific concerns.
d. Prepare for the conduct of sustainability evaluation. Ensure that the following tasks are accomplished before the actual sustainability evaluation:
i. Attended to, and finalized, logistics and administrative requirements and coordination arrangements List of participants have been prepared, invitation letters are distributed, and attendance of participants confirmed. The conduct of sustainability evaluation has been coordinated with the O&M group.
Note: FGD participants should not exceed 15 participants to promote maximum participation among respondents. FGD participants shall compose of organization officers, selected direct beneficiaries and members of the Barangay council. Procedures for the Conduct of Sustainability Evaluation Page 3
Information on the sustainability evaluation are disseminated/posted in public places, and different means are used to generate awareness about the event. Selected date and time, and venue of the sustainability evaluation are confirmed. The venue is ready and well-arranged; enough tables and chairs, chalkboards/whiteboards, and sufficient space for small group discussion are available. Activity flow and program are prepared and finalized. Logistics and administrative preparations (MSIT team composition, refreshments, supplies, sound systems, tasking, funding and other requirements) have been attended to. A documenter has been assigned.
ii. Documents and other reference materials have been secured/prepared by the MSIT/MSIT Secretariat. Documents include: Inventory of completed subprojects, and O&M and financial arrangements per subproject (i.e., O&M may be mainstreamed under the LGU. In this case, supporting documents include (1) minutes of Barangay Assembly meeting approving such arrangement, (2) LGU resolution accepting the responsibility of taking care of the operation and maintenance, and (3) O&M funds appropriated in the B/AIP). Copy of the Subproject Completion Reports. MSIT Secretariat shall fill-up the Basic Information section of the SET using the SPCR. Copy of the previous sustainability evaluation. This shall be the basis of determining compliance with recommended areas for improvement. Sustainability evaluation tool printed in manila paper. The blow-up SET form will be posted on a wall. This will focus attention of respondents - community O&M groups, Barangay Council and other participating beneficiaries.
iii. Team building of the MSIT has been conducted to level-off on all concerns needed for the activity. During the team building, the MSIT members should have decided on the following: Composition of evaluation teams (in cases where there is more than one team which will conduct sustainability evaluation) and tasks of each team member during the actual conduct of SET have been identified. Methodology to adopt during the filling up of rating forms (i.e., consensus method or individual method). The Consensus method requires the group to deliberate the observations then agree on what appropriate rating would correspond to the indicator/s. In Individual method, each member is required to have his/her own rating, after which, results will be consolidated by taking the average to come up with the final rating for each indicator.
2. Conducting the Sustainability Evaluation
Procedures for the Conduct of Sustainability Evaluation Page 4
It is envisioned that the conduct of Sustainability Evaluation and O&M monitoring shall be replicated and sustained by the Multi-Inspectorate Team, even after phase-out of KALAHI-CIDSS. KALAHI-CIDSS Project shall continue to assist the communities and LGUs during the 2 nd and 3 rd cycle. This period provides adequate time to orient and capacitate the MSITs.
a. Be at the venue early. All MSIT members/secretariat are at the venue early to ensure that all is set for the activity, and unforeseen problems/issues are addressed. b. As much as possible, start the activity on time. The suggested activity flow is detailed below:
i. Opening Program Request the present Barangay Official or O&M group officer holding the highest position to give his/her opening remark and, together with the AC, acknowledge the participants. Introduce the members of the MSIT.
ii. Present the objectives of the activity Explain the objectives of the activity clearly, preferably using the local dialect to promote better understanding among participants. In line with the transparency principle of the KC project, emphasize the importance of generating accurate information in the areas for evaluation to arrive at the true status of the subproject and O&M group. Also, emphasize that the sustainability evaluation does not intend to find faults in the O&M but rather identify the areas for improvement in the components being evaluated. Enjoin everyone to speak freely on the areas of evaluation and on the status of the subprojects being evaluated to surface good practices as well as problem areas which need to be addressed for continued benefits derived from the subprojects.
iii. Explain the procedures of the activity and their participation Explain that inspection of subprojects, Focus Group Discussions (FGD), Plenary and meetings shall be used to collect data, information, and to verify observations made. Also explain how the Team intends to carry out the sequencing of activities. Mention that record review shall also be done to check the soundness and responsiveness of required documents. Records include organizational requirements such as VMGO, Constitution and By-laws, operational policies, ordinance; financial records, O&M plan, and other pertinent documents as may be applicable. Emphasize that the presence or absence of such documents will affect the individual scores of pertinent indicators. Inform the participants of their expected roles in each of the procedures.
iv. Undertake actual inspection of all structures and sub-structures Procedures for the Conduct of Sustainability Evaluation Page 5
Actual inspection of all structures and sub-structures of the subproject should be conducted to assess its physical-technical condition. Ideally, the group should do the activity as one team for them to understand and appreciate the process. Depending on the complexity (e.g., roads and water system) and location of the subprojects, the team may divide into smaller groups to allow the simultaneous conduct of the physical-technical evaluation. Each sub- group must have a technical representative, or one that could understand the sub-project structures, to guide the sub-group members in understanding the technical conditions. Ideally, a maximum of half-day shall be spent for one subproject. Include photo documentation.
v. Do the Focused group discussion Explain to the participants the components of the tool, and generate responses for each of the indicator. Translate technical terms to facilitate understanding by the participants. Thoroughly explain the questions to generate the appropriate responses. Answers may be written first in meta cards prior to consolidation of answers. FGD participants should not exceed 15 participants to promote maximum participation among respondents.
vi. Conduct exit conference Immediately after generating the computation of the Final Rating, the MSIT shall conduct an exit conference with the FGD participants. The MSIT shall present the findings, observations and recommendations to the organization/O&M group/community. Lessons learned and challenges should be provided with reflection-analysis and proper guidance. Observed gaps and action points shall serve as inputs during the Action Planning. The MSIT and O&M groups jointly analyze the gaps and weaknesses identified during the evaluation. The group reflects on the causes and the ill effects of such deficiencies to subproject sustainability, and come up with recommendations and agreements. The facilitator should encourage the participants to share their experiences, thoughts and comments on the evaluation activity. He/she should encourage sharing of techniques, ways and means applied to a resolve problem or issue. The facilitator should stimulate discussion by encouraging participants to affirm or reinforce experience shared or to present other experiences where different techniques or ways of dealing with the identified problem proved to be a success. Factors that made the techniques/measures a success or failure should be surfaced by the facilitator.
vii. Conduct Action Planning and Tactic Session Procedures for the Conduct of Sustainability Evaluation Page 6
After the discussions, the FGD participants shall proceed to the preparation of an Action Plan to address the findings presented during the exit conference. The Action Plan (format attached) should detail the activities/recommendations, materials and resources needed, timeframe to address the need or gaps, and responsible person/group needed to accomplish the planned activities.
viii. Wrap-up/Summarize Agreements and schedule the succeeding evaluation Wrap-up the activity by summarizing the agreements reached during the activity and placing emphasis on the critical follow-through activities. Before dispersing the group, set the schedule of the next sustainability evaluation.
c. It is important that documenters record the lessons learned and the good/best practices that surfaced throughout the activity. This can be replicated to the other Barangays, as may be applicable.
3. Conducting the post-Sustainability Evaluation activities
Immediately after the conduct of the sustainability evaluation, the AC/MAC shall:
1. Call on the MSIT members for a brief meeting to reflect on the conduct of the sustainability evaluation. During the meeting, learning and insights shall be gathered, gaps identified, and recommendations for improving the conduct of future sustainability evaluations generated. 2. Assist the MSIT in consolidating the evaluation results. Sit down with the MSIT members to finalize the evaluation findings and recommendations. Assist the MSIT members come up with a comprehensive report. 3. Ensure that the proceedings have been properly documented. Sit down with the MSIT Secretariat or the designated documenter to finalize the documentation. Assist the documenter in filling out details that may have been missed or overlooked. 4. Gather copies of the Process Documentation, summary of SET results and Action Plan per subproject and file these neatly for future reference. Where possible, store copies digitally. 5. Furnish the RPMO and NPMO a copy of the Process Documentation, consolidated/summary of SET result and Action Plan per Subproject within one week of the conduct of the activity, and duly accomplished and signed SET forms within two weeks of the conduct of the sustainability evaluation. 6. Ensure that the MSIT and the CF in-charge of the Barangay shall monitor the implementation of the Action plan using the monthly O&M monitoring tool.
Procedures for the Conduct of Sustainability Evaluation Page 7
Procedures for the Conduct of Sustainability Evaluation Page 8
ACTION PLAN For the period ______________________
Name of Subproject: _____________________________________ Location: _____________________________________ Date of Evaluation: _____________________________________
Components/ Key Areas Findings Recommended Actions/ Specific Activities Timeframe Materials/Tools Needed Amount Source of Fund Responsible Person Subproject Utilization