Country: Philippines
Closing date: 20 Sep 2017
1.1.Purpose: International Federation of Red Cross and Red Crescent Societies (IFRC) seeks to evaluate the impact and effectiveness of the Haiyan Operation from November 2013 – March 2017 as well as identify the key lessons and recommendations to improve ongoing and informed future response.
1.2.Audience: The findings will be the basis for Philippine Red Cross (PRC), IFRC and other Partner National Societies (PNS) to inform future planning and response preparedness for similar operations in the future.
1.3.Commissioners: This evaluation is being commissioned by the Deputy Regional Director IFRC Asia Pacific Regional Office (APRO).
1.4.Reports to: The evaluators will report to the evaluation management team (EMT). Whilst in Philippines, the evaluators will be under the management of the Head of Country Office.
1.5.Duration: 4 weeks (one month)
1.6.Timeframe: 25 September – 25 October 2017
1.7.Methodology summary: Secondary and literature review, field work, interactions with stakeholders across levels, internal discussions, analysing the quantitative data available from the project interventions and other evaluations over the life of the project etc.
1.8.Location: IFRC APRO in Kuala Lumpur (Malaysia), PRC National Headquarters and IFRC Country Office (in Manila), with field visits to Leyte and Panay (Philippines)**Team Composition:** The team will consist of a Team Leader and three members. The team requires people with understanding of the International Red Cross and Red Crescent Movement and experience of evaluating post-disaster relief and recovery programmes, preferably in island nations. The team will put major focus on community engagement and accountability, gender and diversity, quality and efficiency of interventions and learning for the future in similar type of disasters. The team leader will be responsible of the outcome of the evaluation.**Background**
On Friday, 8 November 2013, Typhoon Haiyan (locally known as Yolanda) tore through Central Philippines with an unprecedented fury — a combination of cyclonic winds (of more than 275 kph), heavy rains and tsunami-like storm surges. Described as one of the most powerful tropical cyclones ever, Haiyan caused colossal devastation. In its aftermath, more than 6,300 people had lost their lives while there was extensive destruction and damage to housing, livelihoods and infrastructure, leading to a drastic reduction in living conditions, income, and access to basic services. In all, more than 16 million people (some 3.4 million households) were affected, with more than 1 million houses destroyed or damaged.
A state of national calamity was declared by the President of the Philippines on 11 November 2013, signalling a request for and acceptance of international assistance. Subsequently, the Inter-Agency Standing Committee (IASC) system-wide Level 3 emergency response (the highest category, requiring global mobilization and response) was formally activated. In response, local communities, the authorities, humanitarian actors, the civil society and corporate players mounted interventions in affected areas. Humanitarian actors — including the International Red Cross and Red Crescent Movement family — mobilized/deployed maximum resources and launched operations of a scale that they had not mounted before in the Philippines in recent decades.
The International Federation of Red Cross and Red Crescent Societies (IFRC), the International Committee of Red Cross (ICRC) and other Movement partners have committed to support Philippine Red Cross (PRC) efforts to ensure that assistance reaches the most vulnerable survivors, where efforts were guided by the Movement-Wide Operational Framework.
On 8 November 2013, IFRC allocated CHF 475,495 from its Disaster Relief Emergency Fund (DREF) to support the PRC in delivering immediate assistance and undertaking initial needs. On 12 November 2013, IFRC launched an emergency appeal with – after undergoing several revisions – a budget of CHF 94.53 million to support 100,000 households (500,000 people). During the relief phase, the appeal supported the worst-affected areas in the provinces of Aklan, Antique, Capiz, Cebu, Iloilo, Leyte and Palawan, while the provinces of Aklan, Antique, Capiz, Cebu and Leyte were supported during the recovery phase.
The overall objective of the appeal operation was to assist typhoon-affected communities to recover, adapt, and learn improved coping strategies to become less vulnerable to future disasters. This was achieved through: utilizing ‘build back better and safer’ shelter approaches; supporting restoration, strengthening and protection of livelihoods; supporting improved access to health services, water and sanitation; and promoting improved behavioural practices for sanitation, health and disaster preparedness.
At its completion, the operation has supported many affected communities across eight integrated sectors. These are relief, emergency shelter, shelter repair and rebuilding; livelihoods; water, sanitation and hygiene promotion; health; National Society institutional preparedness and capacity development; and, community preparedness and risk reduction. Please go to this link for reports and update on Haiyan operation http://www.ifrc.org/en/publications-and-reports/appeals/?ac=mdrph014&at=0&c=&co=&dt=1&f=&re=&t=&ti=&zo=
Evaluation Purpose & Scope
Purpose
This evaluation aims to assess the efficiency and quality of operational management, level of community engagement and accountability (CEA), sustainability of the interventions and contribution of the operation in PRC capacity development across levels.
The evaluation will determine the degree of achievement of the humanitarian objectives of the relief and recovery interventions. It will also determine the effectiveness, significance and attribution of the methodologies used in achieving the results. Focus will be endowed to DRR, health, shelter, livelihoods, water and sanitation interventions providing recommendations on how this capacity can be further strengthened at both headquarter and chapter levels. It will also look at the measures put in place to enhance community engagement and accountability to beneficiaries, donors/partners and other stakeholders and provide suggestions for improvement. Attention will be given on the extent to which the response has considered and addressed the needs of vulnerable groups especially women, girls, people with disability, other socially included sections, elderly people, woman headed households etc.
It will review interventions vis-à-vis results achieved and strategies that worked well, methods, approaches and strategies that require improvement and recommend improvements should be made, taking into consideration the context and capacities of Movement components. The evaluation will also suggest options on how organizational development and longer-term community preparedness/risk reduction programmes can be strengthened. The desired result of the evaluation is to provide best practices, lessons learnt, and recommendations that may inform PRC, IFRC and other Movement partners on establishing better guidelines, priorities, plans and implementing ongoing or future operations.
Being the final evaluation, this process will consider other reviews and evaluations, along with the programme monitoring data, to ensure that the evaluation is a part of a long and meticulous scientific fact finding process and not a stand-alone study.
Scope
The evaluation will look over the key highlights throughout the operation. The field visit will be focused on the islands of Leyte and Panay. The evaluation will cover the period from the beginning of relief interventions and end of recovery interventions. It shall consider learnings from evaluations of previous operations as well as implementation and monitoring data and reported coordination mechanisms of other operations that were ongoing at the same time. It shall also note the impact of Typhoon Haiyan response on other operations and longer-term programmes.
Evaluation Criteria – Objectives - Questions
The following are the evaluation criteria by which this evaluation will be conducted.
1) Relevance and appropriateness of the operation in delivering assistance based on needs and context;
2) Coverage in reaching target population across different groups, ethnicity, gender, physical construct etc.
3)Efficiency of the interventions in delivering quality services with minimum resources, effectiveness of the management and systemic mechanisms and strategies; and accountability to the impact population in relation to service delivery, sustainability, inclusion and resilience building.
4) Impact of the interventions on the National Society’s capacities and the level of resilience of the communities and institutions in areas covered by the operation.
5)Connectedness and sustainability of the interventions in developing the organizational capacity of PRC and enhancing the resilience of assisted people against future hazards.
The objectives of this evaluation are:
- To assess the level of achievements and significance of change vis-à-vis the commitments made in plan.
- To assess the relevance and viability of strategies and mechanisms built across different levels of the PRC, which led to enhancement of their programming capacity (in relief and recovery), organizational sustainability and position as a more relevant organization in the Philippines.
- To evaluate the effectiveness of the Movement-wide cooperation and suggest future actions for improving the same.
The following are recommended questions aligned to the evaluation criteria for this evaluation:
Relevance and appropriateness
To what degree the interventions were successful in identifying the most vulnerable people from among the affected population and responded appropriately to their context and situation specific needs?
Was the impact population selection process fair, appropriate and effective?
Were there mechanisms to capture community’s complaints/feedback and how accountable was the project in assisting the community take informed decisions?
Was the assistance provided appropriate and sufficient to meet intended needs?
To what extent the community was engaged in the assessment, planning, design, implementation, and monitoring of the interventions?
Were gender and diversity aspects considered in programming, monitoring and reporting?
Were intervention strategies and priorities in line with local customs and practices of the affected population, the priorities of the Government authorities and other key humanitarian actors?
Were the interventions in line with PRC and IFRC strategies, standard operating procedures (SOPs) and guidelines?
What problems and constraints were faced during implementation and how did the interventions manage these?
What important lessons have been identified that can improve future interventions in the Philippines and beyond?
Did the project consider mid-course corrections and changed direction of programming based on lessons learned from the monitoring data?
Coverage
- Have recovery interventions reached all population groups in need, including those in remote areas who would otherwise not received humanitarian assistance?
- Were there exclusions or differential impact between groups based on their location?
- How could the coverage and distribution/delivery methods be improved?
Efficiency, effectiveness and accountability
Did the interventions meet their immediate, intermediate and developmental results?
Were there adequate resources (financial, human, physical and informational) available and were the resources utilized effectively and efficiently?
Were systems, procedures and control mechanisms adequate to ensure smooth delivery of assistance and protect the National Society and IFRC from financial loss and reputational risk?
Were adequate tracking systems in place to ensure transparency and accountability?
Were complaints/feedback mechanisms put in place for community questions and concerns to be answered? What were the concerns raised by communities during the intervention?
How effective were processes for planning, monitoring and quality management, (e.g. use of assessment data, internal reviews and other quality assurance mechanisms)?
Was there adequate time and effort invested for the integration of interventions across the different operation sectors?
To what degree was integration achieved and how could this be further strengthened?
Would greater investment in preparedness measures have resulted in more efficient, effective and less costly interventions?
How were programme activities managed and coordinated, particularly between PRC, IFRC, other Movement partners, clusters, and local authorities?
Was the capacity of the human resource system enough to fulfil the needs of the interventions and beneficiaries? Were personnel skills utilized in an efficient and effective manner?
Was there adequate and relevant staffing including: a) decisions concerning the number of staff members needed, where, when, with what competences, at what levels, and at required availability and b) decision-making chain regarding staffing?
Did the lesson learned workshop result in PRC addressing the identified capacity and operational gaps for the recovery phase?
Was the operation able to develop a realistic capacity enhancement plan, follow it and reported positive impact of the same on the actual capacity of PRC at different levels?
Impact
How has the operation changed the living condition of the target communities in relation to their health, livelihoods, resilience to any future hazard, enhanced equality, more equitable and inclusive support and service delivery mechanisms etc.?
Did the interventions result in enhanced institutional capacity of PRC across different levels?
How are the communities reached through the interventions not negatively affected, and more prepared, resilient and lesser at-risk, because of the humanitarian action undertaken by the project?
Connectedness and Sustainability
Are there evidences suggesting that the operation outcomes are likely to sustain and what are the methods of measuring the same in future?
Did the operation strengthen the local disaster preparedness and response capacities (including community resilience to disasters and crises)?
Did the interventions result in enhanced institutional capacity of the PRC province and below chapters, in terms of:
a. ability to implement recovery programmes, with minimum support.
b. ability to prepare for and respond to disasters in a timely, efficient, and coordinated manner; and
c. ability to mobilize communities at risk to cope with future disasters?
Did the support of the IFRC strengthen and complement the response of local PRC chapters and coping mechanisms, or hinder them?
Did the support provided to affected communities enable them to enhance their resilience to withstand possible future Typhoon and others hazards?
Evaluation Methodology
The methodology will adhere to the IFRC Framework for Evaluations, with attention to the processes upholding the standards of how evaluations should be planned, managed and conducted.
A desk review will include the analysis of beneficiary data from the operation, reports, secondary data from government and other humanitarian agencies as well as a review of the community feedback from the activities conducted and any monitoring and past review or evaluation data that may be available with PRC and IFRC in the Philippines, at the time of this evaluation
Interviewees can be conducted in person or via Skype and will include PRC, IFRC, and PNS personnel (e.g. managers, field officers, direct implementers, and volunteers), non-beneficiaries (e.g. state/regional officials, township executives/volunteers, ‘most vulnerable’ beneficiaries including children, pregnant and lactating women, the elderly, and families with high dependency ratio). These may be in the form of key informant interviews (KII), focus group discussions (FGDs) or other participatory methods, at the discretion of the evaluation team. Interviews will also take place at Manila and Kuala Lumpur to include perspectives from PRC and IFRC senior management.
The evaluation team will be responsible to clearly outline the support needs in-country in the inception report. This will be agreed with PRC and IFRC based on resources available.
The evaluation team is expected to adapt to a qualitative research and evaluation framework considering the width of information will be available through the secondary and literature review and the evaluation team is expected to assess the operation on the ‘depth’ parameters of quality, sustainability, relevance, efficiency etc.
Case studies, best practices documentation, anecdotes and chronological documentation of the processes will ensure the quality of the outcomes and whether the final products can be published.
Deliverables (or Outputs)
- Inception report. The inception report should include the proposed methodologies, a data collection and reporting plan with identified deliverables, draft data collection tools such as interview guides, and travel and logistical arrangements for the evaluation.
Debriefings. A debriefing should be conducted in IFRC Country Office in Manila and in IFRC Asia Pacific Regional Office in Kuala Lumpur. The debriefings should be formal presentations to update the progress and initial findings of the evaluation.
Initial findings/feedback meetings: The team will share its initial findings with PRC and IFRC in-country presence in an after-review participatory meeting before departs from the Philippines. A similar meeting will be organized at the IFRC Asia Pacific Office in Kuala Lumpur.
Debriefing/feedback to management: The team will report its preliminary findings to the commissioner and management at the IFRC Asia Pacific Office in Kuala Lumpur. The team leader and the three team members should be present.
Draft report. A draft report with the full contents, findings, recommendations and limitations, will be submitted to the IFRC for initial endorsement and feedback. The report should also have the basic contents of the annexes, with or without numbers and photographs. The feedback provided by IFRC and PRC should be validated by the EMT and the same should be incorporate into the final report.
Final report. At a minimum, the report should include an executive summary, background of the intervention evaluated, a description of the evaluation methods and limitations, findings, conclusions, lessons learned, recommendations, and appropriate appendixes, including a copy of the TOR, data collection instruments, and full citations for any cited resources. All products arising from this evaluation will be owned by the IFRC. The evaluators will not be allowed, without prior authorization in writing, to present any of the analytical results as their own work or to make use of the evaluation results for private publication purposes.
The preliminary and final reports will be submitted through the evaluation management team, which will ensure the quality of the report, providing input if necessary. The evaluation management team will submit the report to the IFRC Secretariat stakeholders interviewed for review and clarifications. The Deputy Regional Director will oversee a management response and will ensure subsequent follow up.
A presentation comprising the methodologies, the final findings, limitations and recommendations to be submitted to the EMT, which later can be presented to the PRC and IFRC leadership for their endorsement and comments on the final recommendations. All case studies, anecdotes, any rough tool used, copies of participatory tools and the actual manuscripts and transcripts to be submitted to the EMT to ensure scientific accountability and data protection of the evaluation.
Proposed Timeline (or Schedule)
Time Schedule
Activities
Deliverables
Week 1
(25-30 September)
Desk Review: review documentation, and related primary/secondary resources for the evaluation.
Development of detailed inception report, or data collection/analysis plan and schedule, draft methodology, and data collection tools.
Preparation and pilot of data collection tools.
Data collection from targeted stakeholders in Manila
Inception report, data collection/analysis plan and schedule, draft methodology, and data collection tools.
Week 2
(1-7 October)
- Data collection in targeted communities in Leyte and Panay
Week 3
(8-14 October)
Debriefing – present initial findings, conclusions and recommendations (in Manila and Kuala Lumpur)
Data analysis and drafting evaluation report
Debriefings presentations
Draft version of evaluation report.
Week 4
(15-25 October)
Submit the first draft of the evaluation report and seek feedback from EMT
Final draft of evaluation report.
Revise and submit final evaluation report. Address feedback with revisions in report where appropriate.
Final evaluation report.
Evaluation Quality & Ethical Standards
The evaluators should take all reasonable steps to ensure that the evaluation is designed and conducted to respect and protect the rights and welfare of people and the communities of which they are members, and to ensure that the evaluation is technically accurate, reliable, and legitimate, conducted in a transparent and impartial manner, and contributes to organizational learning and accountability. Therefore, the evaluation team should adhere to the evaluation standards and specific, applicable process outlined in the IFRC Framework for Evaluation.
The IFRC Evaluation Standards are:
Utility: Evaluations must be useful and used.
Feasibility: Evaluations must be realistic, diplomatic, and managed in a sensible, cost effective manner.
Ethics & Legality: Evaluations must be conducted in an ethical and legal manner, with regard for the welfare of those involved in and affected by the evaluation.
Impartiality & Independence; Evaluations should be impartial, providing a comprehensive and unbiased assessment that considers the views of all stakeholders.
Transparency: Evaluation activities should reflect an attitude of openness and transparency.
Accuracy: Evaluations should be technical accurate, providing sufficient information about the data collection, analysis, and interpretation methods so that its worth or merit can be determined.
Participation: Stakeholders should be consulted and meaningfully involved in the evaluation process when feasible and appropriate.
Collaboration: Collaboration between key operating partners in the evaluation process improves the legitimacy and utility of the evaluation.
It is also expected that the evaluation will respect the seven Fundamental Principles of the Red Cross and Red Crescent: 1) humanity, 2) impartiality, 3) neutrality, 4) independence, 5) voluntary service, 6) unity, and 7) universality.
Further information can be obtained about these principles at: www.ifrc.org/what/values/principles/index.asp
The following ethical considerations will be observed throughout planning and execution of the evaluation and during documentation and presentation of the findings:
The process at any point will ensure ‘Do No Harm’ principle. This will not only relate to physical consideration but also emotional and contextual considerations that might affect the well-being or social position of the participating individuals.
The representing National Society (here PRC) will be informed about the purpose, design and the expected outcomes of the evaluation, in advance and the country plan will be finalized in collaboration with PRC and the local IFRC office.
The methodologies and the broad generic question categories will be discussed with the PRC leadership and the process will commence only after receiving an endorsement from them.
Information at all levels will be confidential in nature and the analysis will be done based on blinding of the characters and context; as much as possible.
Each person involved and the participating stakeholders will be completely briefed about the purpose of the process and the expected end products, with all the mediums to be used for documentation and presentation of the same.
Any photograph and contextual information (name, household information, history, locality name, Barangay name etc.) will only be published if all the participating members endorse the process and allow publication of the same in totality.
Any health or social status of the interviewed individuals will not be revealed with his/ her real name on it and without consent.
The data and information collected will be triangulated in totality before publishing and the same will be done for all the secondary information received.
The case study and monographs will use caution to present the cases with dignity and without any personal judgment and bias of the evaluator.
The findings, case studies, photographs, process highlights and recommendations will be shared with the PRC and IFRC officials at the Philippines, before finalizing the document for publication.
The methodologies and the research design to be finalized and endorsed by the Evaluation Management Team, who will act as the upholder of the ethical standards of this report.
The author(s) will not have any personal and patent rights over the raw and refined data and the end-product of this process.
In case the author wishes to use the unpublished datasets or information’s, from this evaluation; (s)he/ they must seek principle approval from IFRC.
The evaluator(s) is not authorized to promise a service or provide solution for any expressed problems. Though (s)he may document the same and make it available to the PRC and IFRC.
Complete orientation of the government laws and policies will be pivotal and at any point the evaluation team must respect and adhere to the local laws and policies.
Evaluator/s & Qualifications
The team will comprise people with understanding of the Movement and experience of evaluating post-disaster relief and recovery programmes, preferably in island nations. One member of the team should have experience in National Society development and in conducting assessment of attribution of response and recovery programmes in capacity and systemic development of National Societies. At least one member of the team needs to have extensive experience of response and recovery operations, preferably with understanding of budget and logistics management in similar type of operations. The team may require a member with understanding of gender and diversity, community engagement and accountability and qualitative research.
The Team Leader will be supported by an evaluation team of three – from PRC, IFRC and PNS – who were not directly involved in the operation. The Team Leader will coordinate directly with the IFRC AP Regional Office and IFRC Country Office in Philippines and will liaise with the EMT and act as the focal point for coordination of internal and external stakeholders, evaluation coordination and final product development.
The following characteristics are highly desirable for the team leader (external evaluator):
· Have experience/significant knowledge of the humanitarian response mechanisms in relief and recovery
· Have previous experience in conducting external evaluations
· Have previous working experience in the Philippines (key requirement)
Team members will support the team leader in conducting a successful and efficient evaluation and to deliver different objectives especially in relation to rehabilitation and recovery program outcomes, impact of the project on National Society development and help capture the successful models, strategies through the medium of chronological process document, case study compendium formation etc.
The following characteristics are highly desirable for the whole team (including team leader):
· Knowledge of activities generally conducted by humanitarian organizations in relief and recovery.
· Knowledge and experience working with the Movement (preferred).
· Previous experience in coordination, design, implementation, monitoring and evaluation of humanitarian programmes, preferably with the Movement.
· Strong analytical skills and ability to clearly synthesize and present findings, draw practical conclusions, make recommendations and to prepare well-written reports in a timely manner.
· Understanding of gender and diversity, community engagement and accountability and qualitative research methodologies.
· Understanding of participatory evaluation methodologies and ethical standards in conducting research with involvement of human subjects
· Ability to work within tight deadlines and manage with available resources.
· Have a relevant degree or equivalent qualifying experience.
· Excellent written and spoken English skills required.
How to apply:
Interested candidates for Team Leader role should submit their expression of interest to pmer.apzo@ifrc.org by 20 September 2017. In the subject line, please state the evaluation you are applying for, your surname and first name. (SUBJECT: Team Lead Haiyan Evaluation - Last Name, First Name).
The application should include:
- Cover letter clearly summarizing experience as it pertains to this assignment, daily rate, and contact details of three professional referees
- Curriculum Vitae (CV)
- Provide samples of previous work (reports of previous evaluations and reviews completed)
Application materials are non-returnable and we thank you in advance for understanding that only short-listed candidates will be contacted.