Nepal: Nepal Earthquake Recovery Operation Movement-wide Meta Evaluation

1. 1. Summary

1.1. Purpose: The meta-evaluation will critically assess linkages between all evaluations and thematic studies undertaken under the Nepal earthquake response operations to the overall objectives of the One Recovery Plan of the Movement. This evaluation will independently analyse the results, learnings and draws out a consolidated synthesis of recommendations and lessons learned from the findings from Movement-wide prospective. Contribute to organizational knowledge and learnings for NRCS and the Movement partners.

1.2. Commissioner: The IFRC Asia Pacific Regional Office (APRO), Director, on behalf of the Movement partners in close collaboration with the NRCS. The findings and products arising from this review will be owned and managed in accordance to NRCS policy and procedures.

1.3. Audience: Findings of this meta-evaluation will be used for organizational knowledge and learnings by the Nepal Red Cross, IFRC and Movement partners to plan future strategies and interventions.

1.4. Evaluation team: An independent meta-evaluation expert (international consultant) drawn from the IFRC membership and/or external agencies will be hired through a competitive process. 1.5. Duration of consultancy: Up to 40 working days (including approximately 20 days in Nepal). 1.6. Estimated dates of consultancy: The exercise will take place from early October 2019.

1.7. Location of consultancy: Nepal and home country.

2. Background

The International Federation of Red Cross and Red Crescent Societies (IFRC) comprises 190-member Red Cross and Red Crescent National Societies and a secretariat in Geneva to support humanitarian activities around the world. The IFRC is committed towards increasing the impact of the Movement in terms of quality assurance, standards and a culture of lesson learning in disaster recovery operations. Those operations that require an international response and meet certain criteria of scale, scope, complexity or risk, as in the Nepal earthquake response, are committed to carrying out this meta-evaluation against One Recovery Plan. The IFRC Framework for Evaluations requires that an evaluation be undertaken for any programme over 24 months in timeframe. On Saturday, 25 April 2015, a powerful earthquake measuring 7.8 magnitudes on the Richter scale struck Nepal. The epicenter was about 76 kilometers northwest of Kathmandu, but its impact was felt in 32 of Nepal’s 75 districts. A series of aftershocks continued to cause further damage and panic, with the strongest aftershock, measuring 7.3 magnitude, hitting the country on 12 May. As a result of combined impacts of 25 April and 12 May 2015 earthquakes, 8,790 people died, and more than 22,300 people were injured. The Post Disaster Recovery Framework (PDRF)1 assessment showed that at least 498,852 private houses and 2,656 government buildings were destroyed. In addition, 260,319 houses were partially damaged, and 19,000 classrooms were completely destroyed and 11,000 damaged. Nepal has not experienThe International Federation of Red Cross and Red Crescent Societies (IFRC) comprises 190-member Red Cross and Red Crescent National Societies and a secretariat in Geneva to support humanitarian activities around the world. The IFRC is committed towards increasing the impact of the Movement in terms of quality assurance, standards and a culture of lesson learning in disaster recovery operations. Those operations that require an international response and meet certain criteria of scale, scope, complexity or risk, as in the Nepal earthquake response, are committed to carrying out this meta-evaluation against One Recovery Plan. The IFRC Framework for Evaluations requires that an evaluation be undertaken for any programme over 24 months in timeframe. On Saturday, 25 April 2015, a powerful earthquake measuring 7.8 magnitudes on the Richter scale struck Nepal. The epicenter was about 76 kilometers northwest of Kathmandu, but its impact was felt in 32 of Nepal’s 75 districts. A series of aftershocks continued to cause further damage and panic, with the strongest aftershock, measuring 7.3 magnitude, hitting the country on 12 May. As a result of combined impacts of 25 April and 12 May 2015 earthquakes, 8,790 people died, and more than 22,300 people were injured. The Post Disaster Recovery Framework (PDRF) assessment showed that at least 498,852 private houses and 2,656 government buildings were destroyed. In addition, 260,319 houses were partially damaged, and 19,000 classrooms were completely destroyed and 11,000 damaged. Nepal has not experienced such a mega disaster for the last 80 years. According to the Government, 14 severely affected districts were classified ‘Category A’, with a further 9 districts, with medium damage were classified ‘Category B’. A state of national emergency was declared by the Government of Nepal (GoN) on 25 April 2015 and an official request for international assistance was made within hours of the first tremor. Many humanitarian actors, including the International Red Cross Red Crescent Movement (RCRC Movement)ced such a mega disaster for the last 80 years. According to the Government, 14 severely affected districts were classified ‘Category A’, with a further 9 districts, with medium damage were classified ‘Category B’. A state of national emergency was declared by the Government of Nepal (GoN) on 25 April 2015 and an official request for international assistance was made within hours of the first tremor. Many humanitarian actors, including the International Red Cross Red Crescent Movement (RCRC Movement) mobilized the full range of their resources to support the relief and recovery efforts in line with Nepal government’s overall strategy. On behalf of the Movement partners, the Nepal Red Cross Society (NRCS) led the implementation and delivered immediate humanitarian assistance to more than 3.5 million people in more than 40 affected districts.
The relief phase was officially closed in early December 2015. The GoN was strict in ensuring that all recovery projects were aligned with Government policy, regulations and guidelines, although these were not in place at the time. This was reinforced by a Government Bill (Reconstruction of Earthquake Affected Structures) and established the National Reconstruction Authority (NRA) in December 2015. In June 2016, Movement partners (NRCS, IFRC, PNSs and ICRC) confirmed their commitment to promote a “one vision, one plan, one structure, one team approach” to recovery and endorsed an Integrated Movement Recovery Plan “One Plan” to guide 4+1 recovery approach (four technical sectors – shelter, WASH, livelihoods and health plus National Society capacity building) in 16 affected districts. Community engagement and accountability (CEA), protection, gender and inclusion (PGI), disaster risk reduction (DRR), planning, monitoring, evaluation, reporting and information management (PMER-IM) were also important cross-cutting elements across all the sectors.
The partners also agreed that NRCS would implement and report on the ‘One Plan’ through a dedicated structed called Earthquake Response Operation (ERO) both at NRCS HQ and district chapters. Since late June 2016, NRCS implemented a wide range of recovery initiatives and the Movement partners supported in different ways, depending on priorities and available resources across 16 worst affected districts. From the beginning, 13 in-country PNSs supported the recovery efforts of NRCS however in line with the One Transitional Framework some of the PNSs phased out their bilateral support starting from June 2018. Currently there are 10 in-country PNSs namely American Red Cross, Australian Red Cross, British Red Cross, The Canadian Red Cross Society, Danish Red Cross, Finnish Red Cross, Japanese Red Cross Society, Qatar Red Crescent Society, Swiss Red Cross and The Republic of Korea National Red Cross. PNSs such as Belgian Red Cross -Flanders, Spanish Red Cross and Norwegian Red Cross closed their Nepal office upon completing a significant portion of recovery commitments and handed over the remaining activities to the Nepal Red Cross Society departments.
The main Red Cross Red Crescent planned actions for recovery are:
• 7,464 households to be provided with Shelter grants,
• 316,193 people to be provided with water and sanitation facilities and hygiene promotion
• 137,625 people to be provided with health services and community health activities,
• 124,696 people to be reached with livelihood activities,
• 16 Nepal Red Cross chapters to be supported with capacity building, including cross-cutting themes such as PGI, CEA, DRR and PMER-IM.
Through the integrated recovery programme, NRCS reached to more than 160,000 families to regain resilience and build back better in line with the overall recovery strategy of the Government. For the district wise Movement-wide achievement data, please click here.
Nepal earthquake response operations has been the largest operation ever managed by the NRCS in its 56 years of history. Given the complexities of this operation with the involvement of 13 in-country PNSs, demanded need for undertaking different types of evaluation, reviews and thematic studies and lessons learnt in order to measure impacts and demonstrate accountability to donors, stakeholders, general public, beneficiaries and media.

3. Purpose, scope and objectives

3.1. Purpose:
In December 2018, NRCS, IFRC and in-country PNSs came to an agreement that individual partners will conduct their own evaluations following common principles, guidance and criteria provided in the Nepal Evaluation Framework (Annex - A). Therefore, IFRC proposes to conduct a meta-evaluation of all reviews and thematic studies that have been conducted between 2015 – 2019 time period.
This meta-evaluation will:
• Provide an independent assessment of all evaluations and studies managed, carried out or commissioned by the in-country Movement partners for earthquake response operations from May 2015 onwards (Annex – D), by assessing the quality and reliability of the findings, to then identify common actions and themes that helped the target communities to recover.
• Analyse key components of One Recovery Plan (4+1) and provide synthesis of combined findings, recommendations and learnings, and draw conclusion from Movement-wide perspective to define success.
• Contribute to NRCS and the Movement partners organizational knowledge and insight into the conceptual and operational model of the One Plan, that can be replicated and scaled up for a greater impact.

3.2. Scope:
This meta-evaluation will review all evaluation and thematic study reports published and/or for internal consumption by the Red Cross Red Crescent Movement. The meta evaluator has to draw conclusion based on thorough review of all relevant documents and must consider the latest final versions. Approximately, 26 such evaluations/reviews were conducted between 2015 - 2019 period would be included in the meta-evaluation analysis. The consultant also needs to consider evaluations carried out by other relevant stakeholders such as Government of Nepal/NRA, HRRP, UN, etc. especially on the Nepal Earthquake Recovery programmes.
Meta expert has to carry out additional exercise such as Key Informant Interviews (KIIs) and Focus Group Discussions (FGDs) including field works to capture Movement perspective such as partnership, coordination, structure and procedures, systems and application of the Principles and Rules for Red Cross Red Crescent humanitarian assistance. Also, interview some of the key informants in Kathmandu from other stakeholders to compare Red Cross interventions to the other responders.

3.3. Specific objectives:
1) To document and consolidate key findings (in terms of relevance, impact, efficiency, effectiveness, sustainability), learnings and recommendations captured through different evaluations and synthesize them into ONE from Movement perspective.
2) To identify key actions, model and/or approaches (in terms of relevance, impact, efficiency, effectiveness, sustainability) that have proven most effective to help target communities to recover across 16 districts and contributing to the SDG priorities in Nepal.
3) To identify learnings and best practices (what has worked well and what should be done differently) from the recovery programmes that can inform NRCS and Movement partners similar interventions in the future.
4) To analyze overall performance of the recovery programme against Movement-wide indicators and to capture exit/sustainability criteria highlighted in all evaluations.
5) Capture the strengths and weaknesses and within the 4+1 models; especially partnership, coordination, structure and procedures followed and provide solutions/recommendations to the NRCS and the Movement partners to address shortfalls.
6) To assess correlation of 4+1 components implemented by NRCS and inclusiveness of cross cutting themes such as PGI, CEA, DRR and PMER-IM.
7) To identify positive and negative aspects of conducting many evaluations for a common programming approach, and to recommend the effective use of evaluation framework including data management, evaluation methods and similar studies.
8) To examine the reliability of the data sources, and feasibility and limitations of methods of the evaluation and related studies.

3.4. Key Questions:
The questions below are initial guidance and should be considered in line with the One Plan approach. These questions are expected to be further elaborated in consultation with the meta-expert while working on the inception report. 1. a) To what extent has the recovery operations achieved the expected objectives and been relevant and appropriate to the needs and context?
b) To what extent contexts and external environments affected the overall success of the recovery progrmame and communities to bounce back from the impacts of earthquakes; especially shelter and construction related interventions?

1. a) What actions/components, model and approaches proved to be most successful in terms of replicability and scalability?
   b) To what extent Red Cross recovery intervention contributed to overall government policies, plans, SDG priorities as well as aligned to the wider practices.
2. Were there any common enabling factors supported effectiveness of community interventions and making the community resilient?
3. To what extent has the programme been effective in reaching the most vulnerable people and communities affected by the earthquake against One Plan commitments?
4. a) To what extent coordination and integration applied in line with the One Plan?
   b) To what extent has the recovery operations ensured optimization of resources and supported implementation of programmes in an effective and efficient manner?
   c) To what extent, recovery interventions took into consideration the long-term needs of NRCS, such as linkages between recovery and longer-term development programmes?
5. To what extent the 4+1 components were correlated?
6. a) What are the positive and negative aspects of conducting many evaluations in terms of standard, process and gaps in all completed evaluations?
   b) what methods could be applied to future evaluations to address evaluation questions on sustainability and exit, as well as replicability and scalability?
7. a) Was reliable data collected consistently throughout the programme cycle? If not, what was the obstacles?
   b) What kind of data was critical to respond to the agreed evaluation criteria and guiding questions of the IFRC evaluation framework?

5. Methodology & process The methodology will adhere to the IFRC Framework for Evaluations4, and OECD criteria of evaluations with particular attention to the processes upholding the standards of how meta-evaluation should be planned, managed, conducted, and utilized.
It is expected that the applicant to propose an approach and methodology for the meta-evaluation during the application stage which will be further developed in consultation between the consultant, IFRC and Movement partners during the inception stage. Some of the primary methods can draw upon as follows:

1. Desktop review of all documents related to evaluation and thematic studies reports and ToRs available from Nepal earthquake response operations, including guidelines and policy documents of the National Reconstruction Authority (NRA) of Nepal government.
2. Qualitative analysis using appropriate tools and software
3. Scoring tools to prepare meta-analysis matrix
4. Key informant interviews of key stakeholders/partners and Focus group discussions, (institutional and beneficiaries) as appropriate.

The consultant will meet with and/or over skype interview key Red Cross Red Crescent stakeholdThe consultant will meet with and/or over skype interview key Red Cross Red Crescent stakeholders at NRCS HQ and district chapters, PNSs, and the relevant IFRC Secretariat offices. The team will also consult with other partners and organizations such as NRA of GoN, the UN, INGOs / NGOs as appropriate.
The evaluation management team (EMT) will manage and oversee the meta-evaluation process including hiring of consultant, ensure required administrative and other supports to the consultant. The consultant will report on progress or challenges to the EMT.
The consultant will provide an independent and objective perspective based on evidences and will be the primary author of the report and recommendations. S/he will submit the draft and final report to the EMT, who will ensure the quality of the report providing input if necessary. The EMT will submit the final report to the Commissioner, IFRC head of country office and partners in Nepal for review, clarifications and management response.ers at NRCS HQ and district chapters, PNSs, and the relevant IFRC Secretariat offices. The team will also consult with other partners and organizations such as NRA of GoN, the UN, INGOs / NGOs as appropriate.

The evaluation management team (EMT) will manage and oversee the meta-evaluation process including hiring of consultant, ensure required administrative and other supports to the consultant. The consultant will report on progress or challenges to the EMT.
The consultant will provide an independent and objective perspective based on evidences and will be the primary author of the report and recommendations. S/he will submit the draft and final report to the EMT, who will ensure the quality of the report providing input if necessary. The EMT will submit the final report to the Commissioner, IFRC head of country office and partners in Nepal for review, clarifications and management response.

6. Deliverables (or outputs)

The Consultant will review all evaluation reports and conduct a meta-analysis of their results and methods. This will involve critically assessing the findings, results obtained and drawing out recommendations and lessons learned from the findings. While drawing the conclusions, the Consultant is expected to consider all the final evaluations and it should be objective and evidence based rather individual opinions and/or generalised statements.
6.1. Inception report: desk review of programme background documents and existing evaluation reports, TORs as well as identify proposed line of inquiry and define detailed methodology and analytical frameworks. The inception report should be finalised within a week of consultancy confirmed.
6.2. Analysis and synthesis: Systematic meta-analysis of existing reports (internal and external), reviews, studies and other relevant documents.
• Debriefings / present initial findings to evaluation management team, NRCS, IFRC and in country Movement partners: The consultant will present its preliminary/initial findings with recommendations for discussion and feedbacks.
• Draft report: A draft report identifying key findings (in terms of relevance, impact, efficiency, effectiveness, sustainability), correlation of 4+1 components, conclusions, recommendations and lessons for the current and future operation, will be submitted to the evaluation management team after three weeks of the consultancy. It is expected that the report should contain a separate analysis of estimated community contributions to overall recovery programmes.
6.3. Validation and finalization
• Final report: The consultant will review and incorporate feedbacks in the final report. The final report will contain a short executive summary (no more than 1,000 words) and a main body of the report (no more than 10,000 words) covering the background of the intervention reviewed, a description of the review methods and limitations, findings, conclusions, lessons learned, and clear recommendations. The report should also contain appropriate appendices, including a copy of the ToR, cited resources, a list of those interviewed, and any other relevant materials. The final report will be submitted within two weeks after receiving the consolidated feedback on draft report.
• Meta-analysis matrix to be annexed and summary of this will be included in the main report.
The findings and all products arising from this evaluation will be jointly owned by the Nepal Red Cross Society and IFRC on behalf of the Movement partners. The reviewers will not be allowed, without prior authorization in writing, to present any of the analytical results as his / her own work or to make use of the review results for private publication purposes.

8. Quality and ethical standards

The reviewer should take all reasonable steps to ensure that the meta-evaluation is designed and conducted to respect and protect the rights and welfare of the people and communities involved and to ensure that the review findings are technically accurate and reliable, is conducted in a transparent and impartial manner, and contributes to organizational learning and accountability. Therefore, the review team should adhere to the standards and applicable practices outlined in the IFRC Framework for Evaluation.
The IFRC evaluation standards are:

1. Utility: Evaluations must be useful and used.
2. Feasibility: Evaluations must be realistic, diplomatic, and managed in a sensible, cost effective manner.
3. Ethics & Legality: Evaluations must be conducted in an ethical and legal manner, with particular regard for the welfare of those involved in and affected by the evaluation.
4. Impartiality & Independence; Evaluations should be impartial, providing a comprehensive and unbiased assessment that takes into account the views of all stakeholders.
5. Transparency: Evaluation activities should reflect an attitude of openness and transparency.
6. Accuracy: Evaluations should be technical accurate, providing sufficient information about the data collection, analysis, and interpretation methods so that its worth or merit can be determined.
7. Participation: Stakeholders should be consulted and meaningfully involved in the evaluation process when feasible and appropriate.
8. Collaboration: Collaboration between key operating partners in the evaluation process improves the legitimacy and utility of the evaluation.It is also expected that the review will respect the seven Fundamental Principles of the Red Cross and Red Crescent: 1) humanity, 2) impartiality, 3) neutrality, 4) independence, 5) voluntary service, 6) unity, and 7) universality. Further information can be obtained about these Principles at: www.ifrc.org/what/values/principles/index.asp
   **
9. Qualifications**

Selection of the international and national consultant will be based on the qualifications outlined below:

1. Demonstrable experience in leading and managing evaluations and review of international recovery operations in complex situations where multiple partners and sectors involved related to major disasters;
2. Knowledge and experience of working with the Red Cross Red Crescent Movement and knowledge of the IFRC’s disaster management systems;
3. Significant professional experience in using meta tools and prior experience of conducting meta-evaluations and/or evidence synthesis is highly desirable.
4. Strong analytical skills and ability to clearly synthesize and present findings, draw practical conclusions, make recommendations and to prepare well-written reports in a timely manner.
5. Proven track record of conducting excellent qualitative research, with emphasis on the design and use of comparative analysis method and tools.
6. Excellent statistical and quantitative analysis skills.
7. Knowledge of strategic and operational management of humanitarian operations and proven ability to provide strategic recommendations and conveying complex information to key stakeholders in a clear and compelling manner.
8. Strong team leadership and management skills including negotiation, facilitation and time management.
9. Demonstrated capacity to work both independently and as part of a team.
10. Excellent writing and presentation skills in English.
11. Immediate availability for the period indicated.

Interested applicants should submit their expression of interest to the following email: pmer.apzo@ifrc.org and dibakar.behera@ifrc.org by 20 September 2019. In the subject line, please state the consultancy you are applying for, your company name or last name and first name. (SUBJECT: Nepal Earthquake Response Operations Movement-wide Meta Evaluation - Last Name, First Name).
Application materials should include:

1. Curriculum Vitae (CVs) of international and national consultants.
2. Cover letter clearly summarizing your experience as it pertains to meta-evaluation, daily consultancy fees (excluding travel and accommodations costs) and three professional references.
3. Two-pager summary of proposed method for meta evaluation - Applicants will be required to provide a 2-pager approach paper with review methodology.
4. Writing samples – Please provide a sample of previous written report that described in this TOR.
   Application material are non-returnable, and we thank you in advance for understanding that only short-listed candidates will be contacted for the next step in the application process. Please take note that incomplete application will be rejected.