Real Time Evaluation - Ecuador

The International Federation of Red Cross and Red Crescent Societies **(IFRC)**

Terms of Reference- **Real Time Evaluation**

Ecuador Earthquake response operation

1. Summary

1.1Purpose: This Real-time Evaluation (RTE) will assess the effectiveness and efficiency of the Red Cross and Red Crescent Movement cooperation and coordination in the IFRC’s response to the earthquake that struck Ecuador on 16 April 2016. The RTE will look at the relevance, effectiveness, efficiency, timeliness and appropriateness of the response. Attention will also be given to the application of the Principles and Rules for Red Cross Red Crescent humanitarian assistance[1] in the response operation.

The outcome of the RTE will inform the Ecuadorian Red Cross (ERC) and IFRC ongoing strategy and response to assist earthquake-affected communities.

1.2 Commissioner: This RTE has been commissioned by the USG, Programme and Operations Division (POD), IFRC, Geneva.

1.3 Audience: This RTE will be used by the ERC, the IFRC’s Regional Office in Panama, the IFRC’s Country Cluster Support Team (CCST) in Peru, the IFRC’s headquarters in Geneva, and the National Societies participating in the response operation.

1.4 Evaluation team: An independent evaluator supported by two/three evaluation practitioners drawn from the IFRC membership.

1.5 Duration of consultancy: Up to 30 days (including approximately 15 days in the field)

1.6 Estimated dates of consultancy: June – July 2016

1.7 Location of consultancy: Ecuador, Panama, and Geneva.

2. Background

A 7.8-magnitude earthquake struck off the coast of northern Ecuador on 16 April 2016, close to the towns of Cojimíes and Pedernales in Manabí Province. On 18 April 2016 a DREF of 405,778 Swiss francs was allocated to support 1,000 families (5,000 people) and to carry out further assessments. On 21 April 2016, an Emergency Appeal was launched for 18,350,836 Swiss francs to support 100,000 people for 12 months.

Based on the initial official data, 570 people died; over 7,000 were injured; and close to 25,000 people were sheltered in collective centres. There were 1,125 buildings destroyed, with 829 buildings and 281 schools affected. Six provinces were initially under red alert: Manabí, Esmeraldas, Santa Elena, Guayas, Santo Domingo and Los Ríos.

Immediately following the earthquake, the ERC, with IFRC support, has been implementing and managing a response effort in the affected areas of the country. The ERC activated all its provincial boards and a general alert was issued for response teams in the provinces of Esmeraldas, Manabí and Guayas, as well as an emergency operations centre (EOC) activated in its national headquarters in Quito. The Ecuadorian Red Cross has 24 provincial boards, 110 local branches, 8,000 volunteers and 200 staff members.

The Emergency Appeal strategy is intended to support the activities being conducted by the Ecuadorian Red Cross in line with its contingency plan for this type of event and in line with its humanitarian role. Based on assessments conducted by the Ecuadorian Red Cross (ERC) and taking into account the evolving nature of the situation, the operation seeks to provide immediate support to the communities most affected by the earthquake. The ERC prioritized communities with minimal access to adequate food, water and sanitation and those not attended by other humanitarian organizations. ERC committed to prioritizing attention for:

· People whose homes have been destroyed;

· Households that include several young children and/or a member with a disability, chronic illness or elderly people; and

· Households led by single-parents and/or with diverse family structures.

Complementing the IFRC support are the following Red Cross Red Crescent Movement partners actively involved in the operation: American, British, Canadian, Colombian, Mexican, Norwegian, Salvadoran, and Spanish Cross, the Red Crescent Society of the Islamic Republic of Iran, and the International Committee of the Red Cross (ICRC).

Operational objective: A total of 100,000 earthquake-affected people in urban and rural areas have their immediate humanitarian needs met, and are supported to take steps towards recovering their livelihoods

Needs assessment and beneficiary selection:

The ERC has carried out a series of initial, rapid assessments, with support from the IFRC and Partner National Societies (PNSs). Information from these assessments was complemented with reviews of secondary data and situation reports of other national and international humanitarian agencies, as well as media reports. Based on the assessments the following sectors were included in the Emergency Plan of Action for the operation:

§ Search and rescue

§ Health and care

§ Water, sanitation and hygiene promotion

§ Shelter

§ Livelihood and food security

§ Restoring family links (RFL)

§ National Society Organizational Capacity Building and Institutional Preparedness

§ Disaster response preparedness; Early warning; Risk reduction

§ Community engagement and accountability

3. Evaluation purpose and scope

The IFRC is committed to quality assurance, standards and a culture of lesson learning in its disaster response and, as such, is committed to carrying out RTEs during all major disasters requiring an international response and meeting certain criteria of scale, scope, complexity or risk. All RTEs aim to improve service delivery and accountability to beneficiaries, donors and other stakeholders and to build lessons for the improvement of the IFRC disaster response system. The Ecuador earthquake response operation falls within these criteria.

This RTE will review what is working well and what requires improvement, taking into consideration the context and capacities of the ERC and other Movement components, via the following:

i. Relevance and appropriateness:** The delivery of humanitarian assistance to the target population based on needs and context.

ii. Efficiency and effectiveness: The efficiency and effectiveness of the IFRC response, timeliness and appropriateness of the assessment, planning and management processes and systems put into place, resources used, from the outset and as the context / needs evolved, including identification of critical gaps and bottlenecks.

iii. Coverage: Which population groups are included in or excluded from the intervention. Special attention will be given by the evaluators to the extent the response has considered and addressed the needs of vulnerable groups and in particular women, girls, boys and people living with a disability.

iv. Connectedness: Ensuring that short-term emergency activities are implemented taking longer-term and interconnected factors into account

1. Cooperation and coordination: Cooperation and coordination within the Movement, and with other partners and external actors. The internal Disaster Management system will also be reviewed.

2. Application of thePrinciples and Rules for Red Cross Red Crescent humanitarian assistance: To what extent and how were the Principles and Rules for RCRC Humanitarian assistance used in this response

Scope of the evaluation:

The evaluation will cover the ERC’s and IFRC’s initial response from the moment when the earthquake struck on 16 April 2016 until July 2016, taking into account the earlier existing contingency planning and the current situation for the ongoing operation.

4. Evaluation objectives and key questions

The specific objectives and possible key questions to be addressed in this RTE are listed below. The RTE is also required to propose possible operational options and directions for the ongoing operation based on the findings. The questions below provide an initial guidance and are expected to be further elaborated by the RTE team.

1.To what extent has the response achieved the expected results and been relevant and appropriate to the needs of the target groups?

a) Did the needs assessment take into account the vulnerabilities and capacities of groups in the communities?

b) To what extent has the design of the operation taken into account the capacities of the ERC, both at HQ and branch level?

c) Did the response adapt to changes in need, capacities and context?

d) What successes and gaps can be identified in the response and are there ways these gaps could have been addressed or could be addressed in future?

2.To what extent has the response achieved its intended immediate results in an effective and efficient manner?

a) Have immediate results been achieved according to the intervention design, based on the indicators?

b) Did the target population receive assistance in a coordinated manner (within the different sectors of intervention and with other partners)?

c) How timely and relevant were the different plans, appeals, and reports?

d) To what extent have plans been developed based on thorough, participatory needs assessments and if not, what were the constraints? Determine quality and timeliness of the needs assessment.

e) Was there adequate time and effort invested for the integration of interventions across the different operation sectors and how could this be further strengthened?

f) How timely and effective was the response against the needs and stated objectives?

g) How effective were the systems to mobilize resources – financial, human resources, communications/media, logistics etc.? How adequate was the mobilization of human resources? And what challenges were faced in delivering the appropriate support?

h) How effective were the contributions of regional assets (RIT, etc) and how efficient was the cooperation and coordination with NS from the region acting internationally?

i) Was the IFRC’s and the ERC’s operational structure well geared to deliver timely, efficient and effective disaster response (including RRU’s)? The evaluation will also consider the findings of the Canadian Red Cross Regional Response Unit Evaluation. The RRU evaluation findings should feed into RTE to ensure complementarity and to avoid duplication of efforts.

j) To what extent were the Principles and Rules for RCRC Humanitarian Assistance adhered to and were these Principles and Rules effective as a coordination tool to improve the delivery of humanitarian assistance?

3. Determine to what extent there are appropriate coordination and cooperation mechanisms in place for this operation and determine their effectiveness.

a) How effectively did the IFRC coordinate the operation and has it been able to adhere to and support the following joint working modalities:

ü With the Ecuadorian Red Cross.

ü Within the IFRC (country/regional/Geneva level). Coordination between regional and global levels

ü Within the wider IFRC membership.

ü With ICRC.

b) Decision making process and coordination for mobilizing regional and global tools.

c) What coordination has there been with non-Movement actors at national and regional levels? How has this worked in relation to the National Society’s auxiliary role with the Government. d) What systems for communication and information management have been used to improve IFRC and Movement coordination? 4. To what extend is the intervention taking into consideration long-term needs?

a) How is the response building, in an inclusive way, on the capacity of local organisations and structures including the ERC?

b) How is the intervention building on and preserving the structures and systems in place prior to the earthquake?

c) How has the response resulted in enhanced institutional capacity of ERC?

Additional questions: It is recognized that emerging questions related to those framed above may arise in the course of the RTE.

5. Evaluation methodology & process

The methodology will adhere to the IFRC Framework for Evaluations[2], with particular attention to the processes upholding the standards of how evaluations should be planned, managed, conducted, and utilized.

An IFRC evaluation management team will manage and oversee the evaluation and, with the evaluators, ensure that it upholds the IFRC Management Policy for Evaluation. The evaluation management team will consist of three people not directly involved with the operation. One member is from the IFRC PMER team in Geneva, one from the Americas Regional office and one from the Disaster and Crisis Prevention, Response and Recovery in Geneva (DCPRR).

The evaluation team will consist of up to three/four people: one external evaluator as team leader and two or three partner National Society evaluation practitioners who will also provide the interface with the IFRC offices in country and will help to clarify internal processes and approaches for the team. The team will be gender balanced. The team leader should have regional knowledge / experience and ideally all candidates will have experience with evaluation practices and the IFRC disaster response systems.

The external evaluator will provide an independent, objective perspective as well as technical experience on evaluations, and will be the primary author of the evaluation report. S/he will not have been involved or have a vested interest in the IFRC operation being evaluated, and will be hired through a transparent recruitment process, based on professional experience, competence, ethics and integrity for this evaluation. The RTE team leader will report on progress or challenges to the management team and will be the primary author of the evaluation report.

The National Society staff and volunteers will work with the external evaluator in the evaluation process, and will be able to provide perspectives on the RCRC actors and interactions in the operation. It is expected that the team will be able to conduct a reliable and informed evaluation of the emergency operation that has legitimacy and credibility with stakeholders.

The specific evaluation methodology will be detailed in close consultation between the RTE team and IFRC, but will draw upon the following primary methods:

1. Desktop review of operation background documents, relevant organizational background and history, including prior IFRC RTE evaluation reports, and any relevant sources of secondary data, such exist surveys from IFRC participants in the operation.
2. Field visits/observations to selected sites and to the Country / Regional offices.
3. Key informant interviews (institutional and target population as appropriate).
4. Focus group discussions, (institutional and target population) as time and capacity allow.

The RTE team will meet with and interview key Red Cross Red Crescent stakeholders in the ERC, Partner National Societies, and the relevant IFRC offices. The team will also consult with other partners and organizations such as governments, the UN, INGOs / NGOs as appropriate to the evaluation’s objectives, including the target population.

The team will be briefed in Geneva and Panama.

Initial findings will be shared through a findings workshop with the Ecuadorian Red Cross and the Partner Operational Response teams in Ecuador.

Consultancy Timeframe

The evaluation is expected to last a maximum of 30 days

A draft report will be prepared for review. This review process should occur within 4 weeks of submitting the draft report to the evaluation management team, and will involve the following stakeholders in the following order:

· Week 1-2 post review: The evaluation management team to check content is in line with this TOR and IFRC evaluation standards. Stakeholders who participated in the evaluation to provide feedback on any inaccuracies or clarifications (differences of opinion should not be put forward here but outlined in the management response). Following this, a final draft is prepared.

· Week 3-4 post review: an evaluation management response team from within the IFRC will review the report and a management response will be compiled by POD to be included as an appendix to the final published RTE report.

The draft IFRC Real-time Evaluation Management Guide will be used for this RTE and made available to the evaluation team.

6. Evaluation deliverables

Inception Report– The inception report will be a scoping exercise for the RTE and will include the proposed methodologies, data collection and reporting plans with draft data collection tools such as interview guides, the allocation of roles and responsibilities within the team, a timeframe with firm dates for deliverables, and the travel and logistical arrangements for the team.

Debriefings / feedback to management at all levels: The team will report its preliminary findings to the field and IFRC in Panama (Regional Office). The team or team leader will debrief in Geneva in a timely manner and will adhere to the above mentioned review process.

Draft report: A draft report identifying key findings, conclusions, recommendations and lessons for the current and future operation, will be submitted by the team leader within two weeks of the evaluation team’s return from the field.

Final report: The final report will contain a short executive summary (no more than 1,000 words) and a main body of the report (no more than 10,000 words) covering the background of the intervention evaluated, a description of the evaluation methods and limitations, findings, conclusions, lessons learned, clear recommendations. Recommendations should be specific and feasible. The report should also contain appropriate appendices, including a copy of the ToR, cited resources or bibliography, a list of those interviewed and any other relevant materials. The final RTE report will be submitted one week after receipt of the consolidated feedback from IFRC.

All products arising from this evaluation will be owned by the IFRC. The evaluators will not be allowed, without prior authorization in writing, to present any of the analytical results as his / her own work or to make use of the evaluation results for private publication purposes.

The preliminary and final reports will be submitted through the RTE management group, who will ensure the quality of the report providing input if necessary. The management group will submit the report to the IFRC Secretariat stakeholders interviewed for review and clarifications. The final report will be submitted to the USG, POD along with the proposed management response for approval. The USG POD will ensure subsequent dissemination and follow-up.

7. Evaluation quality and ethical standards

The evaluators should take all reasonable steps to ensure that the evaluation is designed and conducted to respect and protect the rights and welfare of the people and communities involved and to ensure that the evaluation is technically accurate and reliable, is conducted in a transparent and impartial manner, and contributes to organizational learning and accountability. Therefore, the evaluation team should adhere to the evaluation standards and applicable practices outlined in the IFRC Framework for Evaluation.

The IFRC evaluation standards are:

1. Utility: Evaluations must be useful and used.
2. Feasibility: Evaluations must be realistic, diplomatic, and managed in a sensible, cost effective manner.
3. Ethics & Legality: Evaluations must be conducted in an ethical and legal manner, with particular regard for the welfare of those involved in and affected by the evaluation.
4. Impartiality & Independence; Evaluations should be impartial, providing a comprehensive and unbiased assessment that takes into account the views of all stakeholders.
5. Transparency: Evaluation activities should reflect an attitude of openness and transparency.
6. Accuracy: Evaluations should be technical accurate, providing sufficient information about the data collection, analysis, and interpretation methods so that its worth or merit can be determined.
7. Participation: Stakeholders should be consulted and meaningfully involved in the evaluation process when feasible and appropriate.
8. Collaboration: Collaboration between key operating partners in the evaluation process improves the legitimacy and utility of the evaluation.

It is also expected that the evaluation will respect the seven Fundamental Principles of the Red Cross and Red Crescent: 1) humanity, 2) impartiality, 3) neutrality, 4) independence, 5) voluntary service, 6) unity, and 7) universality. Further information can be obtained about these Principles at:

www.ifrc.org/what/values/principles/index.asp

8. Qualifications

Selection of the external evaluation consultant will be based on the qualifications outlined below:

1. Demonstrable experience in leading evaluations of humanitarian programs responding to major disasters, with specific experience in RTEs preferred;

2. Knowledge of strategic and operational management of humanitarian operations and proven ability to provide strategic recommendations to key stakeholders;

3. Strong analytical skills and ability to clearly synthesize and present findings, draw practical conclusions, make recommendations and to prepare well-written reports in a timely manner;

4. Experience in qualitative data collection and data analysis techniques, especially in emergency operations;

5. Knowledge and experience working with the Red Cross Red Crescent Movement and knowledge of the IFRC’s disaster management systems;

6. Demonstrated capacity to work both independently and as part of a team;

7. Must be fluent in Spanish (verbal and written);

8. Excellent English writing and presentation skills, with relevant writing samples of similar evaluation reports;

9. Knowledge of and experience working in the Americas region;

10. Minimum qualification of a master’s degree or equivalent combination of education and relevant work experience;

11. Immediate availability for the period indicated.

The IFRC would also look for the two to three NS team members to have relevant evaluation and disaster response experience. The profile of the NS team members is as follows:

1. Experience in assessment/review of the emergency operations (preferably of similar scale).
2. Experience in qualitative data collection and data analysis techniques, especially in emergency operations useful;
3. Experience of a major humanitarian response.
4. Knowledge and experience working with the Red Cross Red Crescent Movement and knowledge of the IFRC’s disaster management systems.
5. Strong operational experience, but also analytical abilities.
6. Knowledge of the Americas region, but not involved in the operation.
7. English and Spanish (reasonable working level of written and verbal skills).
8. Immediate availability from end June to end of July 2016.
9. Completed IFRC security training.

9. Application procedures

Interested candidates should submit their application material by Friday 24 June 2016 to the following email: pmer.support@ifrc.org. Application material is non-returnable, and we thank you in advance for understanding that only short-listed candidates will be contacted for the next step in the application process.

Application materials should include:

1. Curriculum Vitae (CV) in English and Spanish

2. Cover letter clearly summarizing your experience as it pertains to this RTE, your daily rate, and three professional references.

3. At least one example of an evaluation report most similar to that described in this TOR.

[1] The Principles and Rules for Red Cross and Red Crescent Humanitarian Assistance govern National Societies and their International Federation in international humanitarian assistance. The Principles and Rules encompass preparedness for response, disaster relief and early recovery activities. They establish a coordinated and agreed approach to quality and accountability, and recognize partnerships with public authorities, humanitarian actors and other organizations external to the Movement

[2] http://www.ifrc.org/Global/Publications/monitoring/IFRC-Framework-for-Evaluation.pdf

Application procedures

Interested candidates should submit their application material by Friday 24 June 2016 to the following email: pmer.support@ifrc.org. Application material is non-returnable, and we thank you in advance for understanding that only short-listed candidates will be contacted for the next step in the application process.

Application materials should include:

1. Curriculum Vitae (CV) in English and Spanish

2. Cover letter clearly summarizing your experience as it pertains to this RTE, your daily rate, and three professional references.

3. At least one example of an evaluation report most similar to that described in this TOR.