Final Evaluation of the Monarch Butterfly Program Colombia and Venezuela

Evaluation Purpose & Scope
This final evaluation aims to evaluate the relevance, effectiveness, systems management, and coordination of the Monarch Butterfly Program between 2018 and 2021 responding to the migrant´s needs. Identification of lessons learned and concrete recommendations that contribute in the future to a better program design approach are expected.
Evaluation Objectives - Criteria - Questions
*Objectives:

Objective 1: Evaluate outcomes, and their impact on the target population, achieved by the National
Societies during the timeframe of the program with particular emphasis on:
• Health: how the primary health services as psychosocial support, sexual health care, and nutrition were
adapted in the HSP and communities to reach the target population and how its welfare was impacted. Also,
evaluate how the pandemic affected the program and the delivery process.
• Migration: what profile of migrants were supported; how the approach during the program contributed to the
strengthening of the two National Societies’ approaches to migration and HSP.
• Protection, Gender, and Inclusion (PGI): how the PGI approach was employed and its impact on different
population groups in the HSP; launch of school and community interventions; the relevance of restoring family
links in relation to protection.
Community Engagement and Accountability (CEA), as a cross-cutting approach, will be evaluated. Emphasis
should focus on the participation of the target population in the creation and implementation of the programmatic
activities.
Objective 2: To assess how the National Societies has strengthened its knowledge and capacities in the area of
migration through the program.

Evaluation Criteria
The following are the evaluation criteria by which this evaluation will be conducted. These criteria should be
used as applicable for every question and objective, based on their relevance for the given objective and the
questions related to the objective.

1. Relevance of activities and humanitarian services delivered by the Red Cross to mitigate the needs identified
   in the target population.
2. Quality of humanitarian services delivered by the Red Cross both in HSP and communities.
3. impact of the program in migrants and communities in delivering assistance based on needs and context;
   and the impact of the program on the National Societies, their branches and volunteers
4. Coverage in reaching target populations across different groups, considering ethnicity, age, gender,
   disability, migratory status, etc.
5. Efficiency of the interventions in delivering quality services with minimum resources, the effectiveness of
   the management and systemic mechanisms and strategies; and accountability to the affected population in
   relation to service delivery, sustainability, engagement, inclusion, and resilience building.
6. Sustainability achieved through the program within the National Societies and their branches, and at the
   community level of the interventions

Recommended Questions
The following are recommended questions listed per objective to be assessed along with the above evaluation
criteria. However, the consultancy should contribute additional questions in order to strengthen the evaluation
results.
Objective 1: Activities, outputs, outcomes, and their impact on a target population, achieved by the
National Societies National.

1. Were the interventions according to the context, migration dynamics, and people's needs?
2. Were there adequate resources (technical, financial, human, physical, and informational) available and
   were the resources used effectively and efficiently during the program?
3. Were needs assessments conducted to justify interventions? What was the quality of these
   assessments? How were they used?
4. the program during its various phases was implemented according to its design, how the program's
   challenges were managed?
5. Were there mechanisms to capture the community’s suggestions, questions, complaints/feedback, was
   done?
6. How does the program integrate this into the plans and the implementation and how accountable was the
   program in assisting the community to make informed decisions?
7. How do the people reached rate the assistance received?
8. Were develop some capacity-building in migrants, communities, school environments?
9. How sustainable were and is the capacity-building (people and communities)?
10. Any transitions or transfers with other projects or programs within the national societies were carried out

Objective 2: To assess how the National Societies has strengthened its knowledge and capacities in the
area of migration through the program

1. How did the program contribute to improving the NS’s involvement in coordination spaces with local
   authorities, between National Societies, IFRC, other Red Cross Movement partners, clusters / technical
   working groups?
2. Did the program contribute to strengthening the migration approach in National Societies and IFRC?
3. Were some capacity-building efforts developed with volunteers, branches, NS, IFRC, and Movement
   network?

Evaluation Methodology
The lead evaluator is expected to develop a detailed methodology for this evaluation in the inception report, which
needs to be approved by the Evaluation Management Team.
The methodology is expected to include review and analysis of key documents, primary and secondary
information, key informant interviews (including those with the operational NS and branches), volunteer
interviews, and interviews with people reached by the programme. The evaluator can suggest other
methodologies. Sampling, as well as data collection methods and pace, are to be decided in consultation with the
National Societies and the Evaluation Management Team. Potential ideas to this effect should be reflected on in
the cover letter, and if selected, are required to be described in detail in the inception report.
• Desk review: Conducting a desk review of documentation, including the programme proposals for each
phase, interim and final reports proposal, Plan of Action, proposals, updates, pledge-based reports, M&E
data, final reports to donors, and other reviews and lessons learned papers as well as secondary analysis of
the appropriateness of surveys and reports conducted during the program. Most of this work can be done
from outside the countries.
• Key informant interviews: the evaluators need to interview different persons involved in the program to have
a solid overview of the different phases. This includes persons from IFRC, the two National Societies as well
as migrants, people from communities and schools, and local actors as health secretaries and organizations
with which the Red Cross collaborated during the programme. Digital/online interviews can be arranged with
the persons who have already left the program. A list of possible interviewees will be shared upon selection.
• Field visits and primary data collection (interviews with people reached and/or community leaders):
Depending on feasibility due to COVID-19 related restrictions during the evaluation timeframe, the evaluator
should aim to assess the impact and/or to validate monitoring survey results, by interviewing a sufficient
number of migrants (at the HSP) and people in communities who received support through the programme.
If field visits are not possible, coordinate with country teams for remote interviewing options.
The Evaluation Management Team will provide support in developing an arrangement in which primary data
collection with people reached and/or community leaders is feasible, involving the National Societies capacities
if possible.

Evaluation Team
There are no limits on the number of people necessary for the Evaluation Team, including the option for a sole
investigator. All people on the team must be listed in the proposal. The evaluator (team leader) will have the
ultimate responsibility to lead the evaluation process, design the methodology and deliver the outputs as
described below.
Evaluation Management Team
The Evaluation Management Team (EMT) consists of the ARO PMER Regional Manager, Migration and
Protection, Gender and Inclusion Regional Coordinator. This could include IFRC staff from other clusters or
country delegations. The EMT will ensure coordination between the IFRC delegation in the country and
representatives from the implementing National Societies. The EMT will support the Evaluation Team with
establishing contact with relevant stakeholders in the implementing National Society, and with developing
modalities to ensure remote and, if feasible, on-site access for conducting the evaluation. The EMT is tasked with
reviewing and approving the deliverables listed in the following section.

Deliverables (or Outputs)
Inception report - The inception report should include the proposed methodologies, a data collection and
reporting plan with identified deliverables, draft data collection tools such as interview guides, questionnaire,
sampling method, a timeframe with firm dates for deliverables and travel (if applicable), and logistical
arrangements for the evaluation.
Debriefing - A debriefing will be conducted with the IFRC and NS after data collection. The debriefing is to update
the progress and initial findings with recommendations of the evaluation.
Draft report: The consultant will produce a draft report in Spanish (identifying key findings based on facts, lessons
learned, conclusions, and recommendations to be used on future programmes) which will be reviewed by the
IFRC EMT and the cluster programs team. The consultant will be given feedback after 10 working days to
incorporate into the final report. The draft report will be also shared with the National Societies for comments.
Final report. A Final report highlighting key findings, conclusions, lessons learned, and recommendations will be
submitted within 10 days of receiving the feedback on the draft report. Final evaluation report of no more than 40
pages (excluding executive summary and annexes such as a copy of the ToR, cited resources, a list of those
interviewed, and any other relevant materials). Likewise, a presentation (ppt or other tools) should also be made
to IFRC and the NS of the results of the evaluation.
The findings and all products arising from this evaluation will be jointly owned by the National Societies involved
and IFRC. The evaluator will not be allowed, without prior authorization in writing, to present any of the analytical
results as his / her own work or to make use of the review results for private publication purposes. All case studies,
anecdotes, any rough tool used, and copies of participatory tools to be submitted to the IFRC teams to ensure
scientific accountability and data protection of the people involved in the evaluation.

Evaluation Quality & Ethical Standards
The evaluators should take all reasonable steps to ensure that the evaluation is designed and conducted to
respect and protect the rights and welfare of people and the communities of which they are members, and to
ensure that the evaluation is technically accurate, reliable, and legitimate, conducted in a transparent and impartial manner, and contributes to organizational learning and accountability. Therefore, the evaluation team should
adhere to the evaluation standards and specific, applicable processes outlined in the IFRC Framework for
Evaluation. The IFRC Evaluation Standards are:

1. Utility: Evaluations must be useful and used.
2. Feasibility: Evaluations must be realistic, diplomatic, and managed in a sensible, cost-effective manner.
3. Ethics & Legality: Evaluations must be conducted in an ethical and legal manner, with particular regard
   for the welfare of those involved in and affected by the evaluation.
4. Impartiality & Independence; Evaluations should be impartial, providing a comprehensive and unbiased
   assessment that takes into account the views of all stakeholders.
5. Transparency: Evaluation activities should reflect an attitude of openness and transparency.
6. Accuracy: Evaluations should be technically accurate, providing sufficient information about the data
   collection, analysis, and interpretation methods so that its worth or merit can be determined.
7. Participation: Stakeholders should be consulted and meaningfully involved in the evaluation process
   when feasible and appropriate.

8. Collaboration: Collaboration between key operating partners in the evaluation process improves the
   legitimacy and utility of the evaluation.
   It is also expected that the evaluation will respect the seven Fundamental Principles of the Red Cross and
   Red Crescent: 1) humanity, 2) impartiality, 3) neutrality, 4) independence, 5) voluntary service, 6) unity, and 7)
   universality.

   Evaluator & Qualifications
   The evaluator must have experience or significant knowledge of the humanitarian response mechanisms and
   have previous experience in conducting evaluations for multi-year programmes. It is preferred for the evaluator
   to have a good understanding of interventions with migrants and communities.
   The evaluator will coordinate directly with the EMT and the IFRC team in Colombia and Venezuela when
   evaluating the activities in the country with Programmes Coordinator oversight.
   The evaluator should meet the following requirements:
   Required:
   • 5- 7 years of demonstratable experience in coordination, design, implementation and monitoring, and
   evaluation of humanitarian programmes
   • Knowledge of activities generally conducted by humanitarian organizations in the sectors of Health,
   Migration and PGI, and CEA, among others.
   • Experience in participatory approaches to evaluations
   • Excellent Spanish writing and presentation skills, with relevant writing samples of similar evaluation
   reports. English is an asset.
   Desirable:
   • Very good understanding of the RC/RC Movement and types of humanitarian response.
   • Field experience in the evaluation of humanitarian or development programs with prior experience of
   evaluating Red Cross programmes.

How to apply:

Applications are to be submitted by the 17th February 2022 to peru.limaregdelegation@ifrc.org stating in the subject line: ‘Monarch Butterfly Program Final Evaluation’. The following should be submitted with the application:

The application should include:

1. Cover letter clearly summarizing experience as it pertains to this assignment, flat rate, and contact details of three professional referees.

2. Technical proposal, including budget: a technical proposal should accompany the application, detailing the consultant’s understanding of the ToR with a detailed budget to undertake the work. Timeline of activities (specifying what part will be done remotely and in the field).

3. Curriculum Vitae (CV) of the consultor or the team if applicable.

4. Provide samples of at least two previous evaluations and reviews completed.