Evaluation Consultant

To conduct a winterization response evaluation by engaging an experienced consultant alongside a Review Board comprising of three members of the Movement Partners with extensive experience in integrated winterisation response in the context of protracted displacement and migration crisis to guide the process. The purpose of the Winterization Response Evaluation is to document the response relevance and effectiveness and to identify lessons for improving sectoral planning for Winterization to ensure continuity of the assistance to the displaced, non-displaced, host and returnees. The review will assess the outcomes of winterization support in terms of quality, accomplishments, timeliness, cost effectiveness and overall accountability with reference to the response structure, system, procedure, and performance by the URCS with support from its Movement and external partners. Considering this crisis as protracted and potential risks and vulnerabilities in terms of settlements and population, key recommendations particularly on areas of winterization package, its coverage as well as advocacy and strategies on balancing the immediate and long-term needs from this winterization response evaluation will guide modifications and improvements required to the ongoing medium- and longer-term interventions.

Purpose:

To improve quality assurance and accountability, this evaluation will examine the Red Cross winterization strategy and the response plan in delivering the winterization services and its medium and longer-term and document the lesson learned so that it can be used to replicate the good practices in other similar kind of crises in reference to the URCS ONE PLAN and sustainability to respond the complex crisis in Ukraine. Also, its findings and recommendations should help the URCS and its partners to make immediate necessary adjustments to both programming and operations for the winterization as the response evolves in order to better meet the needs of the affected population. Furthermore, it also provides clear recommendations on implementation plan in relation to URCS ONE PLAN (2023 – 2025) as the URCS sector/subsector is preparing details sectoral plan including the plan of action.

The scope will focus on determining and streamlining packages on the winterization response in consultation with URCS and supporting partners. It is expected that key findings and recommendations will help to address the relevant needs for the URCS next winter response. The focus of the Winterization Response Evaluation is to evaluate the winterization response of the Red Cross on its relevancy/connectedness, coverage, efficiency, effectiveness, and coordination mechanism between URCS, IFRC, ICRS, PNSs and other partners.

Objectives

• To examine URCS’s, and associated partners’ humanitarian action has timely, at appropriate scale and scope, providing quality technical programming, efficient and effective, and delivered safely and securely to benefit the most vulnerable people for the winterization.
• To assess the extent to which the response has been accountable to the affected population to protect them from the harsh winter in Ukraine and engaging them in effective and efficient manner.
• To recommend deliverable actions and its modality such as cash transfer programmes (for winterization) or in-kind support (Winterization Package) and its relevancy in Ukraine that are grounded in the response’s context and can immediately be implemented.
• To recommend transition strategy (for relief and shelter) from emergency/recovery to a longer-term response as a Red Cross.
• To identify good practices and lessons learned regarding operational coordination (including government, Movement Partners and other humanitarian partners) at national, regional (oblasts) and local (branch) level, including facilitation and management of complementary international assistance in relation to the winterization support.

Evaluation Criteria

The following criteria will be used to guide the evaluation recommendations:

• Relevance and appropriateness
• Coverage
• Efficiency / effectiveness / accountability
• Impact
• Connectedness and sustainability

Evaluation questions

The consultant and URCS team will create specific questions linked to the above objectives link with the crisis, RCRC focused winterization program and cross-cutting themes such as community engagement and accountability (CEA); protection, gender and inclusion (PGI); preparedness for international assistance; monitoring and reporting; and logistics arrangement. Recommendation of overall questions based on the criteria above are outlined below. The details questions as per the evaluation criteria is presented in the Annex:

1. Did the winterization support meet the affected populations’ immediate needs (relevancy) and intended results? (impact)
2. To what extent has the assistance been able to effectively target the population in most need of winterization assistance?
3. In relation to the RC winterization support, to what extent, if any, has the perception of relevance varied between geography, instrument, target group and vulnerability?
4. Were complaints/feedback mechanisms put in place for community questions and concerns to be answered? What were the concerns raised by the affected population during the intervention (Accountability)?
5. To what extent, if any, has the effectiveness varied between different activities, for different target groups and in different areas?
6. To what extent were PGI minimum standards integrated into the winterization response planning and implementation?
7. What have been the (strategic as well as operational) challenges and successes for the procurement and logistics support provided to the winterization support?
8. To what extent have the winterization support included a focus on sustainability (climate/environment) and what are, if any, opportunities are there for delivering a more sustainable response?
9. How well or not well have the coordination between RCRC Movement partners worked for the planning and implementation of the winterization support?
10. What actionable recommendations can be made to improve the relevance and effectiveness of the winterization response for 2023/24 in relation to the URCS ONE PLAN and winterization strategy?

METHODOLOGY

The methodology will adhere to the IFRC Framework for Evaluations, with particular attention to the processes upholding the standards of how evaluations should be planned, managed, conducted, and utilized.

For this evaluation, a ‘Review Board’ will be established to review the findings and endorse it. The Review Board will have members from URCS and Partner National Society Program Lead (Particularly from Shelter and Relief) with the experience of Ukraine Crisis Response. The Review Board will be entrusted to guide the evaluation process, including by provide strategic inputs across the whole process, from the design phase to the delivery and comment on the final report.

The specific evaluation methodology will be detailed in close consultation between the EMT, Commissioner and Review Board, but will draw upon the following primary methods:

• Desk review of operation background documents, relevant organizational background and history, including URCS and National policies and SOPs, prior RCRC reports, and any relevant sources of secondary data, such as Post Distribution Monitoring (PDM), learning review report, and other surveys and report from Movement partners and participants in the operation.
• Field visits/observations Selected locations of the winterization support areas.
• Key informant interviews (institutional and beneficiaries as appropriate).
• Focus group discussions (institutional and beneficiaries) as time and capacity allow.

The successful evaluation team will develop an inception report after the initial desktop review for submission and approval to the EMT. This inception report should outline a detailed proposed methodology that must consider:

• Sampling method is to be recommended by the evaluator, as long the final sample to be evaluated on includes both URCS involved in the winterization operation interventions, and the ‘most vulnerable’ beneficiaries.
• Data collection methods and pace are to be decided by the evaluator, in consultation with the Evaluation Management Team but should consider the reality of difficult-to-reach areas.
• The evaluation consultant should visit a representative number of communities in the operational areas (through winterization support). This will be agreed with Evaluation Management Team (EMT) based on resources available.

The evaluation consultant will need to work together with evaluation management team.

OUTPUTS/DELIVERABLES

The evaluation consultant will deliver the following outputs:

1. Inception Report: The inception report will be a scoping exercise for the review/evaluation and will include the proposed methodologies, data collection and reporting plans with draft data collection tools such as interview guides, the allocation of roles and responsibilities within the team, a timeframe with firm dates for deliverables, and the travel and logistical arrangements for the team.
2. Debriefings / feedback to management at all levels: The team will report its preliminary findings to the URCS prior to leaving the country. This will include a presentation to key stakeholders as determined by the EMT, a detailed PowerPoint presentation and sufficient time for discussion and suggested edits.
3. Draft report: A draft report identifying key findings based on facts and will separate from the report opinions or rumours, conclusions, recommendations, and lessons for the current and future operation, will be submitted by the evaluation consultant within three days after presenting the initial findings.
4. Final report: The final report will contain a short executive summary (no more than 1,000 words) and a main body of the report (no more than 10,000 words) covering the background of the intervention evaluated, a description of the evaluation methods and limitations, findings, conclusions, lessons learned and clear recommendations. Recommendations should be specific and feasible. The report should also contain appropriate appendices, including a copy of the ToR, cited resources or bibliography, a list of those interviewed and any other relevant materials. The final report will be submitted two days after receipt of the consolidated feedback from URCS. Details of the final report are outlined in the table below.

Evaluator(s), with knowledge of local context who will provide an independent, objective perspective as well as technical experience on evaluations, will be hired to conduct the review/evaluation. The evaluator will be the primary author of the evaluation report. S/he will not have been involved or have a vested interest in the URCS operation or context being evaluated, and will be hired through a transparent recruitment process, based on professional experience, competence and ethics and integrity for this evaluation. The evaluator will report on progress or challenges to the evaluation management team. The evaluator should have the following characteristics:

• Demonstrable experience in leading evaluations of humanitarian programmes responding to refugee and displacement crises and preferably previous experience of conducting evaluation in Ukraine or impacted countries.
• Knowledge of activities generally conducted by humanitarian organizations in the shelter and non-food items including winterization of response.
• Field experience in the evaluation of humanitarian or development programmes, with prior experience of evaluating RCRC programmes desirable.
• Strong analytical skills and ability to clearly synthesize and present findings, draw practical conclusions, make recommendations and to prepare well-written reports in a timely manner (examples of previous work).
• Previous experience in coordination, design, implementation, and monitoring and evaluation of humanitarian programmes.
• Experience in qualitative data collection and data analysis techniques, especially in emergency and recovery operations
• Knowledge and experience working with the Red Cross Red Crescent Movement preferred.
• Demonstrated capacity to work both independently and as part of a multi-discipline, multi-national team.
• Ability to work within tight deadlines and manage with available resources.
• Fluent in spoken and written English.
• Strong interpersonal skills.
• Relevant degrees or equivalent experience.
• Availability for the time period indicated

The evaluator(s) should take all reasonable steps to ensure that the review/evaluation is designed and conducted to respect and protect the rights and welfare of people and the communities of which they are members, and to ensure that the evaluation is technically accurate, reliable, and legitimate, conducted in a transparent and impartial manner, and contributes to organizational learning and accountability. Therefore, the evaluator(s) should adhere to the evaluation standards and specific, applicable process outlined in the IFRC Framework for Evaluation. The IFRC Evaluation standards are:

The IFRC Evaluation Standards are:

1. Utility: Evaluations must be useful and used.
2. Feasibility: Evaluations must be realistic, diplomatic, and managed in a sensible, cost-effective manner.
3. Ethics & Legality: Evaluations must be conducted in an ethical and legal manner, with regard for the welfare of those involved in and affected by the evaluation.
4. Impartiality & Independence: Evaluations should be impartial, providing a comprehensive and unbiased assessment that considers the views of all stakeholders.
5. Transparency: Evaluation activities should reflect an attitude of openness and transparency.
6. Accuracy: Evaluations should be technical accurate, providing sufficient information about the data collection, analysis, and interpretation methods so that its worth or merit can be determined.
7. Participation: Stakeholders should be consulted and meaningfully involved in the evaluation process when feasible and appropriate.
8. Collaboration: Collaboration between key operating partners in the evaluation process improves the legitimacy and utility of the evaluation.

It is also expected that the evaluation will respect the seven Fundamental Principles of the Red Cross and Red Crescent: 1) humanity, 2) impartiality, 3) neutrality, 4) independence, 5) voluntary service, 6) unity, and 7) universality.

How to apply

Interested applicants should submit their expression of interest to the following email: zsuzsanna.zakarova@ifrc.org and copy:

i.kletskovskyy@redcross.org.ua and n.p.dahal@redcross.org.ua by 26 June 2023 (Ukraine time). In the subject line, please state the consultancy you are applying for, your company name or last name and first name. (SUBJECT: Ukraine: Red Cross Winterization Response Evaluation – Last Name, First Name).

For more detailed TOR please send a message to zsuzsanna.zakarova@ifrc.org.

Application materials should be submitted in English which will include:

1. Curriculum Vitae (CVs) of consultant.
2. Cover letter clearly summarizing your experience as it pertains to this evaluation, daily consultancy fees inclusive of rate for the national consultant (as relevant) and three professional references.
3. Applicants will be required to provide a 2-page approach paper with review methodology and two samples of previous written report most similar to that described in this ToR.
4. Financial proposal: It is expected from the international consultant to submit the financial bid for themselves (daily rate), full cost of interpretation costs (if needed).

Application material are non-returnable, and we thank you in advance for understanding that only short-listed candidates will be contacted for the next step in the application process. Please take note that incomplete application will be rejected.

APPENDICES (TO BE PRESENTED AFTER APPOINTMENT)

1. Package of Reference Documents for Winterization Response
2. Following URCS Ukraine Crisis response documents (all EA and EPOAs, Situation reports, operation update, URCS ONE PLAN, PDM reports, Satisfaction survey data)
3. Stakeholders list.