A. Background
The conflict in Syria stretches back over 10 years and has resulted in the loss of hundreds of thousands of lives, the displacement of millions within Syria, to neighbouring countries and beyond. Over five million Syrian people are displaced in the Middle East and North Africa, including in Turkey, Lebanon, Jordan, Iraq, and Egypt. Turkey alone currently hosts 4 million refugees and asylum-seekers, including over 3.6 million registered Syrian refugees, and some 370,000 registered refugees and asylum-seekers from other nationalities including Somalia, Iraq, Iran, and Afghanistan – making it the host country with the largest refugee population in the world. Over 90 percent of those displaced are living in urban areas.
The Emergency Social Safety Net (ESSN), launched in 2016, is the largest humanitarian aid programme financed by the European Union and its Member States under the Facility for Refugees in Turkey. Since April 2020, with funding from the European Union, the IFRC and the Turkish Red Crescent Society (Türk Kızılay/TRC) are partnering to implement phase III of the ESSN. As of February 2021, the programme was assisting over 1.8 million refugees living under temporary and international protection in Turkey to meet their basic needs via debit card-based monthly cash assistance, complemented by severe disability and quarterly family size-based top ups.
B. Rationale and purpose for the Evaluative Learning Study
As the largest humanitarian cash programme in the world, the ESSN III is committed to high-quality, institutional learning that contributes to the growing global evidence base on large scale multipurpose cash assistance. The ESSN III programmatic framework emphasises its role in facilitating and embedding this institutional learning in an accountable way, through strategic partnerships, targeted research, and disseminating best practices and lessons learned to all interested stakeholders. This evaluative learning study is an integral part of the ESSN III’s overall learning strategy and in particular its integration of independent analysis, alongside third-party monitoring (TPM), which reflects ECHO guidance on ensuring external perspectives for monitoring, evaluation, and learning in the delivery large scale cash transfers.[1] Third-party monitoring for the ESSN III begins data collection in 2021 and focusses on exploring cross-cutting themes about the operational set-up of the ESSN III (including shock-responsive elements), the effects of Covid-19 on the programme, the ESSN’s effectiveness in enabling recipients to meet basic needs, and its accountability to recipients. The impact or effect of the programme is a separate but complementary component of ESSN III learning, and an important consideration for this evaluative learning study.
An impact evaluation for ESSN I and II conducted by the World Bank, the World Food Programme, and TRC demonstrated that the programme’s impact on recipients was positive overall, particularly with regards to improvements in food consumption, coping with food shortages, and incurring less debt.[1] The evaluation also found “prima facie evidence” that the programme “caused substantial changes in household composition and size,” and observed a “sharp decline in inequality” for both ESSN recipients and ineligible households (the entire study group).[2] The findings, coupled with the effects of the Covid-19 pandemic, generate further questions about refugees’ ability to access income, meet needs, and cope in an evolving situation, all of which have programmatic implications for the newest iteration of the ESSN and its new operational set-up between TRC and IFRC.
The impact evaluation for ESSN I and II was published recently (November 2020) with data collected pre-pandemic; the programme parameters have largely remained constant, or the notable programme shifts, such as the introduction of a complementary cash mechanism (C-ESSN), require more time to materialize/come into effect and to be able to analyse subsequent effects or impact.[3] As such, a similar impact evaluation at this stage is not likely to add value alongside programmatic M&E and the ongoing third-party monitoring.[4] There is also a need to focus on learning beyond traditional impact analysis, in line with the CaLP Network and ALNAP’s recent recommendations on evaluating cash assistance – moving beyond evaluations that confirm providing cash improves people’s ability to meet needs during the period they receive assistance and exploring more complex issues such as the underlying factors that may affect programme outcomes.[5]
In particular, the socioeconomic effects of the Covid-19 pandemic on the global economy and refugees’ situations in Turkey require an in-depth understanding of how ESSN recipients survive and support their families in these challenging conditions. The ability to generate income and adapt income sources is crucial for refugees during the Covid-19 pandemic, and understanding income dynamics is key to identifying where longer-term opportunities may lie.[6] The 2020 Atlantic Council / UNDP report on “Turkey’s Refugee Resilience” underlined the difficulties refugees have in accessing reliable employment opportunities; 92% of Syrians working in Turkey engage in informal labour.[7] However, it is unclear how these findings correspond to the ESSN caseload, which targets the most vulnerable households across Turkey. Exploring income sources and testing assumptions about vulnerability are crucial to the overall success of the ESSN in enabling people to meet their basic needs, and in identifying opportunities to maintain the positive outcomes of the programme as its design and focus shift over time.
Income is also a core metric to understand vulnerability in the ESSN III programme, as it is a key determinant in understanding other variables such as coping strategies, levels of debt, and expenditures. The ESSN III’s Post Distribution Monitoring (PDM 10) found that approximately 80% of refugee households surveyed (ESSN recipients and ineligible households) lost access to informal income because of pandemic restrictions, had to incur greater debt, and were decreasing expenses to improve their families’ situation.
Despite this in-depth analysis, there are still information gaps regarding the types of income sources (especially from informal employment), how they relate to coping (captured by the Intersectoral Vulnerability Study - IVS), and how families have adapted during the pandemic. It is therefore important to have more granular level data, building upon PDMs and IVS findings, that can be used for future
programming decisions, linked to a nuanced analysis of refugees’ access to income and in particular informal labour. It is, however, notoriously difficult to accurately capture and analyse income from self-reported data, particularly at the scale of a programme like the ESSN.[1] Challenges include:
· Trend analysis of income sources (particularly informal ones) is difficult through programme monitoring and evaluation without longitudinal data, to determine the frequency, amount, and sustainability (including seasonality) of income sources.
· Income is self-reported and subject to recall bias; people receiving cash via the ESSN may not be willing to share income-related information if they feel this information could be used to determine their eligibility.
· Correlating income (sources and levels) and coping with other factors (demographic, geographic, etc.) within one dataset is challenging, given the need to conduct remote data collection during the pandemic and keep the tools concise.
The wealth of information and knowledge produced by the ESSN III programme requires a stronger understanding of refugees’ access to income, to generate learning that could inform programme design decisions going forward (targeting, pathways away from assistance that consider people’s capacities, vulnerability, etc.) as well as strengthen the global evidence base on nexus programming more broadly. An independent study is more likely to overcome the challenges of capturing refugees’ income sources if perceived to be autonomous from ESSN programmatic processes, especially eligibility processes. Moreover, the evaluative learning study offers an opportunity to collaborate with experts and facilitate knowledge exchange, thereby strengthening the programme’s internal capacity to collect and analyse socioeconomic data at scale as well as disseminate this learning globally.
C. Scope and methodology
The ESSN III programme therefore seeks a knowledgeable and reputable institution or company to co-design an evaluative learning study exploring refugees’ income sources and the programme’s effect on socioeconomic vulnerability as well as people’s capacity to cope. This will involve:
An inception phase to refine the study scope, research questions, methodology, and analysis plan, as well as identify capacity development priorities with TRC and IFRC
Data collection and analysis
The capacity development/exchange component (design and implementation)
Synthesis of findings and report.
The ESSN programme prioritises institutional capacity strengthening and seeks to learn new analytical approaches from the service provider, as well as provide its contextual knowledge based on years of ESSN implementation and data analysis. Capacity exchange will be an integral part of this study and should be incorporated into the workplan through both structured and informal sessions with the appropriate ESSN workstreams. The study scope is therefore divided into two overall components – the first focussed on the study scope and the second dedicated to strengthening ESSN capacity to develop the current structure with relevant and effective adjusted/new models and conduct robust analysis (A and B below).
Initials consultations between TRC and IFRC, with input from DG-ECHO, identified the following preliminary research questions, which will be revised and consolidated with the contracted provider. The final scope and methodology for the ESSN III Evaluative Learning Study will be collaboratively designed between the selected institution and key stakeholders through an iterative process, documented and refined by the contracted provider.
A) Research questions
Income sources
· What are refugees’ main income sources (aside from ESSN cash assistance)?
· Which sectors do the employment-related sources correspond to, and how secure are these opportunities?[1]
· Which sources have people been able to maintain / rely on during Covid-19, and which have disappeared?
· What, if any, is the relationship between income levels and vulnerability severity (established by the IVS)?
Income, adaptation, and coping
· Is there a relationship between types of income and certain coping strategies, following on the analysis of PDMs and IVS findings?
· Do certain vulnerable groups or households adopt particular strategies that enable them to cope better? (E.g., identifying factors of positive deviance[2])
B) Institutional capacity and improving metrics
· What are useful metrics for analysing refugees’ income in Turkey: sectors, levels/thresholds, frequency, sustainability, etc.?
· How can the ESSN III programme track income more reliably?
· What improvements could be made to ESSN data collection and analysis that would strengthen the ability to understand and programme for socioeconomic vulnerability, based on the findings of this study?
The study is expected to include mixed methods, to enable a better understanding of the factors that influence people’s capacity (or vulnerability), while providing statistically sound, quantitative analysis.[3] Regional disaggregation is crucial to understand the different contexts in which refugees live and work. The methodology will be finalised collaboratively between the contracted institution and ESSN stakeholders once existing datasets and analyses are reviewed for relevance. In the proposal, applicants should demonstrate how they have captured information on income sources in relation to coping/well-being of displaced people for other evaluations or research, and how they propose to approach the topic for this study.
D. Expected deliverables and milestones
The selected Service Provider(s) will be expected to submit the following outputs and deliverables in English:
· ESSN III Evaluative Learning Study Draft Inception report, discussion draft including information regarding:
o proposed study design and methodology for data collection and analysis
o technical approach and tools that will be used, and why these are appropriate
o approach, key steps, and respective roles and responsibilities between the key stakeholders to the project across the data collection and analysis phases
o primary data collection tools and guides (if relevant)
o how restrictions related to COVID-19 will be managed
o detailed timeline and project/work plan supporting the achievement of project milestones and objectives, and any other key actions that the service provider should undertake – including the Capacity Support/Exchange plan (see below)
· Progress updates throughout the work.
· Capacity Support/Exchange Plan and implementation of outlined related activities.
· ESSN III Evaluative Learning Study Final Inception report - capturing the scope of work and methodology and updated following feedback on the draft Inception Report.
· ESSN III Evaluative Learning Study Final Report – Analysis, finding and recommendations/ pathways for implementation.
· Slide decks and other materials as required in order to support the running of review and validation workshop with key stakeholders to enable finalising of all reports; sharing of draft versions of the report with relevant stakeholders, via the IFRC project focal point, for feedback before being finalised.
· Regular reports and updates – written or oral, as required and requested – to the IFRC and TRC, in particular the project focal points, and final wrap-up of the project in an accountable and appropriate manner.
· Progress updates throughout the work.
· Capacity Support/Exchange Plan and implementation of outlined related activities.
· ESSN III Evaluative Learning Study Final Inception report - capturing the scope of work and methodology and updated following feedback on the draft Inception Report.
· ESSN III Evaluative Learning Study Final Report – Analysis, finding and recommendations/ pathways for implementation.
· Slide decks and other materials as required in order to support the running of review and validation workshop with key stakeholders to enable finalising of all reports; sharing of draft versions of the report with relevant stakeholders, via the IFRC project focal point, for feedback before being finalised.
· Regular reports and updates – written or oral, as required and requested – to the IFRC and TRC, in particular the project focal points, and final wrap-up of the project in an accountable and appropriate manner.
Timeframe:
September 2021:
ESSN III Evaluative Learning Study Draft Inception Report: Including briefings and stakeholder meetings, desk review, preparation of study design and methodology, data collection and analysis approach and tools, detailed timeline and project/work plan, preparation, and submission.
October 2021:
ESSN III Evaluative Learning Study Final Inception Report: Finalising of draft inception report, including via review and validation workshop and incorporating feedback received.
November 2021 – March 2022:
Capacity Support/Exchange Component **
Training, workshops and any other related activity outlined in the Capacity Support/Exchange Plan
March 2022:
ESSN III Evaluative Learning Study Final Report
April 2022:
ESSN III Evaluative Learning Study Wrap-Up**:** Including wrap-up workshop, presentation of actual results (once report is finalised) and all other activities required to facilitate correct and accountable closure of the project**
E. Skills and qualifications
The Service Provider should have demonstrable knowledge and experience in the following areas:
· At least 10 years’ experience in research and evaluation of humanitarian cash-based programmes and social safety net programmes (largescale and complex)
· Proven capacity support and strengthening expertise in study design and data analysis (institutional learning/teaching, knowledge exchanges, training design and delivery, etc.)
· Experience with mixed methods approaches; experience with longitudinal studies or ethnographic methodologies (as recommended by ODI[1]) considered an advantage
· Demonstrated experience in designing, planning, organising, and conducting surveys in social safety net programmes/humanitarian cash transfer programme, including expertise and experience in indicator development, testing and data collection / analysis
· Demonstrated understanding, expertise and experience working on issues related to cash transfer programming, social protection, and refugees
· Proven experience in data validation and data quality control for large-scale projects/programmes
· Sufficient resources and staff to carry out the assignment effectively and efficiently in Turkey
· Experience working within Turkey considered an advantage, particularly with the Turkish government and/or other Turkish organisations
· Good communication skills, both orally and in writing
· Fluency in written and spoken English (deliverables), Turkish (meetings / workshops), and Arabic (consultations and data collection)
F. Contract Award Criteria
The contract will be awarded on the following basis:
30 points: Adherence to Terms of References specifications and related requirements, including (i) a clear understanding of required deliverables; and (ii) a robust, appropriately proposed methodology and approach
20 points: Demonstrable ability to delivery coherence and consistency with the existing ESSN III Project Plan.
25 points: Quality of suggested team member(s) to undertake all aspects of the assignment, meeting academic and qualification requirements; minimum 10 years’ experience, demonstrated experience in carrying out similar work
25 points: Financial Proposal: Competitive fee rates and expenses in relation to the market and demonstration of value for money; clear and effective financial plan to deliver output-based deliverables and key performance measures; financial approach and methodology for ensuring the requirements will be delivered on time and in line with agreed costs, highlighting any financial risks.
G. Conditions
· The Service Provider will work on its own computer(s) and use its own office resources and materials in the execution of this assignment. The contractor’s fee shall be inclusive of all office administrative costs
· Local travel and airport transfers (where applicable) will be covered in accordance with IFRC rules and tariffs
· Flight costs will be covered at economy class rate as per IFRC policies
· Any air tickets for travel will be authorised by and paid for by IFRC directly, and will be for the attendance of meetings and workshops, and in agreement with the project’s focal point
· Organisations must declare direct or indirect conflict of interests
· Organisations must also adhere to principles of impartiality, neutrality, integrity, and confidentiality
H. IFRC Recourse in Case of Unsatisfactory Performance
In case of unsatisfactory performance, payment will be withheld until quality deliverables are submitted. If the contractor is unable to complete the assignment, the contract will be terminated by notification letter sent 30 days prior to the termination date. In the meantime, IFRC will initiate another selection in order to identify an appropriate candidate.
How to apply:
Service Providers who are interested in participating in the bidding process for this IFRC tender should express their interest by contacting below focal point before 16th July 2021 in order to obtain the requisite tender documents.
Primary contact: Katalin Romics (e-mail: Katalin.romics@ifrc.org)
Applicants are then expected to submit the completed tender package to gsc.bids@ifrc.org until 23rd July 6pm, Geneva time.