1. Background and purpose
Forecast-based Financing (FbF) or Early Warning Anticipatory Action (EWAA) is a funding modality and way of working whereby resources are released based on forecast information before a climate-related disaster happens, to implement planned activities which reduce risks, enhance preparedness and response, and make disaster risk management overall more effective.
In 2017, the Mongolian Red Cross Society (MRCS) established an FbF mechanism and similarly FAO established its EWAA system to help herder households cope with the anticipated impacts of dzud, based on forecast information and the dzud risk map produced by the government. The mechanism was triggered in January 2020 to anticipate and mitigate the dzud impacts. Both agencies delivered unconditional cash grants and animal care kits to 1,450 of the most vulnerable herders across a total of 12 provinces and 83 soums.
The purpose of this contract is to engage a consultant experienced in quantitative survey data analysis following a quasi-experimental study design. The data has already been collected; data collection is not part of this assignment. The consultant will analyse the existing survey data to assess whether and to what extent forecast-based assistance with cash grants and animal care kits has helped the beneficiaries to avoid or reduce negative impacts of dzud compared to similarly vulnerable and dzud-affected households who did not receive forecast-based assistance.
2. Scope of work and research questions
The consultant is asked to perform a state-of-the-art analysis of the available quantitative data set that has been collected following a quasi-experimental study design. Three different intervention groups and a comparison group are represented in the sample as shown below:
- Group: MRCS / number of beneficiaries: 1,000 households / sample size: 200 households
- FAO / 450 households /150 households
- DREF / 1,750 households / 200 households
- Comparison group / N/A / 200 households
The consultant will analyze the main research questions pertaining to the effectiveness and efficiency of the interventions vis-a-vis the comparison group. Therefore, advanced statistical methods suitable for publication in a peer-reviewed journal are required to test results for statistical significance, and to account for potential sources of bias in the dataset. The two (MRCS and FAO) interventions have to be assessed separately; a comparison between the two is desirable.
Research questions:
To what extent are the MRCS and FAO intervention groups and the comparison group comparable? What are the main differences between the groups? How can they be statistically accounted for?
Have the interventions been effective in helping vulnerable herder households to protect their herds from dzud impacts, as measured by animal morbidity, mortality, maintaining body conditions and offspring survival rates?
Have the interventions been effective in helping the beneficiaries to avoid negative coping strategies, including:
Emergency sales of livestock (destocking) at deflated prices;
Selling other valuable assets (destitution sales);
Taking on - potentially unsustainably high or high-interest - new debt?
To what extent has the Coronavirus pandemic affected the livelihoods, well-being and experience of herders during and following the 2020 dzud period?
What was the return-on-investment, expressed as benefit-to-cost ratio, of the MRCS and FAO interventions, respectively? (Cost information will be shared by each agency when requested).
3. Specific tasks and deliverables
A suitable statistical software package should be used by the contractor to analyse the survey data (Stata, SPSS, R, etc). The syntax files (.do / .SPS / .R) documenting all operations performed on the data set should be stored and submitted to MRCS together with the data analysis report.
Data cleaning: cleaning procedures should be documented in syntax files, including identification of outliers or missing data points.
Balancing and weighting the sample data: Since it can be expected that there will be differences between the two (MRCS and FAO) intervention groups and the comparison group, steps must be taken to account for these differences. In past analyses of FbF projects, propensity score matching (PSM) and the use of bias-corrected matching estimators have been found to be suitable solutions, although the survey contractor may suggest a different approach. All estimates from the survey data must be weighed by applying base weights that are adjusted for non-response error and unequal selection probabilities (a detailed description of the process by which provinces, soums and households were selected will be provided to the survey contractor).
Impact data analysis: For each questionnaire section, the survey contractor will produce summary tables and bar charts for all main variables, showing the difference between the two intervention groups and the comparison households side-by-side (in one table and bar chart, respectively). Tables must clearly indicate the statistical significance of the differences between (intervention and comparison) means, or lack thereof, by including p-values or following a consistent highlighting process, for example, by including asterisks (*).
Cost-benefit / return-on-investment (ROI) analysis: The survey questionnaire includes detailed questions on the household’s socio-economic conditions and herder livelihood, including the number, condition and productivity of animals and the change over time since December 2017 (before the Dzud) until now. The cost of the FbF intervention is known and can be obtained from MRCS. Based on this information, the contractor will calculate a benefit-cost ratio and ROI which can be drawn from the livestock or productivity losses and livelihood impacts that were avoided or mitigated. With this it should be possible to derive a summary estimate along the following lines: “For every one FbF US Dollar spent, the project generated a US$ X.XX return in avoided losses and more resilient livelihoods.”
Final products: The results of the analyses should be summarized in an analytical summary report, plus a PowerPoint presentation, and should be complemented with and all tables and bar charts provided in an annex.
All work must be original and follow the highest research and ethics standards.
List of deliverables (dates tentative and subject to change):
- Complete set of analytical tables and charts for all questionnaire variables & summary PowerPoint presentation, by 31 August 2020
- Summary write-up of key findings on the impact of the interventions on herders’ socio-economic well-being and livelihoods, including Cost-Benefit Analysis / Return on Investment analysis write-up (25 pages max., excluding annexes, by 14 September 2020
- Summary presentation of findings to MRCS/FAO, by 21 September 2020
- Submission of complete, cleaned electronic dataset, statistical syntax files and all supplementary documentation, by 21 September 2020
4. Reporting and work arrangements
This is a remote, home-based assignment. The contractor will formally report to the International Federation of Red Cross Red Crescent Societies (IFRC), Asia Pacific Regional Office (APRO), Kuala Lumpur, Malaysia. The work will be substantively supervised jointly by the MRCS and IFRC focal points for this study. Upon signing the contract, and once the contract is activated, regular progress reports against the consultant’s work plan must be submitted to the supervisor in a form and frequency to be determined by the supervisor.
Payment is subject to satisfactory performance and completion of all deliverables.
All submissions will be made electronically (email, Skype, phone, etc.), unless requested otherwise by the MRCS/IFRC team.
5. Duration of the consultancy
The contract is entered between the International Federation of Red Cross Red Crescent Societies (IFRC), Asia Pacific Regional Office (APRO), Kuala Lumpur, Malaysia, and the contractor. The estimated level of effort is 20 working days. The approximate start date is in September 2020. All tasks are expected to be implemented and completed, with final products delivered by end September 2020.
6. Required qualifications and experience of the consultant
- Graduate or advanced (PhD) degree in relevant disciplines such as statistics, demography, economics, sociology or related disciplines
- Proven experience in quantitative research for impact assessment
- Proven experience in analyzing probabilistic sample survey data, including sample data weighting and propensity score matching (submit work samples together with application)
- Experience managing large datasets
- Knowledge of R, Stata, SPSS (or similar) and quantitative data analysis methods and documenting analytical operations in syntax files
How to apply:
The deadline for applications is 12 August 2020 at 23:59 hrs [Kuala Lumpur time].
Interested applicants should submit the following documents electronically to raymond.zingg@ifrc.org.
Short technical proposal summarizing:
Qualifications and relevant experience with similar assignments
Reflections on the research design
Draft work plan showing the suggested timing of carrying out the specific tasks
List of publications and completed quantitative survey analyses, in tabular format
2 samples of previous work that show the expertise and experience of the consultant; samples must be the original work of the contractor (e.g. analytical reports or peer-reviewed journal articles).
Financial proposal including detailed budgets for professional fees and reimbursable expenses (if any) for all tasks related to this assignment. Kindly also indicate whether contractual arrangements would be with an individual (individual consultant or representative of a firm) or with a firm. Kindly note that there is a preference to contract with individuals. No payments will be made outside the contractually agreed budget.
Incomplete applications or applications that are received late will not be considered. Only shortlisted candidates will be contacted.