Terms of Reference for Endline Assessment for the WASH, Protection and Livelihoods support Project in Lower Juba Region, Somalia, 2021–2023

Terms of Reference for Endline Assessment for the WASH, Protection and Livelihoods support Project in Lower Juba Region, Somalia, 2021–2023, CARE

Background

Somalia remains one of the most complex and long-standing humanitarian crises in the world. Cycles of extreme flooding, spikes in conflict, the worst locust infestation in decades, and the outbreak of COVID-19 have all resulted in unprecedented humanitarian needs. The multi-layered nature of the crisis exceeds the capacity of local resources to respond. The 2021 Somalia Humanitarian Response Plan (HRP) estimated that 5.9 million people will require humanitarian assistance, up from 5.2 million in 2020 and 4.2 million in 2019. Displacement continues to increase, adding to the estimated 2.6 million people living in a state of protracted displacement.

The impact of the multiple humanitarian crises has exacerbated pre-existing vulnerabilities of women and girls and negatively affected livelihoods and coping mechanisms for vulnerable families and internally displaced people (IDPs).

CARE is a humanitarian non-governmental organization committed to working with poor women, men, boys, girls, communities, and institutions to have a significant impact on the underlying causes of poverty. CARE seeks to contribute to economic and social transformation, unleashing the power of the most vulnerable women and girls.



The water, sanitation and hygiene (WASH), protection and livelihoods support project in Lower Juba Region, Somalia, 2021–2023 is a 2-year Global Affairs Canada-funded project running from April 2021, through Mar 2023. The project is implemented by CARE, which plays an overall leadership role and implements field activities. The ultimate outcome of the project is to save lives, alleviate sufferings and maintain human dignity through WASH, livelihoods, and protection interventions for crisis-affected women, men, boys, and girls in Lower Juba, Somalia. To accomplish this, CARE implemented activities that addressed the immediate needs of returnees, IDPs and improve the livelihoods of host communities, addressed the WASH conditions, and provides protection related assistance to a target of 57,258 (22,903 men, 4,008 boys, 25,766 women, 4,581 girls) for a period of two years in the three districts (Kismayo, Afmadow and Badhaadhe districts of Lower Juba region.



Intermediate Outcome

Increased and equitable use of gender-responsive assistance by crisis-affected women, men, boys, and girls to meet their basic WASH, livelihoods and protection need in Lower Juba, Somalia.

Immediate outcomes

• Increased and more equitable access to safe and sustainable WASH supplies and services among vulnerable women, girls, boys, and men in Lower Jubba.
• Increased capacity of vulnerable households in communities in Lower Juba to meet their basic needs.
• Increased and more equitable access to protection prevention and response services for vulnerable women, girls, boys, and men in Lower Juba.

Objectives of the Endline assessment

The overall objective of the endline assessment is to assess and provide reliable endline information on project performance against set parameters on the three project components i.e. livelihoods, (including cash interventions), WASH and protection. The evaluation report will identify best practices and key lessons in technical aspects as well as the program management approach to facilitate continued learning and improvement of humanitarian emergency response. Findings of this final evaluation will be shared with GAC, and with the humanitarian community working in Somalia, through the Somalia NGO Consortium, the UN Cluster system, the wider CARE international family, as well as through local networks.

Evaluation Questions

• Did the project achieve every indicator of the Performance Measurement Framework at outcomes levels (immediate, intermediate, and ultimate)? making use of the output level indicators already measured by the project if needed
• To what extent does the project comply with OECD-DAC criteria on efficiency, effectiveness, appropriateness/relevance, sustainability, and short-term intended and unintended impacts?
• To what extent did the project deliver the required support to enable targeted households to maintain food security?
• What was the impact of project intervention on women, IDPs, returnees and people with disabilities who received direct support through the project?
• How did the intervention support the household’s food access and improved the livelihood recovery of the targeted communities?
• How did the project contribute to increasing the community’s access to clean, safe and affordable water for household and livestock use?
• To what extent did the project provide protection support to vulnerable people in the community and how these services changed the lives of the GBV victims or survivors?
• What are the key lessons learnt and best practices from the program implementation, operating context and response type as well?
• How can these lessons learned (around design, implementation, coordination, flexibility among other aspects) from this intervention be used to inform future programming?

Evaluation Criteria

The evaluation will be done in line with OECD-DAC recommended criteria based on the five criteria relevance, effectiveness, efficiency, sustainability and impact. The end-line assessment will contribute to both strengthening accountability of CARE International in Somalia to its donor and key stakeholders including beneficiaries, and to learn from this experience to inform future programing.

Relevance

• Was the project consistent with the sector policies and strategies of the Government of Somalia more so to Jubba Land state?
• To what extent did the project respond to the needs and priorities of the affected people/ communities it targeted, with specific reference to the experiences and opinions of women, girls and other vulnerable and/or marginalized groups.
• To what extent was the flexible and multiyear funding fit into the needs of the target population and how different was this funding from the annual funding?

Effectiveness

• To what extent has the project increased the capacity of water accessibility and enhanced operation and maintenance of the water points? Are the results satisfactory?
• To what extent has the project enhanced communal and personal hygiene practices through the attainment knowledge, attitude and practice changes?
• What were the key drivers and barriers that impacted on the delivery of the project outputs?
• To what extent has the project used learning to improve delivery?

Efficiency

• Were resources utilized and managed in an efficient manner? Would it have been possible to achieve the same results at a lower cost?
• Were the project activities implemented within the planned time and financial targets?
• What factors and constraints affected project implementation including technical, managerial, organizational, institutional, and socio-economic policy issues in addition to other external factors unforeseen during the project design?

Sustainability

• What is the level of community ownership, as reflected in the participation of the community in the project supported WASH community-based structures?
• Is the project supported by community-based structures adequately and has a long-term vision and strategy for operating, maintaining, and regulating the water supply and sanitation infrastructure/system?
• Do the community have the capacity to maintain the benefits from the project with support from CARE or any other donor? If yes, how? If no, what will be the principal challenges in sustaining the program benefits?

Impact

• Has the project contributed to the improvement in the quality of life of the people in affected districts?
• Have there been any other changes (positive and negative) at the household or community level as a result of the project?
• Has the project affected people in ways that were not originally intended? If yes, what has changed?
• what was the role of the program partners to deliver the assistance?
• What is the gendered impact of the project in the areas of intervention?

Equity

• Did the project contribute to equitable participation and benefits to various groups (men, women, boys and girls and differently abled people)?
• Over the past 2 years, were there any undertakings by the project to promote equity? If yes, how? If no, why not?

Approach and Methodology

This endline study is a performance evaluation that adopts a non-experimental design for simple pre-post comparison of results using mixed methods involving both quantitative and qualitative data. Data collection to involve a quantitative beneficiary household survey, document reviews, and beneficiary and stakeholder interviews. A comparative analysis approach will be used to report on project achievements for selected indicator values.

It’s expected that the technical support team will propose a suiting and detailed evaluation methodology encompassing the appropriate evaluation method, sampling, data analysis, presentation and reporting and ethical consideration of the evaluation.

Scope of the Evaluation

The end-line assessment will cover Kismayo, Afmadow and Badhaadhe districts of Lower Juba, Somalia, Specifically the locations where the project activities has been implemented. The participants of the evaluation survey will include; IDPs, returnees, host rural and urban communities in the target area as well as government authorities and partner agencies.

Responsibilities of CARE

• CARE will provide all logistics support including flights, accommodation, transport, enumerators recruitment, payment of enumerators, etc.
• Commission the consultancy fee for the study
• Introduce the consultant to relevant stakeholders
• Review and approve the study instruments/tools
• Support the consultant in field data collection while supervising the entire process.
• Provide input to the draft report and approve the final report

Responsibilities of the Consultant

• Develop appropriate methodology and tools for the data collection and analysis process. The tools will be shared and approved by CARE before starting field work.
• Develop a practical work plan for the work
• Undertake a desk-based review and lead field level data collection
• Train enumerators who will be involved in the data collection.
• Carry out all Key Informant Interviews and Focus Group Discussions if selected as data collection tools.
• Debrief CARE Somalia/Somaliland after completing the assignment and validate the findings prior to submitting final report.
• Prepare and submit final evaluation report to CARE Somalia/Somaliland as per time frame set in the ToR.

Key Deliverables

The evaluation deliverables are:

1. Inception Report: For review, the evaluation team should submit to CARE an inception report presenting findings from the desk review and/or examination of data to date. The inception Report should include
2. Detailed evaluation methodology
3. Proposed sample size with specific detail on sampling approach and methodology
4. Evaluation Matrix
5. Procedures for ethical consideration
6. Revised work plan
7. Data collection methodology, including data collection tools for all indicators
8. Proposed data analysis methods and the tools and technologies to be used.
9. Draft Evaluation Report: The consultant providing the technical support should share a draft evaluation report that addresses all the questions identified in the ToR and any other issues the team considers having a bearing on the objectives of the evaluation. Once the initial draft evaluation report is submitted, CARE will have 5 working days in which to review, comment on the initial draft and submit the consolidated comments to the evaluation team. The evaluation team will then be asked to submit a revised final draft report within 5 working days, and again CARE will review and send comments on this final draft report within 3 working days of its submission.

• Final Evaluation Report: The consultant will be asked to take no more than 7 days working days (or as agreed upon in the work plan) to respond to and incorporate final evaluation report comments from CARE. The technical lead will then submit the final report to the Emergency Director in CARE International in Somalia/Somaliland. A fact sheet (one page) for each thematic area and a succinct presentation should accompany the final report.

CARE suggests the final evaluation report have the following format/structure,

• Executive Summary (one page written succinctly to summarize the methodology, key findings, and lessons learned)
• Recommendations — list of key recommendations and learning from the evaluation, with brief explanations
• Introduction:
• Project Introduction, purpose and objectives
• Methodology, Scope, and Sampling
• Limitations
• Key Findings analyzed on evaluation criteria and in reference to the evaluation question while also providing gender and geographically disaggregated data.
• Conclusions
• Recommendations for future programming.
• List of interviewees including name, organization and role (if applicable)

1. Submission of Dataset(s) to the Development Data Library: The consultant must submit the report to CARE in a machine-readable, non-proprietary format, any dataset created or obtained in performance of this award. The dataset should be organized and documented for use by even those not fully familiar with the intervention or valuation. The data sets to be submitted include
2. All data collection tools used for the assignment
3. Raw and cleaned data sets (Qualitative and Quantitative)

• Generated codebooks and syntaxes preferably in SPSS or STATA formats

1. Qualitative codebook, audio types, transcriptions and translations.
2. All Photos documented for evaluation purposes
3. GPS Coordinates for all sampled locations

• Presentations

Duration of the Assignment

The duration of the assignment is 40 working days after the signing of the contract. Days are inclusive of traveling, fieldwork, and reporting. The evaluation is expected to take place from 1st February to 12th March 2023.

Skills and Qualifications

The evaluation will be external. The consultant/s will have to be a team or persons with vast experience in performing evaluations for similar projects funded by GAC as well as someone with vast experience in Somalia/Somaliland to better understand the context. Other qualifications for the consultant/s are listed below.

• 10 years’ international humanitarian and development experience particularly in the Somalia context.
• Proven humanitarian evaluation experience
• Solid experience and understanding of cash policy and delivery, including various delivery mechanisms
• Solid experience and understanding of Cash and Voucher Assistance-CVA and its inter-connectedness to other sectors especially food security, nutrition and livelihoods.
• Strong knowledge of the drivers of malnutrition and poor health in the Somalia context and experience in evaluating health and nutrition programs.
• Strong knowledge and experience in the Somalia context (teams including Somali speakers are desirable and if unfeasible, it should be outlined how the team will obtain local feedback in their methodology.
• Knowledge and understanding of the World Humanitarian Summit’s Grand Bargain and OECD DAC Criteria
• Strong experience and knowledge in application of humanitarian standards like SPHERE and Core Humanitarian Standards (CHS)
• Good understanding of integrated approach to programming and ability to make recommendations based on the evaluation findings
• Ability to see the big picture and make practical recommendations for delivery and improvements on the ground
• Ability to absorb, consolidate analyse and communicate large amounts of data and information simply and concisely
• Experience and ability to operate in security volatile environments
• Experience of working with local partners and private sector in particular the financial/payments sector
• Excellent inter-personal skills
• Willingness to travel to Somalia under the CARE international security framework.
• Good participatory facilitation and workshop design skills
• Excellent English report writing skills

How to apply

All applications MUST be accompanied by a technical and financial proposal including a brief outline of the proposed methodology, 3 references with contacts and a tentative work plan as well as the candidate’s availability during the month of February and March 2023. The proposals shouldn’t exceed 10 pages. Deadline for submission of technical and financial proposals is 15th January 2023.

Interested consultants or firms are expected to conduct their own security and accessibility of the target project locations before application. Updated CVs of individuals or profiles of applying companies expressing interest to conduct the assessment should be sent to consultants@som.care.org no later than 15th January 2023. Please indicate “GAC4 End-line assessment’’.

Female applicants with requisite experience are highly encouraged.

CARE is an equal opportunity employer promoting gender, equity, and diversity. Female candidates are strongly encouraged to apply. Our selection process reflects our commitment to the protection of children from abuse

Emailconsultants@som.care.org

Terms of Reference for Endline Assessment for the WASH, Protection and Livelihoods support Project…