CLWR – Ethiopia External Evaluation

CANADIAN LUTHERAN WORLD RELIEF (CLWR) – CONSULTANCY SERVICE

FINAL EVALUATION Protection, WASH and Shelter and Increased Resilience for IDPs, Returnees, and host community members in Tigray 2022-2024
Classification:	CONTRACT
Duration	TBD (expected 30-40 days)
Location	Ethiopia, Tigray Region
Reports to	Program Manager
Department:	Programs

 

SUMMARY

CLWR is seeking an evaluator to conduct an external evaluation of the “Protection, WASH and Shelter and Increased Resilience for IDPs, Returnees, and host community members in Tigray 2022-2024” Global Affairs Canada funded nexus project.

Lutheran World Federation – Ethiopia/Canadian Lutheran World Relief encourages both companies and individual evaluators to apply for this consultancy service. The main requirement is the experience in the region, and usage of innovative and mixed methods approaches. The evaluator (or enumerators engaged in the data collection) is expected to have strong written and spoken English. LWF-Ethiopia can provide translation to local language if required and can also provide support in finding the local enumerators / data collection team in case the lead consultant is not based in the region. Main requirements include:

• At least 15 years’ demonstrated experience in participatory evaluation and research methodologies, preferably in Ethiopia;

• Demonstrated experience in designing age-appropriate, gender sensitive research methodologies, tools and analysis;
• Strong understanding of nexus theory and practice
• Familiarity with Global Affairs Canada and the Feminist International Assistance Policy
• Demonstrated experience in conducting / analyzing outcomes mapping evaluations / outcomes harvesting assessments (Desirable);
• Excellent English communication and writing skills;

 

PLEASE SUBMIT THE FOLLOWING INFORMATION TO [email protected]G BY WEDNESDAY, MAY 8TH.

1. Brief cover letter, explaining how you are uniquely positioned to complete a review of the GRIT project, including an indication of availability to travel to the region.
2. A financial proposal costed by deliverables (see page 6, Expected Outcomes), including daily rate and number of days required to complete each
3. CV detailing relevant education, training, and
4. At least 3 professional references, including contact details (email and telephone number), that can speak to your experience working on related assignments .
5. A relevant writing sample that showcases your previous work and demonstrates your capacity to complete a final evaluation of the Nexus project.

 

Type of evaluation: Final Evaluation

Name of the project: Protection, WASH and Shelter and Increased Resilience for IDPs, Returnees, and host community members in Tigray 2022-2024

Project Start and End dates: April 2022 – June 2024

Project duration 2 years, 3 months

Project locations Ethiopia, Tigray Region, Mekelle City Administration, Ganta Afeshum woreda (includes Adigrat), Kilte Awulaelo, Hawzen woredas (Eastern Zone), Enderta woreda (South Eastern Zone), Raya Azebo and Endamokoni Woredas (includes Maichew) (Southern Zone)

Thematic areas WASH, Protection, Emergency NFIS, Livelihoods, Climate Resilience.

Target Population Conflict affected IDPs and Host Communities in Tigray, Ethiopia

Donor Global Affairs Canada, Canadian Lutheran World Relief.

Projects overall budget 4,050,000 CAD

Overall objective of the project Reduced suffering, increased human dignity, lives saved and improved resilience among IDPs, returnees, and host communities in Tigray, Ethiopia

 

Project Background

The project was designed to respond to the impacts of the Tigray war erupted early in November 2020 between TPLF (Tigray People Liberation Front), the Tigray region ruling party, and the federal government. At the onset of the project critical and unmet needs were identified for IDPs and host communities in the targeted Woredas, with OCHA estimating 5.2 million people in Tigray requiring urgent humanitarian assistance. The project was designed to provide life-saving WASH, Protection and ES/NFI support to reduce the immediate suffering of conflict-affected IDPs and host communities in Mekelle City, Adigrat & Enderta (Eastern), Maichew (Southern). Additionally, the project aims to maintain the dignity of community members, particularly women, by raising awareness of SRHR and hygiene issues and strengthening communities’ capacity to earn income and avoid their reliance on life- threatening coping mechanisms. The project funding was part of a limited pilot series of nexus funding which included both immediate humanitarian assistance activities, coupled with longer term development support.

 

Project Theory of Change:

The is expected to contribute to Reduced suffering, increased human dignity, lives saved and improved resilience among IDPs, returnees, and host communities in Tigray, Ethiopia through five complimentary intermediate outcomes:

• WASH: Improved sanitation and hygiene practices among conflict-displaced individuals and host community members, particularly women and girls,
• Protection: Improved protection of IDPs, returnees, and hosts
• ES/NFI Increased dignity and safety of IDP and host community members, particularly women and girls, in Mekelle, Eastern and Southern Zone, Tigray
• Livelihoods: Improved economic self-resilience among IDP and host communities, particularly women, female youth, and female headed households,
• Climate Action: Enhanced climate change adaptation and disaster resilience of most vulnerable IDPs, returnees and host communities

 

The program was designed through a double nexus¹ framework, focusing on both humanitarian and development needs of the affected population. Through this framework the project initially began to provide life-saving WASH, Protection and ES/NFI support to reduce the immediate suffering of conflict- affected IDPs and host communities). The project has also aimed to maintain the dignity of community members, particularly women, by raising awareness on SRHR and hygiene issues and by strengthening communities’ capacity to earn income and avoid their reliance on life-threatening coping mechanisms.

The original Theory of Change envisioned a nexus implementation to be primarily based around a phased delivery of supports. Humanitarian supports, organized under Intermediate Outcome 1100, 1200, and 1300, to be delivered solely for the project’s first three months. This was based on the understanding that project participants required immediate assistance before they are able to invest time towards development-oriented trainings and livelihoods investments. Following the initial three months of humanitarian support, the project had intended to begin to deliver programming that would contribute to strengthened resilience amongst beneficiaries through livelihood and climate mitigation supports, contributing to increased food security and reduced dependency on humanitarian aid in the long term. This support would then continue with humanitarian backing for nine months to ensure that humanitarian outcomes are achieved and that participants are equipped to participate in development activities. Development support that involves higher levels of capacity building would then continue for an additional twelve months to accomplish this Ultimate Outcome.

This approach was modified due to the ongoing humanitarian situation in Tigray, which included a humanitarian blockade which effectively stopped implementation for 5 months during the first year of programming. A key learning from that experience was that the original nexus theory of change relied heavily on timing, viewing a cascaded delivery of humanitarian and development activities as the most effective approach in this context. This ToC was conceived in a more stable security context, allowing for the timely completion of activities. However, context required the project to deliver most humanitarian and development activities concurrently to complete the program within the expected time frame.

The program also has three distinct cohorts, which exist due to particular needs and service delivery by other humanitarian organizations in various project areas. These cohorts includes participants who are only beneficiating from humanitarian supports, those benefitting only from development supports, and those who

¹ We utilize the UN OCHA definition of nexus: https://www.unocha.org/es/themes/humanitarian-development- nexus

 

Scope of the Evaluation

The final evaluation is being commissioned by Canadian Lutheran World Relief and Lutheran World Federation Ethiopia in order to understand and document the main results and changes brought on by the project in the lives of conflict affected IDPs and host communities, with a particular focus on the changes in the lives of women and girls. This will serve the dual purpose of informing CLWR and LWF – Ethiopia on key impact areas and support in understanding improvement needs for future similar programming. There is also a strong interest in having the evaluation delve specifically into the utility of the nexus approach in this conflict affected setting. This is an impact evaluation and should assess formative aspects (assesses the design and theory of change of the programme), process (assesses the implementation process of the programme) and impact / results brought by it. The period that the final evaluation will cover is focused on the implementation process from the beginning of the program till June 2024. The main audience is CLWR and LWF – Ethiopia, however the results of the evaluation may also be shared with a wider audience for the purposes of policy discussions the utility of nexus programming in general. The evaluation It will reflect on 2 years of implementation, assessing against the OECD DAC criteria of relevance, effectiveness, impact, efficiency, and sustainability of the program.

 

Key Questions

The evaluator is expected to develop methodological approaches and tools that will provide answers to these questions. Throughout the evaluator should also evaluate the following criteria against the three different program cohorts (humanitarian, development, and nexus) with the aim to illicit lessons and recommendations for the future deployment of double nexus programming in conflict situations. While the project was a double nexus and did not have any explicit peace components, it is also of interest for the evaluator to explore whether there were any observable or reported improved interaction and cooperation among different IDPs, returnees, and host community members, or improved relationships or trust among these various groups? Which specific activities or aspects of the project particularly contributed to these outcomes.

The key areas that the evaluation is intended to answer are based on OECD DAC criteria, as follows:

• Relevance – The extent to which the aid activity is suited to the priorities and policies of the target group, recipient and donor. In evaluating the relevance of a programme or a project, it is useful to consider the following questions:
  • To what extent are the objectives of the programme still valid?
  • To what extend is the theory of change still relevant?
  • Are the activities and outputs of the programme consistent with the overall goal and the attainment of its objectives?
  • Are the activities and outputs of the programme consistent with the intended impacts and effects?

• Effectiveness – A measure of the extent to which an aid activity attains its objectives. In evaluating the effectiveness of a programme or a project, it is useful to consider the following questions:
  • To what extent were the objectives achieved / are likely to be achieved?
  • What were the major factors influencing the achievement or non-achievement of different the objectives? Reflection on various factors: internal and external, operational and programmatic, components of the project etc.
  • To what extent did the sequencing of humanitarian and development activities have on the effectiveness of the programs ultimate outcome of increased resilience?
  • Was the project’s adaptability to the unexpected contextual changes in Tigray and the concurrent (as opposed to the planned cascading) implementation of most
  • humanitarian and development activities have any specific or noticeable impact (positive, or negative) on the various project components
• Coherence – The extent to which other interventions support or undermine the programme, and vice versa. Includes internal coherence and external coherence: Internal coherence addresses the synergies and interlinkages between this and other projects of LWF – Ethiopia, while external coherence looks at other actors for value added perspective.
• Efficiency – measuring the outputs – qualitative and quantitative — in relation to the inputs. It is an economic term which signifies that the aid uses the least costly resources possible in order to achieve the desired results. This generally requires comparing alternative approaches to achieving the same outputs, to see whether the most efficient process has been adopted. When evaluating the efficiency of a programme or a project, it is useful to consider the following questions:
  • Were activities cost-efficient?
  • Were objectives achieved on time?
  • Was the programme or project implemented in the most efficient way compared to alternatives?
• Impact – The positive and negative changes produced by a development intervention, directly or indirectly, intended or unintended. The examination should be concerned with both intended and unintended results and must also include the positive and negative impact of external factors. When evaluating the impact of a programme or  aproject, it is useful to consider the following questions:
  • What has happened as a result of the programme or project?
  • What real difference/changes has the activity made to the lives of the target group?
  • How many people have been affected?
  • How do the those targeted by the project see the impact themselves and how do they describe the changes?
  • To what extent can the benefits of a programme continue after the project is finished?
  • What were the major factors which influenced the achievement or non-achievement of sustainability of the programme or project?
  • Due to the nexus aspect of this project, the following key question will also be a part of the evaluation:
    • Are there discernible differences in the potential for long term resilience between the different program cohorts (Humanitarian, Development, Nexus)
    • What factors amongst these cohorts contribute to more or less potential for longer term resilience following the program closure.Sustainability – Sustainability is concerned with measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn. Projects need to be environmentally as well as financially sustainable. When evaluating the sustainability of a programme or a project, it is useful to consider the following questions:

 

Methodology

The Evaluation methodology should be participatory, inclusive (consultations with all stakeholders, such as gender, age, disability and other vulnerability considerations), sensitive of social norms and practices, and ethical in data collection practices (safety, informed consent, etc). Findings and analysis should be disaggregated by age and gender. Evaluators are encouraged to use participatory and qualitative methods for data collection and data analysis, along with quantitative methods for measuring the changes, in line with the overall theory of change of the project – as identified above. The evaluator will be provided with secondary sources (i.e. outcome data collected results, annual reports, review meeting results, results from any tool implemented by CLWR/LWF -Ethiopia teams for data collection, etc), At the same time, the evaluator shall consider other external secondary data sources when / if necessary. The primary sources of data will include LWF-Ethiopia staff and partners, government partners, other INGOs working in similar locations, and project participants. All data collection methods should be age and gender appropriate.

 

Evaluation Management

The evaluation will be jointly managed by the Project Coordination office in Mekelle, Tigray. However, it will also be implemented in close cooperation with the LWF HQ Office in Addis, as well as the CLWR HQ located in Canada. It is expected that the full evaluation will take a maximum of 40 days, with the final report being completed by June 30^th, 2024. Specific milestone dates, listed below, should be suggested by the evaluator.

	DATES
Evaluation TOR publication	April 22nd
Documentation review, desk research & initial consultations	TBD
Inception report, including detailed methodology and tools	TBD
Data collection	TBD
Data management & analysis	TBD
First draft report of evaluation	TBD

Validation workshops with communities, and LWF Ethiopia	TBD
Review/feedback from LWF Ethiopia and CLWR of first draft report	TBD
Final evaluation report	June 30th, 2024

 

Expected Outcomes

Evaluator is expected to deliver following outcomes / deliverables:

1. Inception report:
   • detailed methodology
   • tools: Objectives and key questions, Methodology, Data collection methods and detailed tools actor by actor, sampling considerations if any, timeline and logistics, etc.

Data collection plan, including the logistical arrangement.

2. Validation workshops: This will include representatives from all actors of the project (Women, girls, men, women who directly benefited from the project, government partners, other coordination partners) as well as project teams. Different workshops can be planned with different audience to share the information and validate the findings in most appropriate manner.

3. Evaluation Final Report, not more than 25-30 pages, written in simple Any additional information should be provided in Annexes.

• • Executive Summary (1-2 page)
  • Methodology, including limitations (2-3 pages)
  • Main Findings (15-20 pages)
    • Findings disaggregated by age and
    • Specific analysis of results as they pertain to humanitarian, development, and nexus Including discussion on factors contributing to on any varying levels of resilience amongst different cohorts.
  • Conclusions and Recommendations (including general recommendations for the nexus approach in humanitarian settings) (5 pages)

 

Evaluation Team

LWF-Ethiopia/CLWR encourages both companies and individual evaluators to apply for this consultancy service. The main requirement is the experience in the region, and usage of innovative and mixed methods approaches. The evaluator (or enumerators engaged in the data collection) is expected to have strong written and spoken English. LWF-Ethiopia can provide translation to local language if required, and can also provide support in finding the local enumerators / data collection team, in case the lead consultant is not based in the region. Main requirements include:

• At least 15 years’ demonstrated experience in participatory evaluation and research methodologies, preferably in Ethiopia;
• Demonstrated experience in designing age-appropriate, gender sensitive research methodologies, tools and analysis;
• Strong understanding of nexus theory and practice
• Familiarity with Global Affairs Canada and the Feminist International Assistance Policy
• Demonstrated experience in conducting / analyzing outcomes mapping evaluations / outcomes harvesting assessments (Desirable);
• Excellent English communication and writing skills;

 

To Apply

Please submit:

1. Brief cover letter, explaining how you are uniquely positioned to complete a review of the GRIT project, including an indication of availability to travel to the region
2. A financial proposal costed by deliverable (see page 6, Expected Outcomes), including daily rate and number of days required to complete each deliverable.
3. CV detailing relevant education, training, and experience
4. At least 3 professional references, including contact details (email and telephone number), that can speak to your experience working on related assignments
5. A relevant writing sample that showcases your previous work and demonstrates your capacity to complete a final evaluation of the Nexus project.

 

All applications must be received by Wednesday, May 8^th.