Background
Since 1945, Lutheran World Relief (LWR) has worked with vulnerable communities to end poverty, injustice and human suffering. LWR supports agriculture, climate change, and emergency response programming in 28 countries each year. LWR works in Tanzania with an emphasis on capacity strengthening of farmer organizations and long-term food security. Lutheran World Relief is implementing a project entitled “Pareto ni Pesa” (“Pyrethrum for money”) from January to September 2020. The project is designed strengthen market linkages and business relationships between smallholder pyrethrum farmers in Iringa Region (Kilolo, Mufindi and Iringa Rural Districts) focusing on crop production and post-harvest processing at the producer level and selling/buying at the processor level of the value chain. The overall goal of the project is to improve livelihood (increased income) of smallholder farmers from pyrethrum activities. This objective will be achieved through the following two outcomes:
Outcome 1: Men and women pyrethrum farmers improve productivity and quality of pyrethrum through strong business partnership with PCT
- Output 1.1: Targeted farmers collaborate in solidarity groups.
- Output 1.2: Targeted farmers use best pyrethrum production and harvesting practices.
- Output 1.3: Targeted farmers access labor-saving and cost saving technology.
- Output 1.4: Targeted farmers apply proper method of drying pyrethrum flowers
- Output 1.5: Targeted farmers access high quality seeds
- Output 1.6: Targeted youth gain expertise in pyrethrum value chain
Outcome 2: Targeted men and women pyrethrum farmers increase their income earned through partnership with PCT.
- Output 2.1: Targeted farmers collectively market high quality flowers in batches traceable to small groups.
- Output 2.2: Targeted farmers receive payments (mobile money) that reflect the pyrethrin content of their flowers’ quality within the bulked sale.
- Outcome 3: Targeted farmer organization provides sustainable services that benefit its members.
- Output 3.1: Stronger farmer organizations
LWR seeks to engage a firm or individual(s) knowledgeable and experienced in agricultural and market systems programming and evaluation to conduct a final evaluation for Pareto ni Pesa project focused on its relevance, effectiveness, efficiency, Impact, Sustainability, Replicability/Scalability and Coherence in June through July 2022.
II. ObjectiveThe purpose of this end-of-project evaluation is to assess the performance of the project and capture project achievements, challenges, and best practises. LWR will use the results of this evaluation to inform the design of a possible second phase or iteration of the project and strengthen technical approaches and accompanying materials.
NOTE: Midterm and Baseline data for each indicator will be availed to inform the tools generation The specific objectives of the evaluation are:
- Determine whether the project objectives were achieved as measured by the project’s performance monitoring plan
- Review the outcomes of project implementation activities against established evaluation criteria of relevancy, efficiency, effectiveness, impact, scalability/replicability, coherence, and sustainability.
- Contribute to organizational learning by identifying lessons learned, challenges, and good practices that can be used for designing the follow-on phase or design of similar projects.
Provide recommendations for future interventions contributing to the broader goal of increased incomes amongst target beneficiaries
III. SCOPE OF WORK
The final evaluation will address the following criteria and answer the associated questions. Disaggregation by appropriate comparison groups of data and analysis will be required, including but not limited to gender.
Relevance:
Questions:
- To what extent are the objectives of the project valid? Were the activities, the most relevant activities given the needs of the targeted populations?
- To what extent were the strategies implemented by the project?
- Was the project design most appropriate and relevant to the organizational policies and strategies?
- To what extent did the project integrated programming approach respond to the diverse needs and capacities of the target population? How did this affect project implementation?
- To what extent does the project respond to the specific, and sometimes different, needs of women and men?
Is there scope to increase the relevance of future interventions?
Effectiveness
Questions:
- To what extent has, the project achieved the planned outcome indicators?
- What strategies have proven particularly effective in achieving the outcomes and what were the challenges to effective implementation of such strategies?
- To what extent did project interventions contribute to improved capacity of the target groups?
- Were the interventions timely and appropriate to the need at that point in time?
- What are some of the major factors influencing the achievement or non-achievement of the objectives?
- Has equitable access, participation and benefit been ensured across program components? Are there any relevant gender gaps and if so, why?
- To what extent were the intended outputs achieved and to what extent were the intended results produced?
- Did the performance monitoring system and indicators capture expected results as intended?
- To what extent have the performance indicators been used for systematic monitoring and steering of the project? To what extent did the project staff assure the quality of performance monitoring data for reporting?
Efficiency
Questions:
- Evaluate the efficiency of the organizational set‐up for the project and systems used in the delivery of the project and to what extent these contributed to or inhibited the delivery of the project outcomes.
- Were the project activities implemented within the planned time and financial targets? If not, what factors have led to the change in the timeframe and planned targets?
- How effective were the project management activities in documenting and communicating results?
- To what extent were the modalities used the most appropriate and efficient for the context? What could have been alternatives?
- What effect, if any, did working through partners have on the efficiency of the project implementation process?
Impact
Questions:
- What are the main changes produced by the program, positive or negative and what are the key factors behind these changes?
- What are the key lasting changes brought about by the project?
- Did the project achieve any unintended impact in the context? In addition, what could be the reasons and why?
- What is the project’s overall impact in terms of gender equality?
Sustainability
- Assess the level of sustainability (financial, institutional, and social) of the individual project components, and identify critical areas that may affect sustainability.
- To what extent can the benefit of the project/program continue after the grant period? Is there evidence that project activities will continue?
- Factors that influenced achievement or non-achievement of sustainability.
- What technical, programmatic, and management lessons can LWR and partners including PCT take away to improve the implementation of similar activities in the future?
Scalability/Replicability
Questions:
- What components of the project can be likely scaled up or replicated in other areas?
Documentation of best practices, what worked well? What did not work well? What were the challenges?
Coherence
Questions:
- What is the compatibility of the intervention with other interventions in a country, county, region, sector, and institution?
- To what extent do other interventions (particularly policies) support or undermine the intervention, and vice versa (inward looking to LWR and PCT programming, as well as other actors including Government of Tanzania interventions)
IV. METHODOLOGY
In collaboration with the LWR Senior Program Manager, technical program manager and other project staff, the consultant will be responsible for the development of an overall methodology, including a detailed plan for data collection, analysis, and report writing, that addresses the evaluation objectives. The methodology should be able to clearly address each of the specific objectives and should specify mechanisms to avoid bias. Moreover, the desired representative sample size, sampling methodology and data collection tools will be a critical component of the design. Creative use of data collection, participatory data analysis, and presentation of findings that maximize the usefulness of the evaluation will be very important. The evaluator will work closely with LWR Senior Program Manager to finalize the methodology, data collection tools and the work plan before field work. The evaluator will define the methodology and approach; however, the methodology will likely include:
- Document review: Desk review of all relevant project documentation including project proposal, quarterly reports, baseline survey, mid-term evaluation, M&E reports, Project Design Workbook
- Review project data sources and methodology for collecting both quantitative and qualitative data
- Key informant interviews (KII): Potential key informants including AMCOS, PCT, LGAs and LWR’s Senior Program Manager
- Consultation with beneficiaries (surveys, focus group discussions, observation):
V. DELIVERABLES
All deliverables must be in English.
- Inception report/evaluation plan (using the following outline)
- A Review of the evaluation questions, scope, and consistency
- Data Analysis plan
- Final Evaluation Objectives and Matrix
- Final Evaluation Methodology
- Limitations to the evaluation/Study
- Ethical Considerations
- Timelines, Roles, Level of Effort and Responsibilities of the consultancy team
- Annex 1: Terms of Reference
- Annex 2: Data Collection InstrumentsEvaluation report – MS Word (using the following outline: maximum 25 pages, excluding annexes)
- Executive Summary
- Purpose of the evaluation
- Sampling methodology (updated from the inception report to reflect any changes made during the study).
- Findings and analysis (as defined in inception report)
- Conclusions
Annexes, including (but not exclusively): - Evaluation terms of reference
- Schedule of evaluation activities
- List of people and groups consulted
- Data collection tools
- Tabulated data
- Reference documents
All data sets: Consultant will deliver the data sets (both raw and analyzed data) to LWR (SPSS, Stata or excel for quantitative and for qualitatively transcribed data).
Power point brief presentation: This will include the evaluation key findings, recommendations and lessons learned and will be not more than 12 slides.
VI. IMPLEMENTATION ARRANGEMENTSARRANGEMENTS
The duration of the consultancy will be from June through July 2022. LWR will orient the lead consultant on the scope of the work. The consultant will outline the methodology and develop tools for data collection and submit to LWR for approval. LWR will assign a person to accompany the consultants for field work. The assigned person will assist the consultants to make contacts with key informants and arrange for meetings. The consultant will provide his/her own working space, associated resources, and equipment.
The main contact in planning and finalizing this evaluation methodology, data collection, analysis/interpretation and finalizing main deliverables will be LWR Tanzania
The timeline for the completion of the evaluation is very firm. Delays in submission of the deliverables that are not the fault of LWR will result in a reduction in the final payment to the consultant.
VII. EVALUATION TIMEFRAME
June to July 2022
VIII. EVALUATOR QUALIFICATIONS AND APPLICATION PROCEDURES
Qualification of the Consultant
The required qualifications of consultant(s)
- Graduate(s) or equivalent in fields of agronomy, agriculture economy, natural resource management, sociology, organizational development, or any relevant academic training.
- At least ten years’ experience working with grassroots organizations related to rural agricultural development, with at least five years specifically on monitoring and evaluation.
- Demonstrated experience in social science research, including leading at least 3 high quality final evaluations similar to this assignment.
- Excellent English writing skills and fluent in Swahili.
- Legally entitled to work in Tanzania.
Interested consultants should send the information listed below to:
- Expression of interest (up to 3 pages) – outlining the summary of relevant previous experience, the approach in undertaking the evaluation and daily consultancy fee (inclusive of all office supplies, equipment, etc. and exclusive of reimbursable field travel expenses as listed above).
- Latest curriculum vitae (of consultant and any supporting team members)
- Two-page writing sample
- Contact information of three references for similar services offered.
Application procedure
Complete applications containing elements a) through d) should be submitted as attachments to an email to:
[email protected]
The subject line should read: PnP Final Evaluation.
Complete applications containing elements a) through c) should be submitted as attachments to an email to: [email protected]
Deadline for the submission of bids is on June 20tht, 2022, by 5pm Tanzania time
Applicants who do not follow application instructions will be rejected. ONLY shortlisted candidates will be contacted.
