EVALUATION PURPOSE AND USE
This is the first time LWR will conduct an ex-post evaluation of one of its projects. This is therefore not only an exercise to explore a specific project's results but also to gain institutional experience in conducting ex-post evaluations and interpreting their findings.
Internally, this evaluation will be used to:
• Make informed decisions about future LWR programming that is related to this project's focus of supporting an agriculture value chain with an internationally-traded commodity with high standards for quality;
• Learn more about LWR's sustainability and impact; and
• Improve LWR's project design for greater sustainability and impact.
Externally, this evaluation will be used to:
• Showcase to donors LWR's sustainability and impact with strong evidence;
• Provide examples for best practices (and practices to avoid) with peer organizations, especially related to sustainability and impact; and
• Explore findings with project participants and implementing partners to reflect on this experience.
EVALUATION SCOPE
1. The Evaluation Objectives
a. To evaluate the project results' sustainability since the end of the project
b. To evaluate the project results' impact since the end of the project
2. The Evaluation Questions associated with each objective
a. Sustainability
i. Access to resources from financial institutions was an important aspect of the project but had minimal results by the project's end. Were partners able to leverage the project's training and support in this area afterward? Has access to financial institutions improved, decreased, or ended since the end of the project?
ii. Access to markets for grapes improved during the project. What value chain relationships endured? Have any new relationships evolved?
b. Impact
i. What are the key impacts of the project for the targeted organizations and farmers? Did men, women, boys, and girls experience the effects differently?
1. Some impacts that were mentioned during the final evaluation were the extension services models used, such as Training the Trainer and Farmer Field Schools. Have these models continued or evolved? Have extension services continued on the same trend as during the project?
2. Household income based from grape sales increased according to the project's final evaluation. Has this trend been maintained or improved since the end of the project?
3. The percentage of household incomes spent on food generally decreased during the project. Has this trend been maintained?
4. What other impacts from the project are considered important by the targeted organizations and farmers?
ii. What recommendations from the project's final evaluation and subsequent studies were implemented? What were the results? Did men, women, boys, and girls experience the results differently?
iii. What are the perspectives of implementing partners on the project's long-term effects? Did men and women experience the effects differently?
iv. What are some stories that document shifts in power relationships (between men and women, LWR-Partner, NGO-community members, government-community, buyers-sellers etc.) as a result of project participation?
3. The total beneficiary population: 2000 smallholder grape farming households
4. The beneficiary population's geographical dispersion: Within the Dodoma region, Mbabala, Mpunguzi, Hombolo and Gawaye areas
EVALUATION METHODOLOGY
LWR anticipates using a mixed methodology approach for this evaluation that will include reviewing available project data sources and designing a methodology to collect both quantitative and qualitative data. LWR prefers methods that are rigorous and that allow for some analysis on the part of the people providing the information. Examples of such methods could include, but are not limited to, Most Significant Change, PhotoVoice, or Qualitative Impact Assessment Protocol. While an evaluator may choose to use conventional methods (surveys, key informant interviews, focus group discussions), an ideal methodology will mix those methods with a robust method that invites participatory analysis.
Creative use of data collection, participatory data analysis, and presentation of findings that maximize the usefulness of the evaluation will be very important. The evaluator will work closely with LWR to finalize the methodology, sample size, data collection tools and the work plan before field work. The ex-post evaluator will define the methodology. The evaluator will make selected visits to the sites of his/her choice based on the sample size. The ex-post evaluator may propose other approaches that might more effectively answer the evaluation questions. The evaluator is welcome to propose other methods that align with the evaluation's purpose and provide a platform to answer the evaluation questions. LWR prefers that at least one member of the evaluation team be Tanzanian. We may be able to match international evaluators who do not have their own Tanzanian connections with in-country evaluators and welcome proposals that can accommodate that kind of matching.
EVALUATION DELIVERABLES
Expected deliverables include:
o Inception Plan (LWR will provide a template)
o Draft evaluation report and/or slides explaining high-level findings for review
o Final evaluation report (in narrative form)
o Stand-alone evaluation summary document that clearly identifies main findings and recommendations for actions based on the findings (LWR will provide a template)
o One or more Slide Deck(s) that outline:
Project context/history/main interventions that help make the findings easier to understand
Evaluation methodology
Main Findings linked to evaluation questions
Recommendations based on findings, supported with evidence from the evaluation or from peer-reviewed literature
IMPLEMENTATION ARRANGEMENTS
Planning and coordination ahead of data collection, especially to finalize evaluation methodology, will be managed at LWR HQ by LWR's Monitoring, Evaluation and Learning Manager, Wendi Bevins ([email protected]) and LWR's Program Quality Director, Garrett Schiche ([email protected]), who will participate in data collection and be the main point of contact throughout data analysis/interpretation and finalizing main deliverables. LWR Tanzania's Senior Program Manager Moses Kabogo ([email protected]) will coordinate on-the-ground logistics and participant preparation, with LWR's Africa Regional Program Manager Leah Hoffman ([email protected]) backstopping as necessary. These staff will provide all necessary background documentation, an orientation to LWR and its work in Tanzania, and introductions to relevant AMCO staff and other project stakeholders.
EVALUATION TIMEFRAME
The exact timeframe and activities will be negotiated with the selected evaluator, though the major required deliverables are stated here.
June - July
• Inception Report
• Data Collection
• Share slides or outline outlining high-level findings with LWR and partner
• Meeting to discuss high-level findings with LWR and partner, via slides/outline
• Incorporate feedback and update slides/outline (evaluator), prepare materials for presentation to project participants (LWR Tanzania, partner)
August
• Meeting with project participants to validate findings (partner, LWR Tanzania, evaluator if possible)
• Incorporate feedback from project participants and continue revising report
• First draft written report
• LWR review and provide feedback on report
• Present findings to LWR leadership
September
• Final draft written report, executive summary, slide deck(s)
• Sept 25-26 Board of Directors meeting presentation
| 
