ILRG Revised Monitoring, Evaluation, and Learning (MEL) Plan

The primary objective of the Integrated Land and Resource Governance (ILRG) task order under the Strengthening Tenure and Resource Rights II (STARR II) Indefinite Quantity/Indefinite Delivery (IDIQ) contract is to assist the Land and Resource Governance Office under the Development, Democracy, and Innovation Bureau’s Environment, Energy, and Infrastructure Center to design and implement activities that improve land and resource governance, strengthen property rights, and build resilient livelihoods as the foundation for strong economic growth, stability, and resilience. Strong land and resource governance is important within the broader context of reaching myriad United States Agency for International Development (USAID) goals. In particular, successful implementation of this task order will enable USAID to contribute to the following four broad objectives that assist in ending extreme poverty:

  1. Increase inclusive economic growth, resilience, and food security;
  2. Provide a critical foundation for sustainable natural resource management and biodiversity conservation;
  3. Promote good governance, conflict mitigation, and disaster mitigation and relief; and
  4. Empower women and other vulnerable populations.

The task order is currently implementing ILRG activities in Democratic Republic of Congo (DRC), Ghana, India, Liberia, Madagascar, Malawi, Mozambique, and Zambia. Since the original monitoring, evaluation, and learning (MEL) plan was approved, completion tasks were approved for work in Ghana, Liberia, Madagascar, and DRC, so corresponding MEL information has been added for these activities. A completion task MEL plan for India has also been approved. ILRG received Women’s Economic Empowerment (WEE) funding through the Women’s Global Development and Prosperity Initiative, which focuses on Ghana, India, Malawi, Mozambique, and Zambia; a MEL plan for WEE-funded initiatives was also approved. Indicators from the India MEL plan and the WEE MEL plan are included within this MEL Plan for a comprehensive view of all MEL indicators. Performance indicator reference sheets (PIRSs) which come from the India or WEE MEL plan are noted at the top of the PIRS. Any future MEL revisions/updates for new countries or initiatives are expected to start with revisions to this global plan.

ILAW Complexity Aware Monitoring, Evaluation & Learning (CAMEL) Plan

The Complexity-Aware Monitoring, Evaluation and Learning (CAMEL) Plan is a critical management tool designed to measure progress and assist the implementation of the Improving Land Access for Women (ILAW) Activity. This plan outlines ILAW’s approach to data collection, management, analysis, and reporting of a selection of output- and outcome-level performance indicators. The plan also includes ILAW’s complexity-aware approach to strategic collaboration, continuous learning, and adaptive management (CLA) to obtain feedback from partners and stakeholders, monitor changing conditions in the country, review and discuss successes and challenges, and apply lessons learned to upcoming activities and correct course when necessary.

The CAMEL Plan is a living document, which the ILAW team will update during work planning to reflect continuous learning, with the concurrence of the Task Order Contracting Officer (TOCO). The plan centers on two core concepts:

  • Complexity-Aware Monitoring (CAM). ILAW takes a complexity-aware approach to monitoring and evaluation (M&E), recognizing that continuous information about changing conditions is critical for successful adaptive management. ILAW will supplement traditional performance monitoring with CAM methods to understand and adapt to changing conditions, measure unintended consequences, and improve knowledge of the cause-and-effect relationships inherent to influencing social change.
  • Do No Harm. Existing evidence suggests that specifically supporting only women could have the unintended effect of increasing the risk of women losing their land or of gender-based violence (GBV). In addition, Côte d’Ivoire’s socio-political dynamics related to land are highly sensitive, especially in the Western regions where land-based conflicts are cited as a key driver of violence. The ILAW Chief of Party (COP) and Gender Specialist will train all staff and partners on do no harm principles and integrated into weekly reporting and meetings discussion of specific risks related to social conflicts and GBV. Senior ILAW staff and field partners will instruct field agents to inform supervisors immediately in case of acute risks or threats. Field partners will liaise with the national office and local authorities as appropriate, and ILAW staff will liaise with USAID. Table 1 below shows an indicative list of risks as well as mitigation measures.

Impact Evaluation of the Feed the Future Tanzania Land Tenure Assistance Activity: Endline Evaluation Report

Abstract 

This impact evaluation examined USAID’s Land Tenure Assistance (LTA) activity, which was implemented in Iringa District, Tanzania from 2015 to 2019. LTA assisted in land use planning and delivering formalized documentation of customary rights to village residents, known as Certificates of Customary Rights of Occupancy (CCROs), through the use of the Mobile Application to Secure Tenure application. The evaluation randomized treatment assignment across 60 villages, with half receiving LTA’s activities. The five evaluation questions cover the following household outcomes: documentation and tenure security, land disputes, land use and investment, empowerment, and economic wellbeing.

The evaluation team conducted data collection via a panel survey of 1,361 households over three stages (two baseline phases, an interim midline phase for a subset of households, and an endline phase). The evaluation found that within three years of CCRO receipt, LTA had a large and significant positive impact on household tenure security and documentation of land rights, reduced the likelihood of current and future land disputes, and had a smaller positive impact on use of communal land. LTA did not appear to impact the likelihood of fallowing, crop diversification, household land investments, access to credit, or other indicators of household economic wellbeing during that timeframe. Qualitatively, results suggested tangible and important improvements to women’s empowerment, including women’s increased access to land resources and tenure security. The evaluation results help confirm aspects of LTA’s theory of change and align with literature on the impacts of customary land rights formalization on tenure security and other shorter-term outcomes along the envisioned causal pathways. However, the results also highlight a need to revisit expectations for the time required to achieve downstream impacts in rural smallholder settings as a result of customary land formalization on its own. The lack of downstream impacts related to land investments, agricultural productivity, diversification, and broader economic wellbeing highlight the need for USAID to consider coupling or synchronizing future CCRO provisioning programs with agricultural extension and market linkages support to villagers within identified value chains, and financial literacy, financial services, and business development support, once CCROs are obtained.

Executive Summary

This report presents results of an impact evaluation (IE) of the Feed the Future Tanzania Land Tenure Assistance (LTA) activity. The Office of Land and Urban in the United States Agency for International Development’s Bureau for Economic Growth, Education, and Environment (USAID/E3) commissioned the evaluation. The evaluation used a two-phase randomized controlled trial design to rigorously test how mobile mapping and facilitation of formalized customary land tenure certification affect household tenure security, land disputes, investment, women’s empowerment, longer-term economic wellbeing, and related land use and management issues in Iringa District, Tanzania. This report provides impact estimates at endline on key indicators and descriptive qualitative findings related to five evaluation questions, two to two-and-a-half years after household land mapping and receipt of formalized customary land documentation via Certificates of Customary Right of Occupancy (CCROs).

LTA Activity Description

In recent decades, Tanzania and many other countries across sub-Saharan Africa have undertaken substantial land reforms to formalize customary land rights for village residents, aiming to improve tenure security and unlock economic opportunities for the rural poor. The Tanzanian land rights system is based on public ownership of land, wherein all land is owned by the state and held in trust by the president. Individuals who use or occupy village land have the right to obtain formal documentation of their customary land use rights via a legally valid and transferrable CCRO, which is issued by the local government. However, factors such as insufficient capacity of district land offices (DLOs) to assist villages with land use planning or issue CCROs, villagers’ unfamiliarity with formal land laws, and lack of funds to pay CCRO fees have resulted in few villagers obtaining formal documentation of their land rights and CCROs for their plots. Increasingly, the Government of Tanzania and the donor community recognize that improving the security of land rights is essential to protecting the rights of smallholders, reducing land disputes, and maximizing the region’s economic potential.

USAID/Tanzania awarded the four-year, $6 million LTA activity to DAI in December 2015. The activity sought to clarify and document land ownership, support local land use planning efforts, and increase local understanding of land use and land rights in Tanzania. LTA assisted villages and the DLOs in Iringa and Mbeya districts in completing the land use planning process and delivering CCROs in select villages. It also provided education on land laws, CCROs, and land management. LTA used the Mobile Application to Secure Tenure (MAST), a USAID-developed app and approach that facilitates the land mapping and customary land formalization process. The 2015-2019 phase of LTA in Iringa District was implemented in 36 villages, 6 of which were chosen for initial implementation and an additional 30 were part of this evaluation. According to LTA implementation tracking data, the activity registered 59,354 CCROs in Iringa District villages during that time, of which 51,222 CCROs were collected by 26,436 individual claimants.

 




 

Evaluation Design Proposal Impact Evaluation of the Feed the Future Tanzania Land Tenure Assistance Activity

Introduction 

This Evaluation Design Proposal corresponds to the impact evaluation (IE) of USAID/Tanzania’s Feed the Future Tanzania Land Tenure Assistance (LTA) activity. This IE is being implemented by the E3 Analytics and Evaluation Project1 and commissioned by USAID’s Office of Land and Urban in the Bureau for Economic Growth, Education, and Environment (USAID/E3/LU). USAID’s draft Statement of Work (SOW) for this evaluation is included in Annex A. The IE design incorporates a phase-in randomized controlled trial (RCT) to rigorously test how effective the provision of land certification is in increasing land investment, promoting the empowerment of underrepresented groups, strengthening tenure security, decreasing land disputes, and improving environmentally sustainable practices.

This document provides the framework for the IE design, and was updated in September 2017 to include subsequent details and a revised timeline and estimated budget based on changes to the implementation schedule that occurred after Phase I of the evaluation. First, it describes the Tanzanian context and the main LTA activity components, which have been designed to address the key causal factors of land tenure specific to Tanzania. Then, it reviews the existing evidence and remaining knowledge gaps in the land tenure literature, which provides the motivation for this IE. The document then lays out the evaluation questions, evaluation design, and key outcome measures of interest. Subsequent sections focus on the data collection methods and the sampling plan, followed by a discussion of the data analysis methods and the strengths and limitations of the evaluation design, which includes updates based on changes in LTA activity implementation between Phase I and Phase II of the evaluation.

Background

The Tanzanian land rights system is based on public ownership of land, with all land owned by the state and held in trust by the President. The majority of land in Tanzania is designated as Village Land, which is governed by the 1999 Village Land Act. The Act recognizes the rights of villages to hold and govern land according to customary law. Individuals residing on or using Village Land have the right to obtain formal documentation of their rights in the form of a Certificate of Customary Right of Occupancy (CCRO), which can be issued at the local government level.

In practice, most villagers do not have CCROs for their plots and lack formal documentation of their land rights (Pederson 2010). Land use demarcation and mapping that are required to issue the documents have not yet been completed in many villages. Moreover, the District Land Offices (DLOs) responsible for issuing CCROs frequently lack the capacity to do so, and rural land users are often unaware of their land rights under the law.

 




 

Women’s Global Development and Prosperity Initiative Monitoring, Evaluation, and Learning Plan

Overview of the ILRG W-GDP Activity

Purpose

The primary objective of the Integrated Land and Resource Governance (ILRG) task order under the Strengthening Tenure and Resource Rights II (STARR II) Indefinite Delivery/Indefinite Quantity (IDIQ) contract is to assist the Land and Urban Office in the Bureau for Economic Growth, Education, and Environment (E3/LU) to design and implement activities that improve land and resource governance, strengthen property rights, and build resilient livelihoods as the foundation for strong economic growth, stability, and resilience. Strong land and resource governance is important within the broader context of reaching multiple United States Agency for International Development (USAID) goals. In particular, successful implementation of this task order will enable USAID to contribute to the following four broad objectives that assist in ending extreme poverty:

  1. Increase inclusive economic growth, resilience, and food security
  2. Provide a critical foundation for sustainable natural resource management and biodiversity conservation
  3. Promote good governance, conflict mitigation, and disaster mitigation and relief
  4. Empower women and other vulnerable populations

The task order is currently designed to implement ILRG activities in multiple countries around the world using a variety of funding sources. This monitoring, evaluation, and learning (MEL) plan addresses the Women’s Global Development Prosperity (W-GDP) Initiative-funded activities in Zambia, Mozambique, Malawi, and India, as well as an emerging partnership with a private sector cocoa company. The MEL plan will be adapted to address any additional countries, as well as global interventions.

Near the end of FY 2019, ILRG was informed that it would receive W-GDP funding related with the aim of implementing women’s empowerment and gender equality activities in existing and new ILRG geographies over approximately two years. This MEL plan describes the indicators to be used to track impacts of project activities, how data will be collected and how learning can inform program implementation. The ILRG W-GDP task aims to contribute to the three W-GDP pillars:

  • Pillar 1: Women prospering in the workforce;
  • Pillar 2: Women succeeding as entrepreneurs; and
  • Pillar 3: Women enabled in the economy.

The ILRG W-GDP task will particularly work to support women’s empowerment by contributing to Pillars 2 and 3. This MEL plan also recognizes the importance of coordinating with USAID’s Communications, Evidence and Learning (CEL) program, which will be undertaking W-GDP-funded activities in some overlapping geographies. All indicators in this MEL plan are also included in the broader ILRG MEL plan.

 




 

Evaluation of the “Supporting Deforestation-Free Cocoa in Ghana” Project Bridge Phase: Baseline Report

EXECUTIVE SUMMARY

This report provides baseline context for the mixed-methods impact evaluation (IE) of the “Supporting Deforestation-Free Cocoa in Ghana” project Bridge Phase activity that has been commissioned by the United States Agency for International Development’s (USAID) Office of Land and Urban in the Bureau for Economic Growth, Education, and Environment (USAID/E3/LU). The evaluation aims to provide an evidence base for outcomes of the Bridge Phase activities with respect to strengthening land rights and land governance, reducing deforestation, increasing carbon sequestration and cocoa productivity, and enhancing local livelihoods. The evaluation is led by NORC at the University of Chicago, under the Communications, Evidence and Learning (CEL) Project.

This document provides findings from the baseline data collection for the evaluation, including background context on key demographics, household characteristics, and baseline measures on outcome variables. The report also examines balance across Bridge Phase and comparison group villages for the IE components of the evaluation, revisits power calculations from the Evaluation Design Report (EDR) using parameters from the baseline data, and provides an update on the viability of evaluating the farm rehabilitation component of the Bridge Phase activity, given a reduction in the number of beneficiaries receiving this intervention as learned at baseline.

BRIDGE PHASE ACTIVITY DESCRIPTION

The Integrated Land and Resource Governance (ILRG) Task Order under the Strengthening Tenure and Resource Rights II (STARR II) Indefinite Delivery/Indefinite Quantity (IDIQ), is supporting work on sustainability of deforestation-free cocoa through the “Supporting Deforestation-Free Cocoa in Ghana” (SDFC) project Bridge Phase. The ILRG program is implemented by an international consortium including the prime contractor, Tetra Tech, and core subcontractors, including Winrock.

The ILRG activity seeks to refine three interventions through a “Bridge Phase” period, and then scale up a financially viable farm rehabilitation and land tenure strengthening model for the Ghanaian cocoa sector. In combination with land use planning, the scaled activity aims to result in: reduced deforestation and greenhouse gas (GHG) emissions and increased carbon sequestration in the cocoa landscape, increased cocoa farm productivity and resilience, diversified farmer incomes, and improved livelihoods. Implementation of the Bridge Phase is through a partnership across ILRG and three private sector partners: Ecom Agroindustrial Corp. (ECOM), the Hershey Company (Hershey), and Meridia. The Bridge Phase activity has three intervention components: (1) cocoa farm rehabilitation led by ECOM that is provided to a small group of registered farmers; (2) farm mapping and provisioning of tenure documentation to all interested farmers in the four Bridge Phase communities; and (3) landscape governance and land use planning activities implemented at village and district levels.

EVALUATION QUESTIONS

Table 1 lists the six questions addressed by the SDFC Bridge Phase evaluation. The evaluation questions, which the team developed in collaboration with USAID, focus on meeting USAID’s priority learning interests for this activity and were derived from the SDFC Bridge Phase theory of change.

 




 

Evaluation of the “Supporting Deforestation Free Cocoa in Ghana” Project Bridge Phase: Evaluation Design Report

This document describes the proposed design, research methodology, work plan, team composition, and estimated budget for a mixed-methods performance evaluation of the United States Agency for International Development’s (USAID) “Supporting Deforestation Free Cocoa in Ghana” project Bridge Phase. USAID’s Office of Land and Urban in the Bureau for Economic Growth, Education, and Environment (USAID/E3/LU) commissioned the Communications, Evidence and Learning (CEL) Project to conduct this evaluation.

To meet the learning interests of USAID and IPs for the Bridge Phase activities — and given the complexity of the three Bridge Phase interventions, which vary in scope, geography, selection criteria for beneficiaries, and anticipated timeline for maturation of key outcomes — this mixed-methods evaluation will use different analytic approaches to assess the effects of each of the three Bridge Phase interventions. The three interventions are focused on: (1) cocoa farm rehabilitation; (2) tenure documentation; and (3) village-level land use planning. At USAID’s request, the evaluation is designed to aim for as rigorous a quasi-experimental design as possible, within available budget, recognizing that the small scale of implementation for the Bridge Phase activities is a limitation on evaluation rigor and power. An RCT design was not considered feasible for this Bridge Phase, given the short time frame to finalize implementation design and work with collaborating private-sector partner implementation needs.

To increase the learning potential from the evaluation, the evaluation will use quasi-experimental approaches for the two farmer-level interventions: the farm rehabilitation service and the tenure documentation service. Effects of the farm rehabilitation intervention will be assessed using a regression discontinuity approach. Effects of the tenure documentation intervention will be assessed using a difference-in-difference with statistical matching approach. A pre-post qualitative approach will be used to assess outcomes for the land use planning component, which is not amenable to a quasi-experimental design due to the small number of implementation villages.

The evaluation will collect qualitative and quantitative data at baseline and endline from farmers in the four Bridge Phase villages and in eight comparison group villages. The comparison group villages are communities in the same district, and where the farm rehabilitation service has also been offered, but the tenure documentation and village-level land use planning interventions will not be implemented.

 


 

Monitoring, Evaluation, and Learning (MEL) Plan

Reviewing and Updating the MEL Plan

The Monitoring, Evaluation, and Learning (MEL) Plan serves as a tool to guide overall project performance. As such, the team will update it as necessary to reflect changes in USAID AMPR’s strategy and ongoing tasks. Updates will also incorporate feedback from USAID as well GoCAR. Monitoring, evaluation, and learning are therefore not one-time occurrences, but rather part of an ongoing process of review, revision, and implementation. The MEL team will review the MEL Plan annually, update if necessary, and submit a revised version along with the proposed Annual Work Plan. For this purpose, the monitoring period each year is from October 1 – September 30.

This plan is the second version of the MEL Plan produced in September 2019 and submitted for USAID approval along with the second Annual Work Plan (2019-2020).The main revisions are in annual targets and some baseline values following the studies and data collection conducted in the first year of project implementation.

Performance Indicators

USAID AMPR’s performance indicator table (Table 1) presents a range of both custom and standard indicators at the output, outcome, and impact levels. The table also includes all the standard foreign assistance indicators (“standard indicators” or “F indicators”) relevant to project activities. USAID AMPR will also track custom outcome indicators to measure the eight results listed in the contract plus several custom crosscutting indicators. The team will disaggregate indicator data reported by task/country and all person-level indicators by gender, except where government-provided data does not allow disaggregation.1 The Performance Indicator Reference Sheets (PIRSs) in Annex A contain full details for each indicator, including use of indicator, baseline procedures, data collection methodologies, data quality assurance measures, and justifications for proposed targets.

Artisanal Mining and Property Rights (AMPR) Environmental Management and Mitigation Plan (EMMP) Year I: October 2018 – September 2019

Executive Summary 

The Artisanal Mining and Property Rights (USAID AMPR) project supports the USAID Land and Urban Office in the Bureau for Economic Growth, Education, and Environment (E3/LU) in improving land and resource governance and strengthening property rights for all members of society, especially women. Its specific purpose is to address land and resource governance challenges in the artisanal and small-scale mining (ASM) sector, using a multidisciplinary approach and incorporating appropriate and applicable evidence and tools. The project builds upon activities and lessons from the Property Rights and Artisanal Diamond Development (PRADD I and II) projects. The USAID AMPR contract was signed on September 28, 2018 and will run initially for 3 base years and with two optional years possible. Most project activities will be carried out in the Central African Republic.

The Initial Environmental Examination (IEE) Summary submitted was approved on March 20, 2018. The IEE gave a Negative Determination with Conditions (See Annex for copy of the IEE). Pursuant to the requirement to submit an Environmental Mitigation and Monitoring Plan (EMMP) within 60 days of signature of the contract between Tetra Tech and USAID, this initial statement spells out the ways in which the project will respond to the conditions spelled out in the IEE. In subsequent years, annual reports will indicate how the environmental issues are addressed.

During the project start-up phase, the project management team led by Project Manager Dr. Mark Freudenberger, Technical Deputy Terah DeJong, and Chief of Party Maxie Muwonge will conduct training sessions for all project staff on the intent of EMMP’s and the specific measures the project will take to implement the negative determination with conditions. The project management team is blessed with experiences learned from the Property Rights and Artisanal Diamond Development Project (PRADD II) carried out previously in the Central African Republic, Côte d’Ivoire, and Guinea. In these countries, the project confronted successfully many environmental problematics associated particularly with the artisanal diamond economy.


 

Artisanal Mining and Property Rights (AMPR): Monitoring, Evaluation, and Learning (MEL) Plan

Monitoring and Evaluation Plan 

The Monitoring, Evaluation, and Learning (MEL) Plan serves as a tool to guide overall project performance. As such, the team will update it as necessary to reflect changes in USAID AMPR’s strategy and ongoing tasks. Updates will also incorporate feedback from USAID as well GoCAR. Monitoring, evaluation, and learning are therefore not one-time occurrences, but rather part of an ongoing process of review, revision, and implementation. The MEL team will review the MEL Plan annually, update if necessary, and submit a revised version along with the proposed Annual Work Plan (AWP). For this purpose, the monitoring period each year is from October 1 – September 30th.

Performance Indicators 

USAID AMPR’s performance indicator table (Table 1) presents a range of both custom and standard indicators at the output, outcome, and impact levels. The table also includes all the standard foreign assistance indicators (“standard indicators” or “F indicators”) relevant to project activities. USAID AMPR will also track custom outcome indicators to measure the eight results listed in the contract plus several custom crosscutting indicators. The team will disaggregate indicator data reported by task/country and all person-level indicators by gender, except where government-provided data does not allow disaggregation.1 The Performance Indicator Reference Sheets (PIRSs) in Annex A contain full details for each indicator, including use of indicator, baseline procedures, data collection methodologies, data quality assurance measures, and justifications for proposed targets.