Dr. Heather Huntington is an Impact Evaluation Specialist on USAID’s Evaluation, Research and Communication project, implemented by The Cloudburst Group. She leads the design and implementation of impact evaluations for land tenure and natural resource management projects in Ethiopia, Guinea, Liberia and Zambia. Dr. Huntington is among the authors of the 2015 World Bank Paper: Perceptions of Tenure Security and a presenter at the American Evaluation Association (AEA) Conference discussing impact evaluations testing improvements to land tenure in the context of climate smart agriculture in Zambia, artisanal diamond mining in Guinea and community forestry management in Zambia. As part of our Ask the Expert series, we asked Dr. Huntington to describe some of the processes that go into designing and implementing an impact evaluation for USAID:
Question: Tell us about yourself, what is your professional background?
Answer: I design and implement impact evaluations related to land tenure and natural resource management, service delivery and local governance. I received a PhD in Public Policy and Political Science from the University of Michigan and my dissertation was an impact evaluation of a USAID water demand management project in Southern Kyrgyzstan. After graduate school, I served as a full time Democracy Fellow for USAID’s Center of Excellence on Democracy, Human Rights and Governance, where I continue to have an affiliation. My role in the Center was as an impact evaluation specialist for democracy- and governance-related projects.
Question: Why are impact evaluations significant to USAID’s land tenure work?
Answer: Impact evaluations are an important tool to measure the effectiveness of programs in achieving their desired results. Specifically, they can improve USAID’s programming by refining an intervention to an outcome of interest, such as higher agricultural productivity or lower levels of conflict. Impact evaluations help decision makers identify and address holes in the logic of a program’s design.
In addition, well-designed impact evaluations often involve a large number of household surveys and thus produce a large amount of useful data that can serve multiple purposes. This includes testing the program theory and hypotheses, but also exploring many other research questions in depth.
Question: What are some of the key take-aways from the land tenure impact evaluations you are presenting at the AEA conference?
Answer: One take-away is that it it is important to plan for an impact evaluation at the initial design stage of a program, including conducting a cost-benefit analysis, to help determine the type of impact evaluation the program should use. This is important because rigorous experimental designs require that the program implementation design be written into the fabric of the program design as early as the project’s approval stage. Quasi-experimental impact evaluations are generally the easiest to carry out from implementation standpoint, however, from a research perspective, these methods have numerous drawbacks and present multiple methodological challenges.
Considerations such as if the impact evaluation includes a community listing, or pre-census, of the study areas under evaluation is often imperative for the sampling design and needs to be factored in at the budgeting and planning stage. Additionally, factors including the preferred data collection method, should be examined. In our experience, electronic data generally produces higher quality data than paper data collection but requires significant preparation, programming and training for local data collection firms..
Finally, a rigorous impact evaluation requires very close collaboration between the evaluation team, program implementer and donors. While the evaluation should be led by independent third-party groups, communication and coordination regarding timing, interventions and geographic scope is essential between the evaluation team and implementing partner.
Question: Why is the American Evaluation Association Conference important and why is it significant that USAID’s land tenure impact evaluations are presented at this conference?
Answer: I believe that development partners and researchers will be interested in learning about our data and research products and that other evaluation specialists and firms will be interested in learning about our process and methods. In addition to ‘getting the word out’ about what we are doing and what we have learned, it is helpful for us to know what other work is being done in the field of land tenure and natural resource evaluation.