STEWARD III Evaluation: Mid-Term Performance Evaluation

This mid-term evaluation has three objectives:

  • To review the progress made in achieving the STEWARD III objectives;
  • To identify critical mid-course program changes necessary to ensure sustainability of the program; and
  • To the extent possible within the constraints of time and budget, identify lessons for consideration in future programming.

Intended users include USAID/West Africa, USFS-IP as the implementing body, and the Implementing Partners of STEWARD.

PROJECT BACKGROUND

STEWARD III is a forest conservation and sustainable livelihoods project working in trans-boundary priority zones in the Upper Guinean Forest ecosystem, occurring in Guinea, Sierra Leone, Liberia, and Côte d’Ivoire. It is the third iteration of the STEWARD program. STEWARD I was characterized as a design phase, and STEWARD II as a pilot phase. STEWARD III is intended to be the implementation phase. Its goals are to:

  • Conserve biodiversity and improve rural livelihoods in critical trans-boundary landscapes in the Upper Guinean Forest ecosystem;
  • Produce harmonized policies and legal frameworks for natural resources management (NRM) in a regional context; and contribute to national strategic plans on climate change in the Mano River Union states; and
  • Promote resiliency in the face of climate change.

STEWARD III is implemented by the US Forest Service’s International Program (USFS-IP), which has a history of excellence in technical assistance to USAID through a Participating Agency Partnership Agreement (PAPA). It is worth noting that prior to STEWARD the USFS-IP had not managed large projects in the developing world.

STEWARD III is a complex program, involving multiple funding streams (including biodiversity, adaptation, sustainable landscapes, and WASH funding). Each of these carries specific implementation criteria and reporting requirements. The project is managed through six concurrent sub-agreements, which are functionally separate cooperative agreements. This would pose a sufficiently challenging management task under ideal circumstances, yet the difficult physical and institutional environments in the STEWARD states of Sierra Leone, Guinea, Liberia, and Côte d’Ivoire make it even more demanding. These are among the world’s poorest countries, near the lowest rungs of the development ladder in terms of life expectancy, income, and education. All four countries have a recent history of civil strife, and they are in the midst of the world’s worst-ever Ebola epidemic. The World Bank’s Worldwide Governance Indicators for 2012 (the most recent year available) rank all four countries in the lowest quartile of all nations in the areas of government effectiveness, regulatory quality, rule of law, and control of corruption In short, these are very fragile states.

STEWARD III began in June, 2011, and became fully operational in 2012. In the course of implementation it has faced some serious challenges, including a series of personnel changes, financial fraud, and a major public health crisis. It has also made some important accomplishments in the face of challenging circumstances.

BPRP Report: South Africa Study Tour

The Burundi Policy Reform Project carried out a study tour to South Africa in partial fulfillment of requirements under the project’s Victims of Torture Component Task 5.3.5. The 19 participants from Burundi who visited South Africa from August 1 to August 9 had a series of extraordinary introductions to torture victims in South Africa and community-level efforts to meet the needs of victims along with institutions in Johannesburg and Cape Town that focus on issues related to torture and other forms of violence. The participants also visited judicial and human rights institutions established within the framework of transitional justice for the post-conflict South African society.

At the Khulmani Support Group, the Burundians learned how torture victims and survivors of apartheid-era political violence have organized to lobby and advocate for community-level reparations, prosecutions of perpetuators and investigations of disappeared persons. The Steve Biko Center for Bioethics is a university-based institution focusing on issues related to medical torture. The participants found the Center’s work to be relevant to situations existing in Burundi and were anxious to have additional information, follow-up and future cooperation on the subject of medical torture and related ethical issues. The Trauma Center for Survivors of Violence and Torture impressed upon the Delegation that the healing of memories is a necessary ingredient in the rehabilitation of survivors and victims of torture. Participants expressed their satisfaction with the opportunity to interact with torture victims from District Six in Cape Town. The Burundians learned that South Africa has virtually eliminated torture through specific prohibition at the constitutional level. The point was reinforced that torture issues can be most effectively addressed through the mechanisms of a transitional justice program. At the same time, the Delegation was exposed to varied public, civil society and community-based approaches to treatment, rehabilitation and reparations for survivors and victims of torture.

Through meetings and dialogue with representatives of The South African Human Rights Commission, The Human Rights Institute for South Africa, The Foundation for Human Rights, and The International Center for Transitional Justice, the Burundian Delegation obtained detailed information on the South African Constitution, the role of the Constitutional Court, The Truth and Reconciliation Commission (TRC), and prosecutions for apartheid-era crimes through the justice system. Objective panel presentations and discussions on the strengths and weaknesses of the truth and reconciliation process and the broader transitional justice process were obtained from The Foundation for Human Rights and The International Center for Transitional Justice. The Delegation heard that victims of political violence were not adequately included in the TRC process, and that the TRC emphasized truth seeking and pardons to the extent that perpetuators of political crimes were not prosecuted sufficiently.

The Delegation was visibly impressed with the opportunities to meet with Albie Sachs, Justice on the Constitutional Court of South Africa and Bishop Desmond Tutu, social activist and former Chairman of The Truth and Reconciliation Commission. Both of these famous persons gave advice and counsel to the Delegation on the necessity to end impunity and to work toward peace and reconciliation in Burundi.

The participants in the Study Tour to South Africa were unanimous in their assessment of the trip as highly substantive and motivational toward bolstering their individual and collective efforts against torture, ending impunity and moving towards transitional justice in Burundi. Upon returning to Burundi, the participants immediately began the process of follow-up, information dissemination and recommendations resulting from the South Africa experience. The major issue raised by the participants is that of determining those mechanisms of transitional justice that are appropriate to the Burundian political and social context. They realized that Burundi cannot duplicate the South African experience, but lessons from South Africa are certainly applicable in crafting those transitional justice mechanisms that will serve
Burundi.

BPRP Assessment: Women’s Leadership

This assessment report is a result of the field trip conducted by two consultants from Partners for Democratic Change, Olivia Baciu and Eugene Ntaganda, from November 30-December 26, 2008. The assessment constituted the first activity under the women’s leadership component (5.2) for year two of USAID’s Burundi Policy Reform Program.

The assessment laid the foundation for a targeted approach to helping Burundian women provide leadership in a way that supports a more informed transparent and inclusive approach to policy development. The assessment had two objectives:

  1. To provide data to ensure that all three tasks of the component are geared towards the level of the target groups specified in the work plan, giving a comprehensive picture of women’s current skills and capacity needs in conflict mitigation and anti-corruption programming.
  2. To advance the design of work for all three tasks of the component:
    1. For Task 5.2.1: providing recommendations for participation of target groups of Burundian women at international conferences;
    2. For Task 5.2.2: providing recommendations about the content of training that will help the target groups participate more actively in the political process;
    3. For Task 5.2.3: providing recommendations for the design of the trainings in conflict mitigation and fight against corruption.

This report builds its recommendations on the detailed assessment report attached in Annex 1, in French, which includes data gathered from the target groups through structured interviews and questionnaires. This report presents briefly the methodology, the findings for each group, and recommendations under all three tasks of the Women’s Leadership Component.

BPRP Report: Helping Civil Society Organizations Advocate for Reforms That Eradicate Torture

This report summarizes the work of myself, international consultant Ban Saraf, providing assistance to the Burundi Policy Reform Project in carrying out Task 5.3.3 under the Victims of Torture component. The assignment included several activities and deliverables leading to the engagement and the participation of project partners in the formulation of draft advocacy campaign strategies to eradicate torture. Separate, informal reports were submitted throughout the assignment with findings and recommendations per activity.

Activities required by the terms of reference included roundtable meetings, training of trainers in Bujumbura and the interior in conducting advocacy campaigns, network assessment in the interior (four regions), demonstration in the interior of the development of an advocacy campaign strategy, assistance in a communication plan for International Day in Support of Victims of Torture activities, and assessment of NGOs in Bujumbura. Upon my arrival in Bujumbura, the terms of reference were refined to include a detailed schedule and expected deliverables. Accordingly, the following activities were delivered.

BPRP Report: Cambodia Study Tour

The Burundian Observation/Study Tour to Cambodia was successful in all respects. It was extraordinarily well organized, totally substantive and obviously beneficial to the Burundian participants. The Cambodian host organizations were appropriately selected and well-prepared to contribute to the study tour objectives. In retrospect, Cambodia was an ideal choice for the study tour site. There are many similarities between the two countries, yet the contrasts enabled the Burundians to place their own society in a better context.

Visits to the Genocide Museum and the Killing Fields were stark reminders to the Burundians of the atrocities that occurred in their own country during the 1993-1996 period. With monuments, documentation and an international tribunal underway, Cambodia has a strategy to promote healing and reconciliation from its past. The Burundians recognized the need for their own country to move forward in this manner.

The highlights of the study tour included a model prison and courtrooms, substantive meetings with NGO leaders, visits to sites providing services and rehabilitation to victims, abused women and children. Two trips were made to the Extraordinary Chambers of the Court of Cambodia to observe the ongoing trial of a former Khymer Rouge leader for crimes against humanity. Two visits with villagers in rural communities provided positive examples of community mobilization to defend personal and property rights.

The totality of meetings and site visits in Cambodia enabled the delegation members to objectively assess the strengths and weaknesses of Cambodian society relative to the rule of law and human rights environment in Burundi. The delegation discussed and noted short and long-term initiatives that they could undertake in Burundi at the national and community levels. These efforts include community-level mobilization, broader advocacy, increased services for victims, greater utilization of the media, active pursuit of a tribunal or a truth commission to expose past crimes against humanity within the country, and more coordination with the international community on human rights issues.

The social interaction with the Co-Prosecutor of the International Tribunal in Cambodia was an honor and highly beneficial for the delegation in terms of perspective and information.

The participant’s evaluations of the organization and relevance of the study tour were unanimously laudatory and appreciative.
Recommendations for the South Africa tour include a suggestion to correspond with USAID in country on the visit for input and perspective that may be beneficial. At least two civil society participants from the Cambodia tour should join the delegation to South Africa. This will provide perspective, contrast and comparison between the Cambodian and South African experiences.

BPRP Assessment: Victims of Torture Component

The Burundi Policy Reform Program, requested a consultant from its subcontractor, BlueLaw International, to undertake an assessment and programmatic recommendations for implementation of the Victims of Torture Component. Highlights of the consultant’s scope of work include the following:

  • Undertake an assessment and analysis of the institutional and management capacities of civil society organizations in Burundi that have programs for victims of torture.
  • Recommend a program of sub-grants to bolster and enhance the institutional capacities of civil society organizations to provide services to victims of torture.
  • Recommend ways to promulgate effective advocacy against torture at community and national levels.
  • Recommend a contractual arrangement through local organizations to provide technical assistance to selected civil society organizations for effective management and delivery of VOT services along with along with increased capacities for advocacy against torture.
  • Recommend two sites for study tours in other countries for representatives of Burundian organizations involved in services or advocacy related to torture. Provide visit objectives along with selection criteria for participants.

Background

Burundi is recovering from 14 years of civil strife, armed rebellion and instability. Torture exists in the country and is perceived by the international community as a human rights issue for Burundi. The number of incidents of torture is not known, but indigent human rights organizations are aware of cases throughout the country. Within Burundi’s population there is a cultural acceptance that torture is tool of the powerful and people in uniform. Generally, acts of torture occur during arrests, during transport or while held in temporary lock-up. It rarely occurs in the prisons. Victims cannot, or will not, identify those who commit acts of torture. Normally, torture victims do not have access to medical assistance that can identify torture marks or scars, and perpetuators of torture undertake measures to cover-up or make physical evidence disappear.

Although Burundi is a signatory to the International Convention Against Torture (CAT), the country’s penal code does not include the crime of torture. Burundian judges have not considered torture a serious crime. Many cases are dismissed along with a few lenient sentences. Victims are rarely compensated. A revised penal code that includes torture as a serious crime has been drafted and is expected to become law in the near future.

Review and Update of the AgroInvest Project’s Agriculture Policy Priorities

This report reviews and updates the AgroInvest Ukraine Project’s Strategic Policy Priority Needs paper (Brown 2011). AgroInvest Ukraine has done very significant work to advance policy reform in the agricultural sector of Ukraine and, with some minor changes, can do even more and better.

This review and update performs three tasks. First, it articulates Project goals that shape a positive agenda of policy reform work and presents an initial version of that agenda. Second, it revises the policy “filters” developed in the 2011 paper to make them more useful for sorting through the many requests for policy support that AgroInvest Ukraine, as a successful policy effort, receives from its client, partners and other stakeholders. Third, it considers some issues of how to make AgroInvest Ukraine’s assistance to policy reform and provision of policy advice more successful.

Box 1: Summary of Policy Reform Priorities for AgroInvest Ukraine (See Section C for detailed explanation of the stated priorities)

I. Improve market-based incentives for small and medium farmers to survive and prosper by increasing profitable production and sales

  1. Ensure fair competition
  2. Eliminate price controls
  3. Make certain that grades and standards are in harmony with international ones
  4. Promote the growth and development of household farms
  5. Improve agricultural finance for small and medium farms
  6. Develop market infrastructure

II. Guarantee farmers’ property rights and security of contract

  1. Continue work on land tenure and farm size
  2. Examine issues of land use and conservation regulations
  3. Improve security of farmers’ contracts

III. Promote the shift from governance suited to a command economy to that appropriate for a market one.

  1. Consider the appropriateness of special tax, legal and other regimes for agriculture
  2. Facilitate improvements in the structure and functioning of public administration
  3. Assist the development of an appropriate agricultural knowledge system

The development hypothesis of AgroInvest Ukraine is based on the general international experience that, all other things being equal, farms that are no larger than what a family can operate and manage with limited hired labor are much more efficient than farms that rely on hired managers and workers. Management is the scarce factor in agriculture everywhere, and family management in agriculture has indisputable advantages over hired management. This assumption is congruent with the perception that the prospects for stable democracy are best when property ownership is widespread and most families are middle class.

So AgroInvest Ukraine aims to facilitate the development of a vibrant, prosperous and growing social group of small and medium farmers and to help them organize in civil society to defend their own interests. This is the key to further agricultural development in Ukraine, to increasing the prosperity of Ukrainian society as a whole, and to allowing Ukraine to make the contribution it should to improving the world’s food security.

The Project has done and is doing very useful policy work. The quantity of requests for assistance on policy issues it receives demonstrates its success. However, to avoid purely reacting to events and ad hoc requests for policy analysis, AgroInvest Ukraine should continue to apply the filters outlined in Section E of this document.

AgroInvest Ukraine should concentrate on facilitating policy reforms that advance the three priorities. Section C of the review presents a detailed agenda of activities flowing from those priorities. Box 1 above summarizes that agenda.

Point I in the box, improving incentives for small and medium farmers, is most important. Success there will drive demands from Ukrainian society for reform on the other issues. Within the USAID Ukraine portfolio, point I is purely AgroInvest. Points II (security of tenure and contract) and III (improving governance that affects agriculture) are cross-cutting issues that affect all economic sectors. But they are especially important for the agricultural sector, and AgroInvest Ukraine and its partners should work on them in collaboration with other USAID projects and other stakeholders beyond the sector.

Policy reform is sparked by successful policy advice, which changes what policy is and how policy is implemented. It affects not only policy makers’ statements and national laws, it also changes administrative practice and citizens’ activities.

Policy advice includes two activities: policy-oriented research, understanding the facts and trends of the situation, and offering advice based on that knowledge to a principal with direct influence on policy such as government officials, parliamentarians, or leaders of civil society organizations.

Three qualities are critical to successful policy advising:

  • Access to policy-makers
  • Policy-makers’ trust
  • Vision and knowledge

These qualities take time and effort to develop, as discussed later in this report.

Additionally, the Ministry of Agrarian Policy and Food of Ukraine has asked AgroInvest Ukraine for assistance in doing a new strategy for the sector, which would replace the 2007 “State Target Program for the Development of Rural Areas.” The officials concerned have asked specifically for help from AgroInvest and its partners. The request in itself demonstrates that the Project’s policy advice is respected, listened to and effective. AgroInvest should work closely with the Ministry to develop the country’s next multi-year sectoral strategy. This is an opportunity to have a major policy reform impact.

Caribbean Open Trade Support (COTS) Report: Dominica Organic Agriculture Movement

A 2004 article about organic production in Dominica concluded with an assessment of the problems facing organic farmers, including general skepticism about the possibilities of success, difficulties in obtaining inputs, and issues of certification, policy, and regulation. Access to markets was noted as the greatest deterrent for farmers to engage in organic production. Since that time, a number of steps have been taken to address these problems. A February 2005 United Nations Environment Programme/Dominican government report, “Transforming Dominica into an Organic Island,” provides a thorough analysis of why organic production is appropriate and essential for Dominica’s environmental and economic goals. A June 2007 National Organic Agriculture Enhancement Project report provides a complete outline and budget projections for the Dominica Ministry of Agriculture, Fisheries and the Environment to implement a sustainable organic agriculture sector. The creation of an “organic desk” in the Ministry of Agriculture and the official launch of the Dominica Organic Agriculture Movement (DOAM) are signs of progress toward these goals. Progress, though, means continuing to implement the National Organic Agriculture Enhancement Project and building market opportunities for organic products.

Success in Dominica hinges on public and private participation in developing and expanding organic production. Relying on market forces and the private sector alone will not work because of Dominica’s size, scale of production, and access to transportation, and because organic production is not just a market opportunity, it is a method to protect, restore, and enhance the environment, which requires government participation and commitment. The United States and the European Union (EU) lead the world in organic production and organic market share as a result of more than 30 years of public and private initiatives and investments. Active involvement by farmers, businesses, citizens, and government toward a common goal for organics will make a difference in Dominica.

Dominica is poised to move forward as DOAM organic production standards near completion and move through the process for public review and approval by the Bureau of Standards. National legislation that includes organic as a good agricultural practice is being considered and may be the precursor to mandatory national organic standards and certification. The Ministry of Agriculture is receptive to increasing its involvement by providing information and advice to farmers on organic production methods as good agricultural practices, regardless of whether farms fully convert to organic. Other services— such as regional compost operations, cooperative composting near farm areas with multiple individual plots, and restoration of traditional animal breeds — support expansion of organic production, and the Ministry of Agriculture should be encouraged to expand its work in these areas.

DOAM is the hub of the public/private partnership and needs continued support to build its capacity to promote organic production and standards. Bringing together Dominica’s organic activities, initiatives, and opportunities is DOAM’s core role. Presently, DOAM operates as a volunteer organization and therefore there see only incremental gains toward its goals. A volunteer coordinator, empowered by members, must be identified or step forward to manage a business, communication, and advocacy plan. As funds are available, the position should transition from volunteer to paid, which will accelerate the development of organic agricultural production. Regardless of the pace, there are viable activities that will sustain forward movement. This report proposes components of a communication and advocacy plan for DOAM, including a “Buy Organic, Locally Grown” campaign that will promote awareness and educate the public and media about organic agriculture and its benefits. Recommendations are made to develop a diverse base of funding for DOAM, including government funds, foundation or NGO grants, business initiatives such as organic farm supply sales, fees for membership, and organic certification or registration.

Based on an analysis of the current capacity and resources required to establish an organic certification program, an initial, simple registration program that includes DOAM and the Ministry of Agriculture is recommended. A registration program would serve organic farmers and processors by reducing the costs to organic entry, encouraging the adoption of organic practices, building capacity in DOAM and within the government, preparing producers and processors for organic certification requirements, and eliminating the need for two (or more) certifications for export products. Ideally, the Bureau of Standards would take up organic certification and inspection when mandatory national organic legislation is in place. Group certification and a regional organic certification and inspection program are alternatives to a Dominican certification program.

Revisions to DOAM standards are proposed based on the idea that organic standards should reflect the unique situation of Dominican producers and consumers and be oriented toward stimulating both organic production and marketing. Dominica’s standards should be established for internal markets but compatible with U.S. and EU standards so that producers and handlers interested in export markets could obtain the applicable certifications. Standards should primarily serve an educational function, and therefore are proposed as relatively simple and readily understood. The proposed standards do not attempt to encompass all possible sectors of organic production and processing, especially those that are minimally developed for non-organic agriculture on the island. Livestock standards are intended to be attainable by small producers who sell only to local markets.

Local market opportunities exist for Dominica as much of the fresh fruits and vegetables purchased by the local population and by retail stores, hotels, restaurants, and resorts is locally grown. With no strident opposition to purchasing organic at a reasonable price, there is a ready market, especially for eco-tourists who come to Dominica, the “Nature Island.” For fresh organic produce, the neighboring Caribbean islands — especially those with Green Globe-certified hotels or eco-tourist resorts — are a prime opportunity for Dominica. Competition and lack of infrastructure limit other export markets for fresh organic fruits and vegetables, but processed organic products may be able to find access to the United States and the EU, as well as to neighboring islands’ markets.

The consumers that drive trends are shifting from an interest in anything organic to crafted, artisanal, or unique products that are organically produced. This is the niche for processed or preserved organic products from Dominica. Organic medicinal or herbal teas, dried fruit specialty products (fruit leathers and roll-ups), rum, ornamental flowers, and cocoa are recommended for further exploration and development. Partnerships with entrepreneurs, U.S. or EU companies, and other development initiatives — such as COTS’ Business Support Component and the New Orleans: Dominica/Organisation of Eastern Caribbean States (OECS) Trade Initiative — must be pursued to break down technical, financial, and market barriers.

Dominican organic production can find success, but it will require enthusiasm, persistence, and a willingness to move beyond the comfort zone.

Caribbean Open Trade Support (COTS) Work Report for Green Mountain Flowers

The Caribbean Open Trade Support (COTS) Program of USAID/Barbados is designed to facilitate the transition of selected Organization of Eastern Caribbean States member states to open trade, and to enable the countries to compete more successfully and be more sustainable in the global economy. To this end, COTS will implement a wide range of activities over a four year period geared towards:

  1. Enabling businesses to compete more effectively in the global economy
  2. Enhancing public-private sector interaction and dialogue leading to improved public policy dialogue
  3. Assisting government institutions and agencies to remove administrative barriers to the growth of the private sector
  4. Increasing the private sector and the media’s understanding of the challenges and opportunities of the new trading regimes
  5. Increasing the region’s resilience to natural disasters
  6. Mitigating the impacts of human disturbance on ecosystems and improving the institutional framework for managing protected areas

The COTS Program provides business advisory services under its Private Sector Development Component to client firms to increase sales. COTS assist clients to identify and remove constraints inhibiting sales growth through focused and cost effective support. Specific support from the Program is tailored to removing the constraint or “bottleneck” that inhibits a specific business transaction. Services range from business mentoring and networking assistance to providing local, regional or international consultants. These services that are provided will result in:

  • Significant sales generation for assisted firms in Dominica is one of the primary objectives.

Dominica is presently faced with numerous challenges as they seek to develop the main sectors, namely Agriculture and Tourism, which constitutes the building blocks of our national economy. More competitive enterprises in a number of clusters such as tourism, ICT, food and beverage, agro-processing, light manufacturing, entertainment, financial services, and advertising and public relations (PR) Streamlined, more efficient, pro-biodiversity and more environmentally friendly firms need t be developed. Enhanced private sector knowledge of, and access to, global markets, resulting in increased and sustainable exports. Agriculture seems to be the worst affected and is becoming increasingly difficult to compete at the local, regional and international levels. A primary contributing factor to this problem is the lack of new business involvement in this sector. In this competitive world environment, there arises an urgent need to inject new ideas and technology into the sector if it is to attain sustainability.

This assignment seeks to assist the Client to meet increased demand for its products with the design and installation of an effective water distribution system and implementation of efficient production systems. Also recommendations (See item 7) were discussed and listed below for further expansion, increased volume and effective sales and marketing.

Caribbean Open Trade Support (COTS) Impact Analysis Assessment

Impact analysis assessments are designed to carefully determine how impact, i.e. social or economic change that ultimately results from project activities, can and will be demonstrated, measured, and analyzed during the course of the project. Impact analysis seeks to guide USAID in telling a credible and compelling story about project impacts on the societies and economies in the countries that USAID assists.

This report reviews the components of the COTS project in order to identify opportunities for impact analysis — where analysis includes the identification, measurement, and attribution and may involve simple data collection or more complex studies. This report makes a variety of suggestions of specific measures and studies and of areas of further inquiry where impact analysis may be possible.

This report is not intended to be prescriptive so much as to start the project team thinking and planning around impact analysis. The next step is for the project team to determine which analysis it wants to undertake and then identify and assign resources for those efforts.

This assessment explores a variety of ways to identify and assess impact. In some cases, there may indeed be a quantifiable indicator of impact, for which data can be collected on a periodic basis. In other cases, the project may be limited to case studies or individual interviews in order to draw direct links between project activities and the kinds of behavioral changes that we can call outcomes (as opposed to inputs and outputs). For all components, this assessment presents ways that the impacts can ultimately be monetized so that a return on investment indicator can be calculated.

A general conclusion in this report is that the project will only be able to measure and analyze impact in a comprehensive way if impact analysis is integrated into the project activities for each component. COTS is a complex project composed of a variety of activities, many with separate and distinct objectives. While a few possibilities exist for indicators that can capture impact across a set of activities, in many cases attribution will only be possible if project staff looks for ways to identify impact at the activity level. This will require follow-up efforts for activities; for example, interviews of training participants some time (say, three months) after a training to determine whether their behavior has changed consistent with the training. As noted above, some of these analyses will not produce “indicators” but rather case studies or impact success stories that can still be enormously useful for the project to demonstrate impact.

An integrated approach will require all members of the COTS staff to be involved. The M&E specialist can assist by coordinating, monitoring, and compiling analysis, but each technical specialist will need to be involved in identifying impact and collecting data and information to analyze it. This need not be overly time consuming, and again, project staff will need to decide which impacts it wants to capture.

It should be noted that the measures considered in this report that can be monetized are, in most cases, not economic value-added indicators. That is, just because they are measures that can be represented in dollar terms (and therefore compared to the dollar cost of project activities), they do not necessarily represent increases in GDP. They should each be reported for exactly what they are and not summed unless otherwise recommended in this assessment.