Helping Clients Make Evidence-based Decisions via Rigorous Evaluations

Cloudburst excels at conducting rigorous evaluations of interventions to help clients make more evidence-based decisions, and targeted and sustainable programming investments. One such example was our work with USAID’s E3/Land and Urban Office to design and implement an impact evaluation of the Tenure and Global Climate Change (TGCC) project in Zambia (2014-2017). This project supported agroforestry extension services including establishing farmer groups trained in agroforestry and distributing seedlings, as well as land tenure interventions including boundary demarcation and land administration support. Prior to this evaluation, there was little or no evidence on whether granting customary documentation to a farmer makes her or him more likely to adopt climate smart agricultural practices.

The Cloudburst evaluation team collaborated closely with USAID and program implementers to develop a robust evaluation design. The evaluation was designed as a randomized-control trial, the gold standard of evaluations, to assess the direct and joint impacts of the agroforestry extension intervention and tenure security strengthening intervention across various outcomes. Drawing on extensive survey design experience across our entire evaluation portfolio, Cloudburst designed both quantitative survey instruments and qualitative protocols, with special consideration given to vulnerable populations such as women and youth.

Cloudburst uses cost-effective electronic data collection systems for all our surveys, which improves data quality by reducing survey errors in the field, allows real-time monitoring of data and reduces costs. As part of this evaluation, Cloudburst also built local capacity through training about 100 local enumerators and utilized the ability of electronic data collection tools to record geographic coordinates to allow for geospatial analysis. After analyzing data from 285 communities, Cloudburst produced a detailed report for USAID and program implementers that elaborated findings and policy recommendations.

From this series of evaluations, there is strong evidence that the TGCC process of boundary demarcation and expectation of receiving paper documentation substantially increased perceptions of tenure security. Households were asked to assess the short-term (1-3 years) and long-term (4+ years) likelihood that each plot of land would be reallocated or encroached by various entities including their chief, a neighbor or family. These short-term and long-term measures were combined to create an index for perception of tenure security as shown below.

For the overall household sample, the agroforestry extension services were successful in motivating greater uptake of agroforestry practices, although the actual tree planting and seedling survival rates remain low. In villages that received the agroforestry extension services, vulnerable groups such as female-headed households, youth, elders, poor and land-constrained groups also show significant uptake in agroforestry across fields. The graphic below shows the increase in the average number of agroforestry trees per field across all treatment groups by each vulnerable group.

The last important finding is the weak but positive evidence of a link between strengthening customary tenure and enhanced field investments (such as planting basins, rotating crops, fallowing or fertilizer application) for households receiving the land tenure interventions. As expected, the analysis found no evidence of a TGCC impact on crop yields or livelihoods as these are long-term impacts requiring an additional round of data collection.

For more detailed findings, please see the TGCC final report or this interactive data visualization blog post produced for USAID.

Written by Nicole Walter, Evaluation Specialist. Nicole Walter contributes to the design and analysis for impact and performance evaluations of USAID-funded projects, most notably in the areas of land, land tenure security, rural development and natural resource management. Ms. Walter also coordinates Cloudburst’s geospatial analysis for international land tenure research and supports housing, community development, and disaster-related research projects across the USA.