Performance indicators, like quotas, can get a bad rap. Our first crop of climate change standard indicators facilitated an understanding of the scope of the Global Climate Change Initiative’s activities. But some USAID program managers contend that sometimes they inadvertently fostered a tendency to aim low with the potential to let such indicators guide programming. This means, for example, becoming satisfied with measuring how many people were trained, rather than understanding and quantifying how training actually results in climate benefits.
Parsing out why some programs focus on demonstration projects rather than catalytic or systemic changes can be difficult. Yet at a presentation of draft findings by evaluators of the EC-LEDS program earlier this month, discussion centered on the need to track progress against higher-level indicators that can be mapped against the important climate outcomes that USAID programs strive to contribute to. Such indicators can help spur USAID managers to aim for big impact and sustained change.
In this context, monitoring and evaluation experts at USAID and Department of State are promoting a new generation of indicators, such as ‘projected GHG emissions reduced or avoided through 2030,’ which allow more ambitious, longer-term goals to compete for the attention of those designing and managing development activities. We are excited about the prospect of capturing the potential impact that comes to fruition well beyond the life of the project.
Granted, calculating GHG emissions reduced or avoided after an intervention ends can be challenging enough. We recognize that a multi-year estimate of what will happen in the future is more complex still. To support staff in exploring where and how to apply this new projected GHG emissions reduced or avoided indicator, USAID commissioned a series of case studies from its Resources to Advance LEDS Implementation (RALI) project to demonstrate how to use the Clean Energy Emissions Reduction (CLEER) tool for these calculations for some of our more common assistance approaches to promoting clean energy.
The first case study estimates the impact of the small-scale, hydroelectric and solar power activities under Indonesia’s clean energy program. Three more will follow in the near future—estimating the impact of solar photovoltaic, solar water heating, wind, energy efficiency and anaerobic digester activities (check back for updates).
Yet some of our best and most far-reaching work—efforts to reform policy and build capacity--remains hard to quantify. So the RALI project will also help us design methodologies for capturing these more abstract contributions to climate action. The third-party team currently working on a performance evaluation of the EC-LEDS program zeroed in on this approach and proceeded to project not just GHG emissions reduced through 2030, but also expected megawatts built and investment mobilized.
Attempting to capture the breadth or impact of a program can be fraught with peril. Using high-level, impact indicators like percent of energy supply coming from renewables, or rates of deforestation, invites criticism that we are taking credit for changes--or allowing ourselves to be judged by changes--that we do not control or to which we made but one contribution.
Using low-level, output indicators risks judging our work--and allowing our work to be judged--by changes that we can link directly and predominantly to our assistance program. The validity of the claim will be more solid, yet the result is likely to be underestimated. Combining performance indicators and higher-level indicators represents real progress and learning in capturing USAID’s contribution to climate benefits achieved through its development assistance.