EETD Study for SEE Action Finds Energy Savings From Residential Behavior-Based Energy-Efficiency Programs
A new State and Local Energy Efficiency Action Network (SEE Action) report prepared by Lawrence Berkeley National Laboratory (Berkeley Lab) researchers provides guidance and recommendations on methodologies that can be used to rigorously estimate energy savings from residential behavior-based efficiency programs. The Environmental Energy Technologies Division (EETD) authors designed the report for regulators, evaluation professionals, program administrators, and other energy-efficiency program stakeholders.
Residential behavior-based energy-efficiency programs have been identified as a potential major source of new energy savings. These programs are increasingly being implemented by energy-efficiency program administrators nationwide to help meet many U.S. states' energy saving targets and requirements. They utilize strategies intended to affect consumer energy use behaviors in order to achieve energy and/or peak demand savings. They typically use one or more elements to achieve their goals, including customer outreach, energy usage feedback, competition, rewards, benchmarking, or feedback elements. Such programs may focus on changes to consumers' habitual behaviors (e.g., turning off lights), one-time behaviors (e.g., changing thermostat settings), or purchasing behaviors (e.g., buying energy-efficient appliances).
However, the widespread adoption of these programs faces obstacles, including questions about whether observed energy savings are valid and attributable to the behavior program, the savings persist over time, and the results shown for one program can be applied to another program.
"We need rigorous, objective evaluation methods for these programs," says Malcolm Woolf, Director of the Maryland Energy Administration and a leader of SEE Action for issues related to evaluation, measurement, and verification (EM+V). "Strong standards ensure that program administrators, policy makers, and regulators can be confident that the savings estimates claimed by these programs are valid."
The guidance document identifies evaluation issues and recommends methods that ensure the validity of energy saving estimates.
"We recommend using a scientific experimental design method called a randomized controlled trial," says Annika Todd, the report's lead author. "This method is the gold standard for producing valid energy savings estimates that are robust and unbiased."
Randomized controlled trials randomly assign households into two groups: one that receives the behavior-based program (the "treatment group") and one that does not (the "control group"). This method allows evaluators to determine whether any energy saved by the treatment group was due to the program, as opposed to other factors.
The report discusses methods for ensuring that the estimated savings impacts are valid and robust. Top-tier recommendations include:
- Evaluation design: use randomized controlled trials
- Avoiding potential conflicts of interest: employ an independent third-party evaluator for program evaluation, assignment of households to treatment and control groups, and data selection and analysis
- Accounting for potential double-counting of savings: use rigorous methods to account for energy savings that may be claimed by multiple programs
- Ensuring precision in estimates: achieve 5% statistical significance
- Applying impact estimates to different populations in future years: maintain a control group that is representative of all of the different participating populations for every year in which program energy estimates are being used to claim savings
- In the future, a calibrated analytic model may be created to rigorously and validly predict program savings estimates
This guidance document is especially designed for state utility regulators, who are in a position to approve utility behavior-based energy-efficiency programs. "Speaking as a regulator, we need independent, objective methods that we can turn to, and this provides exactly that," says Phyllis Reha, Commissioner for the Minnesota Public Utilities Commission and a leader of SEE Action on issues related to customer behavior and energy efficiency.
State and Local Energy Efficiency Action Network. 2012. Evaluation, Measurement, and Verification of Residential Behavior-Based Energy Efficiency Programs: Issues and Recommendations. Prepared by A. Todd, E. Stuart, S. Schiller, and C. Goldman, Lawrence Berkeley National Laboratory.