From the Lab to the Marketplace Ten Years Later, Energy Efficient Technologies from Research at the Lawrence Berkeley National Laboratory Berkeley Lab logo (left) with six rows of gray dots transitioning to a line art drawing of a cityscape and residential houses.

Berkeley Lab Data Center Energy Efficiency Research

Data center energy consumption over the past decade has sparked increasing attention from data center facility managers, utility companies, policy makers, energy analysts, and businesses attuned to decreasing costs and achieving sustainability goals. In 2000, U.S. data centers used less than 0.12 percent of the nation's energy; by 2010, that figure had grown to about 2 percent. As more data centers spring up to accommodate the growing needs of our information economy, efficient management of this energy consumption will be essential to maintaining productivity and profitability, and to reducing carbon emissions.

Breaking Ground for Innovative Data Center Efficiency

Blueice Supercomputer (DI01615)

For more than a decade, Lawrence Berkeley National Laboratory (Berkeley Lab) has worked with the data center industry, other research organizations, and government agencies to find and implement solutions and to encourage their wider adoption.

In 2004, Berkley Lab gathered input from industry partners and associations, research organizations, energy consultants, and data center suppliers to develop a 10-year research roadmap outlining data center energy research needs. Years before, the Lab had begun in-depth benchmarking on actual facilities, to identify baseline energy use. Based on the work from those projects, Berkeley Lab set forth to characterize data center energy use (including IT equipment consumption). This helped to identify energy savings strategies and potential. The Lab has conducted numerous assessments that have identified potential data center energy savings of 20 to 40 percent.

Part of Berkeley Lab's data center work focuses on identifying, documenting, and promulgating best practices for data centers and cleanroom energy management nationally and internationally.

Energy Star logo U.S. Department of Energy logo

A 2007 report to congress on national baseline energy use and energy savings potential attracted substantial attention in the national press. Prepared for the U.S. Environmental Protection Agency's ENERGY STAR program, the report led to greater U.S. Department of Energy (DOE) and state-level interest in reducing data center energy use. It also led to greater interest in energy efficiency in the industry and to various industry initiatives such as formation of the Green Grid industry association.

Data Center Energy Efficiency Demonstrations

Data center energy efficiency demonstration projects are an important part of Berkeley Lab's data center efforts. In California, collaborations with private industry currently are showcasing energy gains from different energy savings strategies. The projects focus on energy savings from various technologies and strategies such as using direct current (DC) power distribution in data centers, various types of cooling systems, evaluation of higher-temperature cooling water systems that do not require chillers, and the use of wireless sensors to monitor environmental conditions and building energy use to identify energy saving opportunities.

An early demonstration of the use of DC power was hosted by Sun Microsystems1 and involved collaboration with large number of industry partners interested in seeing DC power adopted for data centers. This project evaluated (1) the effects of delivering DC directly to computers modified to accept it, and (2) the effects of converting AC to DC at the rack level and then distributing DC power directly to servers within the rack. More detailed results are available at the High-Performance Buildings for High-Tech Industries and DC Power for Data Centers of the Future websites.

A second DC power demonstration was held at the University of California, San Diego. There, DC power was distributed to computers in a container. This project again involved industry partners and it illustrated the ability to distribute DC power directly into IT equipment.

DC Power Demonstration Seeks to Reduce Data Center Power Conversions

DC power demonstration seeks to reduce data center power conversions

Berkeley Lab also demonstrates energy saving strategies in its own data centers. In a data center in Building 50B, Room 1275, a series of efficiency measures were undertaken:

  • Installation of a wireless mesh network for monitoring environmental conditions and energy end use
  • Creating a hot air return plenum in the ceiling
  • Ducting computer room air conditioners to the hot air plenum
  • Air management improvements (e.g., adding blanking panels, floor tile placement)
  • Hot aisle containment with strip curtains
  • Maintenance of computer room air conditioning (CRAC) units (replaced slipping fan belts)
  • Disabling humidification/de-humidification
  • Use of rear door radiator cooling with higher-temperature coolant
  • Use of prototype in-row cooler operating with higher temperature water

Researchers studied the improvement in energy performance as each measure was implemented. Selected measures have been documented in case studies such as the one described in the "Wireless Sensors Improve Data Center Energy Efficiency" technology case study bulletin.

Wireless sensor network with intelligent control software

SynapSense Web Console

Visualizing air systems (Click to watch a video.)


Novel Cooling—Clustered Systems

Another demonstration, in Berkeley Lab's National Energy Research Scientific Computing Center (NERSC) Center in Oakland, California, showed that large energy savings are possible by completely isolating the cold and hot airstreams for supercomputers. In one arrangement where cold aisles were isolated and air mixing was reduced, researchers lowered the fan speed and measured a six- fold energy savings in fan energy. In another study isolating hot aisles, three-fold fan energy savings were measured.

Berkeley Lab's research has identified increasing the efficiency of data center power distribution as a promising strategy for reducing electricity losses. Improving efficiencies for IT equipment power supplies, uninterruptible power supplies (UPS), power distribution units (PDU) and their back-up strategies, and DC power distribution have been targeted as energy savers. Berkeley Lab has evaluated server power supplies and UPS system efficiencies and has benchmarked performance of distribution systems in multiple data centers.

Quantitative data center research provides valuable input for efforts to refine metrics and measurement protocols and standards. Berkeley Lab is working with the U.S. Environmental Protection Agency's (U.S. EPA's) ENERGY STAR program, as well as with the DOE to support the establishment of guiding principles and the international harmonization of metrics for data center energy efficiency. The Lab contributed to U.S. EPA efforts to refine metrics and measurement protocols for benchmarking server performance and developed proposed criteria that were submitted to the U.S. Green Building Council for consideration. These recommendations were proposed for adoption into the Leadership in Energy and Environmental Design (LEEDTM) criteria for use with data centers.

Educating and Training Data Center Designers and Managers

Education is an important component of Berkeley Lab's data center work. Lab researchers strive to bring the knowledge they have gained over the past decade to the world, through case studies, training, and demonstrations. They has conducted workshops promoting best practices throughout the United States and internationally.

Berkeley Lab is developing tools and methods to assess the performance of existing data centers as well as showcasing best practices in its own facilities. It is implementing best practices in the design of the new supercomputer facility for the University of California, Berkeley.

Design Tools Build on a Solid Knowledge Base

In 2011, Berkeley Lab provided the technical content for DOE's Data Center DC Pro Software Tool Suite. This collection of tools enables data centers to identify and evaluate energy efficiency opportunities.

Since 2009, Berkeley Lab has been helping to develop the Data Center Energy Practitioner Program (DCEP) with DOE's Advanced Manufacturing Office. An outgrowth of DOE's Save Energy Now program, it qualifies individuals to evaluate the energy status and identify efficiency opportunities in data centers. In developing the certification program, Berkeley Lab established goals, identified participant background and training needs, and set up tests to qualify people. As of 2012, more than 200 people have received DCEP certificates and are working in the field to help achieve the program's goal of accelerating energy savings in data centers. Although the program is now administered by the private sector, Berkeley Lab continues to provide technical support.


CW pumps


Water-cooled processor

Berkeley Lab also provides technical assistance to the Federal Energy Management Program (FEMP) to help other federal agencies such as the Department of Defense (DoD), the U.S. Social Security Administration, the National Oceanic and Atmospheric Administration (NOAA), the General Services Administration (GSA), and others improve their data center efficiency.

The U.S. Office of Management and Budget is requiring federal agencies to consolidate their data centers in a move to reduce data center costs throughout the federal government. Berkeley Lab is helping federal agencies implement their consolidation plans and improve efficiency by participating in the agency consolidation task force that is focusing on this issue and by providing technical input to help optimize the consolidations.

Data center energy education also takes place outside of demonstrations. Berkeley Lab participates in the annual Silicon Valley Leadership Group (SVLG) Data Center Summit-a one-day event where energy-efficiency case studies are presented from a variety of stakeholders; primarily SVLG member companies.

Internationally, Berkeley Lab's efforts have included collaborating to create a public-private partnership in India to assist in market transformation and capacity building, and to identify and promote best practices throughout the world. Similar work in China focuses on raising awareness of efficiency opportunities. In Singapore, Berkeley Lab provided technical input for the establishment of their "Green Mark" evaluation criteria, which is similar to the LEEDTM criteria in the United States.

Uncovering Many Avenues of Opportunities

By taking a broad approach over the past decade that looks at data centers from a component level to a building level, Berkeley Lab's data center research has unearthed efficiency opportunities in almost every aspect of data center design and operations. The strategies identified have already saved data centers a great deal of time and money, and offer a large opportunity for savings for those yet to implement them.

In 2013, Berkeley Lab launched its Center of Expertise (CoE) for Data Center Efficiency. By combining previously disaggregated resources, the CoE will be a centralized source for innovative approaches to achieving high levels of efficiency in data centers. It will supply tools, best practices, analyses, and other resource designed to be utilized by those involved with data center design, commissioning, and operation. The Center will partner with public and private organizations to advance joint initiatives benefiting both sectors.

Related Articles

1 Sun Microsystems was purchased by Oracle after the demonstration.