Identifying Energy Efficiency Opportunities for Small Data Centers

A typical small server room; this one at Stanford University
January 2014

Glossy photos in magazines and on the web tend to portray server rooms as large spaces with gleaming, symmetrical rows of servers on temperature-controlled racks. In reality, however, 57 percent of U.S. servers are housed in small spaces such as server closets and localized data centers, in what are commonly referred to as small server rooms. Such spaces comprise 99.3 percent of all server spaces in the United States. In contrast to large, consolidated server operations, which pursue energy efficiency as a strategy to minimize their operational costs, these small, decentralized server operations do not, and because their configurations are site-specific, it is challenging to develop efficiency measures that can be used widely.

Given the great potential for energy savings, researchers from Lawrence Berkeley National Laboratory (Berkeley Lab) investigated how IT equipment was deployed, powered, and operated in small server rooms, and developed strategies to improve energy efficiency. The team surveyed 30 small server rooms across eight San Francisco Bay Area institutions, ranging from high-tech companies to academic and health care institutions, local governments, and a small business.

"We took a half hour to walk through each space with the owner or operator and collect data on how the equipment was configured and run," explains Environmental Energy Technologies Division (EETD) researcher Iris Cheung. "Our hosts also provided background information on the room, to give us a sense of how the current configuration came to be. This helped us to identify potential barriers to energy efficiency improvements."

Common Attributes, Missed Opportunities

Some commonalities arose as server spaces were surveyed; the most prominent being that most were not originally intended to operate as server spaces, and therefore, the efficiencies inherent in dedicated server spaces were not present. Many also suffered from principal-agent problems, meaning that the utility bill was paid by someone other than the server operators or owners. In addition, server energy use was often not submetered, so server operators received little or no feedback on their energy cost, and therefore had no incentive to pursue energy-efficiency improvements.

Also notable was that company business operations often took precedence over energy efficiency concerns. These priorities were reflected in limited IT budgets that left older, less-efficient equipment in place longer. And even though consolidating the servers could have greatly improved energy efficiency, the motivation to do this was low because often the equipment was not used regularly, they couldn't visualize the potential cost savings, and/or the server owners wanted to keep the equipment close to them.

Not surprisingly, server cooling turned out to be an area that offered potential for great improvement. Many server room temperature set points were lower than necessary, so energy was wasted by overcooling. Some used dedicated cooling systems that ran 24/7, even at temperate climates, while others set the building's cooling lower than it otherwise would be, to ensure that the ambient air was cool enough to cool the servers. Few had separate hot and cool areas to minimize hot/cold air mixing and improve cooling efficiency, as is often the case in large server spaces, and none took advantage of cooler outside air to reduce the amount of mechanical cooling required.

A Closer Look Reveals Significant Inefficiencies

Once the surveys were complete, the team selected four sites-one at Berkeley Lab, one at the city of Walnut Creek, and two at Stanford University-for detailed assessments. "We chose these spaces because they broadly represented other small server room configurations and had high potential for efficiency improvements, good site access, and operator interest."

The goal of assessing those four sites was to examine the IT infrastructure and systems in more detail. Using data-logging power meters on the circuits that supplied power to the equipment, the team measured IT, cooling, and other power-consuming equipment in each space, to determine actual power consumption and efficiency opportunities. To calculate power usage effectiveness (PUE), the researchers measured total server room power use, including lighting, power distribution, and uninterruptible power supply (UPS) losses wherever possible; estimating power consumption or losses if measurements were not possible (see table).

"We found that PUE values ranged from 1.5 to 2.1," says Cheung. "So in the upper range, the server room's total power usage was about twice the amount of power used by its IT equipment."

PUE Breakdown for the Four Sites
Server Room Stanford, University
333 Bon Air Siding
Stanford, University
Alumni Center
LBNL 90-2094 City of Walnut Creek
Cooling, kW 8.5 1 5.5 2 3.3 1 14.9 1
Lighting, kW 0.8 2 0.1 2 0.1 2 0.1 2
UPS loss, kW 1.8 2 1.7 2 0.1 2 1.3 1
Total load, kW 21.3 17.2 10.4 31.3
PUE 2.1 1.8 1.5 2.1

1 Directly measured
2 Assumed or estimated

Identifying Energy Efficiency Strategies

Because businesses and institutions using small server rooms often have limited resources, the project first focused on low- or no-cost measures for improving their server room's energy efficiency, which included raising cooling set points and better airflow management. More involved but still cost-effective measures included server consolidation and virtualization, and dedicated cooling with economizers.

Cheung explains: " We found that inefficiencies mainly resulted from organizational rather than technical issues. Because of the inherent space and resource limitations, the most effective measure is to operate servers through energy-efficient cloud-based services or well-managed larger data centers rather than server rooms. Otherwise, backup power requirement, and IT and cooling efficiency should be evaluated to minimize energy waste in the server space. Utility programs are instrumental in raising awareness and spreading technical knowledge on server operation, and the implementation of energy efficiency measures in small server rooms."

Spreading the Word

To reap the significant energy and cost-saving potential for small server rooms, it's necessary to communicate the energy-efficiency benefits (and how to achieve them) widely across the sector. So the team also presented its findings to server room operators, data center energy-efficiency professionals, industry organizations, utilities, and product vendors, with specific efficiency measures that could be applied to other server spaces. Collaborating with Stanford University and the Natural Resources Defense Council, Berkeley Lab also developed a simple and more detailed version of a fact sheet that summarizes energy-saving solutions for small server room owners and operators (see below). In addition, they conducted workshops at data center conferences.

"Much of the inefficiency in small data centers can be traced to a lack of education about energy-efficient equipment and operation among server owners and operators," says Cheung. "By training operators in energy-efficient IT, cooling, and power distribution, and by facilitating increased energy-efficiency visibility, server room energy efficiencies could improve significantly. We hope to build on this work by evaluating more of these spaces, identifying better, cost-effective tools to track server utilization, and by developing case studies that operators can use to increase the energy efficiency of their server rooms."

Cheung, H. Y. Iris, Steve E. Greenberg, Roozbeh Mahdavi, Richard Brown, and William Tschudi. 2013. Energy Efficiency in Small Server Rooms. California Energy Commission.

Mark Wilson