Step 2: The second issue was the configuration of the racks themselves. Not all racks were fully populated, but equipment was always concentrated at the top of the racks, where it was subject to those higher temperatures. In fact, says Patterson, the hottest running servers tended to be mounted at the top of racks, where cooling efficiency was lowest. To address that, Scottrade had lowered the computer room air conditioning system temperature settings, in effect overchilling the rest of the room. "Scottrade was running the overall data center temperatures colder than necessary to keep the temperatures at the top of the racks within acceptable ranges," Graves says.

Step 3: Finally, the balance between the heat load produced by the server racks and the quantity of air supplied to the cold aisle was out of whack. Engineers redistributed the perforated tiles on the aisle floor to match the output required. "A thermal balance was noticed immediately," Graves says.

Air conditioning systems perform most efficiently when the air temperature differentials are higher, so Glumac implemented changes that made the cold aisles colder and the hot aisles a few degrees warmer. "We weren't optimizing the heat-to-cooling ratio that the AC units needed. You have to get that balance," Patterson says.

To address that thermal layer problem, Glumac engineers adjusted the CRAC system by raising the height of the air-return intakes by 1.5 to 2 feet. That pushed the thermocline layer above the tops of the racks, providing a better thermal environment for equipment located there.

Next up: Once the airflow balance was achieved in the aisles, engineers turned their attention to what was inside the racks. "There's an optimal temperature point where you want your chips running," says Patterson. Scottrade tended to have the hottest, most power-hungry devices in the top of the racks, where they received the warmest air. Scottrade reorganized the racks, moving power-hungry servers lower to balance the heat distribution within the racks.

It also helps that Scottrade's new data center is using energy-efficient servers. The 1U and 2U and Dell PowerEdge blade server models in use at Scottrade sport low-voltage processors, variable-speed fans that operate according to processing power consumption, and high-efficiency power supplies. (Those units come with embedded VMware on ROM, making setup easier.) "It draws less energy, and it keeps the internal temperatures in the boxes cooler," Patterson says.

But there's another advantage to newer servers that data center managers may miss: They run fine at higher operating temperatures than the previous generation of equipment. That means that server racks can run warmer. "Data center operators who take advantage of these higher-temperature capabilities can gain significant energy efficiencies in their cooling infrastructure," says Graves.