Vermont’s Lights Out Data Center: No One’s Home Nights or Weekends

The data center for the state of Vermont provides application and storage for the services provided by the state’s government, from processing registration information for the Department of Motor Vehicles, to ensuring that enrolled citizens benefit from the state’s food subsidy program.

In an effort to cut costs, Joe Ng, the information-technology manager responsible for the government’s data center, embarked on a program two years ago to automate management of the facility, moving from operation around the clock to locking the doors and turning off the lights between midnight and 6 a.m. on weekdays and for the entire weekend.

[ For timely data center news and expert advice on data center strategy, see CIO.com’s Data Center Drilldown section. ]

Given the critical nature of the services, going “lights out” for almost half the week was not an easy sell, Ng says.

“It took a long time to convince people, and the resistance was pretty stiff,” Ng says. “But everything went well, and people are really behind it now.”

After six months operating in its lights-out mode, the data center group was able to cut staff by more than 40 percent and significantly reduce costs.

Automating data center operations and allowing for remote management—the core characteristics of a lights-out facility—are a natural fit for many companies that want to save money and increase security. In 2006, Hewlett Packard heavily pushed the lights-out concept for data centers, and since then it has quietly taken off, says Gary Thome, chief architect for HP’s Infrastructure and Blades group.

“Most customers are really serious about wanting to go lights-out, to fully automated data centers,” Thome says. “The ideal is to just be able to sit around and wait for something to happen.”

Here are five key lessons Ng learned in his Vermont lights-out project.

1. “Lights out” is not all or nothing

While some companies may aim to depopulate their data center around the clock, for most companies being lights out is not a black-and-white concept. The definition of lights out is not zero population all the time, says Ng.

“The concept extends from totally nobody there to lights out for a certain period of time,” Ng says. “For the State of Vermont, we cannot go lights out 24-by-7.”

Increasingly, data centers are being designed assuming that fewer, or no, people will work in them, says Joseph Pucciarelli, program director of technology, financial, and executive strategies for business intelligence firm IDC.

“You used to have a data center with a lot of office space around it filled with IT people,” he says. “Now, you don’t see a lot of office space because they don’t want to have a lot of people in the data center.”

2. Make sure you have support

Working as part of the state government means listening to a lot of constituents. Vermont’s Ng had to get support from not only his supervisors, but also the labor unions that represent the state’s workers.

The result: while his business groups saved money by reducing the number of workers in the data center from 17 to 10, those people were not fired, but moved to other positions.

“Nobody lost their job,” he says. “They got shifted to a new department or agency.”

3. Fewer people does not mean less oversight

While some people might think that having no one physically in the data center would mean a loss of oversight, just the opposite is true, says Ng.

Moving to extended periods with no one allowed in the data center gives IT departments a better handle on what is happening. With various systems in place to notify the data-center group of any incidents and better monitor operations, IT workers have more information, not less.

“By going through the exercise, we have a much better awareness of the system,” Ng says. “When things go bump in the night, we have e-mail notifications. We are much more in tune with the system now.”

4. Fewer people means more security

In addition, a lights-out data center removes the biggest source of errors and threat to the facility: people. While workers are need to maintain servers, removing humans from the equation helps security greatly, says HP’s Thome.

“The most unreliable component in the data center is the people,” Thome says. “Pull the wrong cable and, oops, something goes down that was not supposed to go down.”

The State of Vermont’s IT chief agrees.

“We are much more secure because people are not walking in and out during the night,” Ng says. “The more people that walk into a data center, the more security is a problem.”

5. It doesn’t have to be expensive

The state of Vermont wanted to save money, so a plan that cost a lot to implement would not have worked. Rather than buy a state-of-the-art management system, Ng used the software provided by the manufacturers of the data center’s cooling systems and uninterruptible power supplies.

“We put it together using different systems,” he says. “In the end, we did not spend a lot of money upgrading things. We used what we had at our disposal.”

In the end, what initially was a hard sell has become second nature.

“In about six months, people won’t even remember we were ever 24-7,” Ng says.

Follow everything from CIO.com on Twitter @CIOonline.