Last summer when Wendy Cebula was shopping for a new vehicle,\n energy efficiency and lower emissions topped her list of\n requirements, along with four-wheel drive (her family lives on\n a hill). Cebula, then CIO at VistaPrint, a $152 million online\n supplier of custom print services, eventually chose a hybrid\n model instead of the traditional SUV. Even though she\n didn\u2019t think the incremental savings on gas would make up\n for the higher sticker price, she says \u201cit was the right\n thing to do as a human being.\u201dBut as a corporate executive, Cebula, now VistaPrint\u2019s\n COO, can\u2019t lead with her heart. The right thing to do is\n whatever enables the business to grow. Those decisions come\n down to dollars and sense, not what\u2019s best for the\n planet. But every now and then, the two converge.In late 2005, Cebula noticed her data center\u2019s costs\n were growing. The company, whose operations are almost\n completely automated, was adding 100,000 customers a\n month\u2014and growing at nearly 60 percent a year. There was\n no sign the demand for data necessary to serve those customers\n would stabilize. When Cebula and Aaron Branham,\n VistaPrint\u2019s vice president of technology and operations,\n dug into data center operations in detail, they discovered that\n energy costs were rising significantly.If there were some way to lower power costs or increase\n energy efficiency, they could cut expenses. To Cebula, an avid\n recycler who tries to impart environmental awareness to her\n kids, doing so would be a double win. She could do\n \u201cwhat\u2019s good for the environment and what\u2019s\n good for the bottom line,\u201d she explains.Until recently, the environmental impact of the data center\n was largely ignored. But today, energy experts estimate that\n data centers gobble up somewhere between 1.5 percent and 3\n percent of all electricity generated in the United States. At\n the top of the range, that\u2019s about the amount of\n electricity it takes to power the entire state of Michigan for\n a year. Market research company IDC (a sister company to\n CIO\u2019s publisher) estimates that companies spent $26.1\n billion to power and cool servers worldwide in 2005.\n That\u2019s more than was spent to power all the commercial\n buildings in 17 states\u2014from Delaware to Florida and west\n to Texas, according to the Department of Energy\u2019s most\n recent energy consumption survey.And that\u2019s not all. According to the Uptime Institute,\n a consortium of companies devoted to maximizing efficiency and\n uptime in the data center, more than 60 percent of the power\n used to cool equipment in the data center is completely wasted.\n In fact, notes a recent study by the group, energy costs have\n replaced real estate as the primary data center expense.\n \u201cData centers that used to cost $10 million now cost $100\n million,\u201d says Jonathan Koomey, staff scientist at\n Lawrence Berkeley National Laboratory. \u201cThat kind of\n expenditure gets C-level attention.\u201dIt\u2019s garnered government interest too. A federal law\n enacted in December compels the U.S. Environmental Protection\n Agency to examine power consumption in data centers, evaluate\n what technology manufacturers are doing to increase energy\n efficiency and determine what incentives could convince\n companies to adopt more energy-efficient technology. The\n European Union is studying the level of carbon emissions from\n computer equipment. Down the road, local and federal\n governments in the United States and abroad may end up\n penalizing organizations that operate inefficient data centers,\n according to Rakesh Kumar, Gartner research vice president.The combination of financial, environmental and legislative\n pressure will force IT organizations to develop greener data\n centers, says Kumar. By 2011, Gartner predicts, a quarter of\n new data centers will be designed for maximum energy efficiency\n and minimum negative environmental impact. But what that means\n may vary by organization. \u201cThere\u2019s no generally\n accepted, standardized way to build a green data center,\u201d\n says Kumar.At VistaPrint, becoming green has proven easier than Cebula\n thought. The company bought more energy-efficient servers and\n improved utilization in its primary data center in Bermuda,\n steps that have reduced energy usage by 75 percent. As a\n result, the company expects to save nearly half a million\n dollars over three years and estimates it will reduce its\n output of carbon dioxide emissions by several hundred tons in\n this year alone. That\u2019s equivalent to taking more than\n 100 cars off the road for a year.\nVistaPrint also decided to locate a new data center in Canada, where hydroelectric power\u2014a renewable energy source\u2014keeps power costs stable and has potential to lower VistaPrint\u2019s electricity bills by another 70 percent. \u201cWe were able to reduce our footprint at a time when it was very important to us financially,\u201d says Cebula. \u201cAnd it\u2019s much more green.\u201d \nHigh-Density Power Surge \n\n Back in 2000, when VistaPrint founder and CEO Robert Keane\n moved company headquarters from Paris to Waltham, Mass., the\n fledgling operation was bringing in just over $6 million in\n annual sales (the company makes business cards and other\n printed products to order). During the next five years, the\n company (which is officially based in Bermuda) saw its revenue\n explode. VistaPrint outgrew its 7,000 square feet of suburban\n Boston office space and moved a few miles north to a Lexington,\n Mass., location eight times larger.During those years, the company focused on automating and\n optimizing the product design and manufacturing process at its\n Venlo, Netherlands, plant. The only concern when purchasing\n equipment\u2014whether for the company\u2019s main Bermuda\n data center (which runs VistaPrint\u2019s website and\n transaction systems and is hosted by Cable & Wireless) or\n for the one in Lexington (which supports internal systems and\n IT production)\u2014was that the equipment work. As a result,\n VistaPrint procured a hodgepodge of servers: back-end systems,\n front-end systems and databases each ran on different gear,\n says Cebula. By 2004, VistaPrint got caught up in the blade\n craze, purchasing machines from IBM.In the fall of 2005, shortly before Cebula became CIO,\n VistaPrint\u2019s then-COO, Alex Schowtka, hired Branham as\n director of IT operations. Branham thought blades were a\n mistake. The problem was the total power pull. \u201cBlade\n servers look great on paper,\u201d Branham says, \u201cbut\n you start piling them into a rack and suddenly you\u2019re out\n of power, you\u2019re out of AC.\u201dVistaPrint wasn\u2019t the only company that was getting\n burned by its decision to buy high-density equipment. With\n quality data center space priced at a premium, companies sought\n out more compact gear, and vendors happily obliged. \u201cThe\n focus was on getting as much computer power in as small a\n package as possible,\u201d explains Gartner\u2019s Kumar.\n \u201cEnergy was not part of the design mentality.\u201dBut blade servers need more power than less-dense hardware.\n A full rack of high-density servers requires 20 to 30 kilowatts\n of electricity, while traditional data centers are designed to\n provide 2 to 3 kilowatts per rack. Meanwhile, according to the\n Uptime Institute, using high-density equipment triples or\n quadruples facility cooling costs. Energy prices have risen as\n well.CIOs who assumed that data center costs would decline as\n servers got cheaper and more powerful received a rude\n awakening. In what Kenneth Brill, founder and executive\n director of the Uptime Institute, calls \u201cthe meltdown of\n Moore\u2019s Law,\u201d the energy required to power and cool\n $1,000 worth of server equipment has skyrocketed from 8 watts\n in 2000 to 109 watts today\u2014eroding some of the benefits\n from more powerful chips. In the best-case scenario, says\n Brill, within five years it could take 157 watts to run the\n same $1,000 worth of hardware; in the worst case, 1,650 watts.\n In fact, the blade server\u2019s selling point\u2014its\n size\u2014has become irrelevant for some customers. Data\n center operators find themselves using only two servers in a\n rack designed for 10; packing any more than two high-density\n servers in a rack makes them too hard to cool.That was VistaPrint\u2019s quandary. It was the first Cable\n & Wireless customer in the Bermuda data center to install\n blades. And the outsourcer was none too thrilled. Cable &\n Wireless wanted to limit VistaPrint to one blade server per\n rack. VistaPrint got the vendor to agree to placing two servers\n per rack, but talk started of an energy surcharge. For years,\n most outsourcers charged for data center space based on square\n footage\u2014a metric that did not take into account\n electricity costs. As a result, says Lawrence Berkeley\u2019s\n Koomey, the outsourcers created a perverse incentive. \u201cIf\n you charge by the square foot, of course the customers want\n fully packed racks,\u201d Koomey says.\nA Virtual Solution \n\n For Branham, joining VistaPrint in September 2005 was like\n \u201ca trip back in time.\u201d He had spent the eight\n previous years at Monster.com and had seen data center\n operations expand from one server to 1,000. He had spent the\n past year working on server virtualization. But Branham saw\n bigger challenges at VistaPrint. \u201cFrom afar it looks like\n a print company. But once you get inside and see what\u2019s\n really going on, it\u2019s really a technology company,\u201d\n Branham says, noting that everything from sales to\n manufacturing to shipping is run largely on custom-built\n software. \u201cAt Monster,\u201d says Branham, \u201cwe\n weren\u2019t anywhere near this data-focused.\u201dLike Monster in earlier days, VistaPrint wasn\u2019t making\n the most efficient use of its data center. \u201cThey picked\n technologies that were right at the time, not right for where\n they were going.\u201d Branham knew that virtualization would\n reduce power consumption. But Cebula knew that she\n wouldn\u2019t convince anyone to spend more money on servers\n without a solid business case. So Branham ran the numbers,\n focusing on power expenditures.Calculating the total energy consumption of a server\n isn\u2019t straightforward. How much power a server consumes\n depends on how you use it. \u201cIt\u2019s not something that\n our vendors talk about a lot,\u201d says Cebula. The vendors\n were, nevertheless, able to provide data when asked. Using that\n information, Branham made the case for a greener data center\n crystal clear.Branham calculated that if Vista\u00adPrint continued to use\n blades, this solution (conceived of as four racks housing eight\n IBM blade servers) would eat up approximately 32,000 watts\n (4,000 per server) and require 9.1 tons of air conditioning to\n cool. He estimated that the alternative, using eight HP\n Proliant DL 585 rack-mounted servers and 110 VMware instances,\n would require 5,500 watts (50 watts per virtual machine) and\n just 1.6 tons of AC. In addition, virtualization would enable\n VistaPrint to make better use of its CPU capacity. As is the\n case in many companies, VistaPrint was wasting additional\n energy by using only 20 percent of its server capacity. In a\n 24-hour period, the typical x86 server is used to only 5\n percent or 10 percent of its capacity, says Gartner\u2019s\n Kumar, while energy consumption even in idle mode is 60 percent\n to 80 percent of the energy consumed in use.But the development team was concerned about stability and\n performance in a virtual environment. A year earlier, the IT\n group had attempted virtualization on the blades. Running\n VMware on the high-density hardware had been a bust. Memory\n limitations on the blades limited performance and the number of\n instances that could be run on each server. So when Branham\n used the \u201cV\u201d-word again, the development group got\n nervous. Branham won them over with a pilot project that proved\n the performance case. Swapping in the new servers in the\n Bermuda data center would mean eating the investment VistaPrint\n had made in the blades. But Cebula and the executive management\n team couldn\u2019t argue with the ROI. Over three years, the\n greener solution would save VistaPrint $450,000\u2014more than the cost of the hardware refresh.\nCleaner, Cheaper\n Electricity \n\n By the fall of 2006, competition over parking in\n VistaPrint\u2019s Lexington lot led its leaders to conclude it\n was time to move again, to a bigger space across the\n street.As part of its earlier move in the summer of 2004, the\n company had inherited a data center in the basement of its\n existing building. The \u201cfree data center,\u201d as\n Branham calls it, housed 18 racks of equipment by the time he\n arrived. Most of the gear was used for internal IT systems and\n development. They\u2019d probably want to move that operation\n across the street with them. Or would they? The rework of the\n data center in Bermuda got Cebula and Branham thinking. If they\n could cut energy costs by making different gear choices, was\n there anything to be gained by moving the bulk of the Lexington\n data center elsewhere? While researching the Bermuda data\n center revamp, Cebula had learned that the power market in\n Massachusetts was perhaps the most expensive one in North\n America. \u201cOutside of maybe Hawaii,\u201d she adds.\n \u201cDoing those cost comparisons opened our eyes and got us\n to run some of the numbers on power at different possible\n locations.\u201dVistaPrint had recently opened a 68,000-square-foot\n manufacturing facility in Windsor, Canada, which could be\n expanded to accommodate a 100-rack data center. Cebula charged\n Branham with creating a business case for three options:\n building the new data center in the new Lexington space,\n building it in Windsor, or outsourcing it.Unlike most IT departments, Vista\u00adPrint had all the\n information it needed to make the analysis\u2014from real\n estate costs to electric bills\u2014at hand. The majority of\n IT shops are in the dark about their data center energy costs\n because facilities management pays the bills. Facilities and IT\n tend to operate on different planes. \u201cFacilities people\n know how much power is being used, but they don\u2019t know\n anything about the equipment,\u201d says Koomey. The result is\n what the Uptime Institute\u2019s Brill calls \u201cthe\n invisible crisis.\u201d IT has little incentive to make more\n conservative choices about power consumption, and the results\n are not sustainable. AFCOM, an association for data center\n professionals, predicts that power failures or power limits\n will halt data center operations at more than 90 percent of\n companies over the next five years.\u201cNot a lot of IT people get down into the building of\n data centers,\u201d admits Branham. \u201cWe definitely\n understand the total cost perspective,\u201d says Cebula.\n \u201cBut even we were a bit surprised when doing the analysis\n of actual energy consumption.\u201dBranham\u2019s analyses revealed some good news. Real\n estate costs would be cheaper in Canada ($7 per square foot in\n Windsor versus $27.45 in Lexington). But Windsor was a greener pasture in more ways than one. VistaPrint\u2019s manufacturing\n facility is run on hydroelectric power, a renewable energy\n source that doesn\u2019t entail fuel costs and comes at a\n lower and more stable price. (In the Boston area, electricity\n comes largely from generation facilities that run on coal, oil\n or nuclear power.) That\u2019s why data hogs like Yahoo and\n Google have recently begun building operations in the Northwest\n near the Columbia River, a source of hydroelectric power.\n \u201cAny small difference in energy consumption or energy\n costs can make quite a difference,\u201d says Kumar.\n Hydroelectric power also creates little air pollution because,\n unlike fossil fuel\u2013fired plants, it does not generate\n carbon dioxide. Even nuclear plants generate some thermal\n emissions.Sure enough, thanks to a 12-cent per kilowatt-hour\n difference in electricity prices, Branham figured he would save\n more than $130,000 a year by locating in Canada. Those cost\n savings would improve as VistaPrint added racks to the\n facility\u2014a near certainty at its current growth rate.\n Coupled with the discount on real estate, the Windsor option\n proved 10 percent cheaper than outsourcing\u2014even with the\n capital expenditures.Not everyone can just up and move a data center to a cheaper\n real estate market with a renewable energy source.\n \u201cBanks, for example, may need to be in downtown Manhattan\n or central London,\u201d says Kumar. \u201cBut what\n [VistaPrint is] doing makes a lot of sense for\n them.\u201dThe Benefits of Green \n\n The Bermuda project (which was completed at the end of last\n year) enables VistaPrint to shave 25 percent from its future\n hosting costs. And VistaPrint\u2019s Windsor data center is\n scheduled to open this month. Branham plans to move many of the\n systems hosted in Lexington there (where the new virtual\n machines initially will occupy 10 to 12 racks), and the\n facility will also provide disaster recovery for the Bermuda\n data center (a service the company outsourced to Cable &\n Wireless). Cebula says VistaPrint is trying to make other\n eco-friendly choices, outside of IT. For example, the company\n bought an extraction system for its plant that recovers more\n paper waste, which VistaPrint can resell to recyclers.\n \u201cWhat we\u2019re finding is that the right thing to do\n environmentally and the right thing to do financially often go\n hand in hand,\u201d she says. Some green options, like the new\n servers and the paper waste extractor, may involve up-front\n costs, Cebula says, but they usually pay for themselves and\n reap returns (such as revenue from reselling the excess paper\n and lower energy costs). Of course, not every green product or\n process has a payoff for everyone. Some of the new,\n environmentally friendly innovations in cooling technology,\n heat transfer and power supply for the data center (see\n \u201cFresh, Green Tech, Page 40) didn\u2019t make the list\n at VistaPrint. \u201cI did look at the price for a natural gas\n generator,\u201d Branham says. \u201cBut the prices for\n natural gas fluctuate widely. Most people stay with\n diesel.\u201dDetermining the environmental benefits of VistaPrint\u2019s\n data center revamp is hard because of the complexities in how\n power is produced and consumed. But based on a model for\n calculating carbon emissions offered by Lawrence Berkeley, an\n average U.S. company that cuts its electricity consumption the\n way VistaPrint did in Bermuda would reduce its carbon emissions\n by nearly 612 metric tons.Meanwhile, the benefits of VistaPrint\u2019s data center\n changes go beyond saving money or saving the planet. Server\n standardization has made maintenance and upgrades easier.\n It\u2019s also made IT more responsive to business needs. It\n used to take days to provision a new server in Bermuda. Now it\n takes minutes. \u201cThe business is much more satisfied with\n the time it takes to make changes or get new capacity,\u201d\n says Cebula. \u201cAnd with our better failover capabilities,\n we have fewer reductions in service.\u201dVistaPrint recently had a VMware server go down in Bermuda,\n and all the instances that were running on it restarted on\n another server. Total downtime was less than 10 minutes, with\n no human intervention. The business doesn\u2019t understand\n why service is better, says Cebula, \u201cbut they understand\n the better service.\u201d And the money IT might have been\n throwing away on sky-high electric bills in Massachusetts can\n be fueled into more sound IT investments.\u201cGreen isn\u2019t costing us money. Green is saving\n us money,\u201d says Branham. \u201cInterests are\n aligned.\u201dSenior Editor Stephanie Overby can be\n reached at email@example.com.