Enterprise software has long been available, neatly packaged, for all core corporate functions. And now, in the age of cloud computing, it can be turned on and off like water from a spigot. Outsourcers run our data centers, technology is for sale at discount warehouse clubs. And still, some CIOs choose to build major software from scratch. Why?\n \n Creating competitive advantage is the goal of many IT innovation projects, and in some cases you can\u2019t achieve that without building your own systems. The New York Stock Exchange (NYSE) Euronext and its fierce rival Nasdaq OMX, for example, run custom-built trading systems that they continually tweak for stability and speed.\n \n Other companies elect to build software because they must support unique business processes. That is the case for Alcoa, a $21 billion company that manufactures aluminum sheets that will become products as varied as beer cans and airplane exteriors. \n \n Still others seek to invent new markets or ways to operate\u2014business ideas for which no ready-made software exists. Not everything in IT is a commodity like a box of 500 picnic forks. Consumer products distributor Eurpac, for example, built a logistics system to ensure that items like soft drinks and toothpaste reach customers on military bases in Afghanistan, and that these goods arrive still fresh and in good condition.\n \n \u201cThe [software packages] we saw are very good at making automated business decisions and supporting efficient supply chains in a typical environment,\u201d says Mike Skinner, Eurpac CIO. A war zone, however, \u201cis not typical,\u201d he says. \u201cPackages don\u2019t understand so many exceptions to the rules.\u201d\n \n Sometimes, all three objectives lead a CIO to custom development. Private equity powerhouse Kohlberg, Kravis and Roberts (KKR) wrote a business intelligence system to consolidate, harmonize and analyze monthly financial and operations data from the 70 companies it owns in nine industries. The software also generates metrics that cut across these diverse companies, to show how KKR, on the whole, is doing. No vendor offers this, says CIO Ed Brandman, so he had to build it. (For more on KKR's project, see "How KKR Used Data Mining For Competitive Advantage.")\n \n In general, Brandman says, \u201cthe pendulum has swung very hard to \u2018Why would you ever want to build anymore?\u2019 It was a major commitment by the firm to allow us to go down that road.\u201d\n \n But the payoff is big. Now top KKR executives call the system a competitive weapon. And that is the primary reason a CEO will approve a risky project to write software in-house, says Louis Gutierrez, a consultant at the Exeter Group. \u201cThe only thing they want to invest in is something no one else has.\u201d\n \n Building unique software is certainly not the main business of KKR, nor of Alcoa, Eurpac or NYSE Euronext. And they learned lessons along the way, including how to overcome the difficulties of forecasting project costs, how to choose the right IT staff for such strategic work, and how much collaboration and confidence managing a custom software-development project requires. \n \n \u201cBuilding is not easy,\u201d says Steve Rubinow, CIO of NYSE Euro\u00adnext. \u201cIf it were, everyone would do it and we\u2019d get no edge.\u201d\n Slippery Cost-Benefit Getting approval for a major custom project requires special handling. CIOs must prove that riskier in-house development is worth it, in cost or time savings as well as in competitive advantage, says Exeter Group\u2019s Gutierrez. \u201cIt is on this frontier border where innovative companies will always find the need to develop custom code. Not because they want to return to the days of big application-development shops, but because the stuff just doesn\u2019t exist yet to do what you want to do.\u201d\n \n \n \n And estimating costs and benefits can be like planning a big home renovation\u2014even when you think you\u2019ve accounted for everything, tearing down that first wall can unveil surprises that take your project, and budget, in unexpected directions. When implementing packaged software, even if it requires many modifications, costs are pretty clear. Just add up licensing and maintenance fees, plus the price of associated new hardware, labor, ongoing support and perhaps outside consulting.\n \n In a build-your-own situation, forecasted costs and potential returns are unclear because the project likely stretches a company into new markets or enables it to gain a new competitive edge. As Rubinow notes, it\u2019s hard to predict the value of something you haven\u2019t done yet.\n \n NYSE Euronext built its Universal Trading Platform with C and Java development tools in 2008. It now sells the system to other exchanges, but at the beginning, Rubinow says, there was no way to put a number on the new business. \u201cYou put some estimates and guesses into ROI, but it\u2019s hard to do that. There\u2019s greater risk and greater uncertainty.\u201d Yet systems at the heart of a business sometimes demand customized work, he says. \u201cYou can\u2019t buy those off the shelf and expect to be a leader.\u201d\n \n NYSE Euronext doesn\u2019t reveal sales figures for its trading platform, but its revenue-generating IT services group brought in $444 million in sales last year, up from $363 million in 2009. The exchange remains the biggest in the world, handling trades of 654 billion shares of equities in the United States last year, ahead of Nasdaq OMX\u2019s 475 billion. \n \nKKR also expected big returns from its finished product. Private equity investors certainly monitor the financial health of the companies they own, but KKR wanted to go further by applying analytics and proprietary formulas to that data to come up with a more accurate view of the overall performance of those companies. That is, find new ways to interpret financial indicators that are very different in different industries. For example, while inventory turns are key to toy retailer Toys \u201cR\u201d Us, the magic metric for electricity company Energy Future Holdings may be distribution costs as a percentage of overhead.\n \n \u201cThere aren\u2019t a lot of mega buyout, top-tier private equity, alternative investment managers in the world,\u201d Brandman says. \u201cThere are fewer of them who look to manage information the way we manage it.\u201d\n \n KKR ended up spending $2 million building the system. To estimate expenses before starting the project in 2008, the IT group worked backwards, building a detailed budget, Brandman says. They worked from an estimate for implementing packaged software that, although it didn\u2019t do much of what KKR wanted, was related to the project\u2019s goals. Then came estimates for modifications.\n \n Meanwhile, he projected how much time and effort his staff would need if it started from scratch. The team had built smaller projects with Java, MySQL and Crystal Reports, so Brandman extrapolated. Ultimately, he determined that his team could build a system for the same or less money than heavily customizing a package\u2014and it would do exactly what KKR wanted.\n \n While $2 million may seem a pittance for a company that manages $61 billion in assets, Brandman was concerned. In custom projects that aim to create technology that doesn\u2019t exist in the marketplace, he says, \u201cthere\u2019s a real risk costs will go out of line.\u201d\n \n To help ward off overruns, he kept the team small: three highly experienced developers and project leaders who had worked together for more than 10 years. They did no other work during development. More than half of the $2 million, about $1.6 million, went to labor costs, Brandman says, including bonuses and other compensation for making the project successful. \u201cThey knew this was going to have significant financial benefit if they delivered it. And if they didn\u2019t, there wouldn\u2019t be.\u201d\n Staffing RightNo matter how tricky the financials are on a custom project, staffing can be more so. The skills needed to plan and carry out the creation of a large, strategic piece of custom software may have atrophied since the days when IT built most of its own stuff, Gutierrez says.\n \n \n \n Now, however, because CIOs don\u2019t undertake major in-house development work unless the end product provides a strategic advantage, developers on the project must know more than C or Java, he says. They must understand enterprise architecture, business process management, and the broad business issues facing the company and its competitors.\n \n Brandman was lucky enough to have the three trusted senior developers already on his staff. Other companies have to look for new hires. Recruiter Kristen Lamoreaux, president of Lamoreaux Search and a columnist for CIO magazine, recently helped a large publishing company find a team leader with those kinds of skills to work on key mobile computing projects the company has planned. The publisher had contracted with outside companies to build multiple mobile applications. But because the company deemed mobile platforms critical to its business, the leadership decided to move development in-house, she says.\n \n NYSE Euronext views application development as so strategic that it won\u2019t let even small slices of key projects go to outside contractors. \u201cThere are very few things in most industries that are a sustainable competitive advantage. They erode. Smart people copy what you do,\u201d Rubinow explains. \u201cWe want to be very cautious and make it hard for our intellectual property to leak into another company.\u201d\n \n Rubinow competes for IT staff with Wall Street companies that offer larger salaries. To tap into different talent pools, the exchange set up major IT outposts in Chicago; London; Belfast, Northern Ireland; San Francisco; Paris; New York; and Orlando, Fla. Dispersed developers work together using Microsoft SharePoint collaboration software, videoconferencing and source code control software. \u201cThere\u2019s no substitute for two guys sitting shoulder-to-shoulder and talking out loud to each other. I would want that if I could have it, but it\u2019s not practical,\u201d he says.\n Manage DifferentlyWhen Kevin Horner, CIO of Alcoa, decides to do custom development, he makes sure his team understands both the business problem at hand and the expected payoff, such as financial benefits, competitive advantage or improved customer satisfaction. For Horner and other CIOs, framing the project appropriately at the start and repeating that message to everyone involved helps the work stay on track.\n \n \n \n Alcoa prefers to buy as much software off the shelf as it can, in part to keep IT costs at 1.2 percent to 1.4 percent of revenue, Horner says. But it will build in the right circumstances. For example, the company has augmented its Oracle manufacturing systems by building and integrating capabilities for needs such as monitoring the quality of the aluminum sheets it produces for the aerospace industry, which are made by a variety of machines of different ages. \u201cThe sheet for beer cans has different requirements than the aluminum that ends up being skin for the Airbus 380,\u201d Horner says.\n \n Alcoa also builds software to support aspects of its business where it believes it beats rivals, such as in certain metal-refining processes. IT and engineers code algorithms that control refining machines, he says. \u201cYou don\u2019t just walk into Oracle or SAP or Honeywell to purchase these things.\u201d\n \n \u201cThis kind of custom work should be focused only on innovation, not just because some department likes to do things their own way,\u201d Gutierrez says.\n \n Owens and Minor, an $8.1 billion distributor of medical supplies, manages custom development projects differently from those rolling out packaged applications. In 2006, the time came for Owens and Minor to modernize its core enterprise software. The system, a set of products written in Cobol, ran on outdated IBM mainframes. Rather than replace the ERP suite with packaged software, the company decided to move the system to HP servers running Windows.\n \n As Owens and Minor\u2019s CIO Rick Mears explains it, by rewriting existing software, 25 years worth of business processes and knowledge would not be lost or require expensive and time-consuming revamping. Craig Smith, CEO of Owens and Minor, estimates the project saved at least $100 million. That includes vendor fees and avoiding the disruption to operations that is inevitable when moving to a new vendor\u2019s ERP software, with its attendant business and workflow changes.\n \n Following some of the precepts of agile development, Owens and Minor developers and project leaders inside and outside IT held quick meetings, sometimes twice a day, to update each other on progress, Mears says. The idea is to be brief; no one sits down. The development group has also converted individual offices to be more open, public spaces that facilitate collaboration and impromptu conversations, he says.\n \n Eurpac\u2019s Skinner also advises that, if possible, companies should send application developers to the scene where the software will eventually be deployed or where data for it will come from. He hasn\u2019t sent anyone into Afghanistan because of the danger and travel restrictions, but IT staff have gone to Kuwait and Germany, key staging locations in Eurpac\u2019s supply chain, to observe loading consumer products onto boats and trucks.\n \n Here\u2019s an example of the sort of insight such excursions can bring: Eurpac products move out of Germany\u2019s Rotterdam Harbor on the Rhine River. A recent barge accident crippled traffic on the waterway for several weeks, forcing companies to get products onto trucks for transportation by land. \u201cUntil you\u2019ve been over there and have seen how the Rhine is integral to our supply chain, you don\u2019t know how important it is to build software that can handle exceptions,\u201d he says.\n \n With packaged software, it\u2019s clear how to schedule roll-outs\u2014after this module or that suite is tested, for example. With custom software, Skinner advocates an early launch where improvements are made as soon as users give feedback. After all, the team is presumably building the system to change how well the company works. IT shouldn\u2019t be timid about releasing software, even if developers feel more could be done.\n \n Developers can be perfectionists, however, reluctant to let go until every last zero is paired with its one. This can hurt the project\u2019s ROI, Skinner says. \n \n \u201cLet\u2019s say you envision software that makes you five times more efficient. But if you encourage the group to launch at the time when you\u2019re two times more effective, then you\u2019re living it. Then from there, decide next steps,\u201d Skinner says.\n \n The process of building these critical systems has not wed these CIOs to the idea of custom development\u2014or even to maintaining custom code forever. Horner continually scans vendor products for new modules that could replace those Alcoa built. He already intends to evaluate whether to swap out some custom accounts receivable software, which was built in the early 2000s, for a vendor version. \u201cTen years later, [packaged] software is more mature. We\u2019ll ask ourselves, \u2018Does it make sense to take our customizations out?\u2019\u201d In fact, Alcoa is now replacing several custom-built applications from 15 years ago that monitor safety incidents at its factories in real time. \u201cNow we can get that in standard software.\u201d\n \n Even within a custom project, NYSE Euronext might buy common components from vendors if it sees no business advantage in building the module, Rubinow says. And mixing in packaged pieces may get the project finished faster, so the company can ride the wave of benefits from a key custom system that much longer.\n \n For Brandman, a highly successful project built in-house carries psychological rewards for the development team. \u201cIt\u2019s a boost to the staff when KKR has competitors in to look at what the analytics system can do,\u201d he says, adding that the company has turned down opportunities to sell the software. Company founders Henry Kravis and George Roberts themselves lauded the system in public, crediting the development team by name, he says. \u201cThat\u2019s a big deal.\u201d\n Follow Senior Editor Kim S. Nash on Twitter: @knash99.