There\u2019s a sense of relief in Scott Testa\u2019s voice as he talks about terminating the last of his company\u2019s offshore outsourcing contracts this summer. \n\n\n\nAs COO and CIO of Mindbridge, an intranet software provider, Testa has overseen engagements with a handful of Indian IT service providers since 1999. In the beginning, the lure of lower costs from offshore outsourcing was hard to resist. And indeed, through 2002, Testa couldn\u2019t have been happier with the results. He was saving his mid-market company 30 percent on the application development and maintenance work he otherwise would have sourced domestically. But by 2003, Testa began to see the benefits slip away. Staff turnover at the Indian vendors increased. The quality of work on offshored projects decreased. And Testa\u2019s internal staff was growing weary of the time and travel required to keep the relationships on track.Things finally reached a breaking point last year. "[Offshore outsourcing] made a lot of sense for us at one time," says Testa, who will sever ties with the last remaining Indian vendor in June. "But it made a lot less sense for us in 2003. And by the end of 2004, it was right there in our face. It just wasn\u2019t nearly as cost-effective\u2014or effective\u2014for us anymore. We\u2019d get better quality and lower costs by doing the work domestically."Testa\u2019s experience is a sign of the outsourcing times. In the late \u201990s and early part of this decade, many CIOs jumped on the offshore outsourcing bandwagon. They were either feeling the lure of potential savings or being pushed by CEOs or boards with similar dollar signs in their eyes. A surprising number of companies ended up going offshore first and figuring out a strategy later. Now, as marriages arranged during the heyday of offshore outsourcing have matured, offshore outsourcing satisfaction rates have dropped. Last year, IT consultancy DiamondCluster International reported that the number of buyers satisfied with their offshoring providers fell from 79 percent to 62 percent, and the number of buyers prematurely terminating an outsourcing relationship doubled to 51 percent. Also in 2005, PricewaterhouseCoopers found that half of the financial services executives it surveyed were dissatisfied with offshoring.Several years into the craze, expectations about offshoring have come crashing down to earth. In a recent study of offshore outsourcing results among financial services companies, Deloitte Touche Tohmatsu discovered that although offshore performance during the first few years was consistent with expectations, many companies encountered an alarming drop-off in both cost savings and quality after three years. "It is a lot of work to manage these relationships. If you don\u2019t put the resources and the work into managing this relationship well long-term, you are destined to have issues," says Testa. "That, quite frankly, is what happened to us." Philip Hatch, founder of offshore outsourcing consultancy Ventoro, has also found a maximum ROI point occurs sometime before the first three years are up. "After you hit that [three-year mark], unless there is some additional external force, things get stale," says Hatch, who worked for Russian outsourcer Luxoft from 2000 to 2003. "Turnover rates on the outsourcing team pick up. The methodologies and tools that worked well in the beginning become obsolete. And other soft costs creep in," he says.CIOs who are just beginning to evaluate the offshore option could benefit from the lessons learned by those who have gone before them. Even before they make the decision to offshore, CIOs should factor in the costs involved in keeping a long-term offshoring relationship from becoming stale. Smart CIOs have figured out that continuous tweaking and constant attention, as well as developing the right metrics for judging performance, are keys to long-term offshore success."Unless an outsourcing engagement goes through some kind of reinvention, by five to seven years out, you\u2019re going to see no material cost savings over what you would pay to do the work yourself," Hatch predicts. "The outsourcing engagement will be obsolete."The Honeymoon YearsHatch compares the offshore cycle thus far to the dotcom boom and subsequent bust. In the early part of the decade, "people started screaming offshore. We saw CIOs being forced into offshore outsourcing relationships because of their boards, or executives selecting vendors solely based on their hourly rates," says Hatch. "Ninety-nine percent of them had no real business plan.""A lot of these initiatives were driven by costs, essentially replacing expensive labor with cheap labor," agrees John G. Schmidt, president of the Integration Consortium, whose experience with offshore outsourcing dates back to the 1980s when he worked in the Consulting Services division at Digital Equipment Corp. "But the problem is making [offshore outsourcing] sustainable. The U.S. business mind-set is, What can I do in January to save money this year?"Although many IT leaders understood the effort required in the first year or two to launch an offshore outsourcing relationship, many were not prepared for the long-term effort it takes to sustain the value proposition. "People thought that outsourcing was a panacea to cure all ills," says Testa of Mindbridge. "But we found that while it solves some problems, it causes others."When Joe Drouin was promoted to CIO of car parts manufacturer TRW Automotive at the end of 2002, he inherited his predecessor\u2019s offshore outsourcing relationship with Indian vendor Satyam. The arrangement had originally been sought three years earlier by the company\u2019s then-new CEO. For awhile, the deal had worked well, Drouin says, but by the time Drouin came aboard it was beginning to show some signs of wear and tear. Some projects were still successful; others weren\u2019t. The relationship was managed project by project, using a very formulaic approach for gathering user requirements. "It took a lot of effort to get it right," says Drouin, "and it didn\u2019t always happen." The result: missed deadlines, blown budgets and rework. And the developers Satyam assigned to TRW\u2019s projects were a mixed bag. "Some were good and some were not," recalls Drouin. "And we\u2019d lose the really good ones whenever a project would finish. So every time we had a new project, we had to start all over with someone who had to learn the TRW environment." Drouin renegotiated the contract with Satyam to provide a dedicated offshore development center that would engage vendor staff in long-term commitments to address the turnover issue and remove some of the risk of managing the work project by project. "We started building up a good base of TRW-specific and automotive domain knowledge," says Drouin. "We didn\u2019t have to reintroduce ourselves or start from scratch each time." And for the first year after reorganizing the relationship, the offshore outsourcing worked swimmingly.But it wasn\u2019t to last. Romance FadesIn the beginning of any offshore relationship, both customer and vendor expend a Herculean amount of effort to get the relationship up and running smoothly. "The new customer signs on. There\u2019s lots of fanfare and press. All the vendor employees want the gig, and the vendor puts his aces on your team," Hatch says. "They build out new facilities. They buy new software and hardware. The vendor comes in and spends a significant amount of time and money to close the deal."On the customer side, CIOs and their staffs spend time and money evaluating vendors and putting a new management structure in place to support the offshore environment. A manager is assigned to oversee the relationship, often making repeated trips overseas to oversee vendor performance. "Traveling to India or China or the Philippines every quarter to manage operations or vendor relationships works OK for the first year or 18 months," says Chris Gentle, director of research for Deloitte and Touche. "But when you\u2019re doing it for two or three years, it takes a lot out of people." Several years out, weariness and complacency often sets in, what Gentle calls "offshore fatigue." Mindbridge\u2019s Testa recalls that in his shop, problems would start to come up that couldn\u2019t be solved with IM or e-mail, and one of his managers would have to jump on a plane to India the next day. "It\u2019s definitely fatiguing, physically and psychologically," he says.The key, says Gentle, is to make sure you rotate new people into the offshore relationship manager position every couple of years to prevent burnout. Satyam\u2019s offshore operation for TRW has grown to 150 employees, and Drouin says it\u2019s about time he had a TRW Automotive employee onsite in Chennai, India, to oversee the vendor\u2019s work. "There\u2019s only so much you can do on each visit. They focus on a couple of specific things each time they\u2019re out there," says Drouin. "Once you get to this scale, you need someone on the ground day in and day out making sure you get the most from your investment."Travel is not the only thing that can sap those managing an offshore relationship. TRW hit its ROI sweet spot with Satyam about a year after Drouin set up the offshore center. Productivity hit an all-time high, with the offshore center delivering consistently on-time, on-budget, and on-spec projects and support. "But it took us so much effort to get to that level," Drouin says. "Then we kind of backed off, assuming things would run smoothly." Complacency can set in on the vendor side as well. A. Vinod, vice president of IT for a $1.8 billion manufacturing company that Vinod would prefer not to identify, says that over the years, he\u2019s seen dwindling vendor executive involvement in his four-year relationship with Sierra Atlantic, a Freemont, Calif.-based company with offshore development centers in India. "Early on, when our account was growing, their executives made frequent visits to help cultivate the relationship. There were constant calls and status reports," Vinod recalls. "But over time, executive visibility has shifted quite a bit. Deliverables were never missed. But the visits diminished in frequency and face time with them was reduced." Vinod continues to push for more involvement from Sierra\u2019s executives and advises others in a similar situation to do the same. "You have to demand that," he says. In the event that executives are not responsive, Vinod (who has a small four-person dedicated center at Sierra in Hyderabad, India, but also pays Sierra Atlantic for additional projects and support as needed) puts his money where his mouth is. "I tell my vendor, if you want to know how the relationship is going from my end, just look at quarter over quarter billing," he says. "If you\u2019re making less money on us, you\u2019ve got something to worry about. Come over and talk to us."Long after an offshore relationship has ramped up, quarterly meetings between senior-level management at both customer and vendor are a must, Hatch agrees. And if vendor executives aren\u2019t asking how they can improve performance, there\u2019s a problem. "It\u2019s a huge red flag if the vendor isn\u2019t coming to you periodically with suggestions on how to optimize the engagement," he says.Right-the-First-Time MetricsOne issue offshore vendors are notoriously stubborn about is performance metrics. If left to their own devices, many would just as soon stick with the metrics they brought to the table on day one, particularly if those benchmarks make them look good. But the metrics often offered up by offshore vendors\u2014simple cost or man-hour figures, ratios of onsite to offsite staff, errors per thousand lines of code\u2014may not be useful. Over time, it\u2019s the customer who must push for new, more meaningful metrics. "Trying to figure out what\u2019s the right metric to use is the area where we spent the most hours," says Vinod.The majority of work Sierra Atlantic does for Vinod\u2019s manufacturing company is in the area of application support. Throughout the day, a series of tickets are opened as Vinod\u2019s users report problems with applications (anything from a password that needs changing to a program that malfunctions). Those tickets are passed to the offshore team. They look at the problem and make an attempt to resolve it. The metrics Sierra Atlantic has used all along to measure its application-support effectiveness were things like how long an open ticket sat in a technician\u2019s queue or how many hours that technician worked on the problem. And according to those numbers, the vendor was doing a bang-up job.But on the other side of the world, Vinod was seeing the backlog of new tickets inch up every day. "The Sierra Atlantic team thought they were doing a great job. They were publishing this report that showed they were squeaky clean," he says. "But their metrics didn\u2019t mean anything at all. None of these metrics helped drive the only goal\u2014ticket closure with a satisfied user." And since there was an increasing number of tickets being entered into the system, Vinod suspected problems were not being resolved on the first or even second try. The offshore support team had no way of knowing whether the solution they tried actually resolved the original problem (they worked during the day in India, while it was night back in the United States) and, with the performance metrics Sierra Atlantic had in place, the support staffers had no impetus to follow up and find out the net result. So Vinod brought the entire offshore support team (at considerable cost) to his headquarters in Ohio, where he thought they\u2019d feel more connected to the company and more accountable to users. And sure enough, the backlog decreased. "We got the numbers back, and they were fantastic," he says. "Once in the U.S., they were held to the only metric that was important to us\u2014two-day closure of every ticket." Of course, Vinod can\u2019t keep the entire support team in Ohio full time. He\u2019s still working with Sierra Atlantic to figure out the right mix of offshore and onshore vendor staff and new processes to make it work. Drouin says his team has also had a tough time figuring out what numbers will paint a more meaningful picture. His offshore management team recently added a number of metrics to track resources, projects and network availability, which are delivered monthly to Satyam\u2019s offshore project managers and TRW\u2019s project champions. More importantly, they\u2019re working to finalize a next generation of metrics whose inspiration comes from the world of manufacturing. Drouin calls them right-the-first-time metrics, an IT corollary to the manufacturing metric "first-time yield." "Rather than the number of bugs per line of code, we want to figure out how many times we get something that\u2019s just right out of the box from the vendor, or close enough to just right that we don\u2019t have to kick it back to them," Drouin explains. The Truth About TurnoverLately, Drouin has been focused on turnover metrics. Satyam itself tracks when an employee leaves the company. But for Drouin, it\u2019s when a Satyam employee leaves the TRW account that he feels the pain, even if that employee is still working for the vendor. And like most CIOs who have been outsourcing offshore for more than three years, Drouin has been feeling that pain more than ever lately. He was aware of the well-publicized turnover rates in India, sometimes nearing 25 percent or 30 percent. "That\u2019s bad enough," he says. "But you can have a specific project team and experience 100 percent turnover overnight. That\u2019s a tremendous impact, and projects can ground to a halt."Indeed, one of TRW\u2019s biggest offshore projects came to a dead stop twice last year because the entire project team on a product data management (PDM) system to support TRW\u2019s engineering work left overnight. "They literally walked across the street to join another vendor to work on some giant ERP project," Drouin says. It was particularly costly because of the type of project. "If it\u2019s a SAP project, we know that Satyam has a whole host of SAP talent they can bring to bear," Drouin says. "But if it\u2019s something more specialized like PDM\u2014they didn\u2019t have a wealth of resources in that area. And it took them time to go outside and find people." The defection led to lengthy delays in project completion that made TRW\u2019s VP of engineering none too happy. "It significantly slowed down a project aimed at increasing the efficiency of our engineers," says Drouin. "It also impacted our credibility with the business, who began to doubt our ability to deliver the project."Drouin says Satyam has put some processes in place to help ease the impact of turnover. "The vendor has put a sixth person on a five-person project team as a buffer," he says. "If someone leaves the project, they have a resource ready to jump in who\u2019s already up to speed." Ultimately though, says Drouin, turnover is simply something he has to factor into the cost of doing business offshore. Indeed, Hatch\u2019s research reveals a dramatic upswing in turnover on the vendor\u2019s team during the second and third year of offshore outsourcing engagements. Testa of Mindbridge also found that the replacements his offshore vendors put on his projects were increasingly less skilled and experienced but cost just as much. "The best people were being wooed away for more money," says Testa. "And the quality of people we got in their place wasn\u2019t equal to the price we were paying."Lehman Brothers CIO Jonathan Beyman says turnover rates are part of the reason why he doesn\u2019t send work that requires company-specific knowledge to third-party vendors in India. "They\u2019re not putting someone on my account who\u2019s going to stay there for the rest of his career. After a couple of years, I know there is going to be churn," says Beyman, who has two contracts worth a total of $70 million with Tata Consultancy Services and Wipro. "Having subject matter experts is something that\u2019s very important to us. We\u2019ve got employees internally that have worked on our systems for years and years, and we have not been able to duplicate that with a third party offshore."And that\u2019s a big part of the reason some IT executives eventually turn to a captive model, where the company owns and operates the offshore center as its own subsidiary and employees report directly to them (see "The Captive Option," Page 60). Beyman himself has gone to a hybrid model, with a captive center handling high-level work and vendors working on things like QA testing and infrastructure support.Offshoring Is Hard WorkUltimately, it\u2019s clear that for offshore outsourcing to be successful long term, it requires continued reevaluation and renewal. The hard work is not over after the first couple of years; it\u2019s only just beginning. Hatch advises CIOs to consider that and factor it in before they sign a contract with an offshore IT services provider. "You have to make a holistic evaluation of the offshore proposition that looks at total costs long term," Hatch says. "Not just the launch costs, but the mature operational costs, including the relaunch that needs to take place every three years or so." Drouin builds long-term costs for increased turnover into his metrics so that he isn\u2019t taken by surprise. "Don\u2019t assume that the task of closely monitoring is limited to the startup phase. You\u2019re going to have to maintain that level of attention\u2014or close to it\u2014throughout the relationship," he says. Beyman of Lehman Brothers made headlines in 2003 when he re-insourced the help desk he had offshored to India after nine months of terrible service levels. "The places where it works for us, it works because we spend a lot of time and attention on it," he says. The places where it hasn\u2019t worked, it hasn\u2019t worked because it wasn\u2019t managed well."Gentle of Deloitte and Touche says he saw a broad range of results in the cost savings being achieved offshore and the quality being delivered. Some companies were getting quality offshore that was 15 percent higher than on shore. Others were seeing quality about equal to that delivered domestically. And, "some were seeing quality start to dip below the level of quality available onshore, which defeats the whole value proposition of going offshore in the first place," he says.That\u2019s where Testa found himself at Mindbridge. A once-beneficial offshore outsourcing arrangement deteriorated over time until he ultimately ended up in a situation with inexperienced offshore staff delivering him buggy software at no net cost savings. Unlike Drouin and Vinod who continue to work at offshore outsourcing and derive value from it, Testa called it quits. He is in the process of staffing up internally to meet Mindbridge\u2019s application development and support needs, supplementing that with some domestic outsourcing. "I don\u2019t plan to do any more offshoring in the foreseeable future," Testa says. "It\u2019s time-consuming and draining and, at times, extremely frustrating. You spend half the time patting people on the back and half the time kicking people in the ass."As the offshore outsourcing market has matured, the lesson for CIOs is clear. "Offshoring is not for the fainthearted. You can\u2019t dabble in it," says Deloitte and Touche\u2019s Gentle. "You have to have a long-term strategy."