Service-level agreements (SLAs)\u2014those contractual agreements defining the expected performance of a service provider\u2014 alone will not guarantee success when outsourcing application development and maintenance work. They\u2019re one of many tools to help manage an IT outsourcing deal.\nWhen the proper governance of an application development and maintenance relationship is in place, however, \u201cthe right SLAs are very effective and will set the standard for the quality of work the provider will generate,\u201d says Steven Kirz, managing director for business transformation with outsourcing consultancy Pace Harmon.\nIt\u2019s critical to put in place an effective SLA schedule for outsourced application development and maintenance\u2014one that not only looks good on paper, but translates into accountability and business results in real life. CIO.com talked to Kirz about how SLAs should work, the most common ways IT outsourcing customers use (and abuse them), and how to develop a strategic set of\u00a0SLAs that takes into account the complexities of today's application outsourcing engagements.\nHow important are SLAs to application development and maintenance deals?\nKirz:For any outsourcing agreement, SLAs are important, but they are one of many tools available to help manage the agreement. Keep in mind that while SLAs are helpful in managing the service delivery, they are not a surrogate for strategic governance of the overall relationship, nor do they replace any responsibilities of the retained organization as it pertains to setting architectural standards, policies, and priorities.\u00a0\nAlso, it is important remember that SLAs are more than metrics. The SLA schedule in an agreement includes the measurement protocols, performance targets, relative priorities, and processes to adjust priorities or add or remove metrics. Best in class organizations manage far beyond the metrics and use the SLA regime as more than a \u2018carrot and stick\u2019, but rather as a mechanism to engage in a conversation with providers regarding past performance, priorities, and future direction.\nIn order to meet the SLAs, the provider will deploy the right resources and invest in the improved processes and tools required by the SLAs. Without the right SLAs, the quality of resources, the tenure of those resources, and the process improvements will be demonstrably different.\nWhat\u2019s wrong with the way some IT customers employ SLAs for application development and maintenance?\nKirz: Comprehensive SLAs include metrics, performance targets, at risk amounts tied to each metric, and\u2014in the aggregate\u2014a protocol for adding and removing metrics. It is critical that customers of outsourced services treat the SLA as a living, breathing document that evolves with the relationship.\nCustomers who get the most out of their SLAs are those that view it on a positive trajectory and use it as part of an ongoing process of motivating their service providers to achieve continuous service delivery improvement. On the other hand, customers with a negative perspective tend to treat the SLAs as a static document and a mechanism to recapture funds. These customers typically don\u2019t receive best in class performance from their providers nor do they deliver best in class service to their internal stakeholders and customers.\nWhat are the biggest mistakes you see IT buyers make when it comes to selecting SLAs for their deals?\nKirz: Specific SLA problem areas include:\nBusiness alignment. Many customers still don\u2019t take the time to understand the levels of service and performance that the business requires; instead, they use SLAs they find in other deals. Working with the business to understand what SLAs really matter allows customers to focus SLAs (and therefore their providers) on what really matters and better align with business objectives.\nEarn backs. Earn backs are a mechanism that allow providers to perform at or above the SLA standard for a fixed amount of time in order to make up for an SLA default. When the earn-back requirement is met, the provider no longer has to pay the SLA credit. Providers are often able to convince customers that earn backs are \u2018fair\u2019 or that the provider can agree to more stringent SLAs if there is an earn-back mechanism in place. However, earn backs dilute the incentive for the providers to deliver the resources and process improvements that make SLAs important in the first place.\nInability to measure SLAs. Some customers create SLAs that cannot be measured because the tools with which to perform the measurement are never implemented. Providers should be made responsible for implementing the tools with which to measure SLAs.\n\n\t\n\nInability to validate SLA performance data. Customers count on providers to measure SLAs rather than relying upon objective tools to automatically capture SLA performance data. This is a mistake; automated tools will provide a more accurate representation of SLA performance and lead to less disagreement.\nSLA definition. Providers often edit proposed SLA definitions so that they can more easily meet the SLA. For example, Incident Response Time is an SLA meant to ensure that someone from the provider is addressing an incident within a minimum number of minutes (e.g., 10 minutes). However, some providers meet the SLA 100 percent of the time by putting in place an automatic reply to any email it receives in its incident queue. Customers should carefully define SLAs so that they represent and measure the intent of the service level, not simply the language of the defined term.\nSLAs that have never been achieved. Some providers will balk at market-leading SLAs because the SLAs have not been achieved in a customer environment before, and some customers find that position reasonable. Instead, customers should focus on the SLAs their business requires, regardless of whether such SLAs have been achieved in the past, and select providers partly based upon how the providers propose to meet these new levels of service.\nSLA exceptions process. Once SLAs are in place, some providers will work hard to create exceptions that ensure the provider never misses an SLA. Customers need robust SLA governance in order to avoid excuses such as changing the levels of incident priority in order to reset the clock, invoking \u2018hold time\u2019 as providers wait for clarification, or anything having to do with third-party involvement.\u00a0\nWhat should customers keep in mind in order to develop a strategic set of SLAs that take into account the complexities\u00a0of application development and maintenance (ADM)?\nKirz: Customers need to keep in mind that ADM services are on a continuing path away from staff augmentation and are moving towards outcome-based solutions (or managed services). With this comes a need to move to a portfolio of service-level metrics that align with the service delivery model. This may mean redefining metrics and adjusting priorities. Customers should be prepared for this and drive these conversations with their service providers so that they can deliver the increasing demands of internal stakeholders.\nFirst, SLAs that can and should apply to staff augmentation work. Any provider-controlled staff augmentation can and should have an SLA (for example, time to onboard resources or turnover).\nSecond, SLAs for maintenance are very different from those in development. Each area of work should have its own set of SLAs, amount at risk, and credit point pool.\nThird, SLAs must continuously improve over time. Customers either don\u2019t include clauses that ensure SLAs continuously improve or allow providers to dictate the statistical methodology that allows for only minor improvements year over year. Instead, customers should include language that ensures SLAs will improve every year.\nWhat are some effective ADM SLAs that IT leaders should consider? \nKirz: In staff augmentation, metrics-centric SLAs that focus on staff onboarding, attrition, productivity and coding quality tend to work well. For managed services, we like to see metrics that align with expected outcomes\u2014for example, ticket resolution timeframe, batch process effectiveness, application availability and customer satisfaction. More importantly, customers should recognize that most deals have both managed services and staff augmentation components so a comprehensive metric portfolio is key.\nIn application development\u2014beyond the standard SLAs\u2014customers should also consider including SLAs that manage change management compliance, functional specification compliance, and how far in advance of release are all defects resolved.\nIn application maintenance, beyond the standard SLAs, customers should also consider including SLAs that manage the speed of root cause analysis, synthetic transaction time, and number of re-opened tickets.