Banks, especially retail banks, have to be very risk-averse, especially when it comes to their IT ecosystems. For anyone who has ever worked in IT at retail banks, their deployment model is notorious. Firstly, system changes are implemented six to ten times a year, usually with a \u201cfreeze\u201d over periods of high retail activity, such as Christmas and Thanksgiving.\nSecondly, the movement from development to finally entering production is a long and slow path; some banks have as many as three preproduction environments, where the changes can sit for up to a month before progressing to the next environment. Finally, despite all these precautions, system failures once the changes are applied to the production environment are disturbingly high, with predictably adverse reactions from the bank\u2019s customers and the potential of financial loss.\nThe irony is that all the safeguards that are put in place to avoid risk create situations of high risk.\nHow being ultra-cautious creates risk\nThere is an old saying, \u201cpractice makes perfect.\u201d This is one of the fundamentals of Lean manufacturing or the \u201cToyota Way.\u201d Frequent repetition improves the ability to perform a task.\nOn the face of it, the practices mentioned above would seem to be sound. They are the tried and tested ways of implementing changes using the \u201cwaterfall\u201d or traditional IT project approach and the system development lifecycle. Requirements are painstakingly defined, interpreted as technical specifications, and handed off to developers to write the code. The code then passes into the testing hierarchy that starts with unit testing through to user acceptance testing. Change Management now decide whether to accept the change; then, it\u2019s scheduled to move into production. This has historically been done by taking the system offline at a quiet time, such as midnight on Saturday to 5 a.m. on Sunday.\nWhile this seems a proper and orderly way to implement change, it is full of systemic risk:\n\nIf the changes are being applied to one of the bank\u2019s legacy suites, the application is very large and complex. Patches and modifications have been made over the years, and, if the bank is very lucky, there are still one or two \u201cheroes\u201d who have a thorough understanding of the application. Chances are, there is no one who entirely understands the application.\nThe time lapse from original specification of requirements to delivering a finished product can be multi-year, by which time the requirement has changed.\nIn the time that has elapsed, at least some of the original team have moved on; no amount of documentation can replace the tacit information gathered during development\nEven the team members who have been on the project since inception may have been waiting for three months for the change and have forgotten much of the rationale behind what was developed.\nThere are always a few \u201cminor\u201d changes that are piggybacked into the main request when deployment is infrequent. These add high risk because their impact on the original system change is often not understood, and comprehensive integration testing was not done.\nThe test environment is hardly ever a mirror image of production. Also, with complex systems, different results can happen, even where the test and production environments seem identical.\nBecause changes are only permitted a few times a year, the capability to implement the change without a hitch is something neither Development nor Ops are practised in.\nA poor understanding of the system architecture will result in a delay in trying to identify the root cause of any errors and probably a roll-back, because there is not enough time to fix what ails the application. It must be restored to the version before the changes were applied, if possible, and implementation must be rescheduled after the repairs and testing have been done. This is not always managed in the scheduled timeframe.\n\nWhen one considers these risks, it is no wonder that deployment creates anxiety and tension among all involved.\nBanks are agile, too\nIt must be stressed that this approach is usually limited to the legacy core banking systems and that most banks have different approaches to customer-facing products such as internet banking and mobile apps. These are developed using agile techniques and may even operate under different change management processes. Some banks choose to outsource this work; others build in-house capabilities, but in both cases the bank\u2019s IT marches to different drums. There is a general recognition that the core banking systems have to be replaced in order to have a unified and stable IT infrastructure, but this is not an overnight job, mainly because it involves a major overhaul of all the processes involved.\nAgile is not enough. Introducing devops\nEven if a bank has managed to unify their IT development, they need to take the next leap to continuous delivery. Agile works up to actual deployment; then the wheels come off, because the problem now moves to Operations, who have largely been excluded from the development process. The merging of Development and Operations (not forgetting Security) creates an environment where continuous development, integration and deployment become viable. This is how Amazon, Facebook and other websites that do not have the luxury of downtime operate. They are able to achieve multiple deployments per day of which their customers are totally unaware. While there is a great deal of automation required to achieve this, devops is predominantly a culture shift.\nBanks that still want to be around in the next ten years have to mature to a devops culture. The challenge is to introduce the new way of thinking, which is best managed by bringing in Devops-as-a-Service to gradually implement the new processes and instil the culture change. Where the banks have taken the plunge, they have achieved remarkable breakthroughs.\nSome banks that are changing\nThere are banks that have taken up the challenge across the globe.\nIn 2011, ING Bank had a well-established CMMI and ITIL environment, using Prince2 for project management, but were experiencing instability despite all the governances of these best practices. They gradually moved to Agile, with mixed results initially, and then introduced devops, with spectacular results. They were able to implement more changes by introducing Agile, but still encountered the same number of incidents. Their implementation risk was still high until they introduced devops.\nWhat was more impressive was that, although the number of incidents stayed the same, because the frequency of changes increased, the number of incidents per change decreased dramatically. What ING do emphasize is the scarcity of the best skills for the job, and how important it is to acquire them.\nOtkritie FC Bank, one of Russia\u2019s largest financial corporations, optimized their internal systems and improved business processes by shifting to devops. Their traditional infrastructure was typical of most banks, with quarterly releases, and a major outage compelled Otkritie FC Bank to make the change. They took the plunge, changing their business processes, focussing on automated testing and a robust approach to changing old habits. They achieved remarkable success in transitioning into a digital bank. Kirill Menshov, VP and IT Director of Otkritie FC Banking Group attributes much of their achievement to automation of testing and deployment.\nIn Singapore, DBS is a major bank who also experienced a massive outage in 2010. Although IBM admitted liability for the failure, this did not reassure the bank\u2019s clients. It also did not impress the Monetary Authority of Singapore, who instructed DBS to get their house in order. Seven years later, DBS has made the shift to devops, and are optimistic of a brighter and more stable future, both for their 4.5 million customers and themselves. Their focus is on microservices, which will help them deploy faster and better, and they are obviously doing something right, as they were recognised at the world\u2019s \u201cBest Digital Bank\u201d by Euromoney in 2016.\nWhat\u2019s next: Google and SRE (Site Reliability Engineering)\nDevops is almost a decade old now, and ten years is a long time in IT and technology in general. Google state that they have moved beyond devops to SRE and have published a book describing what SRE is and how to get there. For those banks still struggling with their core banking systems, SRE must seem as attainable as a distant planet, but once they are confident in their devsecops capability, they can aim for the new target.