My last one-on-one interview featured a coding cowboy's successful fast-draw approach to service-oriented architecture. But that's not the only maverick approach in innovative IT shops. At Deutsche Bank in New York, John Stepper is doing the SOA equivalent of changing an engine while the car is running.Stepper is Managing IT Director of Equity Derivatives & Equity Finance. After an influential board member established SOA as an objective to improve the bank's global operations, Stepper stepped in. He kicked-off an expansive yet Agile methodology that is quickly moving the bank to a global service-orientation, with a minimum of fuss and pain.It's still early in the bank's SOA evolution, and Stepper has a few things he'd change if he had the chance to do it all again. But the future looks promising. Moreover, Stepper's early experience reveals several lessons to heed when trying to efficiently get 20,000 feet marching in step.\nCIO: Let's dive into this. Why don't we open up by just telling folks who you are, what your title is, what you do, and then we'll dive into the questioning.Stepper: Okay. I'm responsible for application services for the investment bank. And what that really means is, it's any common software artifacts or for assets for the investment bank; for Deutsche Bank.It's a relatively new assignment for me. Most of my career has been with equities trading technology. And within Deutsche Bank, I think it's fair to say it's been tough. To say it in a positive way, we're extremely business aligned, and as a result we've had great progress on the revenue front, but less progress on defining shared infrastructure or common enterprise architecture. And that's really my job on the software side.CIO: Let's talk about your background first, before we get into all the nitty-gritty SOA details. Tell us a little bit about your career. How long have you been in the business? What are some of the things you've done, some of the battles you've been in?\nStepper: I joined Morgan Stanley in '94. Before that I was at Bell Labs. Between Morgan Stanley, I had a brief stint at Natwest Market, when they were trying to be an investment bank. And then the past 11 years or so at Deutsche Bank. It's almost always been around equities trading; typically around risk management derivatives, execution trading, DMA, and then recently, prime brokerage. And particularly at DB [Deutsche Bank] and Natwest, it was largely about growth.When I joined DB, we really didn't have an investment bank per se. And as an equities division, we grew in terms of revenue by more than 15 times, and became one of the top three divisions. Prime brokerage didn't even exist eight years ago as a business, compared to Morgan and Goldman, who had been around for 30+ years. But now it's also in the top three. And so a lot of the emphasis of what we've been doing has been about growth and time-to-market, and less about building a platform for the long term. Now we're maturing as a bank, and we're absolutely ready to make that shift.CIO: You're based in New York City?\nStepper: That's right.\nCIO: And you talked about business alignment being a characteristic of how the bank operates. You've been there 11 years. How was that business alignment achieved prior to SOA appearing on the horizon? Or was it not being achieved? Or was it a struggle to be achieved? Tell us a little bit about the IT environment prior to SOA becoming part of the equation.Stepper: It certainly was achieved prior to SOA. We read a lot about the importance of business alignment in the literature, but it's a double-edged sword. So for us it largely stems from the funding. Almost all of our initiatives are driven by individual business lines and their appetite for funding, as far as their bottoms-up funding cycle. And as a result you've got teams and infrastructures that are built up specialized to those particular trading businesses. So that was great for time-to-market and agility, particularly in the early days, but [it's] almost antithetical to creating shared assets across businesses.CIO: What was the need to share information assets across business lines? If that was a challenge, clearly they were getting away without having to do that. What was driving more pressure to start doing that?Stepper: Success, really. First, we've grown to such a point that now we have different problems. We have problems of being able to scale, to accommodate the growth we've achieved, and [accommodate] the growth we expect to achieve, and cost.So there's an absence of streamlined efficient operations. You have to have [more and more] people, and that's just not a sustainable model. When our focus was on growth, it was actually an excellent strategy. Now that we've achieved some of the business goals, [the] focus [is] on cost management and scalability. And clearly the cycles\u2014whether it be in credits, revenues, or whether it be in any of the one-time margin products\u2014the cycle of margin is decreasing, going from innovative to commoditized. And that cycle seems to be increasingly shorter. And so there's a recognition that, while we'll always be looking to create new financial instruments and new higher-margin products, that we're going to have to be much, much better at institutionalizing the processing of those.\nCIO: So when would you say the institutionalization of that information sharing, that integration started to happen and tell us about your decision-making process that led you to where you are today. What was on the table? What approaches were you considering?Stepper: This shift is something that senior managers who had IT operations across Deutsche Bank have seen for some time. But in a bank as large as ours, it takes time for that to be relevant to the thousands of applications and thousands of human beings who work in IT operations. In terms of other approaches or why we're looking at becoming a service-oriented enterprise, and using SOA specifically within IT, certainly we've adopted other approaches as well. \nWe've entered into a number of strategic partnerships, both with internal entities but also external, to move all the work off-shore, both within IT and operations. We've definitely been a leader in moving processing and moving work off-shore. And it's just that, at the same time, we haven't waited to create the shared assets, and to streamline all of our processes before doing that. We've kind of done them in parallel.And I think for a number of people, that's somewhat heretical thinking. They'll say you should re-engineer what you're doing first, and then move it off-shore, or you should have all of your things fine-tuned beforehand. I'm one of those people who would say that. But having seen what Deutsche Bank has done, I think we've accrued very real benefit and some very valuable experience in being able to work with people globally, and I think that puts us in a better position to re-engineer our workflows. It's not a textbook way of doing it, but I think it's accrued real benefits now, and I think it's a good way to do it in the future.CIO: The textbook way of doing it works if you're a textbook case.\nStepper: [laughs] That's right. That's exactly right.\nCIO: So let's talk about the real case here, what are some of the shared assets that you initially targeted and tell us a little bit about your journey into SOA. What was the decision-making process that led you there, what approach did you decide to take, you're working with some developers internally as well as externally off-shore. Take us back to, I guess, the moment where SOA became something that was on the radar and you knew that this was the way you had to go and then, of course, once you do that, there's crucial decisions that have to be made because there's so many options there.Stepper: Right. So first I give the impetus to service orientation actually came from a board member, Hermann-Josef Lamberti. He ran IT operations for the bank. He's also the Group COO of Deutsche Bank AG. He was the one to stand up and say 'This is something we have to do.' At the time most people really weren't sure what he meant. But over the past month, we've fleshed out the vision he had.The initial focus is on things that everyone can relate to; largely infrastructure utilities. So whether it's hosting environments, whether it's an enterprise service bus and workflow engine, whether it's other common mechanisms\u2014single sign-on mechanisms and such that are basic shared assets we could embrace across the different businesses\u2014I think that was a good way to gain traction towards creating high-level shared artifacts.And so the first few steps to being a service-oriented enterprise and using SOA actually wasn't SOA at all, but a pre-SOA agenda of being able to create infrastructure utilities that could be shared across the bank. The next wave was introducing a proper SOA infrastructure. We've partnered closely with TIBCO, but also with IBM and a few other folks, to create a global enterprise service bus and workflow infrastructure. With that in place, it's easier now to go through the application portfolio, start to expose content as Web services, and start to introduce workflow to places where it was foreign before. That's the spot we're in right now. We've got on the order of 100 small to large projects in flight, all in relation to the projects across the bank. There are a handful of large flagship initiatives that can show the power of being service oriented whether that's about instrument data, account opening, derivatives, transaction processing front to back. There are a number of large flagships, but also, again, 100 projects of varying size that are making SOA both relevant to a broader set of people and also educating broader chunks of the group at the same time.CIO: Why did adding that enterprise service bus and other technological pieces in place make it easier for you to embark upon these first 100 projects?Stepper: Sometimes those first few steps are the hardest. Instead of spending an inordinate amount of time debating tools and techniques, we got past that within a relatively short amount of time with TIBCO and IBM.When we're talking to applications, there's something for [developers] to actually use; something that works, something that's already production grade, something that's already set up as the utility. So the first 10 projects didn't have to go through the nightmare of on-boarding, and we don't have endless debates about what toolset to use. We had some of the best SOA engineers work that out early. So, instead of building it as they came, they anticipated the demand, built it out in a pretty quick timeframe, and it ended up being production grade. That was a great call. Now, when we approach applications, it's easier to onboard them, it's easier to get [developers] trained, it's ready to be workflow- or Web-service-enabled, and so on.\nCIO: And about when did you do this? And how did you determine that it was, in fact, production grade, when there really wasn't a production-grade SOA in place to throw these things against, and do that kind of performance testing?\nStepper: I'd say the environment was in place at the end of last year. It wasn't ready for prime time, although there were one or two early adopters. It was in the first quarter of this year that Hermann [Lamberti] reiterated, from the top, how important this was. That pretty much aligned everyone to make sure that they were engaged in making this move to a service-oriented enterprise happen.That was a watershed moment in many ways. It was no longer people waiting to see if it would work. We were no longer asking, 'Could we?' but were asking, 'Why wouldn't we?' Hermann's interest got everyone, both on the infrastructure side and on the app\/dev side, to lay out what the demand would be over the course of a year. And it was easier than any other infrastructure project to get the right resources behind it, to flesh it out. So, the building out has been going on over the course of '08. We have a number of significant projects on it now. It's not perfect, but we've got the people lined up, and we're very confident in the platform that we have.CIO: So you're affecting culture change by having a top-down push, putting core platform components in place first to eliminate debate, targeting shared infrastructure as your entry point, and then establishing SOA as a clear direction for the organization. But once that's done, what are the next steps? You now have over 100 service-oriented applications in various stage of development. How did you select them? How have you affected culture change among the application developers, specifically? What type of training has been required? What have you looked at in terms of getting everybody up to speed? I imagine the bank has myriad skill sets among the developers, and yet now everybody has to converge around this common approach.Stepper: We did a number of things. At the end of the first quarter, we did something that's not too common for Deutsche Bank, which is, we created some cross-bank governance structures for our SOA program. Most people will hear the word 'governance' and tune out. But this was a group making sure that a group of people across the bank were able to pick up the early work of the infrastructure, and expend that to training and communications for the flagship project to the service development. \n\nWhat we didn't want to do is just turn all the activity onto SOA and create spaghetti; to let 1,000 flowers bloom into something that, in the end, doesn't really add up to much. What the governance structures did was to build on the work we did on the infrastructure side, and advance it when it comes to things like naming standards. All of the engineering decisions that have to be made\u2014which can make all the difference in how these things are used and operate. To extend that to communications, we had the same governance structure set up. We rolled these programs out to touch the development community in small groups. We made sure they were aware of what was going on, that we listened to the problems that were affecting them, and that we would determine what would make a difference for them. That helped us distill some basic information assets\u2014some basic information services, if you will\u2014that would be relevant to hundreds and hundreds of applications. For example, things around investment data, end-of-day market data, single sign-on, core application libraries that every application would use, entitlements, and the like. And through this, again, instead of having 5,000 services, each with one consumer, we instantiated the service catalog with a smaller number\u2014one the order of tens\u2014that would be relevant to hundreds of applications. Then we again went to a communications program, to get people in tune with those, in touch with how to use them, and educated on how to get the most out of them. The early awareness training has been the easiest to get out there and make common. \nOn the infrastructure and communications, training is actually harder, because it's all sorts of skill sets there. And the detailed engineering training is probably the hardest. That's something we're working with third parties to shape, and we'll be rolling that out in the second half of this year.\n\nCIO: Are you able to say what third parties you're working with?\nStepper: We started with the vendors we have, meaning IBM. But we found that the canned training only goes so far. We're trying to strike a balance between 'what are the service-oriented architecture skill sets we need' with 'How can we apply [these skills] at Deutsche Bank?' That means taking things like instrument data examples, application library examples, our particular implementation of enterprise service bus, etc., and make them as real as possible, so that when people need the training, they can actually do something as opposed to just going to the more generic vendor offering. It's been tough to strike that balance, but we're certainly working on it.\n\nCIO: You have, again, over 100 projects in play, and a couple of those are flagship products for the bank. So clearly you're building. What decisions have you made in terms of languages, interoperability, standards? For argument's sake, as I speak to organizations that are doing SOA, some have chosen to use SOAP, some are going with REST, some aren't using either. Some are using their own custom XML payloads. What's your approach to how these applications actually touch each other? How are they talking?\nStepper: For the most part, we've gone the SOAP route, but I'd say we're not nearly done identifying and promoting specific standards. The governance body that I mentioned was formed in the first quarter? There are two of them. One is a SOA standards board, and that's exactly the kinds of things that they're incrementally defining, rolling out, and promoting in terms of standards. The 100 projects I mentioned are the projects that, over the course of the next 12-plus months, we expect to onboard to our SOA infrastructure. And based on the 20 or so that we have live right now, we're able to get real-world examples, extract what we think are meaningful standards, and then promote them in time for the remainder of the next 12 months.So the answer to your question is, it's a work in progress. We have the people in place, and we think the timing is right, to do it in an academic way, but in a way that is relevant to the live applications that are coming onboard over the next 12 months.CIO: What are some insights you've gained that you didn't anticipate, now that those first 20 applications are actually live? Any surprises? Any challenges? Any brick walls that you ran into?\n\nStepper: As much as I said we think we did a great job trying to get the infrastructure to production grade early on in the overall SOA timeline, in retrospect, I would have done that even earlier. I would have formed those governance structures even earlier, too, because I think we\u2014and I think this is probably true in most organizations\u2014I think we underestimated how large a transformation this could be; that is, going from vertically-aligned IT operations environment to something that's trying to make much greater use of shared assets.It's not just about teaching developers how to use Web services. It's changing how the funding is done for shared services. It's putting governance structures in place. It's defining engineering and process standards. It's creating new roles where you have actual process analysts; roles that just don't exist today. I'm glad we have things that are live, because nothing beats having real projects out there that you've learned from as an organization. But I'd still look to put more weight, more resources, behind the early engineering of the infrastructure, and between the early decisions around process and engineering standards. I would do that a lot earlier than I did.\nCIO: About how much earlier if you had it all to do over again?\nStepper: I would have spent, instead of the first half of this year, I would have done that at least six months earlier.\nCIO: How large is the pool of enterprise application developers working on these projects now?\nStepper: That's an excellent question. So, across Deutsche Bank we have thousands of people just for the investment bank, and if you add in all of the key personnel at DB, it should certainly eclipse 10,000 people.They're not all working on SOA projects per se, but the way we use SOA and why it's a big deal for us is, it's not a narrow definition. It's purposefully a broad definition of what a service is, what a reusable asset is, and by all means we use the traditional SOA tools and infrastructure to connect things. But as we talk about it across DB, it's much more about how to do things once and do them right, and not limit that to a narrow technical interpretation.\nCIO: So this is well beyond the application developers. This is the fundamental go-forward philosophy of how your IT organization is going to align with the business.\n\nStepper: That's right, so when becoming a service-oriented enterprise is something that should be embraced by IT and operations and, really, beyond that. It's one effort. It's not the IT guys doing their thing to the side.\n\nCIO: Can you give us a sense of your IT environment? I realize this is a very large organization and there's tens of thousands of technologies and products in play. But what are you building in? What languages are you preferring? We've talked about TIBCO and IBM. Throw a couple of names around for me, give us a sense of where the major stakes in the ground are from a vendor\/product perspective.\nStepper: You're right; it's difficult to answer the question for the bank. I'm tightly-coupled to the investment bank, so my answer is going to be very different from somebody who sits in Germany where a lot of our transaction processing is done on the mainframe. Having said that, we've had a lot of success with equipment from Sun. They're certainly a major partner of ours. A lot of the development is Java, particularly in the investment bank. Oracle is certainly is a huge partner of ours.\nCIO: I appreciate all the time you've spent with us today.\nStepper: It's been a pleasure.