Move to public cloud displays CIO’s clout

In December 2016, Steve Randich took the stage at Amazon Web Services’ (AWS) re:Invent conference to tell the story of how his organization made a daring move to the public cloud.

He ticked through a series of benefits to the Financial Industry Regulatory Authority (FINRA), the privately held, independent regulator of financial markets. One of them — the performance gain of 400 times — must have been mind-boggling to many (including the headline writer for the official YouTube video, which still states the gain as 400 percent).

All of the results were impressive, particularly because this wasn’t some low-value deployment. Instead, FINRA put its mission-critical market surveillance platforms on AWS, along with 90 percent of its data. In a time where many companies were flirting with the public cloud, Randich and his team were going all-in.

Those results were also a testament to Randich’s leadership — specifically, how he steered the organization toward what was perceived as a risky decision to AWS, and how he continues to tune his IT shop to keep delivering on its success.  

FINRA

FINRA boasts two key organizational missions: investor protection and market integrity. The regulator monitors and scrutinizes data from most every major equity exchange and two-thirds of options markets, processing 37 billion transactions on average per day. (That’s the equivalent, Randich pointed out in a recent interview, of what Visa and MasterCard process over the course of six months.)

When Randich arrived as CIO in March 2013, FINRA’s market surveillance platforms were operating on a collection of data-warehouse appliances from large technology providers. The self-contained hardware, which combined storage and processing, were designed for high performance and high volume.

[Related: Financial regulators use AWS’s cloud to analyze 75 billion trades daily]

But major growth in data volumes began testing the limits of the technology. On top of that, the appliances didn’t communicate with each other, and to maintain them, FINRA’s IT team literally had to call the vendor to fix them on-site.

After spending tens of millions to keep them running, Randich says, he and his team began looking for a change. But, he says, “The only change being offered in the commercial market was more of the same.”

FINRA had a burning platform, and Randich had an idea.

Experience Matters

Randich got his first programming experience in high school and participated in a series of after-school programs at Bell Labs, where his father worked. Randich knew he would be a computer scientist around the same time he got his driver’s license.

After earning his degree, he joined KPMG, where he worked with companies across a variety of industries and geographies. “I got a lot of adaptability,” Randich says of the experience. “I could pretty much go into any situation and not be intimidated.”

Add to that accountability. Randich was 21 when he first started managing people. And in his capacity as a consultant, he was frequently interfacing with senior managers and leaders at client organizations.

Those early experiences played a significant role in his rise to the CIO ranks, first with the Chicago Stock Exchange, and then Nasdaq. He went on to become CIO of Citi’s Institutional Businesses, and then became co-CIO for the entire financial services giant.

By the time he arrived at FINRA, Randich had become one of the most experienced IT leaders in the financial services space. Along the way, he learned that management acumen, communications skills and a focus on building strong teams were among the most important assets any CIO could possess.

In one way or another, all three attributes would play critical roles in fixing FINRA’s burning platform.

Right away, Randich knew a major change was needed. FINRA’s data volumes had grown both systematically (trading volumes were increasing up to 20 percent each year for the past 20 years) and organically (FINRA expanded its reach from just doing surveillance for Nasdaq and the New York Stock Exchange to practically every equity and option exchange).

When he realized that new appliances were not a viable option, he began looking at cloud providers. Remember, this was March 2013, when countless IT and business leaders questioned whether cloud products were enterprise-ready or secure, not to mention reasonably priced. The thought of putting mission-critical assets in a public cloud was a foreign concept to most executive teams. Using a private cloud was more viable, but even then, executives hesitated to do anything beyond simple pilot programs involving non-core assets.

Not surprisingly, FINRA leaders balked as well. At first, many didn’t realize they were on a burning platform. Randich used his skills — and leaned on his clout and experience — to clearly explain the situation.

AWS offered the massive processing scale and storage they needed, at commodity costs and without the on-premises appliances that were bleeding money, he argues. It was a burning platform, plain and simple, and the cloud was the answer. Still, some executives and board members worried about security and over-reliance on a sole provider. 

Again, he was able to beat back the pessimism. “They saw we had a burning platform. The option was either, go get a bigger burning platform, or completely re-do this by going to the cloud,” he says. “Coming in here from Citi and Nasdaq, I had enough credibility that they trusted me.”

Still, in Randich’s words, in was no slam dunk. He got approval within four months to pursue AWS — while continuing to look at offerings from IBM, Google and Microsoft. And the board made it clear that they would be watching closely.

[Related: Cloud is now a core strategy — are you onboard?]

By the end of 2013, he finalized his business case for moving the surveillance platforms to AWS, and the board approved in February 2014. He had his marching orders, and budget, to move those platforms and the vast majority of data to AWS. And he less than two and a half years to do it. 

Keeping It Sold

Less than six months after hitting the July 2016 deadline, Randich delivered his victory speech at AWS’s annual conference. In our recent interview, he outlined a few of the notable results.

On the infrastructure side, FINRA has seen a 40 percent reduction in costs versus running a traditional private data center, including costs for the plant, data center, servers and some labor associated with running all of those. In the past, FINRA would peak at 3,000 logical nodes in the data center. Today, it peaks at three to five times that amount, just to get through momentary processing spikes.

The staggering 400-times performance improvement means improved capacity for FINRA employees running and reacting to surveillance activities.

For information security, patching is now done instantly, and testing them for kinks happens automatically. As for disaster recovery, FINRA no longer needs a dedicated backup site. It replicates data within a different region of AWS. If it were to lose the primary region, FINRA would simply rent more capacity to bring back the data.

And the automation improvements have allowed Randich to eliminate or reduce the number of employees doing testing, builds and deployments. Instead, he has invested those savings and resources to help the organization develop new value-added software development capabilities. Overall, FINRA is looking to move its ratio of software developers to infrastructure staff — which historically has been 50:50 or 60:40 — to 80:20, and is nearly there.

Still, there are some naysayers, Randich says, including some from FINRA’s board of directors. “There’s the never-ending desire for the cyber question, because the public cloud is inherently viewed as being less secure, which is completely wrong at this point,” Randich says. “A year and a half ago, not so much.”

Demonstrating the results is part of the answer. The other comes with education, Randich says. Shortly after our interview, he was planning to bring a handful of board members to Amazon’s headquarters to sit face-to-face with the AWS team, learn more about the offering, and make sure they had a committed partner.

“They empowered me to do it, and they trusted me, but they have by no means been a slam dunk,” Randich says. “At every board meeting, the cloud subject comes up one way or another. So I have to keep it sold.”