How Nasdaq is using data and machine learning to raise the bar on financial services

As real-time financial services proliferate, Nasdaq expands its cloud-based data and machine learning services to meet demand. The move is part of the firm’s commitment in creating an innovative and transparent data marketplace, allowing more people to access financial information.

istock 1202870693

Demand for instant access to financial data from investors and traders around the world has shaken up the financial services industry, and Nasdaq, a pioneer in digitizing the trading process, continues to innovate for customers seeking mobile-first, real-time, mission-critical analytics. Its approach involves embracing the cloud, data and analytics, and an API-first mindset.

The global exchange operator, which encourages greater market participation and innovation as vital to the health of the global economy, was one of the early innovators in providing its clients with a wide variety of financial data and analytics. In 2008, Nasdaq partnered with AWS to provide different Nasdaq programs and data access via the cloud, first with Nasdaq Market Replay, followed by Data-on-Demand. Over the years, the company has delivered additional data services through the cloud, including last year’s NextGen Solutions package and Nasdaq Cloud Data Service, further extending its reach as a provider of technology and analytics services.

Its latest offering, Nasdaq Data Link, consolidates the company’s data products into a centralized location with APIs and data sets to provide a consistent experience for clients. The company’s NextGen Solutions continue to leverage AWS services to take advantage of the layers of security to protect the client and customer data, along with the AWS cloud’s scalability, high availability, and multi- and single-tenant capabilities.

Nasdaq’s transformation mirrors a broader paradigm shift of companies embracing a cloud-first mentality for their IT architecture. “Back in 2008, you probably had to have a good reason why you were deploying in the cloud,” says Michael O’Rourke, senior vice president and head of AI/Technology, Investment Intelligence, at Nasdaq. “Today, you need to have a pretty good reason why you wouldn’t deploy to the cloud.”

Real-time data requests surge

The need for real-time, mission-critical data has been a top demand from Nasdaq’s customers for the past few years. The rise of global investors that want easy access to data from their mobile devices, along with the growth of retail investors spurred by the COVID-19 pandemic, have put pressure on financial services companies to provide fast and easy delivery of data. In addition, as consumers become more comfortable managing their money and investments online, they expect better user experiences and faster access to a wide range of financial information.

“Being able to get financial updates in real time is becoming table stakes,” says O’Rourke. “For trading applications, being able to visualize the analytics and assess the risk requires that you have real-time data in many cases.”   

The demand for real-time access to data has driven Nasdaq’s own digital transformation. Over the past decade, the exchange provided data through an on-premises solution, but gradually began adding cloud access — first with non-critical batch processing, then mission-critical batch processing, then non-critical real-time, and now mission-critical, real-time data access.

“With each one of these product launches, it helped prove out that the cloud was actually the right place for those types of workloads,” says O’Rourke. “Those products allowed us to get those early wins, and we continued to build out our footprint in the cloud.”

As the company offered data access in the cloud, it also continued to offer older products via on-premises but discovered that many new clients want to consume data services, but don’t have the infrastructure required. “For instance, if you wanted the Nasdaq equity data, you could get that on-premises as well,” says O’Rourke. “However, those are low-latency services and what makes them really fast also makes them very hard.”

The flurry of startups looking to capitalize on the broader retail participation were requesting to access Nasdaq data directly through the cloud instead of having to build their own on-premises infrastructure.

“It used to be that if you were a CIO or CTO of a startup, one of the first things you would do is build a data center,” says O’Rourke. “That’s not the case anymore. Startups are going directly to the cloud, and they want services and data to consume natively within the cloud.”

This accelerated Nasdaq’s innovation efforts, specifically with its cloud data services. “It confirmed the assumptions we had that new clients to the platform would lean very heavily to new cloud and API solutions,” he says.      

For global investors that needed mobile application access, Nasdaq used RESTful APIs that could be directly integrated in the code, to help stream real-time data to consumers in the cloud.

One example of this approach is Nasdaq customer Unhedged, which provides professional research and productivity pools for investors. “They wanted to be able to consume our data in real time and provide these kinds of research pools for their client customers in real time, and they wanted to do that natively through the cloud,” says O’Rourke. “They were able to set up, without building any infrastructure, and have their applications go native directly to our services, as opposed to a traditional firm that would need to build a data infrastructure to consume all of this data. They could build their applications to directly interact with our servers.” Time to market for the data access portion of the platform decreased from months to weeks.

This simplicity and ease of integration for clients drives Nasdaq’s API-first mindset, providing a consistency of experience across all of its products. “When clients are building code in order to interact with one of our products, many times they end up needing more data,” says O’Rourke.

“If you’re looking at US equities, you may also want to be able to consume index data or alternative data. With our API-first mentality, the same APIs that you use for one data set can be applied to the others. It simplifies the environment so they can adopt and build new services into their platforms a whole lot faster.”

In addition, the consistency tells a client that once they’ve integrated one service, they will fully understand what needs to be done on others. This helps mitigate a company’s deployment risk versus a more traditional infrastructure approach, O’Rourke explains.

Cloud data supports AI/ML innovation

Artificial intelligence (AI) and machine learning (ML) play an increasingly important role for Nasdaq’s data business. First, the technologies allow them to use data sets that were almost unusable before. Unstructured data found in content such as PDFs, audio, video, earnings transcripts, and reports can be run through machine learning and AI processes, using AWS SageMaker and other AI and cognitive services, to create more structured data that is reliable and consistent.

“Instead of having analysts and people read hundreds of thousands of documents, we can start to have machine learning go through those and create structured data and build applications on top of it,” says O’Rourke.

Nasdaq is also deploying AI and ML services from AWS to increase data quality and reliability. Algorithms clean and vet the data to ensure it’s consistent for all of its downstream customers.

“In the financial industry, the opportunity for AI is enormous,” says O’Rourke. “Just within Nasdaq, every single business line is looking at how they utilize machine learning and AI to make better products, improve productivity, and create new solutions.”

Because a lot of this data is in the cloud, Nasdaq can experiment on new innovations and products much more rapidly and cost-effectively. “Now that data is sitting in the cloud, you can spin up applications, test out hypotheses, and find out what works very economically,” says O’Rourke.

Working with AWS has given Nasdaq the ability to quickly deliver real-time, mission-critical data, and AWS’s extensive partner network allows them to test and compare different technologies to find the right solution. “It’s undeniable that they have an absolutely enormous ecosystem of partners and relationships,” says O’Rourke. “When we go to deploy new solutions, AWS supports the technology stacks that we need.”

For example, when Nasdaq was looking at which streaming publish/subscribe mechanism would be the best fit for customers, they were able to set up test environments, perform head-to-head evaluations of different services, and then deploy the chosen solution because AWS offered all of these services in the cloud.  

“Their services have been very reliable, which makes us increase the amount of services and type of criticality of services that we would put in the cloud,” says O’Rourke. “We’re seeing more and more of our mission-critical services being deployed into the cloud. That’s because of the stability that AWS has provided, along with the agility, breadth of solutions, and reliability. They’ve been great partners for us.”

Learn more about ways to reinvent your business with data.

Copyright © 2021 IDG Communications, Inc.