Utility computing can ease big data pain

Adopting cloud or utility computing models can ease the pain for organisations analysing large amounts unstructured and structured data to gain insights that improve their competitiveness.

This is the view of Matt Wood, Amazon Web Services’ (AWS) global general manager for data science, who is in Australia this week to talk local users.

Dr Wood – who has a PhD in bioinformatics – told CIO that when customers analyse data residing in ‘traditionally-provisioned IT infrastructure’, they are constrained by the amount of storage and compute capacity that is available.

“We saw very early on that customers were able to use this utility computing platform that we built out for a wide breadth of analytics capabilities. In a traditionally-provisioned infrastructure, when customers started thinking about analytics they tried to ‘box in’ the questions that they wanted to ask of their data and the information they want to extract…..that was entirely constrained by the amount of storage and compute capacity they had.”

Utility computing removes these constraints and customers can start asking questions of their data, which will lead to actionable information to help improve their business, he said.

“It [big data] is not about managing exabytes or petabytes of data; it’s about working more productively and extracting the value from the information you already have available. It’s about overlaying different sources of information, finding the correlations and trends, diving into those to pinpoint the exact answers to the questions that customers need.

“Bigcommerce is a good example, these guys have 300,000 e-commerce sites that they run on AWS and they are able to do that at a relatively low cost and experiment quickly with new technologies and ideas as they grow.”

Dr Wood also highlighted Yelp, a global review website which uses AWS’ services to refine the algorithms that place their premium listings within their application.

“They ask retailers, mechanics, hairdressers, whoever else and say ‘you can sponsor this listing,’” he said.

“The biggest outcome they found is if they can place those listings correctly, they get a much higher click through rate from customers that are interested in them, because they are much more relevant.

“Anybody inside the organisation can spin up a new high performance computing cluster to evaluate a change in that algorithm process. They can figure out by looking at the last days’ worth of real data that customers were using……whether that change in the algorithm would lead to a beneficial outcome for the customers.”

AWS has grabbed more than 14,000 customers in Australia and New Zealand, boosted by the opening of its first local data centre in Sydney in November last year.

Related: Amazon set to shakeup Australian cloud market

The cloud computing giant – which analyst Gartner on Wednesday claimed is well ahead of the competition – is seeing a lot of success acquiring customers looking to build large-scale Web apps, said Wood.

Notable names, including the Commonwealth Bank and Suncorp, as well as academic institutions and small start-ups, are using AWS’ compute, storage, and relational database services.

Dr Wood, 34, has contributed code to the ‘Ensembl genome browser’, which is used to retrieve and compare petabytes genomic data. He is also involved in next-generation sequencing and the clinical impacts of genomics at AWS.