Amazon CTO: Startups Lead the Way in Big Data Analytics
Large enterprises have been slow to recognise the benefits of Big Data, but cloud can help
Wed, April 25, 2012
Techworld — Startups and small companies are stealing a march on their larger enterprise counterparts by employing analytics at the heart of their businesses, according to Amazon Web Services.
Speaking at an event in London today, Amazon's CTO Werner Vogels said that business intelligence has historically been the "secret domain of large enterprises". However, the advent of cloud computing has given small businesses access to scale-out applications like Hadoop, allowing them gain a competitive advantage.
"What we would consider a level playing field is actually tilting towards younger businesses," said Vogels. "They are using Big Data much more rapidly at the core of their businesses to continuously improve their services, and enterprises are the ones who are having to catch up."
Vogels said that brand loyalty is decreasing, forcing organisations to experiment more. He said that enterprises could learn a great deal from startups, which often build their products and services in conditions of extreme uncertainty, and understand the importance of getting products to market quickly.
"The key is to experiment often and fail quickly," said Vogels. "In the past some of these experiments were very expensive, and often the budget associated with them was sufficient to kill any new idea. But in the cloud, these price tags have been reduced significantly.
"If the cost of failure is almost zero, then that really motivates you to experiment more often," he added.
Vogels reiterated Amazon's vision of putting a supercomputer within the reach of any developer. He cited the example of Cycle Computing, which helps researchers and businesses run supercomputing applications on Amazon's EC2 infrastructure.
Cycle Computing built a supercomputer with 50,000 cores in the AWS cloud for drug researcher Schroedinger. Where it previously would have taken years to analyse a set of 21 million chemical compounds using a small HPC instance with a few hundred cores, Schroedinger completed the experiment in just three hours.
"Here you have a 50,000 core supercomputer at your fingertips for $5,000 an hour. That really means that supercomputing is available for everyone to use," said Vogels.
The overarching theme of Amazon's conference was that cloud services are now ready for the enterprise, and that enterprise adoption has been growing over the last 18 months. The company cited customers such as Shell, Samsung, SAP and the Guardian newspaper, all of which are using AWS to improve agility and gain a competitive edge.