by Bernard Golden

Amazon Goes All In With Data Warehousing at AWS Conference

Nov 29, 20126 mins
AnalyticsCloud ComputingData Warehousing

The first day of the first Amazon Web Services user conference, known as re: Invent, focused on the way customers are using services such as S3 and CloudFront. Meanwhile, the company unveiled RedShift, which columnist Bernard Golden thinks will disrupt the data warehousing market and challenge IBM, Oracle and Teradata.

LAS VEGAS—The inaugural Amazon Web Services AWS Re: Invent user conference opened yesterday, with 6,000 attendees making for a sold-out event. What follows are a few impressions of mine.

Lower S3 Pricing, New Cloud-Based Data Warehouse

The keynote by Andy Jassy, senior vice president of AWS, was impressive (and entertaining) for several reasons:

First, he announced a 25 percent drop in S3 storage rates, effective Dec. 1. Many people on Twitter said this is in response to Google’s just-announced drop in prices. This is certainly possibility, and it supports a point I made a couple of months ago: We’re on the verge of seeing serious commodity cloud price competition, and it’s going to disrupt a lot of existing plans, both for enterprise cloud providers as well as for end users. This price drop isn’t going to be the last one by a long shot, and users everywhere will benefit from the ongoing battle among providers.

As a side note, Jassy a short while later interviewed AWS customer and Amazon competitor Netflix CEO Reed Hastings, and Hastings (twice!) thanked Jassy for the S3 price cut, saying, “It will save us millions of dollars.” Either Hastings is a great actor, or he really didn’t know about the price cut before Jassy’s announcement. Either way, it was pretty amusing to hear his live reaction. I can understand his delight, given the amount of savings involved.

Analysis: Cloud’s Commodity Pricing Squeezes Service Providers, Creates Opportunities

Jassy also provided some humor when he talked up Amazon’s approach to business: operating efficiently so that the company can prosper on low margins. He noted several competitors, not by name but by quoting their CEOs on how their profitability would improve as they improved operating margins, implying that this good news is achieved at the expense of customers. The funny part was that the quotes were styled in the design of the logos of the large tech companies from which they came: Oracle, IBM and HP.

By far the most interesting element of Jassy’s presentation was the announcement of a new data warehousing service called Redshift. (It’s not clear to me if this is the real name of the service or the code name for the early beta that is available now.)

News: Amazon Launches Cloud Database with Analytics Tools, Lowers S3 Pricing

This service goes right for the jugular of existing big data warehousing players Oracle, IBM and Teradata. Just as other AWS offerings compete not by aping the existing offerings but by rethinking them for a low-cost, low-margin, automated self-service approach, so, too, does Redshift represent a different twist on data warehousing.

Instead of the traditional, expensive star schema approach, Redshift is a compressed columnar database that provides better performance and smaller storage requirements. Moreover, Redshift is cost-competitive— at less than $1,000 per year per terabyte, it’s 90 percent less expensive than the competition, Jassy says, which can run up to $20,000 per year.

From what I see, Redshift is designed to be easy to use, without a huge learning curve or requiring disruption to loading or running the warehouse. Another attractive aspect of Redshift is that supports existing analytics and reporting tools from Microstrategy and Jaspersoft—making the transition dead simple for those data warehouse users.

Jassy offered a true-to-life example of a Redshift customer: Amazon itself. Amazon, as you might imagine, has a ton of data to crunch in analyzing its business, and it has traditionally used one of the big players. It migrated to Redshift, and claims its costs went from “millions of dollars per year” to $32,000. One may be skeptical about the Amazon connection in this example, but the numbers are eye-popping.

My read on Redshift is that it is likely to be highly disruptive to enable a whole new market for data warehouse users who were previously priced out of analytics. It also demonstrates that AWS has no barrier to its ambition—it’s certainly not going to limit its offerings to hosting replacement and complementary services to hosting. If Amazon can figure out a way to standardize and automate a software segment, sooner or later the incumbents will see a commodity player come into their space.

Hudl Shows AWS at Work on Massive Scale

I attended a number of excellent sessions throughout the day as well. Amazon made an effort to have customers offer real-world case studies, which is a welcome change from the usual vendor-heavy pitches of most other cloud conferences.

A case study from the last session of the day that I attended focused on CloudFront, Amazon’s content delivery network (CDN) service. After an overview of the service, the speaker introduced Brian Kaiser, CTO of a football video service called Hudl that’s used by 08 percent of the high schools in America, along with 80 Division I colleges and 10 NFL teams.

I had no idea how important video is. I’d heard of the extensive NFL and college use of video, but apparently if you’re a high school football player who aspires to play college football, you need your video reel for recruitment purposes. Suffice to say, video is important, big-time.

How big? Hudl serves up 600 million videos every month from its store of 500 TB of data. That adds up to 1.8 PB data transmitted every 30 days. No matter how you cut it, that’s a lot of football.

Hudl does more with CloudFront than just serve up video content. It distributes the Javascript portions of its app via the service; this gives better performance and lowers latency. Hudl also uses CloudFront security controls to restrict viewing and access.

News: 5 Things to Watch for at AWS re: Invent

This example reinforces the fact that AWS is being leveraged every day for enormous scale use. Hudl is just one (and by no means the largest) user of CloudFront, which is just one of the many AWS services. Until you hear some of the case studies, it’s easy to underestimate just how large AWS is—and how present it is in today’s computing environment.

Overall, day 1 of AWS re: Invent was pretty amazing. Today Amazon CEO Jeff Bezos is speaking during the keynote session. I imagine he has something to announce. After all, he’s not likely to let Jassy upstage him, and the tradition of entertainment is that it builds to a climax. I just wonder: if Redshift is just the warm-up act for an even bigger announcement, then what could it be?

Bernard Golden is the vice president of Enterprise Solutions for enStratus Networks, a cloud management software company. He is the author of three books on virtualization and cloud computing, including Virtualization for Dummies. Follow Bernard Golden on Twitter @bernardgolden.

Follow everything from on Twitter @CIOonline, on Facebook, and on Google +.