9 Windows Azure HDInsight Big Data Success Stories

Microsoft’s Windows Azure HDInsight Service, designed in partnership with Hadoop specialist Hortonworks, makes standard Apache Hadoop available as a service in Microsoft’s Azure cloud, allowing organizations to deploy Hadoop clusters in minutes and shut them down just as easily.

Integration with the Microsoft data platform means organizations can access and analyze their data with PowerPivot, Power View and other Microsoft BI tools, like Microsoft SQL Server Analysis Services (SSAS).

“Hadoop is a cornerstone of big data,” says Quentin Clark, corporate vice president, Microsoft Data Platform. “There are companies talking to us about how they don’t feel they can even be competitive without embracing the big data phenomenon.”

Here are nine success stories built on Windows Azure HDInsight Service.

The 343 Industries development team hosts and manages the Halo 4 installment of the Halo video game franchise. As it prepared the game for launch in November 2012, the team was given two tasks: gain insight into player behavior and user preferences, and analyze data during the five-week Halo 4 “Infinity Challenge” tournament that would accompany the launch.

Halo 4 game data is housed in Azure, so the team turned to the HDInsight Service to process and analyze the raw data. It could then feed game statistics to the tournament’s operator, Virgin Gaming, which used the data to rank players. The development team also used HDInsight to update Halo 4 every week and support a daily email campaign designed to increase player retention.

Virginia Polytechnic Institute and State University has been using the HDInsight Service to aid its life sciences research in DNA sequencing. Leveraging a grant from the National Science Foundation, Virginia Tech computer scientists developed an on-demand, cloud computing model using Windows Azure HDInsight Service that helps locate undetected genes in a massive genome database.

“Of the estimated 2,000 DNA sequences worldwide, they are generating 15 petabytes of genome data every year,” says Wu Feng, professor of Computer Science at Virginia Tech. “Many life sciences institutions simply do not have access to the computational and storage resources required to work with data sets of this size. We’re generating data faster than we can analyze it.”

Restaurant chain Blackball, which specializes in Taiwanese tea and desserts, expanded from 20 stores in Taiwan to 40 more locations throughout Malaysia. It uses perishable ingredients in its desserts and it is imperative that those ingredients get to the right place at the right time. But until recently, its business intelligence consisted of point-of-sale data sent to the corporate headquarters where it was manually entered into spreadsheets. It was also manually monitoring social media for insight into customer behavior, but found it difficult to draw connections.

Partnering with Fusionex, Blackball built a hybrid cloud solution that leverages Windows Azure HDInsight Service that integrates multiple data sources and allows it to manage its stock more efficiently and better serve its customers.

Leeds Teaching Hospitals in the U.K. is one of the largest providers of emergency care services in Europe. It wanted to gain better access to data to identify trends and improve services, but was stymied by ever-growing volumes of structured and unstructured data, from doctors’ notes to emergency room visit data.

With the help of healthcare-focused IT solutions provider Ascribe, it designed a hybrid cloud solution built on the HDInsight Service, beginning with a pilot project that processed six years’ worth of admissions records in a few hours and identified more than 30 scenarios in which the hospital could have improved health and operational and financial efficiency. Now in full production, the solution promises to save the hospital even more.

IT solutions provider Fusionex worked with a provider of building management products to help it improve energy and operational efficiency in parts of Asia where customers operate businesses that run around the clock and deal with high temperatures and humidity.

“Equipment has different interfaces, and it was a challenge to make sense of all the different data formats,” says Keat Ming Chen, vice president of Fusionex. “Our customer was unable to gain a view of what was happening across a facility.”

Fusionex built a solution on the HDInsight Service that analyzes data from multiple sensors and equipment types, allowing the customer to cut reporting time from three months to less than 30 minutes and reduce energy consumption by 20 percent.

The City of Barcelona, Spain, with 1.6 million residents, prides itself on a reputation as an innovator in municipal technology used to streamline civic planning, spur economic growth and improve quality of life. But each of its agencies was using different systems and third-party tools to manage processes and business intelligence. In addition, it recognized that social media and other new technologies were providing huge volumes of public data that it was unable to fully harness.

With the help of partner Bismart, Barcelona built a dashboard that provides near-real-time views of 120 key performance indicators on areas ranging from administrative procedures to the number of people using bus routes and public bicycles.

Danish research firm Chr. Hansen develops natural ingredients for a range of industries, including food, pharmaceuticals and agriculture. It’s moved to automate its laboratory processes, but the resulting robots, temperature sensors and other devices led to a growing volume of complex data that was difficult to capture, process and make available for use in diverse contexts.

With the help of partner Platon, Chr. Hansen built a solution on HDInsight Service that lets the company collect and process data from 100 times more trials and look at more detailed information as a result.

“This solution enables us to rethink the way we work with data so we can get the most value from it,” says Morten Meldgaard, Chr. Hansen’s project director.

Data services company iTrend tracks and analyzes unstructured data generated by social media. It built its data discovery platform on a hybrid cloud implementation running on Windows Azure that includes an Apache Hadoop cluster for processing raw data and a relational database to work with extracted information. It currently uses an on-premises Apache Hadoop cluster, but plans to migrate it to Windows Azure HDInsight Service.

The platform allows iTrend to provide dynamic reporting tools accessed through a customer portal that customers can use to track campaigns, brands and individual products. Once they specify what they want to monitor, the tool automatically tracks, analyzes and summarizes potentially millions of conversations from multiple sources. It then provides a dashboard view of the data.

The customers of Ferranti Computer Systems, a utilities ISV, struggle with an emerging problem: increasing adoption of smart meters has created massive volumes of structured data that must be processed to support business processes. These meters don’t just produce power usage information like mechanical meters—they also provide information like detailed power-quality data and data on number of outages and surges over time. Whereas a utility company worker may check a mechanical meter once a month, a smart meter can provide continuous power consumption data for a customer every 15 minutes.

With Microsoft’s help, Ferranti updated its MECOMS business support system with a number of technologies, including the HDInsight Service to help it process more than 500 million rows of data per day.