With the release of the Hortonworks Data Platform 2.1 version of its Hadoop distribution, Hortonworks is packing in new enterprise features, including data access, data governance, data management, security and operations. At the sold out Hadoop Summit Europe 2014 today, Hortonworks unveiled Hortonworks Data Platform (HDP) 2.1. The updated version of its Hadoop distribution represents the culmination of its Stinger Initiative for interactive SQL query, as well as layering on a host of new enterprise features like data governance, security, stream processing and search capabilities. “For Hadoop to function as a true enterprise data platform, there are certain requirements that must be met,” says Jim Walker, director of product marketing at Hortonworks. “There’s a very clear definition among practitioners on what that requires: data governance, data access, data management, security and operations. HDP 2.1 packages all this together into a single package, which is enterprise Hadoop.” HDP 2.1 packages the latest stable builds of a whole host of Apache open source projects. For interactive SQL query in Hadoop, it delivers Apache Hive 0.13, the final phase of the Stinger Initiative community effort to deliver interactive SQL query at petabyte scale in Hadoop. For the past 13 months, the Apache Hive community has been focused on the initiative, adding more than 390,000 lines of code to Hive from 145 developers hailing from more than 45 unique organizations, including Microsoft, Teradata and SAP. Walker notes that with Apache Hive 0.13, Hive has received a 100X increase in SQL query performance, enables interactive query at petabyte scale and across a wide range of complex queries and joins and is capable of an expanded range of SQL semantics for analytic applications running in Hadoop. In the data governance and security spheres, HDP 2.1 is drawing on Apache Falcon and Apache Knox. Falcon offers a data processing framework for governing and orchestrating data flows in and around Hadoop. It provides the key governance framework for acquisition and processing of data sets, replication and retention of data sets, redirecting datasets on non-Hadoop extensions and maintaining an audit trail and lineage. Knox extends perimeter security for Hadoop, all fully integrated with frameworks like LDAP and Active Directory for credential management. It provides a common place to preform authentication across Hadoop and all related projects. In data processing, the updated platform includes two new processing engines: Apache Storm and Apache Solr. Storm provides real-time event processing for sensor and business activity monitoring. It’s a key component of building a data lake architecture because it allows you to ingest millions of events per second, enabling fast query on petabytes of data. Meanwhile, Solr, integrated with HDP as part of a deep technical partnership with LucidWorks, provides open source enterprise search — high-performance indexing and sub-second search times over billions of documents. Finally, Apache Ambari, the framework for provisioning, managing and monitoring Apache Hadoop clusters, has been updated to version 1.5.1 in HDP 2.1, adding support for new data access engines, stack extensibility, pluggable views, rolling restarts, maintenance mode and more. “Hortonworks remains committed to innovating and delivering a certified, stable and completely open source enterprise Hadoop platform comprised of the most recent Apache project releases,” says Shaun Connolly, vice president of corporate strategy at Hortonworks. “This HDP 2.1 release delivers a comprehensive set of enterprise Hadoop capabilities that span data management, data access, data governance, security and operations, and our completely open source approach ensures our customers can confidently deploy a Hadoop platform that is not only built for the modern data architectures of tomorrow, but also deeply integrates with existing data center technologies,” Connolly says.HDP 2.1 is now available in technical preview and will become generally available later this month. @ThorOlavsrud. Follow everything from CIO.com on Twitter @CIOonline, Facebook, Google + and LinkedIn. Related content BrandPost Leadership superpower: Succeeding sustainably As today's great leaders recognize, true success is not solely measured by the bottom line but also by the impact a business has on its stakeholders, including employees, partners, and the environment. By Marie Kalliney, Practice Leader, Digital Transformation Services, Broadcom Mar 28, 2023 5 mins IT Leadership BrandPost Helping the C-suite leverage their network as a business-boosting asset By Tanya O'Hara Mar 28, 2023 3 mins IT Leadership Opinion 5 hard questions every IT leader must answer Strong leadership is vital to IT success — and shouldn’t be taken for granted. Continual self-reflection is essential for knowing whether it’s time to restructure your approach to leading IT. By Thornton May Mar 28, 2023 5 mins Business IT Alignment IT Leadership Feature CIOs address the impact of hybrid work Assessing how some of the most progressive CIOs strive to provide both technological and emotional support for a dispersed workforce. By Pat Brans Mar 28, 2023 8 mins CIO Remote Work Employee Experience Podcasts Videos Resources Events SUBSCRIBE TO OUR NEWSLETTER From our editors straight to your inbox Get started by entering your email address below. Please enter a valid email address Subscribe