In the previous edition of this blog, I focused on the challenges around data mobility for analytics, the culprit behind said challenges, data gravity, and the resulting data silos. In this edition, I will focus on the efforts Dell EMC is undertaking and how we are leveraging our storage roots to help vault to the clouds.
We at Dell EMC have been helping clients leverage private and public clouds for several years now. Our on-premise hyper-converged solutions and our public cloud offering, Virtustream, have enabled clients to achieve internal IT transformation leveraging cloud technologies. Over time, this has evolved to customers inquiring about ways to leverage the public cloud as well.
A poignant use case that resonates with clients in the analytics space is the use of the public cloud for burstable capacity. A customer articulately explained that for 28 days of any given month, his company was able to handle the pent-up business demand via the on-premise resources. For the remaining two days of the month, the customer wanted to burst into the cloud at twice the scale of the on-premise computing resources. In order to accomplish this, the client needed to successfully transfer data to the cloud and back. In this particular example, the return of the data from the cloud was the most critical step in the process.
Another common challenge shared by our customers was around data that was not being directly returned to the original source. For instance, a customer would create derivative files in the cloud that needed to be returned to a completely different directory, and in some cases, a totally different data cluster. Customers were unable to correctly script certain manual moves because of the set of parameters that were created as the requirements. Often, the limits of the data movement could be traced back to simplistic human keyboard commands at cloud scale.
Shoulder of Giants
Every few years, technology leaps forward in a radical way. However, it’s typically best to refrain from heralding this progress as a “new” technology. As Clayton Christensen so poignantly penned in his book, “The Innovator’s Dilemma,” technology advances are usually existing technology or concepts from other use cases or sectors that successfully supplant incumbent technology in another sector. For the cloud, this is very much the case. Virtualization, orchestration, and auto provisioning aren’t necessarily new concepts, but combined with the operating cost model and burstable provisioning of resources these features have resonated.
The existing quandary of data silos and data mobility is not a new problem. The leap spurred by the cloud has created a new gap of how data gravity is tackled. Enterprise storage has developed technology over the years to span distances for replication and back up. As the world changes and shifts in the current situation, the question of who owns the technology resources being deployed must be addressed. How to move data to those resources despite their physical or virtual location has not changed. The new challenge is to how to leverage the storage leaps that have come before in a way that suits the new incarnation of the innovation.
Innovation never takes straight lines; it tends to zig zag and jump swim lanes. Data mobility for the cloud is similar. When Isilon launched our offering to use the cloud as a backup target earlier this year, it was positioned as a first generation feature. Our vision and focus were on more strategic topics. The collective terminology our CTOs use to refer to our ultimate goal is “Cloud Share,” or the ability to move data to the cloud for use in your favorite cloud provider’s workload tool of choice as an extension of your physical data center. By empowering clients to leverage the cloud as a natural extension of the on-premise resources, we hope to leap forward in our own innovation.
To that end, we have been testing OneFS (the software magic behind Isilon) in the cloud in a virtualized container. The images below are actual screenshots of the OneFS software being tested in the cloud. Our team is still experimenting with the final incarnation of how OneFS will manifest itself to benefit clients. We continue to work with clients to better understand how we can best help them achieve their cloud roadmaps. If our current tests results prove to be an accurate indication, one possible future outcome is that OneFS might be a daemon on a cloud instance running cloud Hadoop. The OneFS daemon adheres to a rule based approach (as it does on-premise today) to move your data to the cloud and back….
Keith Manthey is the CTO for Analytics at Dell EMC | Emerging Technologies Division.