Managing ETL vendors

It wasn’t but a few short years ago your local technology vendor (IBM, Oracle, SAP, Microsoft, etc.) was always looking to enhance your ETL services on premise so you could be in “full control” of your database operations locally. You invested hundreds of thousands of dollars (if not much more!) to build out your local infrastructure to support your data platform of choice. Not to mention all the other costs for hardware, security, environment, expertise, and the list goes on.

Today, one of the top projects of any technology leader or CXO is navigating their cloud strategy.  Whether you are cloud mature, a hybrid, or really just exploring your cloud options, there is much activity around the cloud. It is more than hype and trend as more and more infrastructure and data get moved to the cloud.  Sure, Gartner has released its “Gartner’s Magic Quadrant for cloud infrastructure” year-over-year for more than 10 years. However, the online transition to the cloud is moving at breakneck speed.  Gartner predicts, “spending in public cloud services will reach $260.2 billion in 2017 alone, an 18.5% increase in U.S. dollars”.

While many companies today are advancing towards managed data platforms to save money, grow market share, and drive innovation and efficiency, leaders must evaluate if moving to a managed data platform is right for their organization. Although, depending on the size of your company, the services you offer, and infrastructure needs, moving to a managed data platform may not be in the cards for you.

It’s more than just moving to the cloud

As with any successful project or long-term strategy, you need to develop a solid foundation.  One key component to this foundation is finding a trusted partner to help you design and prepare this transition.

However, if you haven’t been already discussing this with your vendor rep on your options and moving your ETL data to the cloud, you might be kicking yourself for waiting this long.  Sure, the managed data platform of choice offers you high availability, scalability, and security, but you don’t have all the headaches of managing this infrastructure, not to mention your cost savings.  Many vendors like AWS, Microsoft, Oracle and others can operate your customized environment more efficiently and for pennies on the dollar compared to your traditional ETL tools.

Because your customers live in an on-demand environment, your business and your data need to be relevant, responsive, always accessible and available; and the only way you will successfully do this all the time is when your applications are real time, scalable, distributed and always-on.  Of course, this is understood, but how do you truly get there?  It’s when you have successfully established the strategic partnership with your managed data platform vendor, and allowing them to manage the data and environment for you.  Smart business leaders are acknowledging that managing daily database operations is far from their core competencies. These same leaders recognize that their time is better spent focusing on innovating and bringing better solutions to their customers.  Concentrate on the things that matter (your customer) and let all the database managing (configuring, scaling, etc.) to a trusted fully managed data platform.

What to expect from your managed data platform

So, what could help you decide if a fully managed data platform is right for you and your cloud applications?  In a time where your customers are looking for 99.9% availability, on-demand experiences, and instant gratification, you need a data management platform to run your customer-critical apps.  If you don’t, you will be left behind and eventually out of business.

For example, if you are providing data services or analysis for your customers from a large data source or multiple sources, you need to ensure your managed data platform’s performance can match the requests and demand. More importantly, you need to have already established that solid vendor-partner to be able to provide you real-time tactical support, strategic guidance, and be an innovative solution provider when you need it most. This is especially true when it comes to ETL processes, which typically involve a lot of complexity. You need that solid vendor partnership to mitigate large operational problems if you have design or performance issues within your ETL system or processes.  Furthermore, your design analysis should have established the scalability of an ETL system across its lifespan.  This should include recognizing which volumes of data that must be processed within certain established service level agreements.

As was stated earlier several times, your trusted “vendor-partner”, is truly a partner when they tell you that you have the autonomy and the flexibility to implement a hybrid cloud or multi-cloud to meet the needs of your business.  You shouldn’t have to give up control, or be told how to manage your ETL strategy. Additionally, you should have the last word on the terms of the contract to avoid that vendor lock-in clause.

For many of you that operated your data platforms for years in your local environment or private cloud, and have had experienced a great relationship with a large managed data platform vendor, it makes the transition that much easier. If you haven’t had to work with one of these vendors locally, you are still in the winning seat. There are so many competitive options, and all provide the services to move your ETL data quickly, securely, and can scale your environment faster than you could have in your traditional environment.