Navigating the ever-deepening pools of data in enterprise

Companies are creating new data at an incredible rate. IDC is predicting that by 2025 the amount of data created worldwide will have reached 175 zettabytes – 10 times the amount produced in 2017. However, there are real and measurable benefits to organisations creating and storing as much data as possible.

Data underpins the artificial intelligence (AI) and analytics strategies that are providing businesses with both innovative and competitive advantages. And smaller companies can also benefit from data analysis.

Author Bernard Marr says clever leveraging of data benefits a company in five core ways:

• It helps them make better business decisions
• It helps them understand their customers
• It helps to deliver smarter services or products
• It improves business operations
• It directly generates income

Despite the need to execute a structured data strategy, many Australian organisations are finding the ongoing wrestle with data particularly challenging. A recent report found 90 per cent of IT leaders were concerned about how to deal with increasing quantities of data, and 88 per cent were struggling to manage and secure budget for data strategies.

The importance of categorising data

The foundation of a successful data strategy is in understanding how to effectively categorise data into tiers, so the data can be stored efficiently and according to need. It’s standard knowledge to categorise data according to the importance of fast access, with “tier-1” or even “tier-0” data being critical, typically transactional data, while “tier-2” data represents the less immediately critical types of data within the business.

Managing the data overload requires that organisations firmly and accurately categorise the various data types, so that the expensive, high performance storage can be dedicated to the kinds of data that powers AI, edge computing, and other critical applications from the vast majority of data. In other words, the “tier-1” data. The challenge many businesses face is that they over-categorise data as “tier-1” and therefore try to commit too much data to the high performance storage. Businesses sometimes have the wrong impression that the only data that should be stored as “tier-2” is data for archival and disaster recovery purposes, and any data that’s in active use needs to be kept in the fastest storage available. That’s not the case at all, and indeed trying to store that much data as though it is “tier-1” data is prohibitively expensive given how much data flows through organisations.

A more efficient and innovative approach to data is to be flexible in moving it between these tiers, so that data can be made available to the high performance storage when it’s needed, but seamlessly moved to lower-tier storage when it’s not in active use. To be able to do this, it’s important to purchase storage solutions built around the flexible application of data. While all-flash arrays (AFAs) are gaining traction for their ability to run those tier-1 applications, most organisations will find hybrid flash arrays strike a better balance between performance and affordability when dealing with massive loads of data. The cost of AFAs is falling, but it still comes at a premium. There is a lot of inactive data within organisations which makes AFAs highly cost inefficient.

When it comes to hybrid storage, some IT leaders are concerned about what happens if inactive data becomes mission-critical due to a project or application. A successful hybrid array solution relies is one that allows an organisation to move data back and forth between tiers efficiently, as needed. Not all data needs to be in tier-1 storage all the time, but it is important to be able to move it there quickly when required.

Leading provider of storage technology, Seagate, has tiering software that recognises incoming data patterns and then distributes data to the most appropriate tier automatically. This is done by breaking data down into chunks, or “pages”, and ranking them based on an algorithm which runs every five seconds.

Batch migration may take 24 hours to bring critical data into tier-1 storage but the Seagate solution allows for an almost real-time movement of data, boosting the overall data performance of the organisation yet also providing the cost-effectiveness of a hybrid approach to storage.

Building a hybrid solution

One other inhibitor that many Australian organisations face with making strategic use of their data has to do with ownership and control; 89 per cent of organisations are struggling to manage their data, and 69 per cent of Australian workers believe that not having direct control over their data impacts on their ability to meet strategic objectives.

The best solution to these issues is to hold the data within the organisation (rather than rely exclusively on cloud services), and to leverage software-defined intelligence to allow the IT team to work on opening the data to the entire organisation.

To go with the software solutions mentioned above, Seagate’s hardware allows even small enterprises to maintain their own datacentres, with hybrid solutions that involve two different systems – EXOS and NYTRO.

EXOS systems can stack up to 106 16TB drives, making the system highly scalable and able to meet the expected exponential growth in data predicted by IDC and other analysts within all organisations. With close to threefold growth in total global data expected between 2021 and 2025, the storage needs of every organisation will grow, and rapidly, requiring a system that can be added to quickly and efficiently. This also allows organisations to build large internal datacentres and private Clouds at a very economical rate. Because these systems are so economical, drives can also operate as an archival solution, allowing smaller enterprises to consolidate its systems to a single vendor.

NYTRO systems focus on performance, and are ideal for Edge applications, AI, streaming, and other high-demand solutions. They work seamlessly within the same datacentres that contain EXOS solutions and allow organisations to mix performance, scalability and economy.

Seagate has a long heritage in providing OEM storage solutions to other vendors. Seagate technology powers the storage needs of companies as varied as Amazon, Netflix, Tencent, Alibaba, Facebook and Apple, and Seagate has over 40,000 employees located across six continents and seven AI-enabled manufacturing sites.  Its technology has helped meet the most demanding of storage requirements, no matter the vertical. Whether it’s the transactional nature of retail, the content-rich demands of multimedia, or the sheer load of telecommunications, Seagate works directly with organisations to address escalating and demanding storage needs that are critical to their business’ health.

Visit Seagate to learn more.