The road to creating an integrated data estate

Since the onset of the COVID-19 pandemic in early 2020, organisations have been forced to rethink they way they interact with customers. Companies across every sector have created online channels that offer contactless services.

Clearly, these channels have created more data about customers’ buying behaviours and trends than businesses have seen in the past. The problem is that many of these businesses are still struggling with how to fulfil one basic requirement: creating an integrated data estate using cloud, data analytics, artificial intelligence (AI) and machine learning (ML) technologies to gain insights that improve customer service.

Tech executives gathered recently to discuss how they are democratising access to their data and the roadblocks they face when trying to make useful data insights available across their entire businesses. The discussion was sponsored Google Cloud and TCS.

According to Dipty Ranjan Padhy, engagement director, cloud, data, AI and analytics strategy at the Google Business Unit of Tata Consultancy Services, a data estate is a unified platform where all data across an organisation’s applications, customer and transactional systems is stored.

It provides a single source of truth, reduces the time it takes to get insights that are required to respond to real-time events, and provides a better experience for customers. But the multiple data silos that exist across organisations often lead to inefficiencies and issues with the creation of a data estate, he says.

Andrew Psaltis, technology practice lead, data analytics and AL/ML at Google Cloud, echoes his comments. He says that in addition to dealing with these numerous data silos, many organisations do not have a handle on all of the ways in which data is used or if it is used at all.

“They also often do not have a single golden record of data. When taken together, IT organisations may struggle to show value from the consolidation of all data into an estate. They are able to start to overcome this as they move from a cost centre to a profit centre and morph to pairing closely with the business,” Psaltis says.

There are clear benefits to creating a data estate and ‘democratising’ data to enable all staff, no matter where they sit in the organisation, to access to information they need without running into gatekeepers who block access. When this happens, it is harder for staff to make the right decisions.

Simon Mitchell-Wong, chief digital officer at Melbourne Archdiocese Catholic Schools (MACS), says the organisation has developed a data vision, strategy, architecture, and governance framework aligned with its vision for democratising data.

“We are now implementing our proof of concept [in the cloud] that will aggregate various raw data sources into a common enterprise data model that is made available to all business units, which privacy allows, via Power BI,” he says.

Mitchell-Wong says that this data will eventually be in the hands of staff in its schools.

“Democratisation of data is an embodiment of our core value of subsidiarity, which is to enable decision-making at the lowest capable level.

“We believe that data should drive decisions. The proof of concept covers student learning outcomes data, which will be expanded to other datasets in our roadmap. This student data was chosen based on value; it has the greatest potential to improve lives through learning,” he says.

But there have been roadblocks to success, says Mitchell-Wong.

He says new systems can feel threatening to some people, and historically data was siloed across the organisation and lacking common standards and business rules.

“Over time, staff groups hired their own data and reporting specialists and implemented their own surveys, tools, rules, and validations. A project like ours cloud have been considered a threat to various roles who could then be a potential roadblock.

“Our approach over time has been to engage and build trust and to demonstrate how the future state will enable them to deliver more value,” he says.

Mitchell-Wong adds that business stakeholders are becoming ‘data stewards’, amplifying the value they provide through the data platform to reach more users. Likewise, the users of data – in some cases, the same persons – will have access to even more data sets that are validated, trustworthy, more frequently updated, and reliable. We automate much of the routine work in between so our stakeholders can focus on delivering value.”

Experimenting with AI and ML to gain better insights

Many organisations are at the very least experimenting with artificial intelligence and machine learning technologies to help them gain better insights from their data.

During this process, organisations must also leverage cloud infrastructure to migrate and modernise their data platforms, says Tata’s Padhy.

“A modern data platform typically consists of a central data hub that can ingest and host all the enterprise data and act as a unified platform that provides flexibility, scalability and insights for effective data-driven strategies,” he says.

Google Cloud’s Psaltis adds that organisations need to ensure that data is clean and accurate if their AI/ML projects are to be successful. For example, this data set might include a single customer record, he says.

“They should also look for platforms that have an integrated approach between data and the AI platform. This will provide the agility that is needed and allow them to iterate very fast,” he says.

MACS’ Mitchell-Wong says data connects the organisation’s systems by powering its APIs, providing insights through BI, and is essential to meeting compliance obligations.

Automated movement of data will remove administration and compliance burden on its schools over time. This will free up more time to focus on teaching and learning, he says.

“Eventually, AI and ML will help us identify interventions for ‘at risk’ students, identify effective teaching practices, support personalisation, improve administration and reporting, and improve learning outcomes.”

Find out more about Google Cloud. 

Discover the TCS and Google Cloud partnership.