With the focus that companies have on everything “data” – analytics, reporting, dashboards, Big Data, predictive analytics, decision science and so on – data governance is also increasing in importance. A good data governance program gets business stakeholders involved in deciding on data definitions, and in supporting or sponsoring consistent data usage across an organization.
Data governance models can be loosely or highly structured. Structured data governance typically entails forming a data governance council, a defined charter, identified data stewards, and prescribed policies and procedures. Data stewards “own” particular sets of data or systems and are responsible for determining associated business definitions and standards. The data governance council oversees the broader data governance process to ensure it is being managed and attended to on an ongoing basis. This is a great approach for companies that take their data quality seriously and are willing to invest time and resources to ensuring that quality.
My experiences have been focused around what I call “grass roots” data governance. The main requirement for success with data governance is to have broad business stakeholder representation and involvement. Participants need to be committed to making decisions, communicating these decisions to their organizations as appropriate, and bringing back their groups’ feedback. Ideally, they drive the data governance program. I’ve literally contacted COOs across the company to get individuals assigned to participate.
In my first experience with data governance, back in the 1980’s, we called the group the BAR, which stood for business area reps. The group formed when we were first building our enterprise data warehouse. It met to review the business conceptual model and logical data models. Most of the group members were involved in the initial interviews that lead to the development of these models. It was the group we consulted for business definitions and various requirements. Over the course of time, the BAR helped to identify key performance indicators (KPIs) for the business, and key data quality indicators for the data warehouse. Twelve years later the data warehouse had grown from 3 TB to 84 TB, and the BAR still existed. It continued to provide business decisions on future state vision, business definitions, data quality processes, and development priorities.
In my most recent foray into developing a data governance program, I brought together representatives from all of the main departments in the organization. We reviewed the business conceptual model and logical data models. We provided demos on our data warehouse deliverables, as well as the standard reports we were developing. The group was our main source for decisions on business definitions and requirements.
One recent example demonstrates the data governance group’s effectiveness. We brought to the group a question on business definitions for a term which I’ll call payments. Across three departments in the company, we learned three different definitions were being used for payments. The result was inconsistency in reporting to company leadership. I knew we were succeeding in our data governance efforts when an individual from finance volunteered to meet with the representatives from operations and decision science offline, come up with a recommendation, and bring it back to the data governance group for review and approval. That’s progress!
Having a successful data governance group is a foundational step in enabling agile development. The group becomes a core part of the agile development process, reviewing and approving deliverables every step of the way. It can continue to monitor and ensure that what is being built over time consistently reflects the business of the organization.
Additional steps in building this foundational approach to agile data warehouse development include:
– Developing a high level architecture with repeatable design patterns.
– Ensuring solid testing and tools.
– Implementing a robust data quality program.
– Giving the development team the ability to self manage its agile development approach, incorporating continuous improvement.
I will cover these remaining steps in the next few upcoming articles. Stay tuned.