Despite the investments and effort poured into next-generation data storage systems, data warehouses and data lakes have failed to provide data engineers, data analysts, and data leaders trustworthy and agile business insights to make intelligent business decisions. The answer is Data Mesh – a decentralized, distributed approach to enterprise data management.
Founder of Data Mesh Zhamak Dehghani defines Data Mesh as “a sociotechnical approach to share, access, and manage analytical data in complex and large-scale environments – within or across organizations.” She’s authoring an O’Reilly book, Data Mesh: Delivering Data-Driven Value at Scale and Starburst, the ‘Analytics Engine for Data Mesh,’ happens to be the sole sponsor. In addition to providing a complimentary copy of the book, we’re also sharing chapter summaries so we can read along and educate our readers about this (r)evolutionary paradigm. Enjoy Chapter Seven: Principle of Federated Computational Governance!
We made it!
In the final pre-release chapter of O’Reilly’s book Data Mesh: Delivering Data-Driven Value at Scale, we venture into the fourth founding principle of Data Mesh, the principle of federated computational governance.
Data governance teams have well intended and necessary goals: Ensure that data is usable, accessible, protected, and compliant with regulatory requirements. Traditionally, however, achieving these goals haven’t been as smooth. Zhamak writes,“Governance has relied heavily on manual interventions, complex central processes of data validation and certification, establishing global canonical modeling of data with minimal support for change, and often engaged too late after the fact.” Frankly, this approach won’t work for Data Mesh.
Data Mesh Governance
Data Mesh governance cultivates and embraces constant change within the data landscape. Domains are responsible for data modeling and data quality. Computational instructions are automated to ensure “data is secure, compliant, of quality and usable.” Moreover, this approach also “embeds the computational policies in each and every domain and data product.”
In this chapter, Zhamak introduces how to tailor Data Mesh governance to your organization in accordance to the federated computational model via these three components:
- Systems thinking,
- Federated operating model, and
- Computational policies.
The image below demonstrates how these three components interact within a Data Mesh federated computational governance.
Apply Systems Thinking to Data Mesh Governance
Systems thinking requires viewing parts of an organization to see the whole; and not static parts, but dynamic processes. Data Mesh governance mirrors a similar thought process: more than a sum of its parts, it’s a collection of “interconnected systems — data products, data product providers, data product consumers, and platform teams and services.” A great reference for systems thinking is Peter Senge’s Fifth Discipline, which has been a favourite book of mine for more than a decade. It’s a wonderful book filled with examples about understanding how to break away from limiting beliefs as well as traditional, linear thinking to systems thinking, a more effective way to achieve a common goal.
