People to Watch 2022
Zhamak Dehghani
Director, Emerging Technologies, Thoughtworks
Author, Data Mesh
Zhamak, you are credited with popularizing the data mesh concept, but lots of people seem to have their own ideas of what it means. Please describe for us the core idea behind the data mesh in as few words as possible.
Data mesh is a decentralized sociotechnical approach — concerned with organizational design and technical architecture — to share, access and manage analytical data in complex and large-scale environments – within or across organizations.
It is intended to establish a data ownership, sharing and governance model that scales out to get value from data as the organizations grow and become more complex — it embraces the messiness of enterprise data without compromising data quality, availability, trust and understandability.
Data mesh treats data as a product, considers domains as a first-class concern, applies platform thinking to create self-serve data infrastructure and introduces a federated computational data governance model. With data mesh in place, teams should be able to self-serve, share data and run analytical workloads — e.g., generate reports and train ML models — consuming data in a peer-to-peer and distributed fashion without the need for a centralized data team, or a data lake, warehouse or lakehouse.
Are you at all surprised by the support from the data community around your data mesh concept and the momentum that it has garnered? Why or why not?
I am pleasantly surprised at the speed and the scale of the community’s positively curious response to data mesh; their willingness to embrace an emerging concept. But I’m not surprised that the problems data mesh points out are quite common and people are ready for change.
I first publicly talked about data mesh in 2019 at an O’Reilly conference in New York, in a presentation called “beyond the lake”. This was before I had settled on the name “data mesh”. Then, I was expecting a critical response from the audience as data mesh was challenging a few of the multi-decade assumptions around data architecture, governance and organizational model. To my surprise, the talk was very well received. It was then evident that the pains were real; the pains of data users — data analysts or scientists — struggling to get timely access to high quality and trustworthy data; the pains of data engineers, stuck in the middle between data providers and data users, trying to get meaning out of data from unreliable upstreams and put it in a form that others can use, and doing all that without close contact with the business; and the challenges of data leaders getting middling returns on their data and analytical solution investments.
It felt like I had just said the words that others were quietly thinking.
What do you hope to see from the big data community in the coming year?
I do hope that the community starts asking the deeper questions around data mesh design and implementation – e.g., how to run distributed analytical workloads across the mesh of data products with consistency and repeatability, how to give true autonomy to cross-functional teams sharing and using data, how to create interoperability across a mesh of data products, etc. I discuss many of these design concerns in my upcoming book. I hope that the book provides a framework for having deeper discussions and focuses the community on the job at hand — making data mesh a common reality.
Outside of the professional sphere, what can you share about yourself that your colleagues might be surprised to learn – any unique hobbies or stories?
I am a pretty good skier and love skiing moguls on black diamond slopes, all thanks to a teenage brain and an eight year war.
I grew up in Tehran (Iran) during a revolution and a war. During the war, given the air strikes and land-to-land missiles hitting the capital, often the schools were closed for weeks.
As a teenager, I used to take advantage of the situation. Every morning, I excitedly hopped on a ski bus to the safety of the mountains and skied all day, and every evening I nervously returned not knowing if our home was still in place. Thankfully it always was.
