# Data products and data mesh

A data product is a dataset treated like a product, with an owner, a promise to its consumers and a clear interface; data mesh is an approach that lets the teams who know the data own and publish such products themselves.

> [!NOTE]
>
> **Before reading**
>
> [Medallion layers: bronze, silver and gold](<https://wiki.4tm0.net/data/medallion-layers.html>)

## Why it matters

In many companies one central data team builds every pipeline and report. As sources and requests grow, that team becomes a queue, and it has to model data it does not fully understand. Data products and data mesh address this by moving ownership closer to the source, while shared rules keep naming, security and contracts the same across teams.

## How it works

A data product is more than a table. It has:

- an owner who answers for it and decides how it changes;
- consumers who depend on it;
- an interface, called an output port, usually tables or an API, that consumers read and nothing else;
- a contract that states its schema, meaning, freshness and quality, and how breaking changes are announced;
- documentation and discoverability, so people can find and understand it without asking.

Its internal pipelines and intermediate tables are private. Consumers rely only on the published interface, so the owner can change the internals freely.

Data mesh, a term introduced by Zhamak Dehghani, organises a platform around such products. It rests on four principles:

1. Domain ownership: the team that runs a business area, such as orders or payments, owns the data products for it.
2. Data as a product: those teams publish data with the qualities above, not as a side effect of running their systems.
3. Self-serve platform: a central platform team provides storage, pipelines, access control and tooling, so domain teams do not each build their own.
4. Federated computational governance: shared rules, such as naming, security and contract standards, are agreed by representatives of the domains and the platform team, and enforced automatically by the platform. Domains decide everything else.

```
flowchart LR
  G[Federated rules:<br/>naming, contracts,<br/>security] -.->|enforced on| D
  PL[Self-serve platform:<br/>storage, pipelines,<br/>access, catalog] -.->|used by| D
  subgraph D[Domain teams]
    subgraph Orders[Orders domain]
      O1[(clean_orders,<br/>clean_order_items<br/>private)] --> O2[[fct_daily_sales<br/>published]]
    end
    subgraph Payments[Payments domain]
      P1[(clean_payments<br/>private)] --> P2[[fct_payments<br/>published]]
    end
  end
  O2 --> F[Finance reports]
  P2 --> F
  O2 --> M[Marketing models]
```

Adopting all four principles takes significant organisational change. A common middle ground is to keep a central platform team, move ownership of key datasets to domain teams, and treat shared data, such as customers or products, as products owned by a central or dedicated team. Data products are useful even without a mesh: giving a central team's datasets owners and contracts already makes them more reliable.

## Key terms

- Data product A published dataset with an owner, interface, contract and documentation.
- Domain A business area with its own team, data and language, such as orders or payments.
- Output port The published interface of a data product.
- Self-serve platform Shared infrastructure that lets teams build and publish products on their own.
- Federated governance Shared rules agreed by domain and platform representatives, enforced automatically, with all other decisions left to domains.

## Example

The shop's orders team owns the orders data product. Internally it keeps raw\_orders, clean\_orders and clean\_order\_items, which only its own pipelines read. It publishes fct\_daily\_sales, with a contract: one row per day and product, revenue excludes cancelled orders, refreshed by 07:00, owner: head of sales operations, maintained by the orders-data team. Finance and marketing read only fct\_daily\_sales. When the orders team splits clean\_orders into two tables, nobody outside notices, because the published table and its contract did not change.

## Pitfalls

- Renaming tables as products. Without an owner and a contract, nothing changes except the label.
- Domain ownership without a platform. Each team builds its own pipelines and tools, and costs and inconsistencies multiply.
- Governance only on paper. Shared rules that are not checked automatically are applied differently by each team.
- Ignoring shared data. Customers and products are used by every domain; they need a clear owner too.

## Learn more

- [Zhamak Dehghani, Data Mesh Principles and Logical Architecture](<https://martinfowler.com/articles/data-mesh-principles.html>)
- Zhamak Dehghani, Data Mesh: Delivering Data-Driven Value at Scale (O'Reilly, 2022).

## Related

[Data contracts](<https://wiki.4tm0.net/data/data-contracts.html>) [Governance roles](<https://wiki.4tm0.net/data/governance-roles.html>) [Catalog and lineage](<https://wiki.4tm0.net/data/catalog-and-lineage.html>) [Multi-source conformance](<https://wiki.4tm0.net/data/multi-source-conformance.html>)