# Further reading

The books, specifications and articles cited on the topic pages, grouped by section. Generated by build\_reference.py from each page's Learn more list.

## 01 Foundations

- Martin Kleppmann, Designing Data-Intensive Applications (1st edition, 2017), chapter 3 (storage and retrieval). A 2nd edition appeared in 2026; chapter numbers differ. (from [OLTP vs OLAP](<https://wiki.4tm0.net/data/oltp-vs-olap.html>))
- Ralph Kimball and Margy Ross, The Data Warehouse Toolkit, chapter 1. (from [OLTP vs OLAP](<https://wiki.4tm0.net/data/oltp-vs-olap.html>))
- Armbrust et al., Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics, CIDR 2021: https://www.cidrdb.org/cidr2021/papers/cidr2021\_paper17.pdf (from [Data lake, warehouse and lakehouse](<https://wiki.4tm0.net/data/lake-warehouse-lakehouse.html>))
- Apache Iceberg, table spec overview: https://iceberg.apache.org/spec/ (from [Data lake, warehouse and lakehouse](<https://wiki.4tm0.net/data/lake-warehouse-lakehouse.html>))
- Joe Reis and Matt Housley, Fundamentals of Data Engineering (O'Reilly, 2022), chapter 8 (queries, modeling and transformation). (from [ETL, ELT and EtLT](<https://wiki.4tm0.net/data/etl-elt-etlt.html>))
- dbt Labs, Extract, load, transform: https://www.getdbt.com/blog/extract-load-transform (from [ETL, ELT and EtLT](<https://wiki.4tm0.net/data/etl-elt-etlt.html>))
- Tyler Akidau, Streaming 101: https://www.oreilly.com/radar/the-world-beyond-batch-streaming-101/ (from [Processing patterns: batch, micro-batch and streaming](<https://wiki.4tm0.net/data/processing-patterns.html>))
- Martin Kleppmann, Designing Data-Intensive Applications (1st edition, 2017), chapters 10 and 11. A 2nd edition appeared in 2026; chapter numbers differ. (from [Processing patterns: batch, micro-batch and streaming](<https://wiki.4tm0.net/data/processing-patterns.html>))
- Martin Kleppmann, Designing Data-Intensive Applications (1st edition, 2017), chapter 2 (schema flexibility in the document model). A 2nd edition appeared in 2026; chapter numbers differ. (from [Schema-on-write and schema-on-read](<https://wiki.4tm0.net/data/schema-on-write-vs-read.html>))
- Joe Reis and Matt Housley, Fundamentals of Data Engineering (O'Reilly, 2022), chapter 5 (data generation in source systems: schemas). (from [Schema-on-write and schema-on-read](<https://wiki.4tm0.net/data/schema-on-write-vs-read.html>))
- Databricks, What is a medallion architecture: https://www.databricks.com/glossary/medallion-architecture (from [Medallion layers: bronze, silver and gold](<https://wiki.4tm0.net/data/medallion-layers.html>))
- Databricks documentation, medallion lakehouse architecture: https://docs.databricks.com/aws/lakehouse/medallion (from [Medallion layers: bronze, silver and gold](<https://wiki.4tm0.net/data/medallion-layers.html>))
- Zhamak Dehghani, Data Mesh Principles and Logical Architecture: https://martinfowler.com/articles/data-mesh-principles.html (from [Data products and data mesh](<https://wiki.4tm0.net/data/data-products-and-mesh.html>))
- Zhamak Dehghani, Data Mesh: Delivering Data-Driven Value at Scale (O'Reilly, 2022). (from [Data products and data mesh](<https://wiki.4tm0.net/data/data-products-and-mesh.html>))
- Joe Reis and Matt Housley, Fundamentals of Data Engineering (O'Reilly, 2022), chapter 1, the data engineer and other roles. (from [Data team roles](<https://wiki.4tm0.net/data/data-team-roles.html>))
- dbt Labs, What is analytics engineering: https://www.getdbt.com/what-is-analytics-engineering (from [Data team roles](<https://wiki.4tm0.net/data/data-team-roles.html>))

## 02 Storage

- Amazon S3 consistency model: https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html\#ConsistencyModel (from [Object storage and the S3 API](<https://wiki.4tm0.net/data/object-storage.html>))
- Amazon S3 conditional writes: https://docs.aws.amazon.com/AmazonS3/latest/userguide/conditional-writes.html (from [Object storage and the S3 API](<https://wiki.4tm0.net/data/object-storage.html>))
- Amazon S3 performance guidelines: https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance.html (from [Object storage and the S3 API](<https://wiki.4tm0.net/data/object-storage.html>))
- Apache Parquet, file format documentation: https://parquet.apache.org/docs/file-format/ (from [Columnar files and Parquet](<https://wiki.4tm0.net/data/columnar-files-parquet.html>))
- Apache Parquet format specification: https://github.com/apache/parquet-format (from [Columnar files and Parquet](<https://wiki.4tm0.net/data/columnar-files-parquet.html>))
- Apache Iceberg, partitioning: https://iceberg.apache.org/docs/latest/partitioning/ (from [Partitioning](<https://wiki.4tm0.net/data/partitioning.html>))
- Apache Hive, partitioned tables: https://hive.apache.org/docs/latest/language/languagemanual-ddl/ (from [Partitioning](<https://wiki.4tm0.net/data/partitioning.html>))
- Iceberg table spec: https://iceberg.apache.org/spec/ (from [Apache Iceberg: table format basics](<https://wiki.4tm0.net/data/iceberg-table-format-basics.html>))
- Trino Iceberg connector: https://trino.io/docs/current/connector/iceberg.html (from [Apache Iceberg: table format basics](<https://wiki.4tm0.net/data/iceberg-table-format-basics.html>))
- Iceberg maintenance: https://iceberg.apache.org/docs/latest/maintenance/ (from [Apache Iceberg: table format basics](<https://wiki.4tm0.net/data/iceberg-table-format-basics.html>))
- Apache Iceberg, maintenance: https://iceberg.apache.org/docs/latest/maintenance/ (from [Iceberg maintenance](<https://wiki.4tm0.net/data/iceberg-maintenance.html>))
- Apache Iceberg, Spark procedures: https://iceberg.apache.org/docs/latest/spark-procedures/ (from [Iceberg maintenance](<https://wiki.4tm0.net/data/iceberg-maintenance.html>))
- Trino Iceberg connector, table procedures: https://trino.io/docs/current/connector/iceberg.html (from [Iceberg maintenance](<https://wiki.4tm0.net/data/iceberg-maintenance.html>))
- Iceberg table spec, manifests and scan planning: https://iceberg.apache.org/spec/\#manifests (from [Iceberg internals: manifests and planning](<https://wiki.4tm0.net/data/iceberg-internals.html>))
- Apache Iceberg, table configuration (metrics): https://iceberg.apache.org/docs/latest/configuration/ (from [Iceberg internals: manifests and planning](<https://wiki.4tm0.net/data/iceberg-internals.html>))
- Trino Iceberg connector, metadata tables: https://trino.io/docs/current/connector/iceberg.html (from [Iceberg internals: manifests and planning](<https://wiki.4tm0.net/data/iceberg-internals.html>))
- Trino, EXPLAIN ANALYZE: https://trino.io/docs/current/sql/explain-analyze.html (from [Query performance on lake tables](<https://wiki.4tm0.net/data/query-performance.html>))
- Apache Iceberg, performance: https://iceberg.apache.org/docs/latest/performance/ (from [Query performance on lake tables](<https://wiki.4tm0.net/data/query-performance.html>))
- Iceberg REST catalog specification (OpenAPI): https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml (from [Catalogs](<https://wiki.4tm0.net/data/catalogs.html>))
- Trino, Iceberg REST catalog configuration: https://trino.io/docs/current/object-storage/metastores.html (from [Catalogs](<https://wiki.4tm0.net/data/catalogs.html>))
- Apache Polaris: https://polaris.apache.org/ (from [Catalogs](<https://wiki.4tm0.net/data/catalogs.html>))
- Apache Iceberg, table spec: https://iceberg.apache.org/spec/ (from [Table formats compared: Iceberg, Delta Lake, Hudi and Paimon](<https://wiki.4tm0.net/data/table-format-comparison.html>))
- Delta Lake documentation, UniForm: https://docs.delta.io/delta-uniform/ (from [Table formats compared: Iceberg, Delta Lake, Hudi and Paimon](<https://wiki.4tm0.net/data/table-format-comparison.html>))
- Apache XTable: https://xtable.apache.org/ (from [Table formats compared: Iceberg, Delta Lake, Hudi and Paimon](<https://wiki.4tm0.net/data/table-format-comparison.html>))

## 03 Ingestion

- Joe Reis and Matt Housley, Fundamentals of Data Engineering (O'Reilly, 2022), chapter 7, ingestion. (from [Ingestion patterns](<https://wiki.4tm0.net/data/ingestion-patterns.html>))
- Martin Kleppmann, Designing Data-Intensive Applications (1st edition, 2017), chapter 11, change data capture. (from [Ingestion patterns](<https://wiki.4tm0.net/data/ingestion-patterns.html>))
- Apache Kafka documentation (design and Kafka Connect sections): https://kafka.apache.org/documentation/ (from [Kafka and Kafka Connect](<https://wiki.4tm0.net/data/kafka-and-connect.html>))
- Apache Kafka 4.0 release announcement: https://kafka.apache.org/blog/2025/03/18/apache-kafka-4.0.0-release-announcement/ (from [Kafka and Kafka Connect](<https://wiki.4tm0.net/data/kafka-and-connect.html>))
- Jay Kreps, The Log: https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying (from [Kafka and Kafka Connect](<https://wiki.4tm0.net/data/kafka-and-connect.html>))
- Debezium documentation, PostgreSQL connector: https://debezium.io/documentation/reference/stable/connectors/postgresql.html (from [Change data capture](<https://wiki.4tm0.net/data/change-data-capture.html>))
- Debezium documentation, new record state extraction: https://debezium.io/documentation/reference/stable/transformations/event-flattening.html (from [Change data capture](<https://wiki.4tm0.net/data/change-data-capture.html>))
- Apache Kafka documentation (message delivery semantics section): https://kafka.apache.org/documentation/ (from [Delivery guarantees and idempotency](<https://wiki.4tm0.net/data/delivery-guarantees-idempotency.html>))
- Martin Kleppmann, Designing Data-Intensive Applications (1st edition, 2017), chapter 11, fault tolerance and exactly-once. (from [Delivery guarantees and idempotency](<https://wiki.4tm0.net/data/delivery-guarantees-idempotency.html>))
- Tyler Akidau, Streaming 102: https://www.oreilly.com/radar/the-world-beyond-batch-streaming-102/ (from [Late and out-of-order data](<https://wiki.4tm0.net/data/late-and-out-of-order-data.html>))
- Apache Flink documentation, timely stream processing: https://nightlies.apache.org/flink/flink-docs-stable/docs/concepts/time/ (from [Late and out-of-order data](<https://wiki.4tm0.net/data/late-and-out-of-order-data.html>))
- Confluent documentation, schema evolution and compatibility: https://docs.confluent.io/platform/current/schema-registry/fundamentals/schema-evolution.html (from [Schema evolution](<https://wiki.4tm0.net/data/schema-evolution.html>))
- Apache Iceberg, schema evolution: https://iceberg.apache.org/docs/latest/evolution/ (from [Schema evolution](<https://wiki.4tm0.net/data/schema-evolution.html>))
- CloudEvents specification: https://github.com/cloudevents/spec/blob/main/cloudevents/spec.md (from [Event design](<https://wiki.4tm0.net/data/event-design.html>))
- Martin Fowler, What do you mean by "Event-Driven"?: https://martinfowler.com/articles/201701-event-driven.html (from [Event design](<https://wiki.4tm0.net/data/event-design.html>))
- Apache Iceberg, Kafka Connect: https://iceberg.apache.org/docs/latest/kafka-connect/ (from [Writing to Iceberg from Kafka](<https://wiki.4tm0.net/data/kafka-to-iceberg.html>))
- Apache Iceberg, Flink writes: https://iceberg.apache.org/docs/latest/flink-writes/ (from [Writing to Iceberg from Kafka](<https://wiki.4tm0.net/data/kafka-to-iceberg.html>))
- Chris Richardson, Pattern: Transactional outbox: https://microservices.io/patterns/data/transactional-outbox.html (from [The outbox pattern](<https://wiki.4tm0.net/data/outbox-pattern.html>))
- Debezium documentation, outbox event router: https://debezium.io/documentation/reference/stable/transformations/outbox-event-router.html (from [The outbox pattern](<https://wiki.4tm0.net/data/outbox-pattern.html>))
- OASIS, MQTT version 5.0 specification: https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html (from [IoT and device data](<https://wiki.4tm0.net/data/iot-and-device-data.html>))
- MQTT.org, MQTT overview: https://mqtt.org/ (from [IoT and device data](<https://wiki.4tm0.net/data/iot-and-device-data.html>))
- Apache Flink documentation, stateful stream processing: https://nightlies.apache.org/flink/flink-docs-stable/docs/concepts/stateful-stream-processing/ (from [Stream processing](<https://wiki.4tm0.net/data/stream-processing.html>))
- Apache Flink documentation, window aggregation in SQL: https://nightlies.apache.org/flink/flink-docs-stable/docs/dev/table/sql/queries/window-agg/ (from [Stream processing](<https://wiki.4tm0.net/data/stream-processing.html>))
- Tyler Akidau, Slava Chernyak and Reuven Lax, Streaming Systems (O'Reilly, 2018). (from [Stream processing](<https://wiki.4tm0.net/data/stream-processing.html>))

## 04 Modelling and transformation

- Trino documentation, overview and concepts: https://trino.io/docs/current/overview/concepts.html (from [SQL query engines](<https://wiki.4tm0.net/data/sql-query-engines.html>))
- Apache Spark SQL guide: https://spark.apache.org/docs/latest/sql-programming-guide.html (from [SQL query engines](<https://wiki.4tm0.net/data/sql-query-engines.html>))
- DuckDB documentation: https://duckdb.org/docs/ (from [SQL query engines](<https://wiki.4tm0.net/data/sql-query-engines.html>))
- Kimball Group, dimensional modeling techniques: https://www.kimballgroup.com/data-warehouse-business-intelligence-resources/kimball-techniques/dimensional-modeling-techniques/ (from [Dimensional modelling](<https://wiki.4tm0.net/data/dimensional-modeling.html>))
- Ralph Kimball and Margy Ross, The Data Warehouse Toolkit, 3rd edition (Wiley, 2013). (from [Dimensional modelling](<https://wiki.4tm0.net/data/dimensional-modeling.html>))
- Trino documentation, MERGE: https://trino.io/docs/current/sql/merge.html (from [Deduplication and merge](<https://wiki.4tm0.net/data/deduplication-and-merge.html>))
- Apache Iceberg, Spark writes (MERGE INTO): https://iceberg.apache.org/docs/latest/spark-writes/ (from [Deduplication and merge](<https://wiki.4tm0.net/data/deduplication-and-merge.html>))
- Kimball Group, slowly changing dimension techniques: https://www.kimballgroup.com/data-warehouse-business-intelligence-resources/kimball-techniques/dimensional-modeling-techniques/type-2/ (from [History and slowly changing dimensions](<https://wiki.4tm0.net/data/history-and-scd.html>))
- dbt documentation, snapshots: https://docs.getdbt.com/docs/build/snapshots (from [History and slowly changing dimensions](<https://wiki.4tm0.net/data/history-and-scd.html>))
- DAMA International, DAMA-DMBOK: Data Management Body of Knowledge, 2nd edition (Technics Publications, 2017), chapter 10, reference and master data. (from [Multi-source conformance](<https://wiki.4tm0.net/data/multi-source-conformance.html>))
- Kimball Group, dimensional modeling techniques (conformed dimensions): https://www.kimballgroup.com/data-warehouse-business-intelligence-resources/kimball-techniques/dimensional-modeling-techniques/ (from [Multi-source conformance](<https://wiki.4tm0.net/data/multi-source-conformance.html>))
- dbt documentation, about dbt models: https://docs.getdbt.com/docs/build/models (from [dbt](<https://wiki.4tm0.net/data/dbt.html>))
- dbt documentation, incremental models: https://docs.getdbt.com/docs/build/incremental-models (from [dbt](<https://wiki.4tm0.net/data/dbt.html>))
- dbt documentation, unit tests: https://docs.getdbt.com/docs/build/unit-tests (from [dbt](<https://wiki.4tm0.net/data/dbt.html>))
- dbt documentation, data tests: https://docs.getdbt.com/docs/build/data-tests (from [Testing data pipelines](<https://wiki.4tm0.net/data/testing-data-pipelines.html>))
- Apache Iceberg, branching and tagging: https://iceberg.apache.org/docs/latest/branching/ (from [Write-audit-publish](<https://wiki.4tm0.net/data/write-audit-publish.html>))
- Apache Iceberg, Spark procedures (fast\_forward): https://iceberg.apache.org/docs/latest/spark-procedures/ (from [Write-audit-publish](<https://wiki.4tm0.net/data/write-audit-publish.html>))
- dbt documentation, microbatch incremental models: https://docs.getdbt.com/docs/build/incremental-microbatch (from [Backfill and replay](<https://wiki.4tm0.net/data/backfill-and-replay.html>))
- Maxime Beauchemin, Functional Data Engineering: https://maximebeauchemin.medium.com/functional-data-engineering-a-modern-paradigm-for-batch-data-processing-2327ec32c42a (from [Backfill and replay](<https://wiki.4tm0.net/data/backfill-and-replay.html>))
- dbt documentation, how we structure our dbt projects: https://docs.getdbt.com/best-practices/how-we-structure/1-guide-overview (from [Naming conventions](<https://wiki.4tm0.net/data/naming-conventions.html>))
- SQLFluff documentation: https://docs.sqlfluff.com/ (from [Naming conventions](<https://wiki.4tm0.net/data/naming-conventions.html>))
- Dan Linstedt and Michael Olschimke, Building a Scalable Data Warehouse with Data Vault 2.0 (Morgan Kaufmann, 2015). (from [Other modelling approaches: Data Vault, one big table and activity schema](<https://wiki.4tm0.net/data/other-modelling-approaches.html>))
- Activity Schema specification: https://github.com/ActivitySchema/ActivitySchema (from [Other modelling approaches: Data Vault, one big table and activity schema](<https://wiki.4tm0.net/data/other-modelling-approaches.html>))

## 05 Governance and security

- DAMA International, DAMA-DMBOK: Data Management Body of Knowledge, 2nd edition (Technics Publications, 2017), chapter 3, data governance. (from [Governance roles](<https://wiki.4tm0.net/data/governance-roles.html>))
- Zhamak Dehghani, Data Mesh Principles and Logical Architecture: https://martinfowler.com/articles/data-mesh-principles.html (from [Governance roles](<https://wiki.4tm0.net/data/governance-roles.html>))
- Bitol, Open Data Contract Standard: https://bitol-io.github.io/open-data-contract-standard/ (from [Data contracts](<https://wiki.4tm0.net/data/data-contracts.html>))
- Data Contract CLI documentation: https://cli.datacontract.com/ (from [Data contracts](<https://wiki.4tm0.net/data/data-contracts.html>))
- dbt documentation, model contracts: https://docs.getdbt.com/docs/mesh/govern/model-contracts (from [Data contracts](<https://wiki.4tm0.net/data/data-contracts.html>))
- DAMA International, DAMA-DMBOK, 2nd edition (Technics Publications, 2017), chapter 13, data quality. (from [Data quality dimensions](<https://wiki.4tm0.net/data/data-quality-dimensions.html>))
- Barr Moses, Lior Gavish and Molly Vorwerck, Data Quality Fundamentals (O'Reilly, 2022). (from [Data quality dimensions](<https://wiki.4tm0.net/data/data-quality-dimensions.html>))
- OpenLineage, column lineage facet: https://openlineage.io/docs/spec/facets/dataset-facets/column\_lineage\_facet/ (from [Catalog and lineage](<https://wiki.4tm0.net/data/catalog-and-lineage.html>))
- DataHub documentation: https://docs.datahub.com/ (from [Catalog and lineage](<https://wiki.4tm0.net/data/catalog-and-lineage.html>))
- OpenMetadata documentation: https://docs.open-metadata.org/ (from [Catalog and lineage](<https://wiki.4tm0.net/data/catalog-and-lineage.html>))
- Open Policy Agent documentation: https://www.openpolicyagent.org/docs/latest/ (from [Governance as code](<https://wiki.4tm0.net/data/governance-as-code.html>))
- Trino, Open Policy Agent access control: https://trino.io/docs/current/security/opa-access-control.html (from [Governance as code](<https://wiki.4tm0.net/data/governance-as-code.html>))
- OpenID Connect Core 1.0: https://openid.net/specs/openid-connect-core-1\_0.html (from [Authentication and authorization: OIDC and JWT](<https://wiki.4tm0.net/data/authn-vs-authz-oidc-jwt.html>))
- RFC 7519, JSON Web Token: https://www.rfc-editor.org/rfc/rfc7519 (from [Authentication and authorization: OIDC and JWT](<https://wiki.4tm0.net/data/authn-vs-authz-oidc-jwt.html>))
- RFC 9700, OAuth 2.0 Security Best Current Practice: https://www.rfc-editor.org/rfc/rfc9700 (from [Authentication and authorization: OIDC and JWT](<https://wiki.4tm0.net/data/authn-vs-authz-oidc-jwt.html>))
- NIST SP 800-162, Guide to Attribute Based Access Control: https://csrc.nist.gov/pubs/sp/800/162/upd2/final (from [RBAC and ABAC](<https://wiki.4tm0.net/data/rbac-vs-abac.html>))
- Zanzibar: Google's Consistent, Global Authorization System: https://research.google/pubs/zanzibar-googles-consistent-global-authorization-system/ (from [RBAC and ABAC](<https://wiki.4tm0.net/data/rbac-vs-abac.html>))
- Trino, Open Policy Agent access control (row filters and column masks): https://trino.io/docs/current/security/opa-access-control.html (from [Row-level security and masking](<https://wiki.4tm0.net/data/row-level-security-masking.html>))
- Google Open Source Blog, Standardizing fine-grained access control in Apache Iceberg's REST catalog (October 2026): https://opensource.googleblog.com/2026/10/standardizing-fine-grained-access-control-in-apache-icebergs-rest-catalog.html (from [Row-level security and masking](<https://wiki.4tm0.net/data/row-level-security-masking.html>))
- AWS Well-Architected, SaaS Lens: tenant isolation: https://docs.aws.amazon.com/wellarchitected/latest/saas-lens/tenant-isolation.html (from [Multi-tenancy](<https://wiki.4tm0.net/data/multi-tenancy.html>))
- Trino documentation, resource groups: https://trino.io/docs/current/admin/resource-groups.html (from [Multi-tenancy](<https://wiki.4tm0.net/data/multi-tenancy.html>))
- GDPR, Article 4 (definitions) and Recital 26: https://eur-lex.europa.eu/eli/reg/2016/679/oj (from [PII and data classification](<https://wiki.4tm0.net/data/pii-and-classification.html>))
- EDPB, Guidelines 01/2025 on pseudonymisation (version for public consultation): https://www.edpb.europa.eu/our-work-tools/documents/public-consultations/2025/guidelines-012025-pseudonymisation\_en (from [PII and data classification](<https://wiki.4tm0.net/data/pii-and-classification.html>))
- GDPR, Article 5(1)(e) storage limitation, Article 12(3) time limits and Article 17 right to erasure: https://eur-lex.europa.eu/eli/reg/2016/679/oj (from [Privacy lifecycle](<https://wiki.4tm0.net/data/privacy-lifecycle.html>))
- European Data Protection Board, right to erasure: https://www.edpb.europa.eu/sme-data-protection-guide/respect-individuals-rights\_en (from [Privacy lifecycle](<https://wiki.4tm0.net/data/privacy-lifecycle.html>))
- European Commission, international data transfers under the GDPR: https://commission.europa.eu/law/law-topic/data-protection/international-dimension-data-protection\_en (from [Residency and regions](<https://wiki.4tm0.net/data/residency-and-regions.html>))
- GDPR, Chapter V, transfers to third countries: https://eur-lex.europa.eu/eli/reg/2016/679/oj (from [Residency and regions](<https://wiki.4tm0.net/data/residency-and-regions.html>))
- Apache Iceberg, encryption: https://iceberg.apache.org/docs/latest/encryption/ (from [Encryption and key management](<https://wiki.4tm0.net/data/encryption-and-key-management.html>))
- Apache Parquet, modular encryption: https://parquet.apache.org/docs/file-format/data-pages/encryption/ (from [Encryption and key management](<https://wiki.4tm0.net/data/encryption-and-key-management.html>))
- NIST SP 800-57 Part 1, recommendation for key management: https://csrc.nist.gov/pubs/sp/800/57/pt1/r5/final (from [Encryption and key management](<https://wiki.4tm0.net/data/encryption-and-key-management.html>))
- NIST SP 800-92, Guide to Computer Security Log Management: https://csrc.nist.gov/pubs/sp/800/92/final (from [Audit logging](<https://wiki.4tm0.net/data/audit-logging.html>))
- Trino documentation, event listeners: https://trino.io/docs/current/develop/event-listener.html (from [Audit logging](<https://wiki.4tm0.net/data/audit-logging.html>))

## 06 Serving

- dbt documentation, about MetricFlow: https://docs.getdbt.com/docs/build/about-metricflow (from [Semantic layer](<https://wiki.4tm0.net/data/semantic-layer.html>))
- Cube documentation, semantic layer: https://cube.dev/docs/product/data-modeling/overview (from [Semantic layer](<https://wiki.4tm0.net/data/semantic-layer.html>))
- Apache Ossie (incubating), the new name for Open Semantic Interchange: https://ossie.apache.org/updates/ossie-enters-apache-incubator/ (from [Semantic layer](<https://wiki.4tm0.net/data/semantic-layer.html>))
- Sam Newman, Backends For Frontends pattern: https://samnewman.io/patterns/architectural/bff/ (from [Data APIs and the backend-for-frontend pattern](<https://wiki.4tm0.net/data/data-api-and-bff.html>))
- Microsoft Azure Architecture Center, Backends for Frontends pattern: https://learn.microsoft.com/en-us/azure/architecture/patterns/backends-for-frontends (from [Data APIs and the backend-for-frontend pattern](<https://wiki.4tm0.net/data/data-api-and-bff.html>))
- Hightouch, What is reverse ETL: https://hightouch.com/blog/reverse-etl (from [Reverse ETL and activation](<https://wiki.4tm0.net/data/reverse-etl-activation.html>))
- Fivetran Activations documentation (formerly Census): https://fivetran.com/docs/activations (from [Reverse ETL and activation](<https://wiki.4tm0.net/data/reverse-etl-activation.html>))
- Delta Sharing protocol: https://github.com/delta-io/delta-sharing/blob/main/PROTOCOL.md (from [Data sharing](<https://wiki.4tm0.net/data/data-sharing.html>))
- Databricks, OpenSharing announcement (June 2026): https://www.databricks.com/company/newsroom/press-releases/databricks-announces-opensharing (from [Data sharing](<https://wiki.4tm0.net/data/data-sharing.html>))
- ClickHouse documentation, introduction: https://clickhouse.com/docs/intro (from [Low-latency serving stores](<https://wiki.4tm0.net/data/low-latency-serving-stores.html>))
- Apache Pinot documentation: https://docs.pinot.apache.org/ (from [Low-latency serving stores](<https://wiki.4tm0.net/data/low-latency-serving-stores.html>))
- Martin Kleppmann and Chris Riccomini, Designing Data-Intensive Applications, 2nd edition (O'Reilly, 2026), the chapters on derived data and stream processing: https://dataintensive.net/ (from [Low-latency serving stores](<https://wiki.4tm0.net/data/low-latency-serving-stores.html>))
- Model Context Protocol specification: https://modelcontextprotocol.io/specification/ (from [Agents and MCP](<https://wiki.4tm0.net/data/agents-and-mcp.html>))
- OWASP, Top 10 for LLM applications (prompt injection): https://genai.owasp.org/llmrisk/llm01-prompt-injection/ (from [Agents and MCP](<https://wiki.4tm0.net/data/agents-and-mcp.html>))
- Stephen Few, Information Dashboard Design, 2nd edition (Analytics Press, 2013). (from [BI and dashboards](<https://wiki.4tm0.net/data/bi-and-dashboards.html>))
- Apache Superset documentation: https://superset.apache.org/docs/intro (from [BI and dashboards](<https://wiki.4tm0.net/data/bi-and-dashboards.html>))
- Microsoft Learn, DirectQuery in Power BI (compares live and import modes): https://learn.microsoft.com/en-us/power-bi/connect-data/desktop-directquery-about (from [BI and dashboards](<https://wiki.4tm0.net/data/bi-and-dashboards.html>))

## 07 Operations

- Apache Airflow documentation, assets and data-aware scheduling: https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/asset-scheduling.html (from [Orchestration](<https://wiki.4tm0.net/data/orchestration.html>))
- Dagster documentation, assets: https://docs.dagster.io/guides/build/assets (from [Orchestration](<https://wiki.4tm0.net/data/orchestration.html>))
- dbt documentation, continuous integration: https://docs.getdbt.com/docs/deploy/continuous-integration (from [Environments and CI](<https://wiki.4tm0.net/data/environments-and-ci.html>))
- dbt documentation, defer: https://docs.getdbt.com/reference/node-selection/defer (from [Environments and CI](<https://wiki.4tm0.net/data/environments-and-ci.html>))
- Google, Site Reliability Engineering, chapter 4, service level objectives: https://sre.google/sre-book/service-level-objectives/ (from [Observability and SLOs](<https://wiki.4tm0.net/data/observability-and-slos.html>))
- OpenTelemetry documentation, observability primer: https://opentelemetry.io/docs/concepts/observability-primer/ (from [Observability and SLOs](<https://wiki.4tm0.net/data/observability-and-slos.html>))
- FinOps Foundation, FinOps framework: https://www.finops.org/framework/ (from [Cost and FinOps](<https://wiki.4tm0.net/data/cost-finops.html>))
- Apache Iceberg, maintenance: https://iceberg.apache.org/docs/latest/maintenance/ (from [Cost and FinOps](<https://wiki.4tm0.net/data/cost-finops.html>))
- Amazon S3, using versioning: https://docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.html (from [Disaster recovery](<https://wiki.4tm0.net/data/disaster-recovery.html>))
- Apache Iceberg, Spark procedures (rollback and register\_table): https://iceberg.apache.org/docs/latest/spark-procedures/ (from [Disaster recovery](<https://wiki.4tm0.net/data/disaster-recovery.html>))
- OpenTofu documentation: https://opentofu.org/docs/ (from [Infrastructure as code](<https://wiki.4tm0.net/data/infrastructure-as-code.html>))
- Terraform documentation: https://developer.hashicorp.com/terraform/docs (from [Infrastructure as code](<https://wiki.4tm0.net/data/infrastructure-as-code.html>))
- Kief Morris, Infrastructure as Code, 3rd edition (O'Reilly, 2025). (from [Infrastructure as code](<https://wiki.4tm0.net/data/infrastructure-as-code.html>))
- Martin Fowler, Strangler Fig Application: https://martinfowler.com/bliki/StranglerFigApplication.html (from [Platform migration](<https://wiki.4tm0.net/data/platform-migration.html>))
- dbt documentation, migrating from stored procedures: https://docs.getdbt.com/guides/migrate-from-stored-procedures (from [Platform migration](<https://wiki.4tm0.net/data/platform-migration.html>))

## 08 Data science and ML

- Hadley Wickham, Mine Çetinkaya-Rundel and Garrett Grolemund, R for Data Science, 2nd edition, the whole game: https://r4ds.hadley.nz/whole-game (from [The data science workflow](<https://wiki.4tm0.net/data/data-science-workflow.html>))
- Cassie Kozyrkov, What great data analysts do and why every organisation needs them: https://hbr.org/2018/12/what-great-data-analysts-do-and-why-every-organization-needs-them (from [The data science workflow](<https://wiki.4tm0.net/data/data-science-workflow.html>))
- PyIceberg documentation: https://py.iceberg.apache.org/ (from [Python on the lakehouse](<https://wiki.4tm0.net/data/python-on-the-lakehouse.html>))
- DuckDB documentation, Iceberg extension: https://duckdb.org/docs/stable/core\_extensions/iceberg/overview (from [Python on the lakehouse](<https://wiki.4tm0.net/data/python-on-the-lakehouse.html>))
- Polars user guide: https://docs.pola.rs/ (from [Python on the lakehouse](<https://wiki.4tm0.net/data/python-on-the-lakehouse.html>))
- Ron Kohavi, Diane Tang and Ya Xu, Trustworthy Online Controlled Experiments (Cambridge University Press, 2020). (from [Experimentation and A/B testing](<https://wiki.4tm0.net/data/experimentation-ab-testing.html>))
- Fabijan et al., Diagnosing Sample Ratio Mismatch in Online Controlled Experiments: A Taxonomy and Rules of Thumb for Practitioners, KDD 2019: https://www.microsoft.com/en-us/research/?p=651990 (from [Experimentation and A/B testing](<https://wiki.4tm0.net/data/experimentation-ab-testing.html>))
- Feast documentation, concepts: https://docs.feast.dev/getting-started/concepts (from [Feature engineering and feature stores](<https://wiki.4tm0.net/data/feature-engineering-feature-stores.html>))
- Chip Huyen, Designing Machine Learning Systems (O'Reilly, 2022), chapter 5, feature engineering. (from [Feature engineering and feature stores](<https://wiki.4tm0.net/data/feature-engineering-feature-stores.html>))
- Gebru et al., Datasheets for Datasets: https://arxiv.org/abs/1803.09010 (from [Training data management](<https://wiki.4tm0.net/data/training-data-management.html>))
- Apache Iceberg, branching and tagging: https://iceberg.apache.org/docs/latest/branching/ (from [Training data management](<https://wiki.4tm0.net/data/training-data-management.html>))
- MLflow documentation: https://mlflow.org/docs/latest/ (from [The ML lifecycle and MLOps](<https://wiki.4tm0.net/data/ml-lifecycle-mlops.html>))
- Google Cloud, MLOps: continuous delivery and automation pipelines in machine learning: https://cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning (from [The ML lifecycle and MLOps](<https://wiki.4tm0.net/data/ml-lifecycle-mlops.html>))
- Chip Huyen, Designing Machine Learning Systems (O'Reilly, 2022). (from [The ML lifecycle and MLOps](<https://wiki.4tm0.net/data/ml-lifecycle-mlops.html>))
- Google, Rules of Machine Learning, monitoring and training-serving skew: https://developers.google.com/machine-learning/guides/rules-of-ml (from [Model serving and monitoring](<https://wiki.4tm0.net/data/model-serving-monitoring.html>))
- Chip Huyen, Designing Machine Learning Systems (O'Reilly, 2022), chapters 7 and 8. (from [Model serving and monitoring](<https://wiki.4tm0.net/data/model-serving-monitoring.html>))

## 09 Generative AI

- Lance documentation: https://lance.org/ (from [Unstructured and multimodal data](<https://wiki.4tm0.net/data/unstructured-multimodal-data.html>))
- Gradient Flow, Multimodal lakehouses: https://gradientflow.com/multimodal-lakehouse-lance/ (from [Unstructured and multimodal data](<https://wiki.4tm0.net/data/unstructured-multimodal-data.html>))
- Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, NeurIPS 2020: https://arxiv.org/abs/2005.11401 (from [Embeddings, vector search and RAG](<https://wiki.4tm0.net/data/embeddings-vector-search-rag.html>))
- Malkov and Yashunin, Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs: https://arxiv.org/abs/1603.09320 (from [Embeddings, vector search and RAG](<https://wiki.4tm0.net/data/embeddings-vector-search-rag.html>))
- pgvector documentation: https://github.com/pgvector/pgvector (from [Embeddings, vector search and RAG](<https://wiki.4tm0.net/data/embeddings-vector-search-rag.html>))
- Hamel Husain, Your AI product needs evals: https://hamel.dev/blog/posts/evals/ (from [LLM applications and evaluation](<https://wiki.4tm0.net/data/llm-applications-evaluation.html>))
- MLflow documentation, GenAI evaluation and tracing: https://mlflow.org/docs/latest/genai/ (from [LLM applications and evaluation](<https://wiki.4tm0.net/data/llm-applications-evaluation.html>))
- OWASP, Top 10 for LLM applications: https://genai.owasp.org/llm-top-10/ (from [LLM applications and evaluation](<https://wiki.4tm0.net/data/llm-applications-evaluation.html>))
- Hu et al., LoRA: Low-Rank Adaptation of Large Language Models: https://arxiv.org/abs/2106.09685 (from [Fine-tuning or RAG](<https://wiki.4tm0.net/data/fine-tuning-vs-rag.html>))
- Hinton, Vinyals and Dean, Distilling the Knowledge in a Neural Network: https://arxiv.org/abs/1503.02531 (from [Fine-tuning or RAG](<https://wiki.4tm0.net/data/fine-tuning-vs-rag.html>))
- Mitchell et al., Model Cards for Model Reporting: https://arxiv.org/abs/1810.03993 (from [Responsible AI and AI governance](<https://wiki.4tm0.net/data/responsible-ai-governance.html>))
- NIST AI Risk Management Framework: https://www.nist.gov/itl/ai-risk-management-framework (from [Responsible AI and AI governance](<https://wiki.4tm0.net/data/responsible-ai-governance.html>))
- Hunton Andrews Kurth, EU Digital Omnibus on AI enters into force: https://www.hunton.com/privacy-and-cybersecurity-law-blog/eu-digital-omnibus-on-ai-enters-into-force (from [Responsible AI and AI governance](<https://wiki.4tm0.net/data/responsible-ai-governance.html>))