# Environments and CI Separate environments, such as development, test and production, let changes to pipelines be built and checked before they affect real consumers; continuous integration (CI) runs those checks automatically on every proposed change. > [!NOTE] > > **Before reading** > > [dbt]() ยท [Testing data pipelines]() ## Why it matters A change to one model can break dozens of downstream tables, dashboards and exports. If changes are made directly in production, consumers find the breakage. Environments and CI move that discovery to before the merge, and make releases routine instead of risky. ## How it works Data environments differ from application environments in one important way: the code can be copied easily, but the data cannot. A test environment with no data tests nothing; a full copy of production data is expensive and may expose personal data. The usual answer is a mix: - Development: each developer builds models into their own schema, such as dev\_dana, reading production raw data or a sample of it, with their own compute. - CI: each pull request builds the changed models and everything downstream into a temporary schema, such as ci\_pr\_412, on a limited sample or a limited time range, then runs the tests and removes the schema. - Production: the main branch, built by the orchestrator into the real schemas. Environments are selected by configuration, not by code. The same dbt project runs everywhere; a target or variable decides the catalog or schema it writes to and the compute it uses. Table names never contain the environment. CI runs on each pull request and checks several things: the code compiles; linting and naming rules pass; unit tests pass; the changed models and their downstream models build; data tests pass on the CI data; contracts on published models are unchanged, or changed with a proper version; and governance rules hold, such as every new table having an owner. A change that fails any check cannot be merged. Building only what changed keeps CI fast. dbt can compare the pull request with the production state, select modified models and their downstream models, and read unchanged upstream models from production instead of rebuilding them. This is called slim CI or state-based selection. A useful extra check compares results: build the changed model in CI and compare it with production, row counts and key metrics by day. A change that was meant only to rename a column but changes revenue by 3% is caught before merge. ``` flowchart TB PR[Pull request:
change to fct_daily_sales] --> C1[Compile, lint,
unit tests] C1 --> C2[Build changed models
and downstream into ci_pr_412,
upstream read from prod] C2 --> C3[Data tests,
contract check,
compare with prod] C3 -->|all pass, reviewed| M[Merge] M --> D[Deploy: prod build
by orchestrator] ``` Deploying is the orchestrator picking up the merged code for the next production run. Changes that need more than that, such as a backfill of history or a new column in a published table, follow their own procedures: backfill, write-audit-publish or contract versioning. ## Key terms - Environment An isolated place to build and run pipelines, such as dev, CI or prod. - Continuous integration (CI) Automatic checks run on every proposed change. - Pull request A proposed change, reviewed and checked before merging. - Target The configuration that decides where and how a run writes. - Slim CI Building only changed models and their dependants, reusing production for the rest. - Data diff Comparing the output of a changed model with production. ## Example Dana changes fct\_daily\_sales to exclude refunded orders. The pull request triggers CI: ``` dbt build --target ci --select state:modified+ --defer --state prod-manifest/ ``` state:modified+ selects fct\_daily\_sales and everything downstream; --defer reads clean\_orders and other unchanged models from production. The models build into ci\_pr\_412 on the last seven days of data. The reconciliation test passes. No schema check fails, because revenue keeps its name and type, but the change alters what revenue means, which the contract's versioning rule treats as breaking. The pull request therefore also updates the contract's description, bumps its major version and notifies finance. The data diff shows revenue 1.8% lower on each day, which matches the refund rate, and the reviewer approves. After merge, the next nightly run builds the change in production, and a backfill for earlier months is scheduled separately. ## Pitfalls - Changing production directly. Even small fixes skip every check. - Environment names in table names or code. Configuration should decide the target, so code is identical everywhere. - CI that rebuilds everything. It becomes too slow to run on every change, so people stop waiting for it. - Production personal data in development. Use samples with sensitive columns masked or removed. - No comparison with production. Tests pass while the numbers change unexpectedly. ## Learn more - [dbt documentation, continuous integration]() - [dbt documentation, defer]() ## Related [Orchestration]() [Data contracts]() [Governance as code]() [Write-audit-publish]()