Sources, ref(), and the Dependency Graph

A Source Is Raw Data dbt Reads, Never Data dbt Creates

The previous module showed dbt models as SELECT statements saved to `.sql` files, each one building on another. But every dependency chain has to start somewhere — a model's SELECT has to read from a table that already exists in the warehouse before dbt ever ran. That starting table is a source. A source is a named reference, declared in a `.yml` file, to a raw table that was loaded by something outside dbt entirely — a Fivetran connector, an Airbyte sync, a Snowpipe or COPY INTO job, a batch ETL script, an application's change-data-capture stream landing directly in the warehouse.

This is the single most important distinction to internalize before writing a real dbt project: sources are raw tables dbt reads but does not own. dbt never runs `CREATE TABLE` for a source. It never runs `INSERT` into a source. It never truncates, drops, or modifies a source in any way. A model is something dbt builds — dbt owns the DDL, the schema, the refresh schedule, everything about its lifecycle. A source is something dbt merely acknowledges exists, so that it can be referenced safely, tracked in lineage, and checked for freshness.

version: 2

sources:
  - name: raw
    database: analytics
    schema: raw
    tables:
      - name: orders
        description: "Raw order events loaded by Fivetran from the app's Postgres database, roughly every 15 minutes."
        loaded_at_field: _fivetran_synced
      - name: customers
        description: "Raw customer records, loaded by the same Fivetran connector."
        loaded_at_field: _fivetran_synced

Notice what this YAML file does not contain: no `CREATE TABLE` statement, no column definitions, no partitioning strategy, nothing that would actually build `raw.orders`. It only names the database, schema, and table that some other system is responsible for populating. dbt's only job here is to remember that this table exists, under this name, so that any model referencing it through source() can be validated, documented, and tracked.

Why Not Just Write FROM raw.orders Directly?

A reasonable first question: if a source declaration doesn't create anything, why not skip the YAML entirely and just write FROM raw.orders directly in a model's SQL? The table exists either way — the query would run and return the same rows. The answer is that declaring the source buys you three things a hardcoded table reference cannot give you, and all three matter more as a project grows past a handful of models.

1. Lineage tracking

dbt builds a dependency graph (covered in full in Part 05) by statically parsing every model's compiled SQL for ref() and source() calls. If a model reads a raw table through a hardcoded string like FROM analytics.raw.orders, dbt has no way to know that model depends on that raw table at all — as far as dbt's graph is concerned, that model has no upstream dependency, it just materializes out of nowhere. Run dbt docs generate and the generated lineage graph will show that model floating with no incoming edge, which makes the generated documentation actively misleading about where the data really comes from.

2. Source freshness checks

dbt can only check whether a raw table is being refreshed on schedule if it knows that table exists and where its "last loaded" timestamp lives. This is the dbt source freshness command, covered in depth in Part 03 — it is only possible at all because the source was declared with aloaded_at_field. A hardcoded table reference gives dbt nothing to check against.

3. A single place to change the underlying location

Raw tables move. A company migrates from one Fivetran connector to a custom CDC pipeline landing data in a differently named schema. A database gets renamed during a platform migration. A team consolidates three raw schemas into one. If every model hardcodes FROM raw.orders, that string has to be found and changed in every single model file that touches it — easy to miss one, and each miss is a silent bug where half your models point at the old table and half at the new one. If every model instead calls {{ source('raw', 'orders') }}, the fix is one line changed in one YAML file. Every model that reads through that source call picks up the new location automatically the next time dbt compiles.

-- Anti-pattern: hardcoded raw table reference
SELECT order_id, customer_id, order_ts, status
FROM analytics.raw.orders          -- dbt has no idea this dependency exists
WHERE order_id IS NOT NULL

-- Correct: source() reference
SELECT order_id, customer_id, order_ts, status
FROM {{ source('raw', 'orders') }}  -- dbt tracks this in lineage and freshness
WHERE order_id IS NOT NULL

dbt source freshness — Catching a Stalled Pipeline Before It Poisons Your Marts

Upstream pipelines fail silently more often than they fail loudly. A Fivetran connector's credentials expire. A CDC replication slot falls behind and eventually gets dropped. A batch job that used to run every hour gets orphaned after an infrastructure migration and nobody notices for three days. In every one of these cases, the raw table does not disappear and does not error — it simply stops receiving new rows. Every downstream dbt model keeps running successfully, producing a mart that looks complete and correct, built entirely from data that is now three days stale. Nothing in that pipeline raises an error, because nothing is technically broken — new rows just aren't arriving.

Source freshness closes this gap. When a source table is declared with aloaded_at_field — a column recording when each row was loaded, not when the business event happened — dbt can compare the most recent value of that field against the current time and decide whether the table is fresh enough to trust.

version: 2

sources:
  - name: raw
    database: analytics
    schema: raw
    tables:
      - name: orders
        loaded_at_field: _fivetran_synced
        freshness:
          warn_after: {count: 6, period: hour}
          error_after: {count: 24, period: hour}
      - name: customers
        loaded_at_field: _fivetran_synced
        freshness:
          warn_after: {count: 12, period: hour}
          error_after: {count: 48, period: hour}

Running dbt source freshness executes, for each declared source table, roughly the equivalent of SELECT MAX(_fivetran_synced) FROM raw.orders, then compares the age of that maximum timestamp against warn_after and error_after. If the newest row is older than warn_after, the check reports a warning but does not fail the run. If it is older than error_after, the check fails outright — and in a well-built pipeline, that failure is wired to stop the rest of the run before any mart gets rebuilt from stale data.

dbt source freshness

14:02:11  Concurrency: 4 threads (target='prod')
14:02:12  1 of 2 START freshness of raw.orders ......................... [RUN]
14:02:12  2 of 2 START freshness of raw.customers ...................... [RUN]
14:02:13  1 of 2 WARN freshness of raw.orders .......................... [WARN in 0.41s]
14:02:13  2 of 2 PASS freshness of raw.customers ....................... [PASS in 0.38s]
14:02:13
14:02:13  Done.
14:02:13  Warnings: raw.orders is 7 hours, 12 minutes past its warn_after threshold of 6 hours.

The practical value here is timing. Without a freshness check, the first sign of a stalled orders pipeline is usually a confused Slack message from a business stakeholder asking why yesterday's revenue dashboard looks flat — hours or days after the actual problem started, and after several rebuilt marts have already baked stale data into reports people made decisions from. With a freshness check running on a schedule (typically right before the main dbt build), the warning shows up as soon as the raw pipeline first falls behind schedule, before a single downstream mart has been touched.

ref() Resolves a Model's Fully-Qualified Name — And Adapts to Whatever Environment Is Running

Where source() points at raw tables dbt did not build, ref() points at another dbt model — something dbt itself will build (or has already built) during this same run. Mechanically, ref() is a Jinja function. When dbt compiles a model's SQL, every{{ ref('some_model') }} call is replaced with the fully-qualified database, schema, and table (or view) name that some_model will resolve to, for whichever environment this specific run is targeting.

That last clause is the part that makes ref() more than a glorified string substitution. The same model file, containing the exact same ref('stg_orders') call, compiles to a completely different fully-qualified name depending on who is running it and against which target. A developer running dbt run --target dev on their own laptop getsdev_asil.stg_orders. The scheduled production job runningdbt run --target prod gets prod.stg_orders. Nobody wrote environment-specific branching logic to make that happen — it falls directly out of howref() resolves names using the active connection profile's configured schema.

-- models/marts/daily_revenue.sql (the file, unchanged across environments)
SELECT
  DATE(order_ts) AS order_date,
  COUNT(*) AS order_count,
  SUM(total_usd) AS revenue_usd
FROM {{ ref('stg_orders') }}
WHERE status NOT IN ('cancelled', 'fraud')
GROUP BY 1

-- compiled output when run with: dbt run --target dev
SELECT
  DATE(order_ts) AS order_date,
  COUNT(*) AS order_count,
  SUM(total_usd) AS revenue_usd
FROM dev_asil.stg_orders
WHERE status NOT IN ('cancelled', 'fraud')
GROUP BY 1

-- compiled output when run with: dbt run --target prod
SELECT
  DATE(order_ts) AS order_date,
  COUNT(*) AS order_count,
  SUM(total_usd) AS revenue_usd
FROM prod.stg_orders
WHERE status NOT IN ('cancelled', 'fraud')
GROUP BY 1

Without this, every team running dbt across dev, CI, and prod would need either separate copies of every model file per environment, or hand-written Jinja conditionals scattered through every model checking which target is active — both approaches that scale terribly and invite exactly the kind of silent divergence between environments that ref() is designed to prevent.ref() means one canonical model file works correctly, unmodified, in every environment the project is ever run against, including one that does not exist yet — a new CI schema stood up next month resolves ref('stg_orders') correctly without a single line of that model's SQL changing.

Static Parsing: How dbt Knows Execution Order Before Running Anything

A dbt project can contain hundreds of models. Before dbt runs a single one of them, it has to decide the order — stg_orders must finish before fct_orders starts, which must finish before daily_revenue starts. dbt figures this out without executing any SQL at all, through a step called parsing: before any model runs, dbt reads every model file's Jinja and finds every ref() and source() call inside it, without evaluating the SQL itself.

Each ref('model_x') found inside model_y's file becomes a directed edge in a graph: an arrow from model_x to model_y, meaning model_xmust be built first. Do this across every model file in the project and the result is the DAG — Directed Acyclic Graph — a complete map of what depends on what, built entirely from these statically-discovered function calls, before a single CREATE TABLE orCREATE VIEW is executed against the warehouse.

# models/staging/sources.yml
version: 2
sources:
  - name: raw
    database: analytics
    schema: raw
    tables:
      - name: orders
        loaded_at_field: _fivetran_synced

-- models/staging/stg_orders.sql
SELECT
  order_id,
  customer_id,
  order_ts,
  LOWER(status) AS status,
  total_usd
FROM {{ source('raw', 'orders') }}
WHERE order_id IS NOT NULL

-- models/marts/fct_orders.sql
SELECT
  order_id,
  customer_id,
  order_ts,
  status,
  total_usd
FROM {{ ref('stg_orders') }}
WHERE status != 'test_order'

-- models/marts/daily_revenue.sql
SELECT
  DATE(order_ts) AS order_date,
  COUNT(*) AS order_count,
  SUM(total_usd) AS revenue_usd
FROM {{ ref('fct_orders') }}
WHERE status NOT IN ('cancelled', 'fraud')
GROUP BY 1

Parsing this project finds one source() call (in stg_orders) and tworef() calls (in fct_orders and daily_revenue). From just those three function calls, dbt derives the entire execution order without being told it explicitly by any config file or orchestration script:

raw.orders (source, not built by dbt)
   │
   ▼  {{ source('raw', 'orders') }}
stg_orders
   │
   ▼  {{ ref('stg_orders') }}
fct_orders
   │
   ▼  {{ ref('fct_orders') }}
daily_revenue

dbt run execution order: stg_orders -> fct_orders -> daily_revenue
(raw.orders is never "run" -- it is read, checked for freshness, never built)

This is exactly why the anti-pattern from Part 02 — a hardcoded FROM prod.stg_ordersinstead of {{ ref('stg_orders') }} — is not a stylistic shortcut but a correctness bug waiting to happen. If fct_orders hardcodes the reference instead of calling ref(), dbt's parser finds no dependency edge between the two models at all. Rundbt run and there is no guarantee stg_orders finishes — or even runs — before fct_orders does; they could execute in either order, or in parallel across threads, and fct_orders could read a stale or half-built version of the table it silently depends on. The bug does not show up every time — it shows up intermittently, exactly the kind of failure that is hardest to reproduce and debug.

Once dbt Has the Graph, You Can Run Slices of It

The practical payoff of having a real, statically-derived DAG (rather than a project where humans manually order scripts) is that dbt's --select flag can run precise subsets of it using graph operators, without you ever having to reason about ordering by hand.

dbt run --select stg_orders           # just this one model
dbt run --select stg_orders+          # this model and everything downstream of it
dbt run --select +fct_orders          # this model and everything upstream of it
dbt run --select stg_orders+2         # downstream, but only 2 levels deep
dbt run --select tag:finance          # every model tagged "finance" in its config
dbt build --select state:modified+    # anything changed since a saved manifest, plus its downstream

Every one of these operators is only possible because the DAG already exists before the run starts.stg_orders+ means "walk every downstream edge from this node" — dbt can answer that instantly because the edges were already computed during parsing. None of this works if models reference each other through hardcoded table names instead of ref()/source()— there would be no graph to walk in the first place, just a pile of SQL files dbt has no way to relate to one another.

A Complete Source-to-Staging-to-Mart Chain, End to End

Putting everything in this module together: a full, runnable three-model chain, starting from a raw source table nobody at the dbt layer built, through a staging model, to a mart a BI tool would actually query.

Step 1 — declare the source

version: 2

sources:
  - name: raw
    database: analytics
    schema: raw
    tables:
      - name: orders
        description: "Raw order events, loaded by Fivetran roughly every 15 minutes."
        loaded_at_field: _fivetran_synced
        freshness:
          warn_after: {count: 6, period: hour}
          error_after: {count: 24, period: hour}

Step 2 — build the staging model, referencing the source

SELECT
  order_id::varchar        AS order_id,
  customer_id::varchar     AS customer_id,
  order_ts::timestamp      AS order_ts,
  LOWER(status)            AS status,
  total_usd::numeric(12,2) AS total_usd,
  _fivetran_synced         AS loaded_at
FROM {{ source('raw', 'orders') }}
WHERE order_id IS NOT NULL

Step 3 — build a mart, referencing the staging model

SELECT
  order_id,
  customer_id,
  order_ts,
  status,
  total_usd
FROM {{ ref('stg_orders') }}
WHERE status != 'test_order'

Step 4 — build a second mart, referencing the first mart

SELECT
  DATE(order_ts) AS order_date,
  COUNT(*)       AS order_count,
  SUM(total_usd) AS revenue_usd
FROM {{ ref('fct_orders') }}
WHERE status NOT IN ('cancelled', 'fraud')
GROUP BY 1
ORDER BY 1

Running dbt build against this project executes, in order: the source freshness check against raw.orders, then stg_orders, then fct_orders, thendaily_revenue — every step of that order derived purely from thesource() and ref() calls above, with zero manual sequencing logic written anywhere in the project.

dbt build

14:10:02  Running with dbt=1.8.0
14:10:02  Found 3 models, 1 source, 0 tests
14:10:03  1 of 4 START freshness of raw.orders .......................... [RUN]
14:10:03  1 of 4 PASS freshness of raw.orders ........................... [PASS in 0.35s]
14:10:04  2 of 4 START sql view model dev_asil.stg_orders ............... [RUN]
14:10:04  2 of 4 OK created sql view model dev_asil.stg_orders .......... [SUCCESS 1 in 0.62s]
14:10:05  3 of 4 START sql table model dev_asil.fct_orders .............. [RUN]
14:10:06  3 of 4 OK created sql table model dev_asil.fct_orders ......... [SUCCESS 1 in 1.14s]
14:10:07  4 of 4 START sql table model dev_asil.daily_revenue ........... [RUN]
14:10:08  4 of 4 OK created sql table model dev_asil.daily_revenue ...... [SUCCESS 1 in 0.89s]
14:10:08
14:10:08  Completed successfully

schema.yml — Attaching Descriptions and Tests to the Same Nodes ref() and source() Point At

Declaring a source in `sources.yml` is also where documentation and tests attach. The same `version: 2` YAML file that names a source table can carry a human-readable description for every table and column, plus generic tests like `not_null` and `unique` — all keyed to the exact same node that `ref()` and `source()` resolve. This matters because it means documentation is never disconnected from the dependency graph: the lineage graph, the freshness check, and the documentation site are all built from the same declared source and model definitions, not three separate systems that could drift out of sync with each other.

version: 2

sources:
  - name: raw
    database: analytics
    schema: raw
    tables:
      - name: orders
        description: "Raw order events loaded by Fivetran from the app's Postgres database every 15 minutes."
        loaded_at_field: _fivetran_synced
        columns:
          - name: order_id
            description: "Primary key of the orders table in the source application."
            tests:
              - unique
              - not_null
          - name: customer_id
            description: "Foreign key to the customers table."
            tests:
              - not_null

version: 2

models:
  - name: stg_orders
    description: "One row per order, lightly typed and cleaned from raw.orders. No business logic applied here — see fct_orders for filtered, business-ready order data."
    columns:
      - name: order_id
        description: "Primary key, inherited from the raw source."
        tests:
          - unique
          - not_null
      - name: status
        description: "Lowercased order status, one of the values checked by fct_orders downstream."

Notice the description on `stg_orders` explicitly points a future reader toward `fct_orders` for business logic — this is a small but real habit worth building. Because `dbt docs generate` renders every one of these descriptions directly onto the lineage graph, a well-written description turns the generated docs site into an actual map of the project a new team member can read, rather than a bare diagram of boxes and arrows with no explanation of what each box means or why it exists.

dbt docs generate — Seeing the DAG You Built From ref() and source()

Everything this module has covered so far — sources declared in YAML, models chained together with `ref()`, the DAG derived by static parsing — culminates in a single command: `dbt docs generate` followed by `dbt docs serve`. This builds a static documentation site with an interactive lineage graph, rendering every node your project's `ref()` and `source()` calls describe as a box, and every dependency edge as an arrow between them.

dbt docs generate
dbt docs serve

14:22:01  Running with dbt=1.8.0
14:22:01  Found 3 models, 1 source, 0 tests
14:22:02  Building catalog
14:22:03  Catalog written to ./target/catalog.json
14:22:03
14:22:03  Serving docs at 0.0.0.0:8080
14:22:03  To access from a remote machine, you must specify the --host flag

Opening the served site and clicking into the lineage graph shows exactly the picture this module built by hand in Part 05's worked example — `raw.orders` (rendered distinctly as a source, not a model) with an arrow into `stg_orders`, an arrow into `fct_orders`, an arrow into `daily_revenue`. Every arrow in that rendered graph corresponds to one `ref()` or `source()` call somewhere in the project's SQL files — nothing else produces an edge.

This is also the fastest way to catch the anti-pattern from Part 02 in a real project: open the generated graph and look for any mart or model with no incoming edges at all, sitting disconnected from everything else. A model with genuinely no dependencies is rare — almost always, a disconnected node in the generated graph means some upstream reference in that model's SQL is hardcoded instead of calling `ref()`/`source()`, exactly as Part 05's callout describes.

Real Projects Have More Than One Source — Naming and Organizing Them

The worked example in Part 07 used one source (`raw`) with one table (`orders`). Real projects almost always have several distinct raw data providers landing in different schemas or even different databases — a Fivetran connector syncing the application's Postgres database, a separate Stripe connector landing payment data, a marketing team's ad-spend data loaded by a completely different tool. Each of these is typically declared as its own named source, even when they live in the same physical database, because the `name` in a source declaration is what `source()` calls use to disambiguate between them.

version: 2

sources:
  - name: app_postgres
    database: analytics
    schema: raw_postgres
    tables:
      - name: orders
        loaded_at_field: _fivetran_synced
      - name: customers
        loaded_at_field: _fivetran_synced

  - name: stripe
    database: analytics
    schema: raw_stripe
    tables:
      - name: charges
        loaded_at_field: _fivetran_synced
        freshness:
          warn_after: {count: 2, period: hour}
          error_after: {count: 6, period: hour}
      - name: refunds
        loaded_at_field: _fivetran_synced

  - name: marketing
    database: analytics
    schema: raw_marketing
    tables:
      - name: ad_spend
        loaded_at_field: loaded_at
        freshness:
          warn_after: {count: 24, period: hour}

Referencing any one of these from a model is unambiguous because `source()` always takes two arguments — the source name, then the table name — so `{{ source('stripe', 'charges') }}` and `{{ source('app_postgres', 'customers') }}` can never be confused with each other even though both ultimately live in the same `analytics` database. This is also why freshness thresholds are set per source table rather than globally — a payment provider's data landing every few minutes reasonably has a much tighter `warn_after` than a marketing team's daily ad-spend export, and declaring them separately is what makes that distinction possible.

-- models/staging/stg_charges.sql
SELECT
  charge_id,
  order_id,
  amount_usd,
  charge_status,
  _fivetran_synced AS loaded_at
FROM {{ source('stripe', 'charges') }}
WHERE charge_id IS NOT NULL

Sources Can Carry Tests Too, Not Just Models

It is easy to assume tests only belong on models, since that is where most of a project's YAML testing lives. But the same generic tests covered in Module 08 of this track (Testing: Generic and Singular Tests) — `unique`, `not_null`, `accepted_values`, `relationships` — can be attached directly to a source's columns, as shown briefly in Part 08. This matters because it lets you catch a problem in the raw data itself, before it ever reaches a single dbt-built model.

dbt test --select source:raw

14:31:02  Running with dbt=1.8.0
14:31:02  Found 2 models, 1 source, 3 tests
14:31:03  1 of 3 START test not_null_raw_orders_order_id ................ [RUN]
14:31:03  1 of 3 PASS not_null_raw_orders_order_id ...................... [PASS in 0.28s]
14:31:03  2 of 3 START test unique_raw_orders_order_id .................. [RUN]
14:31:04  2 of 3 PASS unique_raw_orders_order_id ......................... [PASS in 0.31s]
14:31:04  3 of 3 START test not_null_raw_orders_customer_id ............. [RUN]
14:31:04  3 of 3 FAIL 14 not_null_raw_orders_customer_id ................ [FAIL 14 in 0.29s]
14:31:04
14:31:04  Done. PASS=2 WARN=0 ERROR=0 FAIL=1 TOTAL=3

That failing test — 14 rows in `raw.orders` with a null `customer_id` — is exactly the kind of problem you want surfaced at the source, before it silently flows through `stg_orders` into `fct_orders` and eventually corrupts a join or an aggregation several models downstream, where it would be far harder to trace back to its actual origin. Testing at the source boundary is the earliest and cheapest point in the whole DAG to catch a data quality problem.

Freshness Thresholds Can Live at the Source Level or the Table Level — And --select source: Targets Both

Part 03 showed `freshness` configured per table. In a project with many tables under one source that mostly share the same acceptable staleness, repeating the same `warn_after`/`error_after` block on every table is unnecessary. A `freshness` block can also be set once at the source level, and every table under that source inherits it unless a specific table overrides it with its own block — the same override precedence pattern Part 07 described for `+materialized` in `dbt_project.yml`.

version: 2

sources:
  - name: raw
    database: analytics
    schema: raw
    freshness:
      warn_after: {count: 24, period: hour}
      error_after: {count: 48, period: hour}
    loaded_at_field: _fivetran_synced
    tables:
      - name: orders
        # no override -- inherits the 24h/48h default above
      - name: customers
        # no override -- inherits the 24h/48h default above
      - name: payments
        freshness:
          warn_after: {count: 2, period: hour}
          error_after: {count: 6, period: hour}
        # payments overrides the default: this data needs to be much fresher

This mirrors exactly why the per-model/per-folder override pattern from Part 07 exists for materializations: set a sensible default once, and only write out the exception for the one table that genuinely needs different treatment, rather than repeating the common case everywhere and risking it silently drifting out of sync across dozens of near-identical table declarations.

The same `--select` graph operators covered in Part 06 also understand sources directly, using the `source:` prefix — useful for running freshness checks or tests scoped to exactly one source rather than the whole project.

dbt source freshness --select source:raw          # only the raw source's tables
dbt source freshness --select source:stripe        # only the stripe source's tables
dbt test --select source:raw+                       # source:raw's tests, plus everything downstream

Five Misconceptions About Sources and ref()

What This Looks Like on Day One

5 Interview Questions — With Complete Answers

Mistakes Beginners Make Constantly

Errors You Will Hit — And Exactly Why They Happen