What is dbt?

What a "Transformation Layer" Even Is

Before you can understand what dbt is, you need to understand the gap in a data stack that it fills. Picture a modern company — call it a mid-size grocery delivery startup, FreshCart. FreshCart has a production Postgres database recording every order, a Salesforce instance tracking sales leads, a Stripe account processing payments, and a mobile app sending click events. None of that data lives together, none of it is shaped for answering business questions, and most of it is structured for the needs of the application that produced it, not for analysis.

Getting from "data exists somewhere, in whatever shape the source system produced it" to "a clean table an analyst can query to answer 'what was our revenue by region last month'" requires three distinct kinds of work. First, the data has to be extracted from each source system. Second, it has to be loaded somewhere central — a data warehouse. Third, it has to be transformed: cleaned, renamed, joined across sources, deduplicated, aggregated, and reshaped into tables that actually answer business questions rather than just mirroring whatever an application's internal database schema happened to look like.

That transformation layer is not a nice-to-have step you could skip. Raw data straight from a production database is usually unusable directly for analysis: it might have technical column names like usr_stat_cd instead of user_status, it might spread related facts across a dozen normalized tables that need joining, it might contain soft-deleted rows that should be filtered out, duplicate records from a flaky sync process, or nested JSON blobs that need to be flattened into columns. Someone — or something — has to do that reshaping work before a business question can be answered reliably. That "someone" is the transformation layer, and dbt is the tool built specifically to be that layer.

A concrete before-and-after, in one table

It helps to see the raw-versus-transformed contrast side by side, using the same FreshCart orders data. The left column below is roughly what a row looks like fresh out of the production database. The right column is what an analyst actually wants to query.

ord_id | cust_id | ord_stat | ord_placed_ts          | amt_cents | is_deleted
78341  | 9042    | 3        | 2026-03-11T14:02:33Z   | 4599      | false

order_id | customer_id | order_status | order_placed_at      | amount_usd
78341    | 9042        | completed    | 2026-03-11 14:02:33  | 45.99

Notice everything that changed between the two: cryptic column names became readable ones, a numeric status code (3) became a human-readable label (completed), cents became dollars, and the soft-deleted flag was used to filter the row rather than exposed as a column an analyst has to remember to check every time. None of that is exotic engineering — it's exactly the kind of mechanical, repetitive cleanup that used to live scattered across dozens of individual analysts' personal SQL scripts before a shared transformation layer existed.

ETL vs ELT — The Shift That Created the Need for dbt

If you take away only one piece of context from this entire module, make it this one: dbt exists because the data industry moved from ETL to ELT, and dbt is purpose-built to be the "T" in that new order. Understanding why that shift happened tells you why dbt looks the way it does.

ETL — transform before loading

For a long time, the standard architecture was extract, transform, load. Data was extracted from source systems, piped through a separate transformation engine or ETL tool (Informatica, Talend, SSIS) running on its own dedicated servers, and only the final, cleaned, business-ready tables were loaded into the data warehouse. The warehouse itself was treated as a precious, expensive resource — on-premises warehouses had fixed hardware, licensing costs tied to server capacity, and limited, hard-to-scale compute. You did not want to waste that scarce warehouse compute on the "dirty work" of transformation; you did that work elsewhere and only loaded the finished product.

ELT — load first, transform where the data lives

Cloud data warehouses changed the economics entirely. Snowflake, BigQuery, and Redshift decoupled storage from compute and made compute elastic — you can spin up more processing power for the duration of a transformation job and pay only for what you used, rather than provisioning a fixed server for peak load year-round. Storage itself also became cheap enough that landing raw, unprocessed data and keeping it around indefinitely was no longer wasteful.

Once compute is cheap, elastic, and lives inside the warehouse itself, there is no longer a reason to transform data on a separate system before loading it. It becomes simpler and more flexible to load raw data into the warehouse immediately after extraction, and then run transformations as SQL queries against the warehouse's own compute — extract, load, transform. This has real practical advantages beyond just cost: raw data is preserved in the warehouse in its original form, so if a transformation turns out to be wrong, you can fix the SQL and simply re-run it against the still-intact raw data, rather than having to re-extract from the source system.

ETL (older model):
  source system --extract--> ETL server/tool --transform--> warehouse
                                    ^
                            transformation happens HERE,
                            on a separate system, before loading

ELT (current model, what dbt is built for):
  source system --extract--> warehouse (raw data lands as-is)
                                  |
                                  v
                          dbt runs SQL transformations
                          HERE, inside the warehouse itself,
                          using the warehouse's own compute

Why "transform where the data already lives" is not just a cost story

The cost argument is the headline reason ELT became viable, but there is a second, quieter benefit that turned out to matter just as much in practice: keeping raw data around, untouched, inside the warehouse. Under classic ETL, once the transformation server produced its final output and that intermediate raw data was discarded or archived elsewhere, discovering that a transformation step three years ago dropped a column you now need meant re-extracting from the source system — which might no longer even exist in the same form, or at all.

Under ELT, the raw freshcart_raw.orders table sits in the warehouse indefinitely (or for as long as retention policy allows), completely unaffected by any bug in a dbt model built on top of it. If a staging model turns out to have miscalculated a column, you fix the SQL and simply re-run dbt — the raw source data needed to recompute the correct answer never went anywhere. This re-runnability is a direct, practical consequence of the ELT ordering, not a separate feature dbt had to build.

What dbt Actually Is, and Its Exact Scope Boundary

At its core, dbt is a command-line tool and framework that lets you write data transformations as SQL SELECT statements — called models — and handles everything around turning those SELECT statements into real tables and views in your warehouse: figuring out the order to run them in based on their dependencies, running automated data quality tests against them, and generating documentation and a visual dependency graph from the same codebase.

Concretely, a dbt model is a single .sql file containing one SELECT statement. You do not write CREATE TABLE or INSERT statements yourself — dbt wraps your SELECT in the appropriate DDL/DML automatically, based on how you've configured that model (typically as a table or a view). You reference other models not by hardcoding a schema and table name, but through a function called ref(), which dbt resolves at compile time and uses to build a dependency graph automatically — covered in full mechanical detail in the next module.

select
    order_id,
    customer_id,
    order_status,
    cast(order_placed_at as timestamp) as order_placed_at,
    amount_cents / 100.0 as amount_usd
from {{ source('freshcart_raw', 'orders') }}
where order_status is not null

Running dbt run takes that file, resolves the source() call into the actual raw table name, and executes the equivalent of CREATE OR REPLACE VIEW stg_orders AS <that SELECT> against your warehouse. That is the entire mechanic, at a beginner level — the deep mechanics of compilation and execution order are the subject of Module 02.

The critical scope boundary: dbt does not extract or load data

This is the single most common misunderstanding beginners bring to dbt, so it's worth stating as plainly as possible: dbt has no functionality whatsoever for connecting to source systems like a production application database, a SaaS API, or a file drop. It cannot pull data out of Postgres, Salesforce, or Stripe. Its only interaction with the world is running SQL against a warehouse it is already connected to, on data that is already sitting there.

A quick self-check: is this dbt's job or not?

Because the scope boundary is so frequently misunderstood, it's worth running through a few concrete requests and sorting which layer each one actually belongs to.

The Three Pillars: Models, Tests, and Documentation

dbt's value comes from bundling three things that used to live separately (if they existed at all) into one workflow built around the same codebase: the transformation logic itself, automated checks on that logic's output, and documentation describing what everything means.

Models — SQL as version-controlled code

A model is just a SELECT statement saved as a .sql file inside a dbt project, which is itself an ordinary git repository. This single fact is what unlocks everything else: because your transformation logic is plain text in version control, you get pull requests, code review, diffs that show exactly what changed in a model, a full history of who changed what and why, and the ability to revert a bad change in seconds. None of that is possible when transformation logic lives as ad hoc scripts on someone's laptop or buried inside a GUI tool's proprietary project file.

Tests — catching bad data before it reaches a dashboard

dbt lets you declare tests against your models directly alongside the model definitions — for example, asserting that order_id is unique and never null, or that every customer_id in the orders table actually exists in the customers table. Running dbt test executes these as SQL queries that should return zero failing rows; if they return any rows, the test fails and you know immediately, rather than an analyst noticing a dashboard number looks wrong three weeks later.

models:
  - name: stg_orders
    columns:
      - name: order_id
        tests:
          - unique
          - not_null
      - name: customer_id
        tests:
          - not_null
          - relationships:
              to: ref('stg_customers')
              field: customer_id

$ dbt test
Running with dbt=1.8.0
Found 2 models, 3 tests

1 of 3 START test unique_stg_orders_order_id ................. [RUN]
1 of 3 PASS unique_stg_orders_order_id ....................... [PASS in 0.41s]
2 of 3 START test not_null_stg_orders_order_id ............... [RUN]
2 of 3 PASS not_null_stg_orders_order_id ..................... [PASS in 0.38s]
3 of 3 START test relationships_stg_orders_customer_id ....... [RUN]
3 of 3 FAIL 4 relationships_stg_orders_customer_id ........... [FAIL 4 in 0.52s]

Completed with 1 error and 0 warnings:
  Failure in test relationships_stg_orders_customer_id (models/schema.yml)
  Got 4 results, configured to fail if != 0

That failing test just caught four orders referencing a customer_id that doesn't exist in the customers table — a real data quality problem, caught automatically, before anyone built a dashboard on top of bad data.

Documentation — generated from the same source of truth

Because models, their columns, and their tests are all declared in the project, dbt can generate a searchable documentation site and a visual DAG (directed acyclic graph, covered in depth in Module 02) directly from that same code — no separate wiki to keep in sync, no diagram that goes stale the moment someone adds a model.

models:
  - name: stg_orders
    description: >
      One row per order, cleaned and standardized from the raw
      orders table. Soft-deleted (test) orders are filtered out.
    columns:
      - name: order_id
        description: Primary key. Unique identifier for an order.
      - name: amount_usd
        description: Order total, converted from cents to dollars.

Running dbt docs generate followed by dbt docs serve reads all of this — model descriptions, column descriptions, declared tests, and the dependency graph built from ref() calls — and produces a browsable, searchable site automatically. Anyone on the team can look up what amount_usd means, which tests run against it, and which models depend on it, without asking the person who wrote it or hunting through old Slack threads.

The kinds of tests you'll actually reach for

dbt ships a small set of built-in "generic" tests that cover the overwhelming majority of everyday data quality checks, and the community and dbt Labs both maintain packages of additional ones for more specific situations.

Beyond these built-ins, teams commonly write custom "singular" tests — a plain SQL query saved under a tests folder that should return zero rows if the data is healthy, for business rules too specific for a generic test to express, like "refunded orders should never have a positive revenue amount."

-- This test fails if it returns any rows at all
select
    order_id,
    amount_usd
from {{ ref('stg_orders') }}
where order_status = 'refunded'
  and amount_usd > 0

Why Companies Adopted dbt — and the Rise of "Analytics Engineering"

Before tools like dbt existed, transformation logic tended to live in one of two uncomfortable places. Either it lived in data engineers' pipeline code — Python or Spark jobs maintained by people whose primary job was infrastructure, not business logic, creating a bottleneck whenever a business question needed a new metric or a new join — or it lived in individual analysts' personal SQL scripts, copy-pasted and slightly modified from person to person, with no shared version history, no tests, and no way to know if changing one query would break someone else's downstream report.

dbt gave the SQL transformation layer the same tooling that software engineers had taken for granted for years: version control, code review, automated testing, CI/CD, and documentation generated from code. That combination let a new discipline take shape — analytics engineering — sitting between data engineering and data analysis. An analytics engineer owns the transformation layer: turning raw ingested data into clean, tested, documented models that analysts and BI tools can trust and query directly.

The business case companies made for adopting dbt typically comes down to trust and speed: trust, because tested and documented models catch data quality problems before they reach an executive dashboard; speed, because a well-organized dbt project with modular, reusable models lets a team answer a new business question by composing existing models rather than starting a transformation from scratch every time.

What changed day-to-day for the people doing the work

It's worth being concrete about what adopting dbt actually changes for an analyst or engineer's daily workflow, rather than leaving it abstract. Before, a typical change — say, fixing a miscalculated revenue metric — might mean finding whichever personal SQL script or dashboard query computed that metric, editing it in place, and hoping no one else's separate copy of similar logic needed the same fix. There was often no reliable way to know who else depended on that logic, and no record of why the number had changed if someone asked next quarter.

After adopting dbt, the same fix is made once, in one model, as a pull request. Anyone who depends on that model — tracked explicitly through ref(), as later modules cover — is automatically affected the next time the project runs, with no separate copies to hunt down. The change has a commit message, an author, a timestamp, and (ideally) a reviewer, forming a durable record of exactly what changed and why, which is precisely the kind of change-tracking software engineers have relied on for decades and which analytics work mostly lacked before this generation of tooling.

Why "dbt" Stands for "Data Build Tool" — and a Short, Accurate History

dbt is short for "data build tool" — a deliberately literal, unglamorous name reflecting what the tool does: it builds your data (models) the way a build tool like Make or Maven builds software, resolving dependencies and running steps in the right order.

dbt was created inside a consulting company called Fishtown Analytics, founded by Tristan Handy and Drew Banin, who were doing analytics consulting work for clients and kept reinventing the same SQL-organization patterns project after project — dbt began as an internal tool to standardize that work before being open-sourced. As dbt grew into its own product and business, Fishtown Analytics renamed itself dbt Labs to reflect that dbt had become the company's core focus rather than a side tool supporting consulting work.

dbt Core vs dbt Cloud

dbt Core is the open-source command-line engine — the actual compiler and executor that reads your project, resolves Jinja and ref()/source() calls, builds the dependency graph, and runs SQL against your warehouse. It is free, runs anywhere you can run a terminal, and is what most of the mechanics in this track describe.

dbt Cloud is dbt Labs' commercial, hosted product built around that same dbt Core engine. It adds a browser-based IDE for writing and running models, a built-in job scheduler so you don't need to wire up your own cron or Airflow trigger just to run dbt on a schedule, hosted documentation, and collaboration features aimed at teams. The underlying transformation logic — the SQL, the compilation, the DAG — is the same engine either way; dbt Cloud is infrastructure and convenience wrapped around dbt Core, not a different transformation language or a different set of capabilities.

A brief, accurate timeline

dbt's early versions in the mid-2010s were used internally at Fishtown Analytics for consulting engagements before being released as an open-source project. Adoption grew steadily through the later 2010s as more data teams moved onto cloud warehouses and needed exactly the kind of in-warehouse transformation tool dbt provided. dbt Labs later introduced dbt Cloud as a commercial product layered on top of the same open-source engine, and the company has continued to expand the core engine's capabilities (incremental models, more materializations, a growing package ecosystem) alongside dbt Cloud's own feature set.

dbt vs. Its Alternatives — A Fair Comparison

dbt is not the only way to solve the transformation problem, and it's worth understanding its real alternatives honestly rather than assuming it's an obviously correct default in every situation.

Hand-written SQL scripts, no framework

The simplest possible approach: a folder of .sql files, run manually or via a basic cron job, with no dependency tracking, no built-in testing, and no documentation generation. This is genuinely fine for a very small team with a handful of models and low complexity. It stops scaling quickly once you have dozens of interdependent models, because nothing tracks which models depend on which, nothing warns you when a change to one query breaks a downstream one, and there's no standardized way to test data quality.

GUI-based ETL/ELT tools

Tools with drag-and-drop transformation canvases let you build pipelines visually rather than in SQL files. This can be more approachable for teams with less SQL fluency, and some of these tools bundle ingestion and transformation together. The trade-off is that the transformation logic usually lives inside the tool's own proprietary project format rather than as portable, plain-text SQL, which makes version control, code review, and migrating away from the tool harder.

Dataform — Google's similar product

Dataform is conceptually very close to dbt: SQL-based models, a dependency graph built from ref()-style references, and built-in testing. Google acquired Dataform and integrated it directly into BigQuery, so it has a natural advantage for teams already standardized on BigQuery. dbt is warehouse-agnostic and has a larger, longer-established open-source community and ecosystem of packages. Neither tool has a decisive technical advantage over the other for most use cases — the practical deciding factor is usually which warehouse a team is on and which tool the team already knows, not a meaningful gap in capability.

A realistic decision scenario

Consider a hypothetical 12-person data team at a mid-size subscription business, currently maintaining about 40 hand-written SQL scripts that build their reporting tables, run via a single cron job on a shared server. Two engineers recently left, and no one else fully understands the dependency order those 40 scripts need to run in — the order was tribal knowledge those two engineers carried in their heads. A change to script 12 recently broke script 31 downstream, and nobody noticed for two weeks because there was no test that would have caught it.

This is close to the textbook scenario dbt was built for: the team is already firmly on a cloud warehouse (so ELT compute economics apply), the transformation logic already exists as SQL (so the migration is organizational, not a rewrite into a new language), and the pain points — unclear dependency order, no tests, tribal knowledge walking out the door — are exactly what a dependency graph, automated tests, and version control solve. A team in this position adopting dbt is not choosing it over a meaningfully better alternative in the abstract; they're choosing it because their actual, concrete problems match what dbt is built to address.

Contrast that with a two-person startup with three simple reporting queries and no reporting complexity to speak of yet. Introducing dbt there has a real cost — a new tool, a new project structure, a new deployment step — for a problem (uncoordinated, poorly-tested SQL at scale) that doesn't exist yet at that size. This is exactly why "should we adopt dbt" is not a yes/no question with one universal answer — it depends on whether the team's actual pain points match what the tool solves, the same honest evaluation this Part has tried to walk through rather than assert.

Who Actually Uses dbt, and What a Normal Day Looks Like

It helps to ground all of this in what using dbt actually looks like for the people who do it daily, rather than leaving the whole module at the level of architecture diagrams. The primary users of dbt are analytics engineers, though data analysts and data engineers both interact with dbt projects regularly too, just with different emphasis.

A typical day for an analytics engineer working in dbt involves picking up a request — a new metric, a bug in an existing model, a data quality alert from a failed test — checking out a branch, editing or adding a .sql model file (and its accompanying YAML for tests and documentation), running dbt locally against a personal development schema to verify the change works and doesn't break downstream models, and opening a pull request for review before merging. None of this requires touching a warehouse console directly or writing raw DDL by hand — the whole workflow happens through SQL files and git, exactly the way software engineers work with application code.

What this means for how you should learn dbt from here

Because the daily workflow is fundamentally "edit a SQL file, run it, check the result," the most effective way to build real fluency with dbt is not to memorize every configuration option up front, but to get a small project running locally as early as possible and start making small, concrete changes to it — exactly the sequence the next module in this track walks through, moving from this conceptual foundation into the mechanical details of compilation, the dependency graph, and execution order.

Common Misconceptions About dbt

Five beliefs that trip up almost everyone new to dbt — each one traces back to a specific Part above where the correction is explained in full.

Why This Actually Matters — Three Real Scenarios

Interview Questions You Should Be Able to Answer

Common Mistakes Beginners Make

Errors and Mix-Ups You'll Actually Hit

Module 01 is largely conceptual, so most of the "errors" a true beginner runs into are not stack traces from dbt itself yet — they're the mix-ups and false starts that come from applying the wrong mental model before you've run dbt for real. The list below mixes those early conceptual errors with the first genuine tool errors most learners hit in their first week.

`Compilation Error: Model 'stg_orders' depends on a node named 'orders' which was not found`

`Database Error: Table "RAW_DB"."RAW_SCHEMA"."ORDERS" does not exist`

`Runtime Error: Parser Error: syntax error at or near "AS"` (or similar SQL syntax errors)

`Warning: Nothing to do. Try checking your model configs and model specification args` (empty run)

`Encountered an error: Could not find profile named 'my_project'`

A sixth error, common in the first real run: a missing or misconfigured profile

Runtime Error
  Could not run dbt
  ProfileConfigError: The profile 'freshcart' in profiles.yml has invalid keys

A dbt project's dbt_project.yml declares which profile (a named connection configuration — warehouse type, account, credentials) it expects to use, and the actual connection details live in a separate profiles.yml file, deliberately kept outside the git-tracked project so credentials never end up committed to version control. This error means the profile name was found but something inside it doesn't match what dbt's adapter for that warehouse expects — usually a typo'd key name copied from an example for a different warehouse type (a BigQuery example field pasted into a Snowflake profile, for instance).

A seventh, very common early symptom: "I don't see my data anywhere"

A learner installs dbt, connects it to a warehouse, and runs their first model successfully — dbt reports success — but then can't find "their data" when they look around, because they were expecting to find it inside dbt somewhere rather than in the warehouse itself. This isn't a dbt error at all; it's the scope-boundary misconception from Part 03 showing up as a support question. The fix is simply to look in the actual warehouse (the schema dbt was configured to build into), where the table or view genuinely was created.