dbt

I tested dbt by setting up a Cloud project on a Snowflake warehouse, modelling a staging-to-marts pipeline of 35 SQL models with referential lineage, writing schema and uniqueness tests, and running scheduled jobs over a week. The cohort I had in mind was data-engineering and analytics-engineering teams running modern data stack workflows with a warehouse as the source of truth. This is a fresh cohort for the catalog: no prior hands-on review covered data transformation and analytics engineering tooling in the prior 65.

The ref-and-source paradigm is the actual unlock. Writing each model as a SQL file with `ref()` references built a dependency graph that dbt traversed correctly on every build, which removed the order-of-execution headache that plagues hand-rolled SQL pipelines. The lineage UI in dbt Cloud surfaced upstream impact when I changed a column type in a staging model, showing exactly which downstream marts and exposures would break before I shipped. Tests (not_null, unique, accepted_values) caught a bad join in a customer-revenue mart on the first run, which is the kind of silent data quality bug that ruins downstream reports for weeks before anyone notices.

What broke me was the SQL-fluency requirement and the dbt-Cloud-versus-self-hosted decision. dbt is unforgiving if your team is not comfortable with SQL: business users wanting to "build a model" cannot do that without learning Jinja plus dbt patterns, which means the analytics-engineer role becomes a permanent dependency. The Cloud product pricing per developer seat plus job-execution minute scales aggressively past 10 developers, and several teams I have seen end up running a self-hosted dbt-core setup to dodge the cost, which then requires owning the infrastructure operationally. The IDE in Cloud is functional but lags VSCode plus the dbt extension on serious development experience.

dbt is the right pick for analytics engineering teams running a modern data stack with a warehouse, dedicated data engineers, and a strong SQL-fluency culture. Skip it if your team includes business users wanting to model data without SQL (use Coalesce, Y42, or even Looker PDTs for friendlier surfaces), if your data volume is small enough that a few stored procedures handle it (dbt is over-engineered for sub-100GB warehouses), or if you cannot afford the Cloud per-seat cost and lack the bandwidth to operate self-hosted dbt-core (you will end up with a half-maintained pipeline).