Pipelines with lineage you can trace.

It ran green.
The numbers were wrong.

Connectors or change data capture with schema-change handling, so an upstream column rename is caught rather than silently dropping data.

Raw, staging and mart layers with documented grain per table, so nobody is joining production replicas and guessing at cardinality.

One definition per metric, versioned and reviewed, consumed by every dashboard and every API rather than reimplemented per tool.

Assertions on freshness, volume, uniqueness and business rules, wired into the run so a failure stops the pipeline before a reader sees it.

Automatic lineage from source to dashboard, so the impact of a change is answerable in seconds and incidents are traceable.

Dependency-aware scheduling with retries and backfill, and alerts that page someone when a business-critical table is stale.

Two dashboards disagree, so every meeting starts by litigating which figure is right instead of deciding anything.

A job broke on Friday and nobody noticed until Wednesday, by which time three reports had been circulated.

The platform runs on knowledge held by a single engineer, and nobody else can safely change a model.

A model or agent project has stalled because the data behind it is inconsistent, undocumented or unavailable.

Compute spend grows faster than data volume, which is nearly always a modelling and query problem.

An auditor or regulator wants to know where a number came from, and the answer currently requires a person.

Four hundred models,documented and tested.

Staging layers from the schema

Staging models, type casting and naming standardisation generated per source table. Hand-writing the two-hundredth staging model is where consistency and attention both fail.

Every column, not the important ones

Tests and descriptions generated per column from the schema and observed data, then reviewed. Previously the work that got trimmed under deadline and never came back.

Legacy queries, translated consistently

Stored procedures and legacy SQL translated into modelled, tested transformations across hundreds of files, uniformly rather than by attrition.

Business definitions are agreed with people, not inferred.

What counts as an active user or recognised revenue is a commercial decision, settled with finance and product in a room. Generated volume is what makes room for that conversation inside the budget.

When a warehouse
is not the answer

For a small team with a single source, a read replica and good SQL is genuinely the right answer. A warehouse adds cost and a maintainer.

You cannot model behaviour nobody instrumented. If the product does not emit the events, tracking comes first.

Transformation code without a maintainer drifts within two quarters and then nobody trusts it. The owner matters more than the tooling.

Real-time infrastructure costs several times more to run and operate. Very few decisions genuinely need second-level freshness.

Whether business-critical tables were ready when they were needed, per table rather than as an aggregate.

Share of models with meaningful assertions, and how often they catch something. Tests that never fail are usually testing nothing.

Metrics in exec reporting with a single documented definition in code. Anything uncovered is named rather than assumed consistent.

Distance between reported revenue and the ledger. The number that decides whether finance trusts the platform.

Compute attributed to transformations, so an expensive model is visible before the quarterly bill.

Days from request to a tested, documented, modelled table. It should be days rather than a quarter.

What the data layer
makes possible