Dashboard stability audit — dbt freshness checks and schema contracts for a finance dashboard

Context

A Series B SaaS with a healthy analytics function. dbt models in production, a Looker dashboard the CFO opened every Monday morning. The numbers were usually right. They were also wrong often enough (three or four times a month) that the analytics lead had started prefacing every Monday standup with “the dashboard looks weird, give me an hour.” Nobody could predict which Monday it would be.

Problem

Three upstream tables (one from the product backend, one from a billing tool, one from the CRM) were silently shape-shifting. A column would get renamed in the product database during a Friday deploy; dbt would happily ingest the rename as a new column with NULLs for the prior data; the model would compile clean; the dashboard would show -38% revenue on Monday. The analytics team had no way to see this coming because dbt was running on a schedule and the test suite didn’t cover schema drift.

Approach

Two-week scope, no model rewrites. The work was three things: identify which upstreams were the actual sources of the Monday surprises, add freshness and schema-shape checks that would catch the drift at the right layer, and hand the analytics team a one-page playbook for what to do when the checks fire.

Solution

• Upstream audit. Ran a git-blame across the last 90 days of dbt model failures and analytics-channel incidents to identify the three tables responsible for the recurring Monday drift.
• dbt freshness + schema-shape tests. Added dbt source freshness checks on the three tables, plus custom schema tests that fail loudly if column names, types, or NOT-NULL constraints change unexpectedly. Failures gate the dbt run so a bad upstream stops the pipeline instead of corrupting the dashboard.
• Schema contracts. Documented the three upstream contracts in the dbt project README: names, types, allowed-null columns, deprecation policy. The contracts got posted to the product team’s Slack so backend changes to those tables hit a review step.
• Playbook. One page: what each check covers, what the failure modes look like, who owns each upstream, how to silence a known-good change without disabling the check entirely.

Outcome

The Monday surprises stopped. The analytics lead reported zero “the dashboard is wrong” threads in the 60 days after handoff. More importantly, three upstream changes that would have broken the dashboard pre-audit were caught and resolved at the contract layer before they hit production. The CFO signed off on a move from monthly to weekly close on the back of the audit; the dashboard was finally trustworthy on the cadence the business actually wanted.