Skip to content

fix(dbt): honour on_low_score="fail" in FreshDataDbtTransform; unique audit files for same-alias models - #391

Merged
kevincostner17 merged 1 commit into
mainfrom
fix/dbt-transform-fail-audit
Sep 15, 2026
Merged

kevincostner17 merged 1 commit into
mainfrom
fix/dbt-transform-fail-audit

Conversation

@kevincostner17

Copy link
Copy Markdown
Contributor

Builds on #351 (merged); rebased onto main.

Summary

on_low_score="fail" ignored by FreshDataDbtTransform (#343)

  • run() raised TrustGateError only when fail_on_low_score=True, so a failing gate under on_low_score="fail" returned should_fail=True and the pipeline carried on. It now raises when result.should_fail, or when fail_on_low_score is set and the gate did not pass. The audit file is still written first.
  • New keyword-only run(raise_on_fail=True). gate_manifest calls run(raise_on_fail=False), so a failing model is recorded as failed and the run continues. The summary shape and fix(cli): clean errors for malformed inputs and honest dbt-gate exit codes #351's skipped / all_passed rules are unchanged.

Same-alias models overwrote each other's audit file (#344)

  • FreshDataDbtTransform gains audit_name, used as the audit file stem. It is checked with _validate_audit_table_name.
  • With output_dir, gate_manifest counts aliases across gated models (case-insensitively). Models that share an alias write <schema>.<alias>_audit.json, or <unique_id>_audit.json when the schema is missing or still not unique. Other models keep <alias>_audit.json. A schema that is not a valid file name is rejected and recorded as that model's error.
  • docs/integrations.md documents on_low_score="fail", raise_on_fail and the audit file names.

Behaviour change: FreshDataDbtTransform(on_low_score="fail").run() now raises on a failing gate; pass raise_on_fail=False to get the result back.

Tests

New tests/test_integrations/test_dbt_transform_fail_audit.py (real SQLAlchemy + sqlite, schemas via ATTACH, tmp_path):

  • on_low_score="fail" raises from run() and still writes the audit; run(raise_on_fail=False) returns the failing result; "warn"/"skip" don't raise; a passing gate never raises under either switch.
  • gate_manifest under on_low_score="fail" records failing models and gates every model.
  • Same alias in staging and marts writes two audit files with the right row counts; non-colliding models keep <alias>_audit.json; a schema-less collision falls back to unique_id, including a case-only collision.
  • Invalid schema and audit_name values ("", ., .., ../orders, a/b, a\b, /abs) raise ValueError and nothing is written outside the output directory.

Verification

  • ruff check .: passed; changed files are ruff format-clean
  • mypy src/freshdata: no issues
  • pytest -m "not online and not large" on the rebased branch: Python 3.12: 5150 passed, 13 skipped; Python 3.9: 5146 passed, 17 skipped

Closes #343
Closes #344

… audit files for same-alias models

on_low_score="fail" was ignored by FreshDataDbtTransform (#343):
- run() only raised TrustGateError when fail_on_low_score=True, so with
  on_low_score="fail" a failing gate returned should_fail=True and the
  pipeline carried on. run() now raises when result.should_fail or when
  fail_on_low_score is set and the gate did not pass. The audit file is
  still written before raising.
- run() takes a keyword-only raise_on_fail=True. gate_manifest calls
  run(raise_on_fail=False), so a failing model under on_low_score="fail" is
  recorded as failed and the run continues. The summary shape and the
  skipped/all_passed semantics are unchanged.

Same-alias models overwrote each other's audit file (#344):
- FreshDataDbtTransform gains audit_name, used as the audit file stem
  (<audit_name>_audit.json) instead of the table name. It is checked with
  _validate_audit_table_name when the transform is configured.
- When output_dir is set, gate_manifest counts aliases across the models it
  gates (case-insensitively, since audit files may land on a
  case-insensitive filesystem). Models whose alias is shared are written to
  <schema>.<alias>_audit.json, or to <unique_id>_audit.json when the schema
  is missing or that name is still not unique. Other models keep
  <alias>_audit.json. An unsafe schema is rejected by the validator and
  recorded as that model's error.

Closes #343
Closes #344
@coderabbitai

coderabbitai Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: be8fa6ac-0c69-4c0d-8269-441978a0fa99


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown

FreshData benchmark report — performance

  • freshdata: ?
  • python: ?
  • platform: ?
fixture n_rows n_cols p50 s p95 s peak MB repair % false-repair % preserve % trust monotonic export %

Authored-code reduction (Metric 6)

@kevincostner17
kevincostner17 merged commit 7b03498 into main Sep 15, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant