fix(semantic): stop unsafe auto-repairs and fix memory replay expert lookup - #352
Merged
Merged
Conversation
…t regrouping Four accuracy fixes to semantic repairs that were auto-applied, or never replayed, incorrectly. Memory replay auto-applied fuzzy matches (#252). A learned repair replayed onto any value with difflib similarity >= 0.92 at confidence min(learned, similarity), so a one-character-different, already-valid email (0.96) cleared the 0.95 auto threshold and was rewritten to the learned target. Only exact normalized matches keep the learned confidence now. Fuzzy matches are also capped at semantic_auto_threshold - 0.01, so they are suggested for review and never auto-applied. Memory proposals have no packaged calibration curve, so calibration cannot raise them back over the threshold. Percent stragglers ignored the column's scale (#253). NumericFormatExpert treated every percent|pct|rate|ratio column as percent-denominated, so '45%' in a column of 0-1 fractions became 45.0. The expert now reads the column's plain numeric values. When at least 3 exist and all lie in [0, 1], it proposes value/100 in the review band (0.80). A rate/ratio column with too few plain numbers to show its scale is also held for review. Percent-scale columns, and percent/pct columns without scale evidence, keep today's automatic repair (TruthBench edu-07 unchanged). Shape alignment re-split digits across groups (#254). ShapeAlignmentExpert re-rendered the alphanumeric payload into the dominant template, so '1.10' became '1.1.0' among semver values. Equal payloads do not mean equal values when the separators move. Alignment now applies only when the value's separator-delimited groups match the template's groups one-to-one (same count and lengths). A value whose groups differ is left alone. A value with no separators, which could be split several ways, is suggested for review (0.75) instead of auto-applied. Separator drift such as '555 0101' -> '555-0101' (TruthBench crm-04) still auto-applies. Memory replay used the wrong expert per issue type (#300). _EXPERTS_BY_ISSUE kept only the last expert per issue_type. Five experts share format_alignment, so every learned repair of that type was gated by IsoInstantDateExpert.applies. Learned NFC and shape-alignment repairs therefore never replayed on their own columns. Replay now looks up the expert by the stored expert name. It falls back to the issue type only when exactly one expert handles that type. Regression tests are in tests/test_semantic_repair_safety.py. Closes #252 Closes #253 Closes #254 Closes #300
Contributor
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
FreshData benchmark report —
|
| fixture | n_rows | n_cols | p50 s | p95 s | peak MB | repair % | false-repair % | preserve % | trust | monotonic | export % |
|---|
Authored-code reduction (Metric 6)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
These are four accuracy fixes for semantic repairs that were either auto-applied when they shouldn't be, or never replayed.
min(learned, similarity), so a one-character-different email (similarity 0.96) cleared the 0.95 auto threshold.semantic_auto_threshold - 0.01, so they are suggested for review and never auto-applied.'45%'became45.0in a rate column of fractions.NumericFormatExpertnow reads the column's plain numeric values.rate/ratiocolumn with too little evidence is also held for review.percent/pctcolumns without evidence, still auto-repair as before (TruthBench edu-07 unchanged).'1.10'as'1.1.0'.'555 0101'→'555-0101'(TruthBench crm-04) still auto-applies.format_alignmentexpert. Learned NFC and shape-alignment repairs were checked as if the column were a date, so they never replayed.Behaviour changes:
docs/semantic-cleaning.mdnow says fuzzy memory matches are only suggested.Tests
tests/test_semantic_repair_safety.py(37 tests):main. The rest guard repairs that must keep working.Verification
ruff check .: all checks passedmypy src/freshdata: no issues in 202 source files-m "not online and not large"), py3.12: 4276 passed, 6 skippedCloses #252
Closes #253
Closes #254
Closes #300