Skip to content

fix(explain): non-string column labels and duplicate-label errors in explain_clean and infer_roles - #381

Merged
kevincostner17 merged 1 commit into
mainfrom
fix/explain-column-labels
Sep 15, 2026
Merged

kevincostner17 merged 1 commit into
mainfrom
fix/explain-column-labels

Conversation

@kevincostner17

Copy link
Copy Markdown
Contributor

Summary

Fixes non-string and duplicate column-label handling in fd.explain_clean, fd.infer_roles and the explain HTML renderer.

  • Integer / tuple labels (Non-string column labels crash or silently misbehave in six APIs #232 part 3): ExplainReport keyed before_stats/after_stats by str(label) but cell_changes by the raw label, and render_explain looked stats up by the raw label. Integer-labelled columns showed changed_cells = 0 and blank HTML dtypes; MultiIndex tuple keys made to_dict()/to_html() fail in json.dumps (_repr_html_() returned None). All per-column mappings are now keyed by str(label), to_frame()/to_dict() stringify keys, and the renderer looks up by str(label). Narratives now match actions (which record str(label)), so integer-labelled columns are no longer silently omitted.
  • Mixed int/str labels (Non-string column labels crash or silently misbehave in six APIs #232 part 6): explain_clean and infer_roles sorted contexts by raw label and raised TypeError; both now sort by str(label). infer_roles keeps the original label in its column output (so df[row["column"]] still works), passes str(label) as the semantic-type name, and accepts semantic_context column hints keyed by either form.
  • Duplicate labels (Duplicate column labels crash detect_pii/anonymize, fd.validate and explain_clean #265 part 3): both functions now raise ValueError("<func> requires unique column labels; duplicated: [...]"), matching fill_missing, instead of AttributeError/TypeError. explain_clean also raises a ValueError naming labels whose string forms collide (e.g. 1 and "1") rather than silently merging them in its string-keyed report; infer_roles accepts such frames.

Behaviour change: ExplainReport.cell_changes keys are now always strings (previously raw labels for non-string columns).

Tests

New tests/test_explain_column_labels.py: integer labels (keys, to_frame, HTML dtypes, narratives, _cell_changes), MultiIndex labels (to_dict JSON, _repr_html_, infer_roles), to_dict/to_frame with tuple keys, mixed int/str labels for both APIs, semantic hints keyed by string for integer labels, duplicate-label ValueError for both APIs, and the 1/"1" collision.

Verification

  • ruff check .: pass
  • mypy src/freshdata: no issues (202 files)
  • pytest -m "not online and not large": py3.12 4260 passed, 6 skipped; py3.9 4256 passed, 10 skipped (before rebasing onto current main; explain/API tests re-run after the rebase on both)

Refs #232 (parts 3, 6)
Refs #265 (part 3)

…abels in explain and infer_roles

explain_clean keyed before_stats/after_stats by str(label) but cell_changes
by the raw label, and the HTML renderer looked up stats with the raw label.
For integer labels (read_csv(header=None)) to_frame() reported 0 changed
cells and the HTML view showed blank dtypes; tuple MultiIndex keys made
to_dict()/to_html() fail in json.dumps, so _repr_html_() returned None.
Every per-column mapping in ExplainReport is now keyed by str(label),
to_frame()/to_dict() stringify keys defensively, and render_explain looks
stats up by str(label). Narratives now match actions (which record
str(label)), so integer-labelled columns are no longer silently omitted.

explain_clean and infer_roles sorted per-column contexts by raw label,
raising TypeError on mixed int/str labels. Both now sort by str(label).
infer_roles keeps the original label in its "column" output so it still
indexes the frame, passes str(label) as the semantic-type name, and
accepts semantic_context column hints keyed by either form.

Duplicate column labels made df[col] return a DataFrame and crashed both
functions with AttributeError/TypeError. They now raise a clear ValueError
("... requires unique column labels; duplicated: [...]"), matching
fill_missing. explain_clean also raises a ValueError naming labels whose
string forms collide (1 and "1") rather than silently merging them in its
str-keyed report; infer_roles, which keeps original labels, accepts them.

Refs #232 (parts 3, 6)
Refs #265 (part 3)
@coderabbitai

coderabbitai Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 7efa90c3-1174-4fd4-9be1-0771ff6a9737


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown

FreshData benchmark report — performance

  • freshdata: ?
  • python: ?
  • platform: ?
fixture n_rows n_cols p50 s p95 s peak MB repair % false-repair % preserve % trust monotonic export %

Authored-code reduction (Metric 6)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant