test: negative API-surface and configuration-interaction suites (Phase 18/19)

[kevincostner17]

What

Two new test suites for the production-readiness program: Phase 19 — negative
API testing and Phase 18 — configuration interaction testing. No
production code changes.

tests/test_negative_api_surface.py (160 tests)

Deliberately invalid input to the public entry points in freshdata.__all__:
clean, clean_text, validate, validate_fields, suggest_plan, plan,
apply_plan, clean_csv, clean_excel, profile, infer_roles,
explain_clean, compile_context, run_suite.

Every case asserts four things, not just "something raised":

• the exact exception class (type(exc) is ..., never a bare Exception);
• a message that names the offending parameter and the expected form —
  the bar being the existing impute must be one of (None, 'auto', 'mean', 'median', 'mode', 'missforest'), got 'knn';
• no partial mutation of the caller's frame, checked with a digest over
  values, column labels, dtypes, the index and its name, and .attrs —
  assert_frame_equal ignores the last two, so it cannot catch a step that
  stashed metadata on the input;
• determinism — the same bad input raises the same class and the same
  message twice in a row.

Covered: wrong frame type (list / dict / None / ndarray / scalar), duplicate
and missing columns, out-of-range and mis-ordered thresholds, every enum-valued
CleanConfig option, unknown domains and engines, contradictory configuration,
corrupted policy / memory / suite / profile files, and unsupported
engine + operation combinations.

tests/test_config_interactions.py (28 tests)

The high-risk combinations, not each option in isolation:
domain + semantic_mode + context; profile + policy + memory;
target_column + semantic_mode; id_columns + clean_text; strict=True +
fallback_policy="error"; streaming + semantic; engine="polars" +
domain; contract + a non-pandas engine; preserve_columns + a
context-declared protected column.

Findings pinned by these tests

Tests that pin weaker-than-documented behaviour carry an S2:/S3: docstring
naming the gap. They are regression anchors, not endorsements — each is
expected to be rewritten when the gap is closed.

Sev	Finding
S2	domain= short-circuits the execution dispatch. engine=, output_format=, engine_config= and fallback_policy="error" are silently discarded: the run executes on pandas, returns pandas, records no fallback event, and the documented "strict out-of-core guarantee" never fires. Every other pandas-only feature (contract=/memory=/profile=/context=) raises TypeError on a native engine instead.
S2	fd.plan(df, engine=...) never validates the backend name. fd.clean(engine="sqlite") raises; fd.plan echoes any string back as plan.backend with a plausible fallback_reason, so a typo yields a confident, wrong execution verdict from the very function documented to answer that question up front.
S2	An unknown semantic_type in semantic_context["columns"] is adopted verbatim at confidence 1.0 — no warning, no report entry, not even under strict=True. semantic_backends warns and raises PolicyError under strict for the same class of typo, and both FieldSpec and config_for_field warn.
S3	suggest_plan, plan, clean_text, validate_fields and compile_context leak AttributeError: 'list' object has no attribute 'columns' for non-frame input, where clean/profile/infer_roles/explain_clean/validate/run_suite raise TypeError naming the accepted types.
S3	A non-numeric threshold (missing_threshold_low="big") surfaces '<' not supported between instances of 'float' and 'str' — neither parameter nor expected type named.
S3	A structurally wrong .json policy or .fdmem memory loads as an empty object. A truncated or foreign memory file silently replays nothing and reports no problem.
S3	A contract mapping or suite JSON missing name raises a bare KeyError: 'name'.
S3	run_suite(df, <not a suite>) leaks AttributeError: ... has no attribute 'to_contract'; fd.validate(suite=...), the other front door onto the same engine, type-checks it properly.
S3	fd.clean_csv(df) — a natural slip — fails with argument of type 'method' is not iterable from the leading-zero pre-scan.
S3	fd.validate_fields range-checks neither rare_threshold nor outlier_fence; outlier_fence=-3.0 silently classifies every value as a statistical_outlier.
S3	clean_text(config={...}) / validate_fields(policy={...}) are not type-checked and leak AttributeError.
S3	A hint for a column that does not exist in semantic_context["columns"] is dropped in silence, while the same unresolved reference in context= raises PolicyError under strict=True.
S4	Spec gap: fd.clean_text has no id-column channel at all — id_columns never reaches it, and field_types is the only protection.
S4	Spec gap: string_case is a layer-1 repair that ignores id_columns / preserve_columns / sensitive_columns. Only a context policy stops it. Consistent with each option's narrow documented meaning, but the obvious way to say "leave my identifiers alone" does not work.

Verified as not defects

• preserve_columns and a context-declared protected column are unioned,
  not overridden, and the policy protection additionally lowers to an immutable
  semantic hint.
• target_column, id_columns and preserve_columns each stop the semantic
  layer — sensitive_columns deliberately does not (it redacts report text).
• StreamingCleaner runs the semantic layer, honours the column protections and
  reports an unknown backend exactly as fd.clean does; semantic_mode is not
  dropped.
• profile= adopts its embedded memory only when no memory= was supplied.
• domain + semantic_mode + context all stay active in one run.
• clean/profile accept duplicate column labels because column_names
  deduplicates them as step 1 — not an inconsistency with the entry points that
  reject them.
• On a native engine, context= is refused before it is compiled, so the error
  is the same whether the rule text resolves or not.

Verification

• pytest -m "not online and not large" -p no:randomly — full suite green.
• ruff check . — clean over the whole repo.
• Local mypy skipped (numpy 2.5 stubs); CI covers it.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: cc0dd2c0-d98e-4c15-8a52-d1d20aa9b421

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}