feat: define public generation invocation contract

[andreatnvidia]

📋 Summary

Defines a stable public generation contract for optional execution packages so they can configure, validate, invoke, and classify Data Designer runs without importing data_designer.engine. It also normalizes builder-loading failures and exposes public outcome metadata for complete, partial, resumed, and early-shutdown invocations.

🔗 Related Issue

Closes #851

Related to #850

🔄 Changes

• Document the supported public imports, serialized builder boundary, validation ownership, model binding, runtime configuration, results, errors, and known limits.
• Export InvalidConfigError, InvalidFileFormatError, and InvalidFilePathError from data_designer.config, and ArtifactStorageError from data_designer.interface.
• Normalize missing, unreadable, and malformed local builder configuration failures to public error types.
• Expose resolved dataset path, requested and actual record counts, partial and early-shutdown status, and requested and effective resume modes on DatasetCreationResults.
• Preserve requested record counts when reconstructing completed workflow stages and cover collision, resume, partial, early-shutdown, and failure behavior.

🔍 Attention Areas

⚠️ Reviewers: Please pay special attention to the following:

• DatasetCreationResults - public completion and resume semantics.
• Public invocation contract - supported imports, validation boundaries, and explicitly unsupported behavior.

🧪 Testing

• make test passes - not run; targeted package suites were used.
• .venv/bin/pytest packages/data-designer-config/tests packages/data-designer/tests -p no:cacheprovider (1772 passed, 1 skipped)
• Unit tests added/updated.
• E2E tests added/updated - N/A, no E2E behavior changed.
• .venv/bin/ruff check --fix .
• .venv/bin/ruff format .
• Existing import-performance threshold.

✅ Checklist

• Follows commit message conventions.
• Commits are signed off (DCO Assistant passes).
• Architecture docs updated with the public contract proposal.

[github-actions]

Fern preview: https://nvidia-preview-pr-854.docs.buildwithfern.com/nemo/datadesigner

Fern previews include the docs-website version archive with PR changes synced into latest. Notebook tutorials are rendered without execution outputs in previews.

[greptile-apps]

Greptile Summary

The PR defines a public generation invocation contract and exposes typed configuration, artifact, completion, partial-run, early-shutdown, and resume metadata.

• Normalizes missing, unreadable, and malformed builder configuration failures.
• Adds public outcome fields to DatasetCreationResults.
• Preserves requested record counts when reconstructing completed workflow stages.
• Documents supported imports and invocation semantics.

Confidence Score: 5/5

The PR appears safe to merge because the previously reported path-classification failures are fixed and no blocking failure remains.

No blocking failure remains.

Important Files Changed

Filename	Overview
packages/data-designer-config/src/data_designer/config/utils/io_helpers.py	The revised path heuristics fix the previously reported mixed-case, malformed-filename, and mapping-like absolute-path classification failures.
packages/data-designer-config/src/data_designer/config/config_builder.py	Builder loading now translates filesystem and YAML decoding failures into public configuration error types.
packages/data-designer/src/data_designer/interface/results.py	Adds public dataset outcome metadata and normalizes missing or unreadable artifact failures.
packages/data-designer/src/data_designer/interface/composite_workflow.py	Reconstructed completed-stage results now retain the relevant requested record count.
packages/data-designer/src/data_designer/interface/data_designer.py	Creation results now receive invocation record counts, early-shutdown state, and requested and effective resume modes.

_{Reviews (9): Last reviewed commit: "fix: tighten generation result contract" | Re-trigger Greptile}

[nabinchha]

Thanks for putting this together, @andreatnvidia!

Summary

This PR defines the optional-package invocation boundary, promotes the relevant public errors, and adds result metadata for record counts, partial completion, early shutdown, artifact resolution, and resume decisions. The implementation mostly matches that intent, but a valid inline-config case now breaks and a few documented public error/outcome guarantees are not yet enforced.

Findings

Critical — Let's fix these before merge

packages/data-designer-config/src/data_designer/config/utils/io_helpers.py:193 — Valid inline YAML ending in .json is treated as a missing file

• What: The new suffix check runs before parsing and classifies any string whose final characters are .json as a file path. A valid inline builder whose last scalar is a JSON seed path (for example, a YAML document ending in path: /tmp/seed.json) now raises InvalidFilePathError. I reproduced this with DataDesignerConfigBuilder.from_config(...); the same payload is accepted on main.
• Why: Inline YAML is an existing supported input to from_config(), so this breaks valid configurations while trying to normalize nonexistent JSON paths.
• Suggestion: Distinguish a parsed mapping from a filename before raising FileNotFoundError—for example, parse string input first and apply the missing-path classification only when the parsed value is not a mapping and the original string looks like a supported filename. Please add a regression test with a valid inline builder ending in a .json field value (and ideally cover the existing .yaml/.yml ambiguity too).

Warnings — Worth addressing

packages/data-designer-config/src/data_designer/config/config_builder.py:129 — Undecodable local configs bypass the public error contract

• What: The normalization catches OSError and yaml.YAMLError, but reading invalid UTF-8 raises UnicodeDecodeError, which escapes unchanged. I reproduced this with an existing .yaml file containing invalid UTF-8 bytes.
• Why: The PR promises public error types for unreadable/malformed local builder configs; callers following that contract cannot handle this case without depending on another implementation exception.
• Suggestion: Normalize UnicodeDecodeError (or the appropriate UnicodeError family) to InvalidFileFormatError and add a binary-file regression test.

packages/data-designer/src/data_designer/interface/results.py:94 — Missing result artifacts are reported as zero records

• What: The new actual_num_records property delegates to count_records(), whose empty glob sums to 0. With an empty final dataset directory, I observed actual_num_records == 0, is_partial is True, and load_dataset() returning an empty frame instead of the ArtifactStorageError promised by the new contract.
• Why: An embedding package can misclassify missing or moved artifacts as a legitimate partial/zero-record outcome, undermining the outcome metadata this PR introduces.
• Suggestion: Have count_records()/actual_num_records raise public ArtifactStorageError when no final batches exist and normalize missing-dataset failures from load_dataset() at the public results boundary. If zero is intentionally supported, narrow the contract and add tests that make the distinction explicit.

fern/versions/latest/pages/concepts/architecture-and-performance.mdx:390 — Published early-shutdown guidance contradicts the new contract

• What: Fern says crossing the threshold always raises DataDesignerEarlyShutdownError, while this PR formalizes two outcomes: zero records raises, but surviving records return a partial DatasetCreationResults with early_shutdown=True. The example also imports the error from the internal data_designer.interface.errors module, and no Fern page documents the newly public outcome fields.
• Why: Users following the published guidance may wait for an exception and silently accept a partial return without checking it.
• Suggestion: Update this page in the same PR to use the public root import and demonstrate is_partial/early_shutdown (plus the requested/effective resume fields and dataset_path where resume is documented).

What Looks Good

• The lazy root exports preserve the package layering and import-time pattern while giving callers stable import locations.
• The requested-versus-effective resume distinction is derived from the engine's resolved storage mode, and the tests cover successful resume, incompatible IF_POSSIBLE fallback, collisions, partial results, and reconstructed workflow stages.
• The new tests are behavior-oriented; all current GitHub checks are green, and changed-file Ruff checks plus 309 focused config/interface tests pass locally.

Verdict

Needs changes — fix the inline-YAML regression, complete the promised public error normalization for malformed/missing artifacts, and align the published run-outcome documentation with the new contract.

---

This review was generated by an AI assistant.

[andreatnvidia]

Thanks for the detailed review, @nabinchha. I pushed cf0f0bdf to address each finding.

The config loader now parses inline YAML before deciding that a string is a missing file path, and invalid UTF-8 is normalized to InvalidFileFormatError. The public results boundary now raises ArtifactStorageError when final batch artifacts are missing or inaccessible. I also updated the Fern guidance to reflect both early-shutdown outcomes and to use the public imports and result metadata.

I added focused regression coverage for these cases. Ruff, the 839 config/interface tests, and the Fern docs checks pass locally.

[nabinchha]

Thanks for the quick follow-up, @andreatnvidia!

Summary

The three new commits address the four findings from my earlier review and the subsequent Greptile path-classification comment, with focused tests and updated Fern guidance. Two edge cases still leave the public error boundary dependent on input shape or raw filesystem exceptions.

Findings

Warnings — Worth addressing

packages/data-designer-config/src/data_designer/config/utils/io_helpers.py:197 — Malformed inline configs ending in a supported extension are reported as missing files

• What: On a YAML parser error, is_local_config_path is determined only from the string suffix. As a result, DataDesignerConfigBuilder.from_config("data_designer: [.json") raises InvalidFilePathError (caused by FileNotFoundError), while the same malformed inline document without that suffix raises InvalidFileFormatError.
• Why: The new contract promises InvalidFileFormatError for malformed inline builder data, so classification currently changes based on the final characters of the malformed payload.
• Suggestion: Could we distinguish path-shaped strings from inline documents before converting parser errors—using an explicit Path, absolute/path-separator evidence, or another unambiguous heuristic—and add a malformed-inline regression case ending in .json/.yaml?

packages/data-designer/src/data_designer/interface/results.py:158 — Record counting still leaks raw filesystem errors

• What: _get_batch_files() now handles an empty glob, but the subsequent lazy.pq.read_metadata() calls are outside the new OSError normalization. I reproduced count_records() raising raw PermissionError from an unreadable batch_00000.parquet; actual_num_records and is_partial inherit the same behavior.
• Why: load_dataset() now exposes ArtifactStorageError for inaccessible artifacts, but the other public outcome properties still require callers to know about filesystem exceptions.
• Suggestion: Wrap OSError from metadata reads as ArtifactStorageError (and apply the same boundary consistently to export reads if inaccessible artifacts are part of that contract), with a regression test for an unreadable or disappearing batch.

What Looks Good

• Valid inline mappings ending in .json/.yaml/.yml now load correctly, and invalid UTF-8 is normalized to the public format error.
• Missing or empty final-batch directories now fail through a shared ArtifactStorageError path, and the Fern outcome/resume guidance matches the new API.
• Both Greptile threads are resolved and outdated; all current CI checks are green, and 316 focused tests plus changed-file Ruff checks pass locally.

Verdict

Needs changes — clarify the remaining path-versus-inline parser ambiguity and finish normalizing inaccessible batch reads across the public result-counting boundary.

---

This review was generated by an AI assistant.

[andreatnvidia]

@nabinchha I tightened up both remaining edge cases in 40ab4050.

Malformed inline YAML is no longer mistaken for a missing file based on its suffix. Missing-file handling now requires clear path evidence, with regression coverage for .json, .yaml, and .yml.

I also wrapped failures while reading batch metadata and export inputs in ArtifactStorageError, so the public result APIs no longer expose raw filesystem errors.

Mind taking another look?

[nabinchha]

Thanks for another focused follow-up, @andreatnvidia!

Summary

Commit 40ab4050 resolves both findings from my previous pass: malformed inline configs ending in supported extensions now remain format errors, and ordinary filesystem failures during count/export reads are wrapped publicly. I independently reproduced the active Greptile path-classification finding, so I am not duplicating it below; one additional unreadable-artifact case still falls outside the documented boundary.

Findings

Warnings — Worth addressing

plans/850/data-designer-contract.md:260 — Unreadable artifacts can still escape as implementation exceptions

• What: This line now guarantees ArtifactStorageError for unreadable dataset and processor artifacts, but the new wrappers catch only OSError. With a corrupt batch_00000.parquet, I reproduced load_dataset(), count_records(), and export() all raising raw pyarrow.lib.ArrowInvalid. DatasetCreationResults.load_processor_dataset() also delegates without a public boundary, so an unreadable processor artifact can still leak PermissionError or a PyArrow exception.
• Why: Callers following the new contract still need knowledge of filesystem and PyArrow implementation exceptions to handle artifact failures consistently.
• Suggestion: Could we normalize source-Parquet read/decode failures and processor-dataset read failures at the results boundary? If corrupt Parquet should instead map to public InvalidFileFormatError, documenting and testing that distinction would also satisfy the boundary without exposing PyArrow types.

What Looks Good

• The malformed-inline regression is fixed across .json, .yaml, and .yml, while clear missing paths still use InvalidFilePathError.
• Permission and disappearance failures during metadata reads and all three export formats now become ArtifactStorageError with preserved causes.
• The branch merges cleanly with the current feat/slurm-execution base; 322 focused branch tests, 129 merged-base integration tests, and changed-file Ruff checks pass locally.

Verdict

Needs changes — resolve the existing Greptile path-syntax thread and complete the public normalization for corrupt dataset and unreadable processor artifacts.

---

This review was generated by an AI assistant.

[andreatnvidia]

@nabinchha I tightened the results boundary in 8ad60d2: corrupt Parquet reads and processor output read failures now surface as ArtifactStorageError. Export-time schema incompatibilities still use InvalidFileFormatError. I also fixed the absolute-path classification Greptile called out. Could you take another look?

[nabinchha]

Thanks for the careful follow-up, @andreatnvidia!

Summary

Commits 8ad60d21 and 4de14e9e resolve the previously reported corrupt-Parquet, processor-read, and absolute-path classification cases, and they tighten the requested-count invariant in reconstructed workflow results. The implementation matches the stated intent; one pre-existing processor-artifact edge case may be worth handling in a follow-up.

Findings

Suggestions — Take it or leave it

plans/850/data-designer-contract.md:260 — Consider covering empty processor artifact directories in a follow-up

• What: The pre-existing processor loader treats an existing processors-files/<name>/ directory containing no Parquet files as a successful empty (0, 0) DataFrame. This behavior is unchanged by the PR, whose new normalization correctly handles missing paths, filesystem failures, and corrupt Parquet.
• Why: An empty directory left by an interrupted or moved processor run could still be mistaken for a valid processor result, although this is not a regression introduced by these changes.
• Suggestion: As a follow-up, we could have the storage loader treat a processor directory with no Parquet files as missing while continuing to allow a genuine zero-row Parquet file.

What Looks Good

• Corrupt Parquet now consistently becomes ArtifactStorageError across dataset loading, counting, all export paths, and processor reads, with the underlying cause preserved.
• Absolute config paths now take precedence over inline-YAML heuristics, including mapping-like filenames, and the early-shutdown docstring accurately distinguishes zero-record failures from partial returns.
• Making requested_num_records required removes the tri-state is_partial result, while the workflow reuse guard safely reruns IF_POSSIBLE stages whose older metadata lacks the required count.

Verdict

Ship it (with nits) — the new commits resolve the outstanding review findings; the empty processor-directory behavior is pre-existing and can be handled separately.

---

This review was generated by an AI assistant.