feat: mlbundle v0.1 — pickle-free model bundles

[UlloaSP]

First complete version of mlbundle: one pickle-free .mlbundle file with the model in its native format, a rich manifest, a locally inferred schema and the model's structure, plus the validating reader.

What is included

• Format contract (docs/format.md and linked sections): closed member list, strict manifest, canonical JSON, integrity records, limits, format version 1.
• Secure container: in-memory zip reader with closed member names, size/ratio limits, strict ZIP64/descriptor handling and deterministic atomic writer; metadata-only inspect.
• Public API: mlbundle.export, mlbundle.load, mlbundle.inspect; CLI mlbundle inspect / mlbundle verify.
• Adapters: scikit-learn (skops, mlbundle-owned trusted lists, closed class/attribute catalog), XGBoost (native UBJSON, user-supplied class labels bound to the model), CatBoost (native CBM, numeric features, symmetric trees), ONNX (original bytes unchanged, validated before a CPU session is created).
• Composite pipelines (feat: support composite native pipelines #16): a scikit-learn Pipeline whose last step is an XGBoost or CatBoost estimator is stored as model.skops (transformers) plus the native member, each loaded and verified by its own adapter, then rebuilt as a real Pipeline. Transformer output width is proven from fitted state without data.
• Verified claims: everything derived from the model is recomputed on load with the writer's extraction code; a state commitment covers all prediction-relevant state.
• Schema: optional mlbundle[schema] extra computes schema.json locally with mlschema; sample rows only with explicit sample_rows=.
• Tooling: uv, ruff, pyright, pytest with coverage; CI for lint, types, tests and a base-wheel smoke check on Ubuntu and Windows.

Added after the first review round

• fix: support real ONNX and MLP models #11: real ONNX tree ensembles and scikit-learn MLP models (wire field ceiling tied to the member byte cap, one validation pass per operation, producer strings up to 65,536 characters, AdamOptimizer / SGDOptimizer trusted).
• feat: add native numeric CatBoost adapter #12: CatBoost adapter.
• fix: export everyday sklearn pipelines #13: everyday scikit-learn transformers (fitted numpy dtypes, numpy scalar-type hyperparameters, scikit-learn score functions and a closed list of numpy functions in FunctionTransformer).
• perf: speed up verified structure loading #14, perf: reuse verified tree adapter snapshots #15: verification performance (shared canonical JSON path, scikit-learn tree payloads, XGBoost structure) with byte-identical output.
• feat: support composite native pipelines #16: composite pipelines; member ceiling raised from 5 to 6.
• ci: prepare manual pre-alpha PyPI publishing #17: version 0.1.0a1; manual publish.yml (Trusted Publishing, main only, pypi_env approval) reusing CI, which now builds one wheel/sdist and tests that exact wheel on both OSes.

Verification

• uv run ruff check ., uv run ruff format --check ., uv run pyright: clean.
• uv run pytest: 1,476 passed, 91.88% coverage, on Ubuntu and Windows in CI.
• Owner's real models (not in the repository): 10 ONNX files, one scikit-learn MLP pipeline and four CatBoost pipelines (2,000 trees each) export, load and predict identically to the originals; the largest CatBoost bundle loads in about 1 s.
• 18 of 19 everyday scikit-learn pipelines and 17 pipelines ending in XGBoost or CatBoost export and predict identically; four unsupported native placements are rejected at export.

Points for review

• Independent security review not completed. Two attempts were stopped by the reviewing tool's content filter. Seven defects found by its partial probes were fixed with regression tests (fix: harden reader prediction state and bounds #8). The zip parser, the ONNX wire validator (rewritten in fix: support real ONNX and MLP models #11), the CatBoost CBM preflight, the trusted lists added in fix: support real ONNX and MLP models #11 and fix: export everyday sklearn pipelines #13 (optimizers, dtypes, named functions) and the composite reader have not had an independent review.
• Exact scikit-learn version coupling. The sklearn class/attribute catalog is audited against scikit-learn 1.9.1 and the recorded version takes part in the state commitment, so a bundle loads only under the scikit-learn version that exported it.
• Functions admitted by name: scikit-learn score functions and nine elementwise numpy functions, from closed lists in mlbundle. User functions and lambdas are rejected at export.
• CatBoost scope: categorical, text and embedding features and Depthwise / Lossguide trees are rejected; state extraction uses CatBoost's private _serialize_model() because there is no public in-memory save; the extra adds about 444 MB to an environment.
• Composite pipelines: trees describe transformed features, with no mapping back to input columns; step names and outer options are declarations; memory, transform_input and metadata routing are rejected; KNNImputer and FeatureUnion have no width rule and are rejected in composites; all-scikit-learn pipelines still carry no structure.
• HistGradientBoosting* is rejected at export (needs an extra skops trusted type).
• Performance: a 200-tree random forest loads in about 3 s and a 600-tree XGBoost model in about 2 s on the development machine.
• mlschema is an optional extra, not a core dependency.
• Writer compression uses DEFLATE with the RLE strategy for byte-identical output across platforms, at the cost of larger JSON members.
• AGENTS.md "A note from the maintainer" was drafted during bootstrap and needs the owner's own wording.

The owner decided to reserve the PyPI name by publishing this pre-alpha as 0.1.0a1 after this merge; see docs/publishing.md. The independent security review is still pending and the README says so.

Not included: GitHub releases, version tags, bundle signing, sklearn-to-ONNX conversion, MLSuite integration.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: c7d702da-7f12-4599-ab32-be970af0fe09

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	onnxruntime@​1.30.0
	onnx@​1.23.0
	pandas@​3.0.6
	scikit-learn@​1.9.1
	catboost@​1.2.10
	skops@​0.16.0
	xgboost@​3.4.1
	mlschema@​0.2.1
	pydantic@​2.14.1

View full report