Skip to content

chore(deps): strip leaf packages from dependencies, restore extras-only model (closes #326)#327

Draft
ywatanabe1989 wants to merge 1 commit into
mainfrom
chore/extras-refactor-issue-326
Draft

chore(deps): strip leaf packages from dependencies, restore extras-only model (closes #326)#327
ywatanabe1989 wants to merge 1 commit into
mainfrom
chore/extras-refactor-issue-326

Conversation

@ywatanabe1989

Copy link
Copy Markdown
Owner

Summary

Closes #326. Strips the 30+ scitex-* leaf packages from [project].dependencies so pip install scitex becomes core-minimal (8 entries) and the documented scitex[<module>] / scitex[all] opt-in model actually works.

Root cause

[project].dependencies silently grew to 38 entries — every scitex-* leaf was duplicated there alongside its own [project.optional-dependencies].<name> extras entry. Result: every pip install scitex pulled the entire ecosystem as REQUIRED deps, ignoring the documented opt-in model. Network cost, install time, and ABI-tagged C-extension wheel churn for users who only need one module.

Bucket assignments

Operator decisions (Telegram msg 664, 661, 660, 2026-06-09):

Core ([project].dependencies) — 8 entries:

  • numpy / pandas / PyYAML / tqdm / packaging / natsort — every scitex helper touches at least one; near-universal in the Python scientific stack.
  • scitex-config==0.3.6 — env / config loader (scitex.helpers.load_scitex_env delegates here; required even before any module is imported).
  • scitex-dev>=0.17.8 — operator-confirmed core: "コアに scitex-dev は必要だと思います、サイテクスコンフィグあればどうせ入りますが" (Telegram msg 664). Pulls in the audit / skill discovery machinery + the project's quality-rule corpus.

Moved to [project.optional-dependencies]:

  • All scitex-* leaves (~30 packages): each already had a <name> extra entry. 7 of them (benchmark, context, cv, introspect, msword, os, tex) were inconsistently missing their own wheel pin — added here so pip install scitex[benchmark] actually installs scitex-benchmark.
  • bs4 / beautifulsoup4 — only used by helpers/_optional_deps.py check_optional_deps wrapper for the scholar stack; already declared in [scholar] / [io] where it's actually needed (operator confirmed: "コア不要、optional 扱い").
  • figrecipe / scitex-ui — already in [plt] / [ui] extras.

[all] extra unchanged — already aggregates via scitex[<name>] self-references, so pip install scitex[all] still produces the bare-equivalent of the pre-refactor install for users who want everything.

Test plan

  • CI pytest matrix (3.11/3.12/3.13) installs via pip install -e ".[all,dev]" (already the existing fallback chain); should still resolve the same wheel set.
  • pip install scitex on a fresh venv resolves only the 8 core entries (verify the size shrink).
  • pip install scitex[stats,io] resolves the documented per-module extras.
  • scitex.helpers.load_scitex_env() still works (scitex-config in core).
  • scitex-dev CLI still discoverable post-install (scitex-dev in core).
  • Lazy-attrs imports like scitex.audio, scitex.cv, etc. raise a friendly ImportError pointing at pip install scitex[<name>] when the corresponding extra is not installed (verified via _AVAILABLE flags + existing _install_guide.py).

Notes

…only model (#326)

Per issue #326 (proj-paper-scitex-clew → operator → me, 2026-06-09):

> "bare `pip install scitex` で 30+ sub-package が direct dep として
>  入ります (app/audio/browser/cv/msword/genai/scholar/ui 等)"
> "文書化された extras model と齟齬しています"

The umbrella's `[project].dependencies` list silently grew to 38
entries — every scitex-* leaf package was duplicated there alongside
its own `[project.optional-dependencies].<name>` extras entry. Result:
every `pip install scitex` pulled the entire ecosystem as REQUIRED
deps, ignoring the documented `scitex[stats,io]` / `scitex[all]`
opt-in model. Network cost, install time, and ABI-tagged C-extension
wheel churn for users who only need one module.

Operator's bucket call (2026-06-09 Telegram msg 664, 661):
  - keep `scitex-dev` in core ("コアに scitex-dev は必要")
  - drop `beautifulsoup4` from core ("コア不要、optional 扱い")
  - `crawl4ai` not used at runtime, optional-only

**Core (`[project].dependencies`) — 8 entries:**
  - numpy / pandas / PyYAML / tqdm / packaging / natsort
  - scitex-config (env / config loader; foundational)
  - scitex-dev>=0.17.8 (audit + skill discovery; operator-confirmed core)

**Moved to `[project.optional-dependencies]`:**
  - All scitex-* leaves (~30): each already had a `<name>` extra entry;
    7 of them (`benchmark` / `context` / `cv` / `introspect` / `msword` /
    `os` / `tex`) were inconsistently missing their own wheel — added.
  - bs4 / beautifulsoup4 — only used by `helpers/_optional_deps.py`
    check_optional_deps wrapper for the scholar stack (already declared
    in `[scholar]` / `[io]` where actually needed).
  - figrecipe / scitex-ui — already in `[plt]` / `[ui]` extras.

`[all]` extra is unchanged (already aggregates via `scitex[<name>]`
self-references) so `pip install scitex[all]` still gives the bare-
equivalent of the pre-refactor install.

Coordinated with companion PR #323 (scitex-dev pin bump to >=0.17.8)
and the consumer-fleet quality-audit template rollout that just
landed.

operator-SSoT repo — opening as DRAFT, NOT merging. Operator reviews
the bucket assignments + merges when ready.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

scitex 2.30.1 umbrella pulls scitex-app/audio/browser/cv/genai/msword/notebook/scholar/ui as required deps (not extras) — defeats the [extras] model

1 participant