diff --git a/TRACKER.md b/TRACKER.md index feec7dd..47f22a2 100644 --- a/TRACKER.md +++ b/TRACKER.md @@ -9,8 +9,8 @@ > CI matrix always includes `ubuntu-22.04`, `macos-14`, and `windows-latest`. **Last updated:** 2026-05-24 -**Current phase:** Phase 0 — Foundation -**Next action:** Phase 0 Step 0.10 — ragctl CLI scaffold +**Current phase:** Phase 1 — Ingestion + Knowledge Store +**Next action:** Phase 1 Step 1.1 — Storage backends --- @@ -29,7 +29,7 @@ | Phase | Title | Steps | ✅ Done | Remaining | |-------|-------|------:|-------:|----------:| -| 0 | Foundation | 13 | **12** | 1 | +| 0 | Foundation | 13 | **13** | 0 | | 1 | Ingestion + Knowledge Store | 10 | 0 | 10 | | 2 | Retrieval Engine | 10 | 0 | 10 | | 3 | Gateway & Agent Runtime | 11 | 0 | 11 | @@ -37,7 +37,7 @@ | 5 | Eval & Observability | 7 | 0 | 7 | | 6 | Governance & Tenancy | 10 | 0 | 10 | | 7 | Pilot, Harden, GA | 10 | 0 | 10 | -| **Total** | | **77** | **12** | **65** | +| **Total** | | **77** | **13** | **64** | --- @@ -57,7 +57,7 @@ | 0.7c | Audit log skeleton | ✅ | `build/phase-0/step-0.7c-audit-log-skeleton` | [#31](https://github.com/officialCodeWork/AgentContextOS/pull/31) | `AuditStore` SPI (append/events/verify_chain), `NoopAuditStore` (SHA-256 hash chain), `AuditWriter` facade (store + structured log), 14 conformance tests | | 0.8 | Eval skeleton | ✅ | `build/phase-0/step-0.8-eval-skeleton` | [#33](https://github.com/officialCodeWork/AgentContextOS/pull/33) | `rag_core.eval` domain types (GoldenSample, EvalMetrics, EvalReport), `rag_config.eval` metric functions (recall@k, MRR, citation_precision), RagasAdapter spike, `ragctl eval run/show`, 5-sample golden JSONL fixture, `tests/eval/` harness (39 tests) | | 0.9 | IaC foundation | ✅ | `build/phase-0/step-0.9-iac-foundation` | [#34](https://github.com/officialCodeWork/AgentContextOS/pull/34) | Terraform modules for Postgres/pgvector, Redis, Qdrant, Elasticsearch (Kubernetes-native, Helm provider); `rag-platform` Helm chart (Deployment, Service, ConfigMap, ServiceAccount, HPA, PDB, Ingress); dev + prod environments; `task infra:*` + `task helm:*` targets; ADR-0003 | -| 0.10 | `ragctl` CLI scaffold | ⏳ | — | — | `packages/ragctl/` CLI (Typer), `ragctl ingest/query/eval/logs/traces/config` top-level commands, shell completion | +| 0.10 | `ragctl` CLI scaffold | ✅ | `build/phase-0/step-0.10-ragctl-cli-scaffold` | — | `packages/ragctl/` package (Typer 0.12+), root `ragctl` entry point, working `config`/`eval`/`traces`/`version` groups, scaffolded `ingest`/`query`/`logs`/`tenant`/`plugin`/`secret` groups (announce target step + exit 0), shell completion via `--install-completion`/`--show-completion`; 19 ragctl tests; docs/reference/ragctl.md + docs/guides/ragctl-quickstart.md; py.typed markers added to rag-config + rag-observability | --- diff --git a/docs/README.md b/docs/README.md index 9c89ed8..c0619a0 100644 --- a/docs/README.md +++ b/docs/README.md @@ -9,6 +9,18 @@ | [eval-skeleton.md](architecture/eval-skeleton.md) | Eval framework architecture: golden-set schema, metric functions, RAGAS spike, `ragctl eval` CLI, extension points | | [iac.md](architecture/iac.md) | IaC overview: Terraform module design, Helm chart structure, dev/prod environments, extension points | +## reference/ + +| File | Description | +|------|-------------| +| [ragctl.md](reference/ragctl.md) | Full `ragctl` command reference — public usage, internals, extension points | + +## guides/ + +| File | Description | +|------|-------------| +| [ragctl-quickstart.md](guides/ragctl-quickstart.md) | Five-minute tour of the `ragctl` CLI | + ## adr/ | File | Description | diff --git a/docs/guides/ragctl-quickstart.md b/docs/guides/ragctl-quickstart.md new file mode 100644 index 0000000..65c690f --- /dev/null +++ b/docs/guides/ragctl-quickstart.md @@ -0,0 +1,104 @@ +# `ragctl` quickstart + +A five-minute tour of the control-plane CLI shipped in Step 0.10. + +## Install + +`ragctl` is part of the AgentContextOS uv workspace. Once you've bootstrapped +the repo, the command is on your PATH inside the workspace virtualenv: + +```bash +task bootstrap # one-time: uv sync + pnpm install + pre-commit +uv run ragctl --help +``` + +If you only want the binary outside the workspace (e.g. on an operator +workstation), install the published package once it lands on PyPI: + +```bash +pip install rag-ragctl +ragctl --help +``` + +## Verify your `rag.yaml` + +The most useful command on day one. Run it before applying a config change to +any environment: + +```bash +ragctl config validate ops/configs/dev.yaml +# VALID ops/configs/dev.yaml +# version=v1 env=dev tenants=2 +# embedder=openai llm=openai vector_store=qdrant +``` + +`--json` is the form you want from CI or pre-deploy scripts: + +```bash +ragctl config validate ops/configs/dev.yaml --json +``` + +To inspect what differs between two configs: + +```bash +ragctl config diff ops/configs/dev.yaml ops/configs/prod.yaml +``` + +## Run a golden-set evaluation + +The eval harness ships with a 5-sample fixture so you can verify the pipeline +end-to-end without setting up real backends: + +```bash +ragctl eval run tests/eval/golden/ --top-k 10 --output eval-report.json +ragctl eval show eval-report.json --verbose +``` + +To enable RAGAS faithfulness scoring: + +```bash +pip install 'rag-config[eval]' +ragctl eval run tests/eval/golden/ --ragas +``` + +## Query traces + +With the local dev stack running (`task dev-full`), Jaeger is available at +`http://localhost:16686`: + +```bash +ragctl traces # last 20 traces for rag-platform +ragctl traces --service gateway --limit 50 +``` + +## Set up shell completion + +Pick whichever line matches your shell: + +```bash +ragctl --install-completion # auto-detect from $SHELL +ragctl --show-completion zsh > ~/.zfunc/_ragctl # manual install for zsh +``` + +## What's *not* there yet + +The following groups are scaffolded — they accept `--help`, exit cleanly, +and announce which build step delivers the real implementation: + +| Command | Lands in | +|------------------|------------| +| `ragctl plugin` | Step 1.1 | +| `ragctl ingest` | Step 1.10 | +| `ragctl query` | Step 3.1 | +| `ragctl logs` | Step 5.6 | +| `ragctl tenant` | Step 6.1 | +| `ragctl secret` | Step 6.7 | + +If you script against them today, your script will keep working — the +exit code and command shape will not change when the real implementation lands. + +## See also + +- [Command reference](../reference/ragctl.md) — full surface, internals, + extension points +- [TRACKER.md](../../TRACKER.md) — current build status and what's coming next diff --git a/docs/reference/ragctl.md b/docs/reference/ragctl.md new file mode 100644 index 0000000..4252218 --- /dev/null +++ b/docs/reference/ragctl.md @@ -0,0 +1,151 @@ +# `ragctl` — command reference + +## Overview + +`ragctl` is the operator-facing control-plane CLI for AgentContextOS. It bundles +configuration, retrieval, evaluation, observability, and tenant-management +commands into a single binary that talks to local files, the gateway service, +and the supporting infrastructure (Jaeger, etc.). + +Step 0.10 delivers the consolidated CLI scaffold. Each sub-command group is +wired into the root `ragctl` app so the surface area is discoverable today, +even where the underlying functionality has yet to land. + +## Usage + +```bash +# discover the surface +ragctl --help + +# print the installed version +ragctl version +``` + +## Command groups + +| Group | Step | Status | What it does | +|-----------|---------|----------|--------------| +| `config` | 0.4 | working | Validate and diff `rag.yaml` files. | +| `traces` | 0.7 | working | Query distributed traces from Jaeger. | +| `eval` | 0.8 | working | Run and inspect golden-set evaluations. | +| `version` | 0.10 | working | Print the installed `ragctl` version. | +| `plugin` | 1.1 | scaffold | Manage SPI plugin registration. | +| `ingest` | 1.10 | scaffold | Trigger ingestion pipelines via the gateway. | +| `query` | 3.1 | scaffold | Run a query against the gateway. | +| `logs` | 5.6 | scaffold | Tail structured logs from the platform. | +| `tenant` | 6.1 | scaffold | Manage tenants. | +| `secret` | 6.7 | scaffold | Manage tenant secrets. | + +Scaffold commands exit `0` and print a one-line "delivered in Step X.Y" +notice so operators can probe the planned interface without surprise. + +### `ragctl config` + +```bash +ragctl config validate path/to/rag.yaml # exits 0 on success, 1 otherwise +ragctl config validate path/to/rag.yaml --json # machine-readable result +ragctl config diff path/a.yaml path/b.yaml # show backend differences +``` + +### `ragctl eval` + +```bash +ragctl eval run tests/eval/golden/ --top-k 10 --output report.json +ragctl eval show report.json --verbose +ragctl eval run tests/eval/golden/ --ragas # optional faithfulness scoring +``` + +`--ragas` requires the optional `ragas` extra: + +```bash +pip install 'rag-config[eval]' +``` + +### `ragctl traces` + +```bash +ragctl traces # default service, 20 traces +ragctl traces --service gateway --limit 50 +ragctl traces --url http://jaeger.observability.svc:16686 # remote Jaeger +``` + +### `ragctl version` + +```bash +ragctl version # → ragctl 0.1.0 +``` + +## Shell completion + +`ragctl` ships with Typer-powered completion for bash, zsh, fish, and +PowerShell: + +```bash +ragctl --install-completion # install for the detected shell +ragctl --show-completion # print the completion script to stdout +ragctl --show-completion zsh > _ragctl +``` + +## Internals + +The CLI lives in [`packages/ragctl/`](../../packages/ragctl/). The Typer app +is constructed once in `ragctl.main` and exposed as the `ragctl` entry point +via `[project.scripts]` in `pyproject.toml`. + +### Layout + +``` +packages/ragctl/ +├── pyproject.toml # rag-ragctl, depends on rag-core/config/observability +├── README.md # short package overview +└── src/ragctl/ + ├── __init__.py # exports app, main, __version__ + ├── main.py # root Typer app + all sub-command groups + └── py.typed # type marker for downstream consumers +``` + +### Dependencies + +`rag-ragctl` depends on: +- `rag-core` — domain types and errors +- `rag-config` — `rag.yaml` loader and eval metrics (powers `config` + `eval` groups) +- `rag-observability` — pulled in transitively for log context +- `typer>=0.12` — CLI framework + +The CLI keeps **no business logic** of its own. Each command is a thin +adapter over a function exported from the corresponding domain package. + +### Scaffold sub-apps + +Sub-apps that aren't implemented yet are constructed by the local helper +`_scaffold_app(group, step, help_text)`. The helper creates a Typer sub-app +whose default callback prints a "delivered in Step X.Y" notice and exits 0. +This keeps the published command shape stable from day one — operators can +script against `ragctl ingest ...` today and the same script will keep +working when Step 1.10 lands the real implementation. + +## Extension points + +To add a new sub-command group: + +1. Create or import the domain logic in the appropriate package + (`rag-core`, `rag-config`, future `rag-retrieval`, etc.). +2. In `ragctl/main.py`, build a new `typer.Typer()` sub-app, register + commands on it, and attach it to the root `app` with `app.add_typer(...)`. +3. Update the **Command groups** table in this file and the README. +4. Add tests under `packages/ragctl/tests/`. + +To replace a scaffold with a real implementation: + +1. Remove the `_scaffold_app(...)` call for that group from `main.py`. +2. Build the real sub-app the same way you would for a new group. +3. Update the **Command groups** table — change Status from `scaffold` to + `working` and link to the deeper reference page. + +## See also + +- [Quickstart](../guides/ragctl-quickstart.md) — getting started in 5 minutes +- [Eval framework architecture](../architecture/eval-skeleton.md) — what + drives `ragctl eval` +- [IaC overview](../architecture/iac.md) — infra surfaces the CLI will + manage in later steps diff --git a/packages/config/pyproject.toml b/packages/config/pyproject.toml index ff189c4..886b829 100644 --- a/packages/config/pyproject.toml +++ b/packages/config/pyproject.toml @@ -28,9 +28,6 @@ eval = [ rag-core = { workspace = true } rag-observability = { workspace = true } -[project.scripts] -ragctl = "rag_config.cli:app" - [tool.hatch.build.targets.wheel] packages = ["src/rag_config"] diff --git a/packages/config/src/rag_config/py.typed b/packages/config/src/rag_config/py.typed new file mode 100644 index 0000000..e69de29 diff --git a/packages/observability/src/rag_observability/py.typed b/packages/observability/src/rag_observability/py.typed new file mode 100644 index 0000000..e69de29 diff --git a/packages/ragctl/README.md b/packages/ragctl/README.md new file mode 100644 index 0000000..b5372b0 --- /dev/null +++ b/packages/ragctl/README.md @@ -0,0 +1,46 @@ +# rag-ragctl + +The **`ragctl`** control-plane CLI for AgentContextOS. + +## Install + +`ragctl` is part of the AgentContextOS uv workspace. Once the workspace is bootstrapped +(`task bootstrap`), the command is available on your PATH: + +```bash +ragctl --help +``` + +## Command groups + +| Group | Status | Purpose | +|-------|--------|---------| +| `config` | ✅ available | Validate and diff `rag.yaml` files (Step 0.4). | +| `eval` | ✅ available | Run and inspect golden-set evaluations (Step 0.8). | +| `traces` | ✅ available | Query distributed traces from Jaeger (Step 0.7). | +| `version` | ✅ available | Print the installed `ragctl` version. | +| `ingest` | 🚧 scaffold | Trigger ingestion via the gateway (lands in Step 1.10). | +| `query` | 🚧 scaffold | Run a query against the gateway (lands in Step 3.1). | +| `logs` | 🚧 scaffold | Tail structured logs (lands in Step 5.6). | +| `tenant` | 🚧 scaffold | Manage tenants (lands in Step 6.1). | +| `plugin` | 🚧 scaffold | Manage SPI plugins (lands in Step 1.1). | +| `secret` | 🚧 scaffold | Manage tenant secrets (lands in Step 6.7). | + +Scaffold commands exit `0` and print which step delivers the real implementation. + +## Shell completion + +`ragctl` ships with Typer-powered shell completion for bash, zsh, fish, and PowerShell: + +```bash +# install completion for the current shell +ragctl --install-completion + +# print the completion script (e.g. to inspect or pipe into a file) +ragctl --show-completion +``` + +## See also + +- [docs/reference/ragctl.md](../../docs/reference/ragctl.md) — full command reference +- [docs/guides/ragctl-quickstart.md](../../docs/guides/ragctl-quickstart.md) — operator quickstart diff --git a/packages/ragctl/pyproject.toml b/packages/ragctl/pyproject.toml new file mode 100644 index 0000000..e43ce0e --- /dev/null +++ b/packages/ragctl/pyproject.toml @@ -0,0 +1,30 @@ +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[project] +name = "rag-ragctl" +version = "0.1.0" +description = "AgentContextOS — ragctl control-plane CLI" +readme = "README.md" +requires-python = ">=3.12" +dependencies = [ + "typer>=0.12", + "rag-core>=0.1.0", + "rag-config>=0.1.0", + "rag-observability>=0.1.0", +] + +[tool.uv.sources] +rag-core = { workspace = true } +rag-config = { workspace = true } +rag-observability = { workspace = true } + +[project.scripts] +ragctl = "ragctl.main:app" + +[tool.hatch.build.targets.wheel] +packages = ["src/ragctl"] + +[tool.hatch.metadata] +allow-direct-references = true diff --git a/packages/ragctl/src/ragctl/__init__.py b/packages/ragctl/src/ragctl/__init__.py new file mode 100644 index 0000000..b9e98d4 --- /dev/null +++ b/packages/ragctl/src/ragctl/__init__.py @@ -0,0 +1,9 @@ +"""ragctl — AgentContextOS control-plane CLI.""" + +from __future__ import annotations + +from ragctl.main import app, main + +__all__ = ["__version__", "app", "main"] + +__version__ = "0.1.0" diff --git a/packages/config/src/rag_config/cli.py b/packages/ragctl/src/ragctl/main.py similarity index 68% rename from packages/config/src/rag_config/cli.py rename to packages/ragctl/src/ragctl/main.py index a04a79b..8375dff 100644 --- a/packages/config/src/rag_config/cli.py +++ b/packages/ragctl/src/ragctl/main.py @@ -1,37 +1,74 @@ """ragctl — AgentContextOS control-plane CLI. -Step 0.4 delivers the ``config`` sub-command group. -Step 0.8 delivers the ``eval`` sub-command group (run + show). -Remaining sub-commands (ingest, query, logs, traces, tenant, secret, -plugin, version) are scaffolded in Step 0.10. - -Entry-point: ``ragctl`` (registered in packages/config/pyproject.toml). +Root entry point for the consolidated CLI. Sub-command groups land step by step: + +| Group | Step delivered | Status | +|-----------|-----------------|----------| +| config | 0.4 | working | +| traces | 0.7 | working | +| eval | 0.8 | working | +| version | 0.10 | working | +| ingest | 1.10 | scaffold | +| plugin | 1.1 | scaffold | +| query | 3.1 | scaffold | +| logs | 5.6 | scaffold | +| tenant | 6.1 | scaffold | +| secret | 6.7 | scaffold | + +Scaffold commands print a one-line "delivered in step X.Y" notice and exit 0 +so operators can discover the planned surface area today. """ from __future__ import annotations +import json +import urllib.request from pathlib import Path import typer -from rag_core.errors import ConfigError - from rag_config.loader import load +from rag_core.errors import ConfigError app = typer.Typer( name="ragctl", help="AgentContextOS control-plane CLI.", no_args_is_help=True, - add_completion=False, + add_completion=True, ) -config_app = typer.Typer(help="Manage and validate rag.yaml configs.", no_args_is_help=True) -app.add_typer(config_app, name="config") -traces_app = typer.Typer(help="Query distributed traces from Jaeger.", no_args_is_help=False) -app.add_typer(traces_app, name="traces") +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + + +def _scaffold_notice(group: str, step: str) -> None: + """Print a uniform 'not implemented yet' message and exit cleanly.""" + typer.echo(f"ragctl {group}: scaffold — full implementation lands in Step {step}.") + + +# --------------------------------------------------------------------------- +# version +# --------------------------------------------------------------------------- + + +@app.command("version") +def version() -> None: + """Print the installed ragctl version.""" + from ragctl import __version__ + + typer.echo(f"ragctl {__version__}") -eval_app = typer.Typer(help="Run and inspect golden-set evaluations.", no_args_is_help=True) -app.add_typer(eval_app, name="eval") + +# --------------------------------------------------------------------------- +# config — Step 0.4 +# --------------------------------------------------------------------------- + +config_app = typer.Typer( + help="Manage and validate rag.yaml configs.", + no_args_is_help=True, +) +app.add_typer(config_app, name="config") @config_app.command("validate") @@ -55,11 +92,9 @@ def config_validate( cfg = load(path) except ConfigError as exc: if json_output: - import json - - typer.echo(json.dumps({"valid": False, "error": str(exc)}, indent=2), err=False) + typer.echo(json.dumps({"valid": False, "error": str(exc)}, indent=2)) else: - typer.echo(f"INVALID {path}", err=False) + typer.echo(f"INVALID {path}") typer.echo(f" {exc.message}", err=True) raise typer.Exit(1) # noqa: B904 @@ -87,8 +122,6 @@ def config_validate( } if json_output: - import json - typer.echo(json.dumps(summary, indent=2)) else: typer.echo(f"VALID {path}") @@ -109,8 +142,6 @@ def config_diff( path_b: Path = typer.Argument(..., help="Second rag.yaml"), ) -> None: """Show backend differences between two rag.yaml files.""" - import json - try: cfg_a = load(path_a) cfg_b = load(path_b) @@ -141,6 +172,17 @@ def _diff(a_node: object, b_node: object, prefix: str = "") -> list[str]: typer.echo(line) +# --------------------------------------------------------------------------- +# traces — Step 0.7 +# --------------------------------------------------------------------------- + +traces_app = typer.Typer( + help="Query distributed traces from Jaeger.", + no_args_is_help=False, +) +app.add_typer(traces_app, name="traces") + + @traces_app.callback(invoke_without_command=True) def traces_list( ctx: typer.Context, @@ -152,9 +194,6 @@ def traces_list( if ctx.invoked_subcommand is not None: return - import json - import urllib.request - api_url = f"{jaeger_url}/api/traces?service={service}&limit={limit}" try: with urllib.request.urlopen(api_url) as resp: # noqa: S310 @@ -183,6 +222,17 @@ def traces_list( typer.echo(f"{trace_id:<20} {op:<40} {duration_us / 1000:>8.1f}ms {len(spans)}") +# --------------------------------------------------------------------------- +# eval — Step 0.8 +# --------------------------------------------------------------------------- + +eval_app = typer.Typer( + help="Run and inspect golden-set evaluations.", + no_args_is_help=True, +) +app.add_typer(eval_app, name="eval") + + @eval_app.command("run") def eval_run( golden: Path = typer.Argument( @@ -210,8 +260,6 @@ def eval_run( ragctl eval run tests/eval/golden/ --top-k 10 --output report.json """ - import json as _json - from rag_config.eval import ragas_available, run_eval if use_ragas and not ragas_available(): @@ -225,7 +273,7 @@ def eval_run( report = run_eval(golden_path=golden, k=k, use_ragas=use_ragas) - report_json = _json.dumps(_json.loads(report.model_dump_json()), indent=2) + report_json = json.dumps(json.loads(report.model_dump_json()), indent=2) output.write_text(report_json, encoding="utf-8") if not quiet: @@ -256,12 +304,10 @@ def eval_show( ragctl eval show eval-report.json --verbose """ - import json as _json - from rag_core.eval import EvalReport try: - data = _json.loads(report.read_text(encoding="utf-8")) + data = json.loads(report.read_text(encoding="utf-8")) rep = EvalReport.model_validate(data) except Exception as exc: # noqa: BLE001 typer.echo(f"ERROR: cannot load report — {exc}", err=True) @@ -292,6 +338,77 @@ def eval_show( ) +# --------------------------------------------------------------------------- +# Scaffold sub-apps — print a "delivered in Step X.Y" notice and exit 0. +# +# Each one is registered as a Typer sub-app with a single default callback so +# the command shape (`ragctl ...`) is correct from day one, even +# though no real work is performed yet. +# --------------------------------------------------------------------------- + + +def _scaffold_app(group: str, step: str, help_text: str) -> typer.Typer: + sub = typer.Typer(help=help_text, no_args_is_help=False) + + @sub.callback(invoke_without_command=True) + def _entry(ctx: typer.Context) -> None: + if ctx.invoked_subcommand is not None: + return + _scaffold_notice(group, step) + + return sub + + +app.add_typer( + _scaffold_app( + "ingest", + "1.10", + "Trigger ingestion pipelines via the gateway (lands in Step 1.10).", + ), + name="ingest", +) +app.add_typer( + _scaffold_app( + "query", + "3.1", + "Run a query against the gateway (lands in Step 3.1).", + ), + name="query", +) +app.add_typer( + _scaffold_app( + "logs", + "5.6", + "Tail structured logs from the platform (lands in Step 5.6).", + ), + name="logs", +) +app.add_typer( + _scaffold_app( + "tenant", + "6.1", + "Manage tenants (lands in Step 6.1).", + ), + name="tenant", +) +app.add_typer( + _scaffold_app( + "plugin", + "1.1", + "Manage SPI plugin registration (lands in Step 1.1).", + ), + name="plugin", +) +app.add_typer( + _scaffold_app( + "secret", + "6.7", + "Manage tenant secrets (lands in Step 6.7).", + ), + name="secret", +) + + def main() -> None: app() diff --git a/packages/ragctl/src/ragctl/py.typed b/packages/ragctl/src/ragctl/py.typed new file mode 100644 index 0000000..e69de29 diff --git a/packages/ragctl/tests/test_completion.py b/packages/ragctl/tests/test_completion.py new file mode 100644 index 0000000..21426cc --- /dev/null +++ b/packages/ragctl/tests/test_completion.py @@ -0,0 +1,34 @@ +"""Shell completion is wired through Typer — Step 0.10. + +We don't try to drive Typer's completion-script generation here: it requires +``shellingham`` to detect a real interactive shell, which doesn't work inside +``CliRunner``. Instead we verify the wiring structurally: + +- the Typer app was constructed with ``add_completion=True`` +- the underlying Click command exposes both completion flags + +This is resilient to how Typer/Rich formats ``--help`` (which wraps text +differently in CI's headless environment than locally). +""" + +from __future__ import annotations + +import typer.main +from ragctl.main import app + + +def test_completion_is_enabled_on_app() -> None: + """Sanity check: the Typer app was constructed with completion enabled.""" + assert app._add_completion is True # noqa: SLF001 + + +def test_install_completion_option_is_registered() -> None: + cmd = typer.main.get_command(app) + flags = {opt for param in cmd.params for opt in param.opts} + assert "--install-completion" in flags + + +def test_show_completion_option_is_registered() -> None: + cmd = typer.main.get_command(app) + flags = {opt for param in cmd.params for opt in param.opts} + assert "--show-completion" in flags diff --git a/packages/ragctl/tests/test_main.py b/packages/ragctl/tests/test_main.py new file mode 100644 index 0000000..5179eeb --- /dev/null +++ b/packages/ragctl/tests/test_main.py @@ -0,0 +1,49 @@ +"""Smoke tests for the ragctl root CLI — Step 0.10.""" + +from __future__ import annotations + +import ragctl +from ragctl.main import app +from typer.testing import CliRunner + +runner = CliRunner() + + +def test_root_help_lists_all_groups() -> None: + """Every advertised sub-command group should appear in --help.""" + result = runner.invoke(app, ["--help"]) + assert result.exit_code == 0, result.output + + expected_groups = ( + "config", + "eval", + "traces", + "ingest", + "query", + "logs", + "tenant", + "plugin", + "secret", + "version", + ) + for group in expected_groups: + assert group in result.output, f"missing group '{group}' in --help output" + + +def test_version_command_prints_package_version() -> None: + result = runner.invoke(app, ["version"]) + assert result.exit_code == 0, result.output + assert ragctl.__version__ in result.output + assert "ragctl" in result.output + + +def test_no_args_shows_help() -> None: + """Running `ragctl` with no arguments should print the help text.""" + result = runner.invoke(app, []) + # Typer exits with code 2 when no_args_is_help displays help; either is OK. + assert result.exit_code in (0, 2), result.output + assert "Usage:" in result.output + + +def test_package_version_is_exported() -> None: + assert isinstance(ragctl.__version__, str) and ragctl.__version__ diff --git a/packages/ragctl/tests/test_scaffold_commands.py b/packages/ragctl/tests/test_scaffold_commands.py new file mode 100644 index 0000000..347f8ca --- /dev/null +++ b/packages/ragctl/tests/test_scaffold_commands.py @@ -0,0 +1,35 @@ +"""Each scaffold sub-app must be reachable and exit cleanly — Step 0.10.""" + +from __future__ import annotations + +import pytest +from ragctl.main import app +from typer.testing import CliRunner + +runner = CliRunner() + +SCAFFOLD_GROUPS = ( + ("ingest", "1.10"), + ("query", "3.1"), + ("logs", "5.6"), + ("tenant", "6.1"), + ("plugin", "1.1"), + ("secret", "6.7"), +) + + +@pytest.mark.parametrize(("group", "step"), SCAFFOLD_GROUPS) +def test_scaffold_group_announces_target_step(group: str, step: str) -> None: + """Invoking a scaffold group prints its 'lands in Step X.Y' notice.""" + result = runner.invoke(app, [group]) + assert result.exit_code == 0, result.output + assert "scaffold" in result.output + assert step in result.output + + +@pytest.mark.parametrize(("group", "_step"), SCAFFOLD_GROUPS) +def test_scaffold_group_help_works(group: str, _step: str) -> None: + """Each scaffold group must support --help.""" + result = runner.invoke(app, [group, "--help"]) + assert result.exit_code == 0, result.output + assert "Usage:" in result.output diff --git a/pyproject.toml b/pyproject.toml index 25a1929..8cead8a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -6,6 +6,7 @@ members = [ "packages/core", "packages/config", "packages/observability", + "packages/ragctl", "apps/gateway", ] @@ -42,6 +43,7 @@ unfixable = ["B"] "**/conftest.py" = ["ANN"] # Typer uses Argument/Option as default values — B008 is a false positive here. "**/cli.py" = ["B008"] +"**/ragctl/main.py" = ["B008"] # Scripts, code generators, and infra setup legitimately write to stdout/stderr. "scripts/**/*.py" = ["T201"] "infra/**/*.py" = ["T201"] @@ -68,7 +70,7 @@ disallow_untyped_defs = false [tool.pytest.ini_options] asyncio_mode = "auto" testpaths = ["tests", "packages"] -pythonpath = ["packages/core/src", "packages/config/src", "packages/observability/src", "apps/gateway/src"] +pythonpath = ["packages/core/src", "packages/config/src", "packages/observability/src", "packages/ragctl/src", "apps/gateway/src"] addopts = "-x -q --tb=short --import-mode=importlib" markers = [ "contract: SPI conformance tests", diff --git a/tests/eval/test_cli.py b/tests/eval/test_cli.py index a61632b..3dbf37a 100644 --- a/tests/eval/test_cli.py +++ b/tests/eval/test_cli.py @@ -6,8 +6,8 @@ from pathlib import Path import pytest -from rag_config.cli import app from rag_core.eval import EvalReport +from ragctl.main import app from typer.testing import CliRunner runner = CliRunner() diff --git a/uv.lock b/uv.lock index 4d3a74b..e151ed4 100644 --- a/uv.lock +++ b/uv.lock @@ -22,6 +22,7 @@ members = [ "rag-core", "rag-gateway", "rag-observability", + "rag-ragctl", ] [manifest.dependency-groups] @@ -2601,6 +2602,25 @@ requires-dist = [ { name = "rag-core", editable = "packages/core" }, ] +[[package]] +name = "rag-ragctl" +version = "0.1.0" +source = { editable = "packages/ragctl" } +dependencies = [ + { name = "rag-config" }, + { name = "rag-core" }, + { name = "rag-observability" }, + { name = "typer" }, +] + +[package.metadata] +requires-dist = [ + { name = "rag-config", editable = "packages/config" }, + { name = "rag-core", editable = "packages/core" }, + { name = "rag-observability", editable = "packages/observability" }, + { name = "typer", specifier = ">=0.12" }, +] + [[package]] name = "ragas" version = "0.4.3"