Add infrahubctl marketplace CLI for fetching schemas and collections

.vale/styles/spelling-exceptions.txt

@@ -111,6 +111,8 @@ REST
 resources
 schema_mapping
 sdk
+stderr
+stdout
 subcommand
 subnet
 subtyping

changelog/952.added.md

@@ -0,0 +1 @@
+Add `infrahubctl marketplace get` for fetching schemas and collections from the Infrahub Marketplace. Auto-detects schemas vs collections by namespace/name, supports `--version` for pinning, `--collection` to force the collection path, `--stdout` to stream content for piping (status messages on stderr), and `--marketplace-url` / `INFRAHUB_MARKETPLACE_URL` to point at staging or local instances.

dev/specs/001-marketplace-api-update/checklists/requirements.md

@@ -0,0 +1,36 @@
+# Specification Quality Checklist: Marketplace Download Command Update
+
+**Purpose**: Validate specification completeness and quality before proceeding to planning
+**Created**: 2026-04-21
+**Feature**: [spec.md](../spec.md)
+
+## Content Quality
+
+- [x] No implementation details (languages, frameworks, APIs)
+- [x] Focused on user value and business needs
+- [x] Written for non-technical stakeholders
+- [x] All mandatory sections completed
+
+## Requirement Completeness
+
+- [x] No [NEEDS CLARIFICATION] markers remain
+- [x] Requirements are testable and unambiguous
+- [x] Success criteria are measurable
+- [x] Success criteria are technology-agnostic (no implementation details)
+- [x] All acceptance scenarios are defined
+- [x] Edge cases are identified
+- [x] Scope is clearly bounded
+- [x] Dependencies and assumptions identified
+
+## Feature Readiness
+
+- [x] All functional requirements have clear acceptance criteria
+- [x] User scenarios cover primary flows
+- [x] Feature meets measurable outcomes defined in Success Criteria
+- [x] No implementation details leak into specification
+
+## Notes
+
+- The spec intentionally references the REST API at a behavioural level only; specific endpoint paths appear only in the informational "Implementation Status" section, which is explicitly outside acceptance and describes what the current branch has already shipped.
+- The "schema wins on collision" precedence is stated as an assumption rather than as a clarification-blocker because it mirrors the pre-existing behaviour; revisit during `/speckit.clarify` if stakeholders want a different default.
+- Items marked incomplete require spec updates before `/speckit.clarify` or `/speckit.plan`.

dev/specs/001-marketplace-api-update/contracts/marketplace-api.md

@@ -0,0 +1,95 @@
+# External Contract: Marketplace REST API
+
+**Scope**: This document reverse-describes the subset of the `marketplace.infrahub.app` REST API that the `infrahubctl marketplace download` command relies on. **This is not a contract we own** — it captures the CLI's assumptions so that drift between client expectation and server reality can be caught in tests.
+
+**Base URL**: `https://marketplace.infrahub.app` (default). Overridable via `--marketplace-url` or the `marketplace_url` setting in `infrahubctl.toml` / `INFRAHUB_MARKETPLACE_URL` env.
+
+---
+
+## Endpoint 1 — Download latest schema
+
+```text
+GET {base_url}/api/v1/schemas/{namespace}/{name}/download
+```
+
+**Success (200)**:
+
+- `Content-Type`: `application/yaml` (or `text/yaml` / `text/plain`)
+- Response body: raw YAML payload of the schema.
+- Response header: `x-schema-version: <semver>` — resolved version the server returned. Required for the CLI to echo the version to the user.
+
+**Not found (404)**: JSON body `{"detail": "<reason>"}`. Treated by the CLI as input to the not-found / auto-detect fallback flow.
+
+**Other 4xx**: JSON body `{"detail": "<reason>"}`; surfaced as an error with the detail message.
+
+**5xx / transport failure**: surfaced as a network-class error (see `research.md` R-4).
+
+---
+
+## Endpoint 2 — Download specific schema version
+
+```text
+GET {base_url}/api/v1/schemas/{namespace}/{name}/versions/{version}/download
+```
+
+Same response contract as Endpoint 1, but `{version}` is a user-supplied semver.
+
+**404 semantics**:
+
+- If the schema itself does not exist, a 404 is returned. Cannot be distinguished from "version missing" by URL alone, so the CLI MUST probe the unversioned endpoint first when constructing a "version-not-found" vs. "not-found" error (see `research.md` R-4).
+- If the schema exists but the specific version is not published, a 404 is returned.
+
+---
+
+## Endpoint 3 — Download collection
+
+```text
+GET {base_url}/api/v1/collections/{namespace}/{name}/download
+```
+
+**Success (200)**: JSON body with shape:
+
+```json
+{
+  "collection": {
+    "namespace": "acme",
+    "name": "starter-pack",
+    "schema_count": 2,
+    "downloaded_count": 2,
+    "skipped": [
+      { "namespace": "acme", "name": "broken", "reason": "no published version" }
+    ]
+  },
+  "schemas": [
+    {
+      "namespace": "acme",
+      "name": "network-base",
+      "semver": "1.0.0",
+      "filename": "acme-network-base-1.0.0.yml",
+      "content": "---\n..."
+    }
+  ]
+}
+```
+
+The CLI writes each `schemas[].content` to disk under `<output_dir>/<collection_name>/<schemas[].name>.yml` (today) or a flat layout under `<output_dir>/` (future — see implementation notes below).
+
+**404 / other errors**: same as Endpoint 1.
+
+---
+
+## Implicit contract for auto-detection
+
+The CLI issues Endpoints 1 and 3 in parallel when the user omits `--collection`. The contract assumed is:
+
+- Both endpoints are idempotent and safe to issue concurrently.
+- Either endpoint responds with a well-formed 404 (not a 5xx) when the identifier is not present as that item type. A 5xx from either probe is treated as a transport failure and aborts auto-detection.
+- If both endpoints return 200, the CLI applies the documented "schema wins" precedence (`research.md` R-3).
+
+**Follow-up (out of scope)**: Request the marketplace team add lightweight metadata endpoints (`GET /api/v1/schemas/{ns}/{name}` and `GET /api/v1/collections/{ns}/{name}` without `/download`) so the CLI can probe cheaply without paying for the payload on the wasted side of a 200-200 collision. When those ship, the CLI swaps its probe targets and keeps the download endpoints only for the winner.
+
+---
+
+## Versions of this contract
+
+This document reflects the API shape observed on or before **2026-04-21**. Any new fields added server-side are expected to be backward compatible; breaking changes require a coordinated CLI update.

dev/specs/001-marketplace-api-update/plan.md

@@ -0,0 +1,102 @@
+# Implementation Plan: Marketplace Download Command Update
+
+**Branch**: `knotty-dibble` | **Date**: 2026-04-21 | **Spec**: [spec.md](spec.md)
+**Input**: Feature specification from `specs/001-marketplace-api-update/spec.md`
+
+## Summary
+
+Migrate `infrahubctl marketplace get` to the public REST API at `marketplace.infrahub.app`, add auto-detection of schema-vs-collection identifiers, keep the `--version` pinning flag for schemas, and retain `--output-dir` (default `./schemas`). The REST migration, `--version`, and `--output-dir` pieces have already shipped in the current branch's `Marketplace` commit; the remaining deltas are auto-detection, the four-class error taxonomy, and the documented precedence rule for namespace/name collisions.
+
+## Technical Context
+
+**Language/Version**: Python 3.10–3.13
+**Primary Dependencies**: `typer` (CLI), `httpx` (HTTP), `rich` (console output), `pydantic` 2.x (config). No new runtime dependencies are expected.
+**Storage**: Files on disk under the user-chosen `--output-dir` (default `./schemas`). No database or server-side state owned by this change.
+**Testing**: `pytest` with `pytest-httpx` for mocking the marketplace REST API; existing tests in `tests/unit/ctl/test_marketplace_app.py` are the template.
+**Target Platform**: Cross-platform CLI (macOS/Linux/Windows), installed via `uv`/`pip` as the `infrahubctl` entry point.
+**Project Type**: Single project (Python SDK + CLI) — uses the `infrahub_sdk` package layout already in place.
+**Performance Goals**: Interactive CLI; target end-to-end command latency dominated by the marketplace round-trip. Auto-detection must not exceed one additional round-trip beyond the download itself in the common (cache-miss, 404-on-schema) case.
+**Constraints**: Must remain backward compatible with scripts that already pass `--collection` / `--output-dir` / `--version`. Must not require any new public SDK surface outside the CLI module. Must not re-introduce GraphQL usage for marketplace calls.
+**Scale/Scope**: Small surface area — a single `infrahub_sdk/ctl/marketplace.py` module (~180 LOC today) plus its test file. Expected diff is additive, likely under ~150 LOC of new code plus tests.
+
+## Constitution Check
+
+*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
+
+This repository ships a template-only `.specify/memory/constitution.md` (unfilled) in sibling worktrees, and none in the current branch. In the absence of concrete articles, the following project-level gates (derived from `AGENTS.md`) are applied:
+
+- **Async/sync dual pattern**: N/A for this change — the marketplace CLI command is already async-only (via `AsyncTyper`) and deliberately does not expose a sync twin. Adding a sync twin is out of scope.
+- **Type hints on all signatures**: PASS — will be enforced on any new helpers.
+- **No modifications to generated code (`protocols.py`)**: PASS — not touched.
+- **No new runtime dependencies without asking first**: PASS — no new dependencies planned.
+- **Lint + format gates (`uv run invoke format lint-code`)**: PASS — will be run before committing.
+- **Tests-first spirit**: Will add unit tests covering each new acceptance scenario before wiring the CLI changes.
+
+**Result**: PASS. No violations to justify.
+
+## Project Structure
+
+### Documentation (this feature)
+
+```text
+specs/001-marketplace-api-update/
+├── spec.md                 # Feature specification (already written)
+├── plan.md                 # This file
+├── research.md             # Phase 0 output
+├── contracts/
+│   └── marketplace-api.md  # Reverse-documented external REST contract we consume
+├── quickstart.md           # Phase 1 output
+├── checklists/
+│   └── requirements.md     # Spec quality validation (already written)
+└── tasks.md                # (Produced later by /speckit.tasks)
+```
+
+`data-model.md` is intentionally omitted: this feature consumes an external REST API and writes YAML to disk, with no owned entity model to design. The Key Entities in `spec.md` are behavioural, not a data-store schema.
+
+### Source Code (repository root)
+
+```text
+infrahub_sdk/
+├── ctl/
+│   ├── marketplace.py         # Primary code under change
+│   ├── config.py              # `marketplace_url` setting (already in place)
+│   └── cli_commands.py        # Registers the marketplace sub-app (no change expected)
+
+tests/
+└── unit/
+    └── ctl/
+        └── test_marketplace_app.py   # Primary test file to extend
+```
+
+**Structure Decision**: Single-project Python layout. All implementation lives in the existing `infrahub_sdk/ctl/marketplace.py` module; tests live alongside existing tests in `tests/unit/ctl/test_marketplace_app.py`. No new packages or modules are introduced.
+
+## Phases
+
+### Phase 0 — Research
+
+See [research.md](research.md). Topics resolved:
+
+1. Auto-detection strategy (schema-probe-first vs. parallel probe vs. metadata endpoint).
+2. Name collision precedence (`schema wins` — see Assumption in spec).
+3. Error taxonomy mapping (not-found vs. version-not-found vs. network vs. invalid-input) onto observable HTTP responses from the marketplace.
+4. Behaviour when a schema exists only as a pre-release (no stable published semver).
+
+### Phase 1 — Design & Contracts
+
+**Prerequisites:** `research.md` complete.
+
+1. **External contract documentation** → [contracts/marketplace-api.md](contracts/marketplace-api.md).
+   Documents the *consumed* REST endpoints (schemas, collections, version pinning), expected response shapes, and HTTP status semantics. This is not an owned contract — it reverse-describes what the CLI assumes about `marketplace.infrahub.app`, so drift can be detected.
+
+2. **Quickstart** → [quickstart.md](quickstart.md).
+   Manual and automated verification steps that exercise each acceptance scenario from the spec.
+
+3. **Agent context update**: not applicable — no `.specify/scripts/bash/update-agent-context.sh` is installed in this branch, and the project-level agent context (`AGENTS.md`) does not require updates for this feature (no new dependencies or commands).
+
+### Phase 2 — Tasks (deferred)
+
+Not produced by this command. Will be generated by `/speckit.tasks` against this plan.
+
+## Complexity Tracking
+
+No constitution violations to justify. The feature fits inside the existing module with no new abstractions.

dev/specs/001-marketplace-api-update/quickstart.md

@@ -0,0 +1,103 @@
+# Quickstart: Marketplace Get Command
+
+End-to-end smoke test for the updated `infrahubctl marketplace get` command. These steps exercise each acceptance scenario in `spec.md` and should be green before merging.
+
+## Prerequisites
+
+```bash
+uv sync --all-groups --all-extras
+```
+
+## Unit tests (fast loop)
+
+```bash
+uv run pytest tests/unit/ctl/test_marketplace_app.py -v
+```
+
+All existing tests must stay green, and new tests MUST be added to cover:
+
+- Auto-detection when identifier is a schema (no `--collection` passed).
+- Auto-detection when identifier is a collection (no `--collection` passed).
+- Auto-detection when both endpoints return 200 — resolved as schema, type printed in output.
+- Auto-detection when both endpoints return 404 — error class "not found".
+- `--version` with an unpublished version — error class "version not found".
+- 5xx on either probe endpoint — error class "network".
+- Invalid identifier (no slash) — error class "invalid input" — caught before any network call.
+
+## Manual verification against the public marketplace
+
+```bash
+# Scenario 1: download a schema by auto-detection
+uv run infrahubctl marketplace get acme/network-base
+ls schemas/
+
+# Scenario 2: download a collection by auto-detection
+uv run infrahubctl marketplace get acme/starter-pack
+ls schemas/
+
+# Scenario 3: pin a specific schema version
+uv run infrahubctl marketplace get acme/network-base --version 0.9.0
+grep '^version:' schemas/network-base.yml
+
+# Scenario 4: custom output directory
+uv run infrahubctl marketplace get acme/network-base --output-dir ./tmp/market-test
+ls ./tmp/market-test
+
+# Scenario 5: explicit --collection still works (override path)
+uv run infrahubctl marketplace get acme/starter-pack --collection
+
+# Scenario 6: version on a collection emits a warning, proceeds
+uv run infrahubctl marketplace get acme/starter-pack --version 1.0.0
+# Expect: "Warning: --version is ignored when downloading a collection." followed by success output.
+```
+
+## Manual verification against a local/staging marketplace
+
+```bash
+uv run infrahubctl marketplace get acme/test \
+  --marketplace-url http://localhost:8000 \
+  --output-dir ./tmp/local-market
+```
+
+This must exercise the same auto-detection behaviour against the overridden host.
+
+## Expected success output shape
+
+For a schema:
+
+```text
+Downloaded schema acme/network-base v1.2.0 -> schemas/network-base.yml
+```
+
+For a collection:
+
+```text
+Downloaded acme/network-base v1.0.0 -> schemas/starter-pack/network-base.yml
+Downloaded acme/dcim v2.1.0 -> schemas/starter-pack/dcim.yml
+
+Collection acme/starter-pack: 2/2 schemas downloaded
+```
+
+The CLI MUST announce the resolved item type (schema vs. collection) explicitly so the user can detect an unintended match in a collision case.
+
+## Expected error output shapes
+
+```text
+# Not found
+No schema or collection named 'acme/missing' found on marketplace.infrahub.app
+
+# Version not found
+Schema 'acme/network-base' has no published version '9.9.9'. Run without --version for the latest.
+
+# Network
+Could not reach marketplace at https://marketplace.infrahub.app: connection timed out. Check your connection or --marketplace-url.
+
+# Invalid input
+Invalid identifier 'acme-network-base'. Expected format: namespace/name
+```
+
+## Lint / format / type gates (must all pass)
+
+```bash
+uv run invoke format lint-code
+```