Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,15 @@ on:
- "pyproject.toml"
- "scripts/validate_docs.py"
- "scripts/validate_changelog.py"
- "scripts/generate_compatibility_dashboard.py"
- ".github/workflows/tests.yml"
- ".github/workflows/dependency-matrix.yml"
- ".github/workflows/compatibility.yml"
- "scripts/generate_api_reference.py"
- "scripts/generate_compatibility_dashboard.py"
- ".github/workflows/tests.yml"
- ".github/workflows/dependency-matrix.yml"
- ".github/workflows/compatibility.yml"
- "tests/validate.sh"
- ".github/workflows/docs.yml"
pull_request:
Expand Down Expand Up @@ -56,6 +64,7 @@ jobs:
run: |
python scripts/validate_docs.py
python scripts/validate_changelog.py
python scripts/generate_compatibility_dashboard.py --check

- name: Validate generated public API reference
run: python scripts/generate_api_reference.py --check
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ jobs:
python scripts/validate_schemas.py
python scripts/validate_contract_fixtures.py
node scripts/validate_contract_fixtures.mjs
python scripts/generate_compatibility_dashboard.py --check
python scripts/benchmark_runtime.py --check
python -m compileall -q examples
- name: Run tests with coverage threshold
Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ and versions are tracked in the repo-root `VERSION` file.

- Automate GitHub Releases from matching version tags with reviewed
distributions, checksums, SBOM metadata, and generated comparison notes.
- Publish a generated dependency and platform compatibility dashboard linked
from the README and documentation site.
- Add framework-specific migration guides for Click, Typer, Cement, and
`argparse`, with rollout and rollback checklists.
- Add golden success, error, inspection, log, NDJSON, and command-protocol
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ Read the complete documentation at

Compatibility results and the permissioned-adopter policy are documented in
the [adoption and compatibility evidence guide](https://basefoundry.github.io/base-cli/adoption-evidence/).
The [compatibility dashboard](https://basefoundry.github.io/base-cli/compatibility-dashboard/)
shows the declared and CI-tested dependency and platform support boundaries.

## Quick start

Expand Down
53 changes: 53 additions & 0 deletions docs/compatibility-dashboard.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Compatibility dashboard

[![Tests](https://img.shields.io/github/actions/workflow/status/basefoundry/base-cli/tests.yml?branch=main&label=tests)](https://github.com/basefoundry/base-cli/actions/workflows/tests.yml)
[![Dependency matrix](https://img.shields.io/github/actions/workflow/status/basefoundry/base-cli/dependency-matrix.yml?branch=main&label=dependencies)](https://github.com/basefoundry/base-cli/actions/workflows/dependency-matrix.yml)
[![Reference consumers](https://img.shields.io/github/actions/workflow/status/basefoundry/base-cli/compatibility.yml?branch=main&label=consumers)](https://github.com/basefoundry/base-cli/actions/workflows/compatibility.yml)

This page is generated from the package metadata and the matrices in
`.github/workflows/tests.yml`, `dependency-matrix.yml`, and
`compatibility.yml`. The badges above reflect the latest completed `main`
workflow runs; a green badge means the declared matrix passed, while a gray or
red badge means that evidence is pending or needs investigation.

## Current support contract

| Surface | Declared support | Completed CI coverage | Interpretation |
| --- | --- | --- | --- |
| Python | `>=3.10,<4` | Python 3.10, Python 3.11, Python 3.12, Python 3.13, Python 3.14 | Supported and tested |
| Click | `click>=8.1,<8.5` | 8.1, 8.2, 8.3, 8.4 | Supported and tested on Python 3.10 and 3.14 |
| PyYAML extra | `PyYAML>=6.0,<7` | 6.0 | Supported when `base-cli[yaml]` is installed |
| Typer extra | `typer>=0.12,<0.28` | 0.25.1, 0.26.0, 0.27.1 | Supported through `attach_typer()` |
| Platforms | Pure-Python core | macos-latest, ubuntu-latest, windows-latest, Debian 12, Fedora latest, WSL2 | Supported tiers documented below |

## What the labels mean

- **Supported and tested** means the combination is declared by package
metadata and exercised by a named CI matrix.
- **Supported, not exhaustive** means the package contract applies, but CI
samples representative versions or platforms rather than every patch level.
- **Untested** means a combination may install but is not a support claim.
- **Unsupported** means package metadata or an explicit policy excludes it;
reports from that combination are welcome but are not release blockers.

The platform boundary is detailed in
[`platform-support.md`](platform-support.md), and dependency-window changes
must follow [`dependency-support.md`](dependency-support.md). Reference
consumer results are compatibility evidence, not customer adoption claims; see
[`adoption-evidence.md`](adoption-evidence.md).

## Updating the dashboard

Do not edit this page's matrix by hand. When a package window or CI matrix
changes, run:

```bash
python scripts/generate_compatibility_dashboard.py
python scripts/generate_compatibility_dashboard.py --check
```

The CI quality and documentation jobs fail when the generated page is stale.
Update the dashboard and the relevant support-policy/release-note entry in the
same pull request. The generator intentionally records declarations and live
workflow links, while GitHub Actions remains the source of truth for the most
recent run outcome.
2 changes: 2 additions & 0 deletions docs/dependency-support.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
This page is the human-readable dependency contract for the current release
line. The package metadata is authoritative for installation; this matrix
documents the versions covered by CI and the process for widening a window.
The generated [compatibility dashboard](compatibility-dashboard.md) combines
these dependency declarations with the platform and adapter matrices.

## Core runtime

Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ nav:
- Framework choice: framework-choice.md
- Adopter readiness: adopter-readiness.md
- Adoption evidence: adoption-evidence.md
- Compatibility dashboard: compatibility-dashboard.md
- Platform support: platform-support.md
- Consumer profiles: consumer-profiles.md
- API and contracts:
Expand Down
176 changes: 176 additions & 0 deletions scripts/generate_compatibility_dashboard.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
#!/usr/bin/env python3
"""Generate the compatibility dashboard from package and CI declarations."""

from __future__ import annotations

import argparse
import re
import sys
from pathlib import Path

MATRIX_PATTERN = re.compile(r"(?m)^[ \t]+(?P<key>[a-z-]+):\s*\[(?P<values>[^\]]*)\]")
QUOTED_VALUE = re.compile(r"['\"]([^'\"]+)['\"]")


def _matrix_values(path: Path, key: str) -> list[str]:
"""Read a simple quoted matrix array from a workflow file."""
lines = path.read_text(encoding="utf-8").splitlines()
inline_pattern = re.compile(rf"^(?P<indent>\s+){re.escape(key)}:\s*\[(?P<values>[^\]]*)\]")
block_pattern = re.compile(rf"^(?P<indent>\s+){re.escape(key)}:\s*$")
for index, line in enumerate(lines):
inline_match = inline_pattern.match(line)
if inline_match is not None:
return QUOTED_VALUE.findall(inline_match.group("values"))
block_match = block_pattern.match(line)
if block_match is None:
continue
base_indent = len(block_match.group("indent"))
values: list[str] = []
for child in lines[index + 1 :]:
if not child.strip():
continue
indent = len(child) - len(child.lstrip())
value_match = re.match(r"^\s+-\s*(?:['\"])?([^'\"\s]+)(?:['\"])?\s*$", child)
if indent <= base_indent:
break
if value_match is not None:
values.append(value_match.group(1))
if values:
return values

text = "\n".join(lines)
for match in MATRIX_PATTERN.finditer(text):
if match.group("key") == key:
return QUOTED_VALUE.findall(match.group("values"))
raise ValueError(f"{path} has no matrix array for {key!r}")


def _dependency_window(dependencies: list[str], name: str) -> str:
for dependency in dependencies:
if dependency.lower().startswith(name.lower()):
return dependency
raise ValueError(f"pyproject.toml has no dependency window for {name!r}")


def _toml_array(text: str, key: str) -> list[str]:
"""Read the quoted array used by this repository's simple TOML metadata."""
match = re.search(rf"(?ms)^{re.escape(key)}\s*=\s*\[(?P<body>.*?)\]", text)
if match is None:
raise ValueError(f"pyproject.toml has no array for {key!r}")
return QUOTED_VALUE.findall(match.group("body"))


def generate_dashboard(root: Path) -> str:
"""Return deterministic Markdown generated from repository declarations."""
pyproject_text = (root / "pyproject.toml").read_text(encoding="utf-8")
dependencies = _toml_array(pyproject_text, "dependencies")
typer_dependencies = _toml_array(pyproject_text, "typer")
yaml_dependencies = _toml_array(pyproject_text, "yaml")

tests_workflow = root / ".github/workflows/tests.yml"
dependency_workflow = root / ".github/workflows/dependency-matrix.yml"
compatibility_workflow = root / ".github/workflows/compatibility.yml"
python_versions = _matrix_values(tests_workflow, "python-version")
native_platforms = _matrix_values(tests_workflow, "os")
click_versions = _matrix_values(dependency_workflow, "click-version")
pyyaml_versions = _matrix_values(dependency_workflow, "pyyaml-version")
typer_versions = _matrix_values(compatibility_workflow, "typer-version")

distribution_names = re.findall(
r"(?m)^\s+- name: (?P<name>(?:Debian|Fedora)[^\n]*)$",
tests_workflow.read_text(encoding="utf-8"),
)
platforms = [*native_platforms, *distribution_names, "WSL2"]
python_text = ", ".join(f"Python {version}" for version in python_versions)
click_text = ", ".join(version.removesuffix(".*") for version in click_versions)
yaml_text = ", ".join(version.removesuffix(".*") for version in pyyaml_versions)
typer_text = ", ".join(typer_versions)
platform_text = ", ".join(platforms)
click_window = _dependency_window(dependencies, "click")
yaml_window = _dependency_window(yaml_dependencies, "PyYAML")
typer_window = _dependency_window(typer_dependencies, "typer")

return f"""# Compatibility dashboard

[![Tests](https://img.shields.io/github/actions/workflow/status/basefoundry/base-cli/tests.yml?branch=main&label=tests)](https://github.com/basefoundry/base-cli/actions/workflows/tests.yml)
[![Dependency matrix](https://img.shields.io/github/actions/workflow/status/basefoundry/base-cli/dependency-matrix.yml?branch=main&label=dependencies)](https://github.com/basefoundry/base-cli/actions/workflows/dependency-matrix.yml)
[![Reference consumers](https://img.shields.io/github/actions/workflow/status/basefoundry/base-cli/compatibility.yml?branch=main&label=consumers)](https://github.com/basefoundry/base-cli/actions/workflows/compatibility.yml)

This page is generated from the package metadata and the matrices in
`.github/workflows/tests.yml`, `dependency-matrix.yml`, and
`compatibility.yml`. The badges above reflect the latest completed `main`
workflow runs; a green badge means the declared matrix passed, while a gray or
red badge means that evidence is pending or needs investigation.

## Current support contract

| Surface | Declared support | Completed CI coverage | Interpretation |
| --- | --- | --- | --- |
| Python | `>=3.10,<4` | {python_text} | Supported and tested |
| Click | `{click_window}` | {click_text} | Supported and tested on Python 3.10 and 3.14 |
| PyYAML extra | `{yaml_window}` | {yaml_text} | Supported when `base-cli[yaml]` is installed |
| Typer extra | `{typer_window}` | {typer_text} | Supported through `attach_typer()` |
| Platforms | Pure-Python core | {platform_text} | Supported tiers documented below |

## What the labels mean

- **Supported and tested** means the combination is declared by package
metadata and exercised by a named CI matrix.
- **Supported, not exhaustive** means the package contract applies, but CI
samples representative versions or platforms rather than every patch level.
- **Untested** means a combination may install but is not a support claim.
- **Unsupported** means package metadata or an explicit policy excludes it;
reports from that combination are welcome but are not release blockers.

The platform boundary is detailed in
[`platform-support.md`](platform-support.md), and dependency-window changes
must follow [`dependency-support.md`](dependency-support.md). Reference
consumer results are compatibility evidence, not customer adoption claims; see
[`adoption-evidence.md`](adoption-evidence.md).

## Updating the dashboard

Do not edit this page's matrix by hand. When a package window or CI matrix
changes, run:

```bash
python scripts/generate_compatibility_dashboard.py
python scripts/generate_compatibility_dashboard.py --check
```

The CI quality and documentation jobs fail when the generated page is stale.
Update the dashboard and the relevant support-policy/release-note entry in the
same pull request. The generator intentionally records declarations and live
workflow links, while GitHub Actions remains the source of truth for the most
recent run outcome.
"""


def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--output", type=Path, default=Path("docs/compatibility-dashboard.md"))
parser.add_argument("--check", action="store_true", help="fail when output is stale")
args = parser.parse_args()
root = Path(__file__).resolve().parents[1]
generated = generate_dashboard(root)
if args.check:
try:
current = args.output.read_text(encoding="utf-8")
except OSError as exc:
print(f"compatibility dashboard validation failed: {exc}", file=sys.stderr)
raise SystemExit(1) from exc
if current != generated:
print(
f"compatibility dashboard validation failed: {args.output} is stale; run the generator",
file=sys.stderr,
)
raise SystemExit(1)
print(f"Validated generated compatibility dashboard: {args.output}")
return
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(generated, encoding="utf-8")
print(f"Generated compatibility dashboard: {args.output}")


if __name__ == "__main__":
main()
1 change: 1 addition & 0 deletions scripts/validate_docs.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
"api-reference.md",
"api-stability.md",
"cache-ownership-and-layout.md",
"compatibility-dashboard.md",
"consumer-profiles.md",
"coverage-policy.md",
"dependency-support.md",
Expand Down
36 changes: 36 additions & 0 deletions tests/test_generate_compatibility_dashboard.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
from __future__ import annotations

import sys
import unittest
from pathlib import Path

sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from scripts import generate_compatibility_dashboard


class CompatibilityDashboardTests(unittest.TestCase):
def test_dashboard_contains_declared_and_tested_surfaces(self) -> None:
root = Path(__file__).resolve().parents[1]
dashboard = generate_compatibility_dashboard.generate_dashboard(root)
for expected in (
"`>=3.10,<4`",
"`click>=8.1,<8.5`",
"`PyYAML>=6.0,<7`",
"`typer>=0.12,<0.28`",
"8.1, 8.2, 8.3, 8.4",
"0.25.1, 0.26.0, 0.27.1",
"Debian 12",
"Fedora latest",
"WSL2",
):
self.assertIn(expected, dashboard)

def test_checked_in_dashboard_is_current(self) -> None:
root = Path(__file__).resolve().parents[1]
expected = generate_compatibility_dashboard.generate_dashboard(root)
actual = (root / "docs/compatibility-dashboard.md").read_text(encoding="utf-8")
self.assertEqual(actual, expected)


if __name__ == "__main__":
unittest.main()
2 changes: 2 additions & 0 deletions tests/validate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ required_files=(
docs/security-review.md
docs/adopter-readiness.md
docs/adoption-evidence.md
docs/compatibility-dashboard.md
MANIFEST.in
scripts/validate_package_artifact.py
scripts/validate_installed_package.py
Expand All @@ -41,6 +42,7 @@ required_files=(
scripts/generate_release_metadata.py
scripts/validate_release_metadata.py
scripts/record_compatibility_evidence.py
scripts/generate_compatibility_dashboard.py
scripts/benchmark_runtime.py
tests/conftest.py
compatibility/README.md
Expand Down
Loading