diff --git a/Agent Skills/open-source-audit/SKILL.md b/Agent Skills/open-source-audit/SKILL.md index 756ca01..340c097 100644 --- a/Agent Skills/open-source-audit/SKILL.md +++ b/Agent Skills/open-source-audit/SKILL.md @@ -379,7 +379,7 @@ publication. Removing the file in a new commit is not sufficient. best proxy for "publishable". - [ ] Version tags follow SemVer if the project has releases: ECMWF production tags use the clean `x.y.z` form (no `v` prefix) - [Codex: Guidelines/External-Contributions.md]. + [Codex: Languages/Versioning.md; Guidelines/External-Contributions.md]. - [ ] A `CHANGELOG.md` is not required — but if one exists, check it is sane: entries match actual tags, no placeholder sections, no internal references. diff --git a/Guidelines/Branching.md b/Guidelines/Branching.md index cf6de6e..822650b 100644 --- a/Guidelines/Branching.md +++ b/Guidelines/Branching.md @@ -1,6 +1,6 @@ # Branching Patterns -This document describes the branching patterns used across ECMWF repositories. [ADR-001: Git Branching Model](../ADR/ADR-001-Git-Branching-Model.md) records the *decision and rationale*; this document is the *normative operational procedure*. Version tags follow Semantic Versioning in the clean `x.y.z` form (no `v` prefix) — see the canonical tag and prerelease (`x.y.z-upstream.N`) conventions in [External Contributions](./External-Contributions.md). +This document describes the branching patterns used across ECMWF repositories. [ADR-001: Git Branching Model](../ADR/ADR-001-Git-Branching-Model.md) records the *decision and rationale*; this document is the *normative operational procedure*. Release version numbers and tags follow the [Versioning](../Languages/Versioning.md) policy (Semantic Versioning, clean `x.y.z`, no `v` prefix; prerelease `x.y.z-upstream.N`) — see also the tagging rules in [External Contributions](./External-Contributions.md). ## Contents diff --git a/Guidelines/External-Contributions.md b/Guidelines/External-Contributions.md index 22c51d6..e583e41 100644 --- a/Guidelines/External-Contributions.md +++ b/Guidelines/External-Contributions.md @@ -89,6 +89,8 @@ Contractors may work in one of two ways: ### Tagging Rules +These tags follow ECMWF's [Versioning](../Languages/Versioning.md) policy (Semantic Versioning, clean `x.y.z`, no `v` prefix); the rules below cover the contractor prerelease and production tags specifically. + Contractors using this model may create prerelease tags for internal testing and staging: ``` diff --git a/Languages/Python-Wheels.md b/Languages/Python-Wheels.md index 3a69efc..0e284df 100644 --- a/Languages/Python-Wheels.md +++ b/Languages/Python-Wheels.md @@ -41,21 +41,21 @@ We could thus switch to exact pinning, ie, say that the mirlib wheel in question However, that introduces a problem once we realize that our compiled stack consists of multiple libraries. Say we release multio with version A, depending on mir B and eckit C, with exact pins. Later on, a new version C+1 of eckit appears, and then we'd like to release multio A+1. -However, if there is no mir of version B+1, we are stuck -- the mirlib-B.wheel already exists, and it depends on eckit-A.wheel, not eckit-A+1.wheel. +However, if there is no mir of version B+1, we are stuck -- the `mirlib-B.wheel` already exists, and it depends on `eckitlib-C.wheel`, not `eckitlib-C+1.wheel`. -To get around that limitation, we switch to 4-dimensional versioning, x.y.z.C, where C is a monotonic counter shared across all packages. -For the example above, we would first release eckit.A.1, mir.B.1 and multio.C.1. -And in the second run, we would release eckit.A+1.2, mir.B.2 and multio.C2, all with exact pins. -The mir.B.1 and mir.B.2 are seemingly the same, from the point of view of the compiled mir code -- but they actually differ in which version of eckit they were build against, and thus justify being separate wheels. +To get around that limitation, we add a fourth component to the wheel version, `x.y.z.N`, where `N` is a monotonic counter shared across all packages (this is a wheel build identifier and does not change the software's `x.y.z` release version — see the [Versioning](./Versioning.md) policy). +For the example above, we would first release `eckit.C.1`, `mir.B.1` and `multio.A.1` (all with build counter `1`). +And in the second run, we would release `eckit.C+1.2`, `mir.B.2` and `multio.A.2` (build counter `2`), all with exact pins. +The `mir.B.1` and `mir.B.2` are seemingly the same, from the point of view of the compiled mir code -- but they actually differ in which version of eckit they were built against, and thus justify being separate wheels. This has an additional benefit of allowing simple check of ABI compatibility of a given python environment. List all installed wheels and check that their fourth versioning number is _exactly_ equal. If not, you are likely to experience "Symbol not found" or worse. The reason for this possibly happening is that pip does not necessarily guarantee to leave your environment in a correct state. -Say you first `pip install multio`, and pip notices that there is multio.C.1, which in turn brings eckit.A.1. +Say you first `pip install multio`, and pip notices that there is `multio.A.1`, which in turn brings `eckit.C.1`. And then later, you `pip install gribjump` -- which is not in any relationship to multio, but depends on eckit. -And say that the most recent gribjump wheel has been released as gribjump.D.2, with eckit.A+1.2 as a dependency. -Pip dully updates eckit, while telling you "oh and btw your environment is broken, multio wheel has unsatisfied dependency". +And say that the most recent gribjump wheel has been released as `gribjump.D.2`, with `eckit.C+1.2` as a dependency. +Pip duly updates eckit, while telling you "oh and btw your environment is broken, multio wheel has unsatisfied dependency". Other package managers like `uv` would refuse to install multio in the first place, but this behaviour can't be relied upon. diff --git a/Languages/README.md b/Languages/README.md index 63542f1..1e3a222 100644 --- a/Languages/README.md +++ b/Languages/README.md @@ -6,4 +6,4 @@ Language-specific guidelines and tooling for ECMWF software projects. - [C++](./C++/README.md) — C++ tooling *(work in progress: currently only the `.git-blame-ignore-revs` convention; coding standards and clang-format/clang-tidy configuration are still to be added)* - [Python Wheels](./Python-Wheels.md) — building and using Python wheels with compiled libraries -- [Versioning](./Versioning.md) — **placeholder / to be written**: versioning guidance for multi-language repositories (see `TODO.md`) +- [Versioning](./Versioning.md) — ECMWF's Semantic Versioning policy: tag format (`x.y.z`, no `v` prefix), MAJOR/MINOR/PATCH bump rules, production vs prerelease tags, and single-version guidance for multi-language / multi-package repositories diff --git a/Languages/Versioning.md b/Languages/Versioning.md index 6561b87..6b4d1e0 100644 --- a/Languages/Versioning.md +++ b/Languages/Versioning.md @@ -1,3 +1,120 @@ -# TODO +# Versioning -How do we manage multi-language repositories with a single version number? \ No newline at end of file +ECMWF software is versioned with **Semantic Versioning (SemVer)**. This document +is the authoritative policy for how to version an ECMWF repository — the tag +format, when to bump each component, prerelease and production tags, and how to +version a repository that contains multiple languages or packages with a single +version number. + +It consolidates the version-tag rules used in +[Branching](../Guidelines/Branching.md) and +[External Contributions](../Guidelines/External-Contributions.md); those +documents govern *how* releases are branched and delivered, while this document +defines *how the version number itself is chosen and applied*. + +## Semantic Versioning + +Versions have the form: + +``` +MAJOR.MINOR.PATCH e.g. 2.4.1 +``` + +Given a public interface (a library API, a CLI, a data/format contract, a wire +protocol), increment the: + +- **MAJOR** version for **incompatible / breaking changes** — anything that + requires downstream users to change their code, commands, configuration, or + data handling. +- **MINOR** version for **backwards-compatible new functionality** (and for + marking features as deprecated). +- **PATCH** version for **backwards-compatible bug fixes** only. + +See [semver.org](https://semver.org) for the full specification. + +### No `v` prefix + +ECMWF version tags use the **clean `x.y.z` form with no `v` prefix** (`2.4.1`, +not `v2.4.1`). This is checked by the `open-source-audit` skill. Note that this +applies to *ECMWF release tags*; third-party GitHub Actions and other external +artefacts keep their own tag conventions (e.g. `actions/checkout@v4`). + +### Initial development (`0.y.z`) + +A project below `1.0.0` is in initial development: **anything may change at any +time** and the public interface is not considered stable. Reach `1.0.0` when the +interface is stable and the software is used in production. New public repositories +usually start at a low [Project Maturity](../Project%20Maturity/README.md) level, +which is a better signal of stability for users than the version number alone. + +### Deprecation and compatibility + +- Prefer to **deprecate before removing**: introduce the replacement, mark the + old interface deprecated (a MINOR release), keep it working for at least one + MINOR cycle, then remove it in a MAJOR release. +- Document breaking changes prominently (release notes / `CHANGELOG`), and a + `CHANGELOG` is recommended so users can see what changed between versions. + +## Production and prerelease tags + +- **Production tags** are clean `x.y.z` tags on `main`/`master`. Only such tags + represent production-ready software. +- **Prerelease tags** use the form `x.y.z-upstream.N` (`N` = a sequential + prerelease number), for contractor-side testing, staging, and CI. They are + **non-production**, must not appear on `main`/`master`, and must never be + deployed beyond development or test environments. + +The full rules, and the delivery/acceptance workflow they belong to, are in +[External Contributions → Tagging Rules](../Guidelines/External-Contributions.md). +The mechanics of creating and pushing release tags are in +[Branching](../Guidelines/Branching.md). + +## Single source of truth + +A repository has **one** version number, and the **git tag is the single source +of truth**. Do not maintain the version in several places that can drift. +Prefer to **derive** the package/library version from the tag automatically +(e.g. `setuptools_scm` for Python, or CMake reading `git describe`) so there is +nothing to keep in sync. If an in-repo version field is unavoidable (for example +`project.version` in `pyproject.toml`, a CMake `project(... VERSION ...)`, or a +top-level `VERSION` file), keep **exactly one** such field and treat it as a +mirror that must always match the tag. A generated version file (e.g. +`_version.py`) is a build artefact and must not be committed (see +[Repository Structure](../Repository%20Structure/README.md)). + +## Multi-language and multi-package repositories + +A repository that contains more than one language or produces more than one +artefact (for example a C/C++ library with Python bindings, or a package plus +its CLI) is versioned as **a single unit under one version number**: + +- **One version for the whole repository.** All components and artefacts built + from a commit share the same `x.y.z`, are tagged once, and are released + together. Do not version the Python bindings independently of the C++ library + they wrap within the same repository. +- **Bump for the repository as a whole.** A breaking change in *any* public + component is a MAJOR bump for the repository; new backwards-compatible + functionality in any component is a MINOR bump; and so on. This keeps a single, + unambiguous compatibility signal for downstream users. +- **If components genuinely need to evolve on independent version lines**, that + is a signal they should live in **separate repositories**, each with its own + single version, rather than being force-fitted into one. + +### Compiled Python wheels + +Where a repository publishes compiled Python wheels split across interdependent +packages (e.g. a library wheel and its Python interface wheel), a +**four-component build identifier `x.y.z.N`** is used for the *wheels*, where `N` +is a monotonic counter shared across the packages so that ABI-compatible builds +can be pinned together. This is a **packaging/build identifier, not a SemVer +version**: the software's release version — and the authoritative git tag — +remains `x.y.z`. The mechanism, and its current limitations, are described in +[Python Wheels](./Python-Wheels.md). + +## Summary + +- Use SemVer `MAJOR.MINOR.PATCH` (`x.y.z`), **no `v` prefix**. +- MAJOR = breaking, MINOR = backwards-compatible feature, PATCH = backwards-compatible fix; `0.y.z` = unstable. +- Production tags are clean `x.y.z` on `main`/`master`; prereleases are `x.y.z-upstream.N` (non-production). +- One version per repository; the git tag is the single source of truth; all languages/artefacts release together. +- Compiled Python wheels may use a `x.y.z.N` build identifier for pinning (see [Python Wheels](./Python-Wheels.md)). diff --git a/Software Management Plan/README.md b/Software Management Plan/README.md index 13845d3..396398f 100644 --- a/Software Management Plan/README.md +++ b/Software Management Plan/README.md @@ -115,7 +115,7 @@ Include: - Branching model - Merge criteria, e.g. pull requests with mandatory X reviews, all CI/CD test pass - **Release strategy** - - Versioning system, e.g. *"Semantic versioning (MAJOR.MINOR.PATCH)"* + - Versioning system — ECMWF uses Semantic Versioning (MAJOR.MINOR.PATCH); see the [Versioning](../Languages/Versioning.md) policy - **Maintenance** - How will future developments, and user requests, be managed? - **Maturity timeline** diff --git a/TODO.md b/TODO.md index 898856f..60efeaf 100644 --- a/TODO.md +++ b/TODO.md @@ -14,7 +14,6 @@ work-in-progress. ## High Priority - [ ] **Guidelines/Testing.md** — Still only covers test-data hosting. Needs unit, integration, and regression testing standards (the README now marks it as draft; other docs' "tests are mandatory" wording depends on this being written). -- [ ] **Languages/Versioning.md** — Still a single-line placeholder. Needs a versioning policy for multi-language repositories. ## Medium Priority