Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,12 @@ takes priority over features, throughput, and presentation.
[docs/release-plan.md](docs/release-plan.md) for the proposed Windows beta.
The release plan describes future work; verify the checkout before claiming
an installer, signing pipeline, or release gate exists.
- Packaging, signing, the bill of materials and the tag-triggered candidate
workflow are in [docs/build-windows.md](docs/build-windows.md); what an
update checks before it runs anything is in
[docs/updates.md](docs/updates.md). Hardware-key signing and clean-machine
qualification are still gates, so a build being implemented is not a
release being possible.
- Inspect the working tree before edits. Preserve unrelated user changes.
Keep this guide concise and link to detailed documentation rather than
copying it wholesale.
Expand Down Expand Up @@ -56,6 +62,7 @@ All module paths below are relative to `src/offloader/`.
| Timeline import | `timeline.py`: optional OpenTimelineIO integration, media resolution, and ambiguity handling |
| Desktop | `gui/main_window.py`, `gui/worker.py`, `gui/queue_view.py`, mode/editor widgets, and `gui/drives.py` |
| Persistent state | `config.py`, `presets.py`, `history.py`; `volumes.py` discovers storage and `naming.py` handles naming |
| Installation and updates | `installation.py` and `installation_lock.py` for transactional maintenance and the shared installed-instance lock; `update.py` finds, verifies and hands over a release, wrapped for the app by `gui/updates.py`. Neither may force-close a running transfer; see [docs/updates.md](docs/updates.md) |

Python 3.10+ is supported. Core dependencies are xxhash and ReportLab; PySide6
is the GUI extra. Timeline dependencies are separate extras. ffmpeg/ffprobe
Expand Down Expand Up @@ -95,8 +102,10 @@ and verification. Do not make a missing media tool block a valid transfer.
writes and unreadable-config fallback. Tests must use temporary state rather
than modifying the user's presets or history.
- Do not represent known gaps as implemented protections: concurrent instances
are not coordinated, size-only skip is not a checksum check, and directory
structure hashes are not yet rechecked. Consult the current safety document
are not coordinated, size-only skip is not a checksum check, the installer
update's last hash-to-launch window is not closed, and the SBOM does not
cover the bundled CPython, OpenSSL, libffi, VC++/UCRT and bootloader
binaries. Consult the current safety document
for the full list before changing claims.

## Development and validation
Expand Down
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ project uses [semantic versioning][semver].

### Fixed

- **Documentation claims were corrected to match the code:** the updater's
re-hash and its remaining window, verify's eviction warning, what an update
closes, the installer's `/D=` handling, and the SBOM's coverage.
- **`offloader update` reported a feed it could not read as "up to date".**
Being offline, an HTTP error or rate limit, a redirect off the allowlist and
malformed JSON all printed "the newest release available" and exited 0, so
Expand Down
58 changes: 54 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -259,13 +259,61 @@ the same destination lists that file with a hash that still matches; a hashed
entry is never superseded. See
[`docs/data-safety.md`](docs/data-safety.md#a-manifest-lists-what-did-not-arrive).
`--allow-cache` skips the page-cache eviction: faster, and may verify memory
rather than the device.
rather than the device. Without it, a failed eviction is printed as a warning
rather than ignored. On macOS eviction is not available, so the warning is
expected there and the result may reflect memory rather than the device.

For an ASC MHL history it also recomputes the directory content and structure
hashes, which is the only check that catches a rename or a moved file — every
file involved still hashes exactly as recorded. See
[`docs/ascmhl.md`](docs/ascmhl.md#directory-hashes).

### `update`

```sh
offloader update # is there a newer release?
offloader update --install # download it, verify it, run the installer
```

GitHub Releases is the feed, so there is no manifest server and no second
place a version number is recorded. Before anything runs, the download has to
be served from an allowlisted host *after* redirects, and carry a valid
Authenticode signature with this project's certificate thumbprint, its
publisher name, and an embedded `FileVersion` matching the release. That last
check is what stops an older but still validly signed installer being
re-served under a newer asset name. The SHA-256 taken while streaming is
compared with a fresh hash of the file on disk before the signature is read,
after it is read, and immediately before launch; a mismatch refuses and
nothing runs. One window remains: a process able to write the download
directory between that last hash and the launch could still swap the file, so
point `--dir` only at a private location (see the known limits in
[`docs/updates.md`](docs/updates.md)).

The feed is the releases collection (`/releases?per_page=30`), not
`/releases/latest`, which GitHub documents as excluding prereleases. Drafts are
ignored, the channel is taken from the version in the tag rather than from
GitHub's prerelease flag, and the greatest eligible version wins rather than
whichever is listed first. So an installed beta such as `0.1.0b1` is offered the
next beta, release candidate or final release, while a stable install is
offered stable releases only.

A feed that could not be read is a failure, not "the newest release
available": the command line prints an error and exits with status 2 (0 is up
to date, 1 is a newer release found), and the desktop app reports the check as
failed.

**It never closes a running transfer.** The installer refuses maintenance
while any Offloader program holds the installation, which is the guarantee
rather than a limitation, so updating means finishing or cancelling the job
first. The command line closes nothing and says so before the elevation prompt
appears. The desktop app declines the update while a job is running or paused,
and otherwise closes itself, after asking, once the installer has started.

The desktop app does the same thing from **Help, Check for updates now**, and
checks once on launch unless that is turned off in Options. Full detail,
including what is deliberately not copied from the reference implementation,
is in [`docs/updates.md`](docs/updates.md).

### Verification modes

| Mode | What it does | Catches |
Expand Down Expand Up @@ -544,7 +592,8 @@ The short version:
was already there and never leaves a plausible-looking filename.
- `--verify full` evicts each file from the page cache before reading it back,
because a read straight after a write is otherwise served from memory and
proves nothing about the device.
proves nothing about the device. If eviction fails, which is always the case
on macOS, that is recorded as a warning.
- Empty files, and verifications that may have been served from cache, are
reported as warnings rather than folded into a "Verified" verdict.
- Reads that fail for a transient reason are retried at the failing chunk, so
Expand Down Expand Up @@ -585,7 +634,8 @@ general-purpose tool reports a filename, a size, and a placeholder icon.
| --- | --- |
| [`ROADMAP.md`](ROADMAP.md) | What is next, why, and what this will not become |
| [`docs/release-plan.md`](docs/release-plan.md) | Windows beta release sequence, packaging, signing, acceptance gates, and recovery |
| [`docs/build-windows.md`](docs/build-windows.md) | Build, sign, and check Windows desktop bundles and installers |
| [`docs/build-windows.md`](docs/build-windows.md) | Build, sign, and check Windows desktop bundles and installers; the bill of materials, and tagging a candidate |
| [`docs/updates.md`](docs/updates.md) | What an update checks before it runs anything, and why it never closes a running transfer |
| [`docs/data-safety.md`](docs/data-safety.md) | Threat model: what is guaranteed, what is not, and the bugs behind each guarantee |
| [`docs/report-layout.md`](docs/report-layout.md) | Every coordinate of the PDF, measured off the reference report |
| [`docs/performance.md`](docs/performance.md) | Why not robocopy, with benchmarks and the confounds that made the first run worthless |
Expand Down Expand Up @@ -630,7 +680,7 @@ ruff check src tests
pytest --cov=offloader --cov-report=term-missing
```

1130 tests at 83% line coverage. They cover formatting against the reference's
1152 tests at 83% line coverage. They cover formatting against the reference's
exact strings, checksum vectors and streaming equivalence, copy/verify
behaviour including simulated destination corruption, pause/resume/cancel
concurrency, retry discrimination, BRAW container parsing, ffprobe parsing,
Expand Down
17 changes: 15 additions & 2 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,8 +124,21 @@ would close it.

- **Email or SMS on completion.** ShotPut Pro has it; a DIT running a long
offload wants to leave the cart.
- **Windows installer and code signing**, so it can be handed to someone who
does not have Python.
- **A signed, qualified Windows release.** The packaging is implemented:
frozen bundle, NSIS installer, transactional maintenance, signing hooks, a
bill of materials for the Python dependency closure, an update client that
reads the releases collection, and a tag-triggered candidate workflow.
Most of what is left is not code. Signing needs the hardware token, the
acceptance matrix needs a clean machine, and Qt's LGPL/GPL terms against
Offloader's MIT licence need a redistribution decision, which the notices
file flags on every build. One part is: the bill of materials covers only the
Python distribution dependencies. The bundle also ships components with no
Python metadata, including the CPython runtime and OpenSSL and libffi DLLs,
the VC++ and UCRT runtimes and the PyInstaller bootloader;
`build/windows/sbom.py` currently reports only the CPython DLL and the
bootloader as uncovered. The release plan keeps a complete third-party
inventory as a gate. See
[`docs/release-plan.md`](docs/release-plan.md).
- **Per-job report templates and custom branding.**

## Not planned
Expand Down
30 changes: 21 additions & 9 deletions docs/build-windows.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,8 @@ coverage, and `SHA256SUMS.txt` for the final installer, ZIP, inventory and
the three bill-of-materials files.
A failed build leaves `.offloader-build-incomplete`; it must not be promoted.
Source changes during a build invalidate the candidate. These inventories are
provenance and tamper checks, not a complete third-party license inventory or SBOM.
provenance and tamper checks. The third-party licence inventory and the SBOM
are separate outputs of the same build, described below.

## Signing

Expand Down Expand Up @@ -146,8 +147,7 @@ PEP 440 forms are rejected rather than silently truncated.

The spec excludes optional timeline import, disables UPX, and includes the
project license and distribution metadata. ffmpeg and ffprobe remain external.
Missing media tools reduce metadata/thumbnails, not copy verification. A
release-ready third-party license inventory and SBOM remain separate work.
Missing media tools reduce metadata/thumbnails, not copy verification.

## Bill of materials and licences

Expand Down Expand Up @@ -195,17 +195,21 @@ actually moved.
### What these three files do not cover

The Python distribution dependencies, and only those. A frozen application also
ships components that have no packaging metadata for the closure to walk: the
CPython runtime DLL that bundle validation requires, and the PyInstaller
bootloader compiled into each of the three executables. All three outputs say
ships components that have no packaging metadata for the closure to walk,
including the CPython runtime DLL that bundle validation requires, the
PyInstaller bootloader compiled into each of the three executables, CPython's
OpenSSL DLLs and libffi, and the VC++ and UCRT runtime DLLs. All three outputs say
so in their own text — a scope line in the notices, a `offloader:scope`
property in the SBOM, a comment in the requirements file — because an inventory
read as complete while missing the interpreter it ships is worse than one that
states its boundary.

`sbom.uncovered_in_bundle()` reports which of those are actually present in a
built bundle, measured against the tree rather than asserted from a list, so
the gap shrinks as it is closed and cannot be closed by editing a constant.
`sbom.uncovered_in_bundle()` reports which of the entries in its `UNCOVERED`
list (currently the CPython DLL and the bootloader executables only) are
present in a built bundle. It is measured against the tree, but it cannot see
a component that list does not name, so the OpenSSL, libffi and VC++/UCRT DLLs
are not reported and the gap is larger than its output. Extend the list before
treating the inventory as complete.
The complete third-party inventory remains a gate in
[release-plan.md](release-plan.md).

Expand Down Expand Up @@ -300,6 +304,14 @@ asserts that a rerun against a published release issues no `gh release edit`.

## Check the artifact

Source validation on 2026-09-21 (Windows x64, Python 3.13.5): 855 tests passed
with 12 skips; lint passed over `src`, `tests`, `scripts` and `build/windows`.
The source archive was built and checked to contain the Windows builder. This
run covers the update client, the tag-triggered candidate workflow and the
bill of materials. **It is a source test run and nothing more:** no bundle was
frozen, nothing was signed, and no installer was executed, so it establishes
none of the gates below.

Installer implementation validation on 2026-09-10 (Windows x64, Python 3.12.10,
NSIS 3.12): 727 tests passed with 5 skips and 85% line coverage. Lint passed.
The unsigned installer and portable bundle built, and the frozen smoke checks
Expand Down
7 changes: 5 additions & 2 deletions docs/data-safety.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,8 +130,11 @@ checksum finds that.

It also reports files present on disk that the manifest does not list, and it
evicts each file before reading so a freshly written tree is read off the device,
and warns when the eviction failed, since the verdicts then say nothing about
the device. The unlisted-file scan starts at the root the manifest belongs to (the folder
and warns when the eviction failed (the CLI prints it as `warning:`, and it is
in the report's warnings), since the verdicts then say nothing about the device.
Eviction uses `posix_fadvise` on POSIX systems, which macOS does not provide, so
on macOS it always fails and the warning is expected; it can also fail
elsewhere. `--allow-cache` skips eviction. The unlisted-file scan starts at the root the manifest belongs to (the folder
holding `<name>_Reports`), not at the common folder of the files it lists, which
for a one-file manifest was the file itself and found nothing.

Expand Down
40 changes: 26 additions & 14 deletions docs/release-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,12 @@ they surface, and that decision has not been made here.

Those three files cover the **Python distribution dependencies** and say so in
their own text. They are not the complete third-party inventory: a frozen
application also ships the CPython runtime DLL and the PyInstaller bootloader
compiled into each executable, neither of which has packaging metadata for the
closure to walk. `sbom.uncovered_in_bundle` reports which of those are present
in a built bundle, so the gap is measured against the tree rather than
asserted.
application also ships the CPython runtime DLL, the PyInstaller bootloader
compiled into each executable, CPython's OpenSSL and libffi DLLs, and the
VC++ and UCRT runtimes, none of which has packaging metadata for the closure
to walk. `sbom.uncovered_in_bundle` reports only the CPython DLL and the
bootloader executables that are present in a built bundle; the rest are not
yet reported.

Hardware-key signing, clean-machine interactive installation and alternate
credential checks, the Qt redistribution decision the notices flag, the
Expand Down Expand Up @@ -121,13 +122,21 @@ credentials, update endpoints, or installation paths.
- **Timeline support:** include and test OpenTimelineIO and the currently
declared adapter in the desktop bundle if timeline import is advertised for
that bundle. Otherwise mark that capability source-only for the beta.
- **Updates:** `offloader update` finds and verifies a release and runs the
signed installer; the in-app check remains deferred. Refuse replacement
while the app or CLI has an active job; never force-kill a copy to install
an update.
- **Scope freeze:** defer new media features, cloud services, notifications,
auto-update, and a marketing website. Fix integrity and packaging blockers
discovered during qualification.
- **Updates:** `offloader update` and the desktop app both check for a
release, verify it, and run the signed installer. Refuse replacement while
the app or CLI has an active job; never force-kill a copy to install an
update. See [updates.md](updates.md). Discovery reads the releases
collection, ignores drafts and takes the channel from the tag's version, so
a beta can find the next beta. What it still needs is the qualification
below: an end-to-end update between two signed published releases.
- **Scope freeze:** defer new media features, cloud services, notifications
and a marketing website. Fix integrity and packaging blockers discovered
during qualification.

Updating is inside the freeze only as far as it goes today: the *check* is
automatic, and installing never is. Nothing is downloaded, replaced or
restarted without someone asking for it, and a silent background update
stays out of scope.

These are working defaults for implementation, not statements that the
packaging or safeguards already exist.
Expand Down Expand Up @@ -225,8 +234,11 @@ The update client is now implemented in `src/offloader/update.py`, wrapped for
the desktop app in `src/offloader/gui/updates.py`, and documented in
[updates.md](updates.md). It meets the conditions this section set: the
signature, publisher and embedded version are all verified before elevation,
and the `/D=` target is computed from the running executable rather than from
the uninstall registry key.
and the `/D=` target is computed from the running executable. The installer
honours an explicit `/D=`, but when it equals the compiled default location it
cannot tell that from no `/D=` and uses the `InstallLocation` registered under
HKLM (writable only by an administrator), so in the common case the registry
value still decides the target; see [updates.md](updates.md).

The app never replaces itself under an active transfer. A running or paused job
declines the update with a reason, the queue is rechecked immediately before
Expand Down
11 changes: 10 additions & 1 deletion docs/updates.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,12 @@ build that Windows then considers older than the one it replaced.
If either version is unreadable the answer is "not newer". String comparison
is what makes `1.0.10` look older than `1.0.9`.

Because the feed is the releases collection, this ordering is applied to
releases the updater can actually see: an installed `0.1.0b1` finds `0.1.0b2`
or `0.1.0rc1` once either is published, and a repository holding only betas
still answers. The channel comes from the version in the tag, not from GitHub's
`prerelease` flag, which the updater does not read.

## What is checked before anything runs

| Check | What it stops |
Expand Down Expand Up @@ -212,7 +218,10 @@ rather than enforced.

## What it deliberately does not do

**It does not close anything.** Offloader's installer refuses maintenance
**It never closes a running transfer.** The command line closes nothing. The
desktop app declines the update while a job is running or paused, and
otherwise closes itself, after asking, once the installer has started.
Offloader's installer refuses maintenance
while any Offloader program (the desktop app or a CLI process, busy or idle)
holds the installation's shared lock, and exits non-zero without changing the
installation; it never force-kills a running copy. That refusal is a promise
Expand Down
Loading