Skip to content

fix(ci): renormalize docker.yml to LF — and fail CI on CRLF blobs #141

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MrChengLen merged 1 commit into from
Sep 26, 2026
+185 −117
Merged

fix(ci): renormalize docker.yml to LF — and fail CI on CRLF blobs #141

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
234 changes: 117 additions & 117 deletions .github/workflows/docker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,117 +1,117 @@
name: Docker
on:
push:
branches: ["main"]
tags: ["v*"]
# Manual rebuild. Without this there is no way to produce an image, or to
# reach `notify-ops` and the deploy that follows it, other than pushing to
# main — and main is protected, so that means opening a PR.
#
# That gap cost three weeks in September 2026: the credential `notify-ops`
# uses had expired, so it failed with HTTP 401 on every run and nothing
# reached production. Once it was replaced there was no way to prove the fix,
# or to redeploy, without inventing a code change. A dispatch here rebuilds
# from the current main and runs the whole chain, which is also what you want
# during an incident.
workflow_dispatch:
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
build-and-push:
runs-on: ubuntu-latest
strategy:
# Build slim + office in parallel; fail-fast off because the two
# variants are independent — a Pango regression in slim shouldn't
# mask an upstream LibreOffice break in office (or vice versa).
fail-fast: false
matrix:
include:
- target: base
suffix: ""
description: "Slim image (mammoth+WeasyPrint docx→pdf path)"
- target: office
suffix: "-office"
description: "Office image (adds LibreOffice for high-fidelity docx→pdf)"
permissions:
contents: read
packages: write
# NEU-B.4: cosign keyless signing uses the GitHub Actions OIDC
# token to prove identity to Sigstore's Fulcio CA — no long-lived
# signing key to manage or rotate. Verifiable later with
# cosign verify ghcr.io/<owner>/<repo>@sha256:<digest> \
# --certificate-identity-regexp "^https://github\\.com/<owner>/<repo>/" \
# --certificate-oidc-issuer https://token.actions.githubusercontent.com
id-token: write
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
# The default ``docker`` driver on GHA runners doesn't support
# ``cache-to: type=gha`` (GitHub Actions cache backend). Switching
# to buildx's ``docker-container`` driver enables the cache export
# so the matrix matrix-base / matrix-office leg can reuse the
# base-stage layers it already pulled.
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
- name: Log in to GitHub Container Registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
# The office variant gets the same tag set as slim but with a
# ``-office`` suffix so a consumer can pick either by tag —
# filemorph:1.1.0 vs filemorph:1.1.0-office. The slim image
# keeps ``:latest`` (default for naive pulls); the office image
# gets ``:office`` for the matching shorthand.
tags: |
type=semver,pattern={{version}},suffix=${{ matrix.suffix }}
type=semver,pattern={{major}}.{{minor}},suffix=${{ matrix.suffix }}
type=raw,value=${{ matrix.target == 'base' && 'latest' || 'office' }}
type=sha,format=short,prefix=sha-,suffix=${{ matrix.suffix }}
- name: Build and push Docker image (${{ matrix.target }})
id: build
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
with:
context: .
target: ${{ matrix.target }}
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
# GHA-cache lets the office stage reuse every layer the base
# stage already produced — without it the office build redoes
# ffmpeg, ghostscript, pip install. Scope is per-target so the
# two matrix legs don't trample each other's cache.
cache-from: type=gha,scope=${{ matrix.target }}
cache-to: type=gha,scope=${{ matrix.target }},mode=max
- name: Install cosign
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
with:
cosign-release: "v2.4.1"
- name: Sign published image (keyless / Sigstore)
env:
COSIGN_EXPERIMENTAL: "true"
DIGEST: ${{ steps.build.outputs.digest }}
TAGS: ${{ steps.meta.outputs.tags }}
# Sign every tag that got pushed at the SAME digest so a downstream
# `cosign verify ghcr.io/.../filemorph:<tag>` works regardless of
# which alias the consumer pulls.
run: |
set -euo pipefail
for tag in $TAGS; do
cosign sign --yes "${tag}@${DIGEST}"
done
name: Docker

on:
push:
branches: ["main"]
tags: ["v*"]
# Manual rebuild. Without this there is no way to produce an image, or to
# reach `notify-ops` and the deploy that follows it, other than pushing to
# main — and main is protected, so that means opening a PR.
#
# That gap cost three weeks in September 2026: the credential `notify-ops`
# uses had expired, so it failed with HTTP 401 on every run and nothing
# reached production. Once it was replaced there was no way to prove the fix,
# or to redeploy, without inventing a code change. A dispatch here rebuilds
# from the current main and runs the whole chain, which is also what you want
# during an incident.
workflow_dispatch:

env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}

jobs:
build-and-push:
runs-on: ubuntu-latest
strategy:
# Build slim + office in parallel; fail-fast off because the two
# variants are independent — a Pango regression in slim shouldn't
# mask an upstream LibreOffice break in office (or vice versa).
fail-fast: false
matrix:
include:
- target: base
suffix: ""
description: "Slim image (mammoth+WeasyPrint docx→pdf path)"
- target: office
suffix: "-office"
description: "Office image (adds LibreOffice for high-fidelity docx→pdf)"
permissions:
contents: read
packages: write
# NEU-B.4: cosign keyless signing uses the GitHub Actions OIDC
# token to prove identity to Sigstore's Fulcio CA — no long-lived
# signing key to manage or rotate. Verifiable later with
# cosign verify ghcr.io/<owner>/<repo>@sha256:<digest> \
# --certificate-identity-regexp "^https://github\\.com/<owner>/<repo>/" \
# --certificate-oidc-issuer https://token.actions.githubusercontent.com
id-token: write

steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

# The default ``docker`` driver on GHA runners doesn't support
# ``cache-to: type=gha`` (GitHub Actions cache backend). Switching
# to buildx's ``docker-container`` driver enables the cache export
# so the matrix matrix-base / matrix-office leg can reuse the
# base-stage layers it already pulled.
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0

- name: Log in to GitHub Container Registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Extract metadata
id: meta
uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
# The office variant gets the same tag set as slim but with a
# ``-office`` suffix so a consumer can pick either by tag —
# filemorph:1.1.0 vs filemorph:1.1.0-office. The slim image
# keeps ``:latest`` (default for naive pulls); the office image
# gets ``:office`` for the matching shorthand.
tags: |
type=semver,pattern={{version}},suffix=${{ matrix.suffix }}
type=semver,pattern={{major}}.{{minor}},suffix=${{ matrix.suffix }}
type=raw,value=${{ matrix.target == 'base' && 'latest' || 'office' }}
type=sha,format=short,prefix=sha-,suffix=${{ matrix.suffix }}

- name: Build and push Docker image (${{ matrix.target }})
id: build
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
with:
context: .
target: ${{ matrix.target }}
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
# GHA-cache lets the office stage reuse every layer the base
# stage already produced — without it the office build redoes
# ffmpeg, ghostscript, pip install. Scope is per-target so the
# two matrix legs don't trample each other's cache.
cache-from: type=gha,scope=${{ matrix.target }}
cache-to: type=gha,scope=${{ matrix.target }},mode=max

- name: Install cosign
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
with:
cosign-release: "v2.4.1"

- name: Sign published image (keyless / Sigstore)
env:
COSIGN_EXPERIMENTAL: "true"
DIGEST: ${{ steps.build.outputs.digest }}
TAGS: ${{ steps.meta.outputs.tags }}
# Sign every tag that got pushed at the SAME digest so a downstream
# `cosign verify ghcr.io/.../filemorph:<tag>` works regardless of
# which alias the consumer pulls.
run: |
set -euo pipefail
for tag in $TAGS; do
cosign sign --yes "${tag}@${DIGEST}"
done
16 changes: 16 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,22 @@ Versions follow [Semantic Versioning](https://semver.org/).

## [Unreleased]

### Fixed — `docker.yml` stored with CRLF line endings; CI now rejects CRLF files

`.github/workflows/docker.yml` had been stored with Windows line endings (CRLF)
since the manual-rebuild change (`f72dedc`), although `.gitattributes` pins YAML
to LF. GitHub runs the workflow either way, so nothing broke, but the next
ordinary edit would have converted all 117 lines and shown up as a whole-file
diff that hides the lines actually changed. The file is back to LF; its content
is unchanged (`git diff -w` is empty).

`.gitattributes` only normalises line endings when git itself stages a file. A
commit made through the GitHub API stores exactly the bytes it is sent, and a
file read from a Windows checkout can carry CRLF, which is how
`requirements.lock` once turned into a 3725-line diff. `tests/test_line_endings.py`
now fails CI when any tracked text file is stored with CRLF, so the next one is
caught in its own pull request rather than by whoever edits the file after it.

### Fixed — long upload names lost their file extension on download

`safe_download_name()` cut the finished download name to 200 characters, so an
Expand Down
52 changes: 52 additions & 0 deletions tests/test_line_endings.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Tracked text files are stored with LF line endings.

``.gitattributes`` normalises text files to LF, but only when git itself stages
them. A commit made through the GitHub API (``createCommitOnBranch``) stores
exactly the bytes it is sent, CRLF included: ``requirements.lock`` landed that
way in 7caa75e (PR #81, repaired in PR #85), ``.github/workflows/docker.yml`` in
f72dedc. Nothing breaks, but that commit is already a whole-file diff, and for a
file pinned to ``eol=lf`` the next ordinary edit renormalises it into another.

This reads the index (the ``i/`` column of ``git ls-files --eol``), not the
working tree: a Windows checkout may hold CRLF on disk, and what matters is the
blob that gets committed.
"""

from __future__ import annotations

import subprocess
from pathlib import Path

import pytest

_REPO_ROOT = Path(__file__).resolve().parent.parent


def test_no_tracked_text_file_is_stored_with_crlf() -> None:
if not (_REPO_ROOT / ".git").exists():
pytest.skip("not a git checkout (e.g. an unpacked release tarball)")
listing = subprocess.run(
["git", "ls-files", "--eol", "-z"],
cwd=_REPO_ROOT,
capture_output=True,
text=True,
encoding="utf-8",
check=True,
).stdout
offenders = []
for entry in filter(None, listing.split("\0")):
# "i/<eol> w/<eol> attr/<attributes>\t<path>"; -z leaves the path unquoted.
info, path = entry.split("\t", 1)
index_eol, _worktree_eol, attr = info.split(maxsplit=2)
# `-text` (or `binary`) in .gitattributes opts a file out of
# normalisation, e.g. an .eml fixture that has to keep CRLF.
if index_eol in ("i/crlf", "i/mixed") and "-text" not in attr:
offenders.append(f"{index_eol:8} {path}")
assert not offenders, (
"stored with CRLF line endings, although .gitattributes normalises them to LF:\n "
+ "\n ".join(offenders)
+ "\nFix: `git add --renormalize <path>`. For a commit through the API, convert "
"\\r\\n to \\n before base64-encoding. A file that has to keep CRLF needs "
"`-text` in .gitattributes."
)
Loading