Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,18 @@ project uses [semantic versioning][semver].
the repository. The desktop app now uses the same mark as its window icon,
built in memory, so the shell entry, the window and the report header finally
agree on what this tool looks like.
- **A file list beside the queue, showing both checksums.** The interface could
say a job was "Verified" but never showed the evidence — the source hash and
the destination's, per file — which was only in the CSV and the PDF. The
selected job's files now sit next to the queue: size, both hashes head:tail
(`3F2A:05D1`, because a prefix cannot distinguish a match from a near-miss),
the full values and every destination path in the tooltip, and a doubled
check mark for a copy that was written *and* read back. That mark depends on
the job's verification mode: a source-only job, whose "verified" never reads
the destination, shows a single check and a summary that says so.
Destinations that
disagree with one another read `mismatch` rather than showing the first and
hiding the rest.
- **A stall is reported instead of looking like a slow link.** A hung network
handle raises nothing — it just stops returning bytes — so nothing could be
retried and the job sat at a stale throughput figure. Source reads are now
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ pip install -e ".[dev]"
`ffmpeg` and `ffprobe` on `PATH` are optional — the suite runs without them.

```sh
pytest # 571 tests
pytest # 593 tests
pytest --fuzz # property tests at 3000 examples each, about 2 min
ruff check src tests
pytest --cov=offloader --cov-report=term-missing
Expand Down
17 changes: 15 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,19 @@ destination file rather than leaving something that looks complete. A source
that stops delivering bytes reads as `Stalled on …` with the time since the
last one, rather than a throughput figure that has quietly stopped being true.

Beside it, the **file list** for the selected job: every file with its size,
the source checksum and the destination's next to each other, and a doubled
check mark for a copy that was both written and read back (full verification).
A source-only job gets a single check and its summary says "verified in transit
(source only, not read back)": the bytes written matched the source, but the
destination was never re-read. Hashes are shown
head:tail (`3F2A:05D1`) because a prefix alone cannot tell a matching pair from
a near-miss; the full values are in the tooltip, along with every destination
path. Destinations that disagree with each other read `mismatch` — both copies
came from one read, so they cannot legitimately differ. This is the same
evidence the CSV and the PDF carry, in front of the person deciding whether to
reformat the card.

Two guards run before anything is queued:

- **Duplicate offload protection.** The source's file listing — names and sizes,
Expand Down Expand Up @@ -375,13 +388,13 @@ what makes the report layer testable without moving bytes.

```sh
pip install -e ".[dev]"
pytest # 571 tests
pytest # 593 tests
pytest --fuzz # same suite, 3000 examples per property (~2 min)
ruff check src tests
pytest --cov=offloader --cov-report=term-missing
```

571 tests at 86% line coverage. They cover formatting against the reference's
593 tests at 86% line coverage. They cover formatting against the reference's
exact strings, checksum vectors and streaming equivalence, copy/verify
behaviour including simulated destination corruption, pause/resume/cancel
concurrency, retry discrimination, BRAW container parsing, ffprobe parsing,
Expand Down
Binary file modified docs/images/app-preset-editor.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/app-preset-mode.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/app-simple-mode.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@ select = ["F", "E9", "I", "B", "UP"]
# Qt's model API is defined with QModelIndex() defaults; matching the framework
# signature matters more here than the general rule against call-in-default.
"src/offloader/gui/queue_view.py" = ["B008"]
"src/offloader/gui/file_list.py" = ["B008"]

[tool.ruff.lint.isort]
known-first-party = ["offloader"]
Expand Down
265 changes: 265 additions & 0 deletions src/offloader/gui/file_list.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,265 @@
"""Per-file detail for one job: what landed, and what it hashed to.

The queue answers "did the job succeed". This answers "which file, and prove
it" — the source checksum and the destination's beside each other, which is the
whole claim the tool makes and the one thing the interface never showed. It is
in the CSV and the PDF; an operator deciding whether to erase a card should not
have to open a report to see it.

Hashes are abbreviated head:tail rather than truncated, because the first four
characters alone cannot distinguish a matching pair from a mismatched one that
happens to share a prefix. The full value is in the tooltip.
"""

from __future__ import annotations

from PySide6.QtCore import QAbstractTableModel, QModelIndex, Qt
from PySide6.QtGui import QColor
from PySide6.QtWidgets import (
QAbstractItemView,
QHeaderView,
QTableView,
QVBoxLayout,
QWidget,
)

from ..models import FileStatus, Job, VerificationMode
from ..util import format_size
from . import theme
from .widgets import label, row

COLUMNS = ("", "File", "Size", "Source", "Destination")
COL_MARK = 0
COL_SIZE = 2
COL_SOURCE = 3
COL_DESTINATION = 4

#: One glyph per verdict. Doubled for a verified copy, because that is two
#: separate facts — it was written, and it was read back and matched. Only a
#: FULL job earns it: see `mark_for`.
MARKS = {
FileStatus.VERIFIED: "✓✓",
Comment on lines +37 to +41

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Make the read-back indicator depend on the job's verification mode.

FileStatus.VERIFIED is also produced by a successful SOURCE_ONLY copy. In that mode the engine compares the source stream with the bytes handed to the destination writer; it never reopens the destination to read it back. This table maps the status directly to ✓✓, while the new README and changelog define that mark as written and read back.

Reproduced by running a source-only data offload and passing the returned job to FileModel: the row displays ✓✓, and the pane says verified 1 of 1, with no mode qualification. An operator therefore sees the same claimed read-back evidence as a full-verification job.

Use job.verification when choosing the mark and label, and distinguish streamed verification from destination read-back in the summary or tooltip. Add an engine-produced source-only job alongside a full-verification job in the display tests; the existing _job() fixture always selects FULL.

FileStatus.COPIED: "✓",
FileStatus.FAILED: "✕",
FileStatus.SKIPPED: "–",
FileStatus.CANCELLED: "⊘",
}


def read_back(job: Job | None) -> bool:
"""Whether this job's VERIFIED status means the destination was re-read.

`SOURCE_ONLY` also produces VERIFIED, but compares the source stream with
the bytes handed to the writer; the destination is never read back.
"""
return job is not None and job.verification is VerificationMode.FULL


def mark_for(status: FileStatus, job: Job | None) -> str:
"""The glyph for a file: the double check only for a read-back job."""
if status is FileStatus.VERIFIED and not read_back(job):
return MARKS[FileStatus.COPIED]
return MARKS.get(status, "?")


def abbreviate(checksum: str | None) -> str:
"""`28f7e67d9b39ea9b` as `28F7:EA9B`.

Head *and* tail: a prefix on its own is not enough to tell a matching pair
from a near-miss, which is the only thing these two columns exist to show.
"""
if not checksum:
return "—"
clean = checksum.strip().upper()
if len(clean) <= 9:
return clean
return f"{clean[:4]}:{clean[-4:]}"


def _destination_checksum(entry) -> tuple[str, bool]:
"""The destinations' checksum and whether they all agree with each other.

Several destinations are written from the same read, so a disagreement
between them is a real finding and must not be hidden by showing only the
first one.
"""
sums = [d.checksum for d in entry.destinations if d.checksum]
if not sums:
return "—", True
first = sums[0]
return first, all(value == first for value in sums)


class FileModel(QAbstractTableModel):
def __init__(self, parent=None) -> None:
super().__init__(parent)
self._job: Job | None = None

def set_job(self, job: Job | None) -> None:
self.beginResetModel()
self._job = job
self.endResetModel()

@property
def job(self) -> Job | None:
return self._job

@property
def files(self) -> list:
return list(self._job.files) if self._job is not None else []

# ---------------------------------------------------------------- Qt API
def rowCount(self, parent=QModelIndex()) -> int: # noqa: N802
return 0 if parent.isValid() else len(self.files)

def columnCount(self, parent=QModelIndex()) -> int: # noqa: N802
return 0 if parent.isValid() else len(COLUMNS)

def headerData(self, section, orientation, role=Qt.DisplayRole): # noqa: N802
if role == Qt.DisplayRole and orientation == Qt.Horizontal:
return COLUMNS[section]
return None

def data(self, index: QModelIndex, role=Qt.DisplayRole):
if not index.isValid():
return None
entry = self.files[index.row()]
column = index.column()
destination, agree = _destination_checksum(entry)
matched = bool(entry.checksum) and destination == entry.checksum

if role == Qt.DisplayRole:
return (
mark_for(entry.status, self._job),
str(entry.relative),
format_size(entry.size),
abbreviate(entry.checksum),
abbreviate(destination) if agree else "mismatch",
)[column]

if role == Qt.ForegroundRole:
if column == COL_MARK:
return QColor(theme.status_color(entry.status.value.lower()))
if column == COL_DESTINATION:
# Green only when this column has actually proved something:
# the two sides agree, and every destination agrees too.
if not agree:
return QColor(theme.status_color("failed"))
if matched and (entry.status is not FileStatus.VERIFIED
or read_back(self._job)):
return QColor(theme.status_color("verified"))
return None

if role == Qt.ToolTipRole:
lines = [str(entry.source)]
if entry.status is FileStatus.VERIFIED:
lines.append(
"verified: written and read back from the destination"
if read_back(self._job) else
"verified in transit (source only): the bytes written "
"matched the source, but the destination was not read back")
if entry.checksum:
lines.append(f"source: {entry.checksum}")
for destination_entry in entry.destinations:
lines.append(
f"{destination_entry.status.value.lower():<11} "
f"{destination_entry.checksum or '—'} "
f"{destination_entry.path}")
if destination_entry.error:
lines.append(f" error: {destination_entry.error}")
if entry.checksum and not matched and destination != "—":
lines.append("the destination does not match the source")
return "\n".join(lines)

if role == Qt.TextAlignmentRole:
if column == COL_MARK:
return int(Qt.AlignCenter)
if column == COL_SIZE:
return int(Qt.AlignRight | Qt.AlignVCenter)
return None


class FileListPanel(QWidget):
"""The selected job's files, or a line saying why there are none to show."""

def __init__(self, parent=None) -> None:
super().__init__(parent)
self.model = FileModel(self)

self.table = QTableView()
self.table.setModel(self.model)
self.table.setSelectionBehavior(QAbstractItemView.SelectRows)
self.table.setSelectionMode(QAbstractItemView.SingleSelection)
self.table.setAlternatingRowColors(True)
self.table.verticalHeader().setVisible(False)
self.table.setShowGrid(False)
self.table.verticalHeader().setDefaultSectionSize(26)

header = self.table.horizontalHeader()
header.setSectionResizeMode(COL_MARK, QHeaderView.Fixed)
header.resizeSection(COL_MARK, 34)
header.setSectionResizeMode(1, QHeaderView.Stretch)
for column in (COL_SIZE, COL_SOURCE, COL_DESTINATION):
header.setSectionResizeMode(column, QHeaderView.Fixed)
header.resizeSection(COL_SIZE, 76)
# Wide enough for "mismatch" as well as a head:tail pair, so the column
# does not jump width when a job goes wrong. Kept as tight as that
# allows, because everything here is width the file name does not get —
# and a clip name truncated to "A002_08…" identifies nothing.
header.resizeSection(COL_SOURCE, 90)
header.resizeSection(COL_DESTINATION, 90)

self._title = label("Files", "heading")
self._summary = label("", "muted")
self._empty = label(self.DEFAULT_MESSAGE, "muted")

layout = QVBoxLayout(self)
layout.setContentsMargins(0, 0, 0, 0)
layout.setSpacing(8)
layout.addWidget(row(self._title, 12, self._summary, None))
layout.addWidget(self._empty)
layout.addWidget(self.table, 1)
self.show_job(None, None)

DEFAULT_MESSAGE = "Select a job to see the files it copied."

def show_job(self, job: Job | None, pending: str | None = None) -> None:
"""Display `job`'s files, or `pending` when there are none yet."""
message = pending or self.DEFAULT_MESSAGE
if job is self.model.job:
# The same job object cannot have grown rows — the engine hands the
# job over once, finished. Only the waiting message can change, and
# rebuilding here would reset the table on every progress event of
# a selected running job.
if job is None:
self._empty.setText(message)
return

self.model.set_job(job)
has_rows = bool(job is not None and job.files)
self.table.setVisible(has_rows)
self._empty.setVisible(not has_rows)
if not has_rows:
self._empty.setText(message)
self._summary.setText("")
return
self._summary.setText(self._describe(job))

@staticmethod
def _describe(job: Job) -> str:
verified = sum(1 for entry in job.files
if entry.status is FileStatus.VERIFIED)
failed = sum(1 for entry in job.files
if entry.status is FileStatus.FAILED)
parts = [f"{len(job.files)} files",
format_size(job.total_bytes),
job.hash_label]
# Counted rather than inferred from the job's verdict: "Verified: 5 of
# 6" is the number that decides whether a card can be erased.
if verified:
how = ("" if read_back(job)
else " in transit (source only, not read back)")
parts.append(f"verified{how} {verified} of {len(job.files)}")
if failed:
parts.append(f"{failed} failed")
return " · ".join(parts)
Loading
Loading