Skip stream entries whose content does not match their trusty URI (fixes #208) - #209
Merged
Conversation
An entry whose signature is valid but whose content does not hash to its trusty URI artifact code (e.g. the RAAAAA... nanopub at counter 88951, 2026-08-29 — knowledgepixels/nanopub-registry#164) used to pass every loader check and reach the repo writers, where NanopubUtils.updateXorChecksum threw ArrayIndexOutOfBoundsException on the short artifact code. The retry loops treated that deterministic error as transient, so every poll re-fetched the same entry and re-failed for minutes at a time, wedging ingestion fleet-wide on one counter (#208). - NanopubLoader: verify TrustyNanopubUtils.isValidTrustyNanopub as the first constructor check; on mismatch, log an ERROR and route the entry through the existing aborted/notes flow so the counter advances past it. Also guards the latent NPE when getArtifactCode returns null. - JellyNanopubLoader.loadUpdates: log a "loader wedged" ERROR with operator guidance once 3 consecutive batch failures pass without the counter moving. Log-only by design: a store outage produces the same signature, and skipping there would silently drop real nanopubs. - Decode-level (!isSuccess) failures still throw and retry on purpose: a truncated stream read is transient, and skipping there would leave holes. - Regression test with the real captured registry segment (poison entry 88951 plus a valid sibling) as fixture. Fixes #208. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause (corrects the diagnosis in #208)
The poison entry (
RAAAAA…, counter 88951) has a valid signature but an invalid trusty hash — its content does not hash to its 43-char artifact code (knowledgepixels/nanopub-registry#164). The Jelly layer decodes it as a success item; nothing in the load path verifies the trusty hash, so it passed every constructor check and died in the repo writers:The generic retry loops (8 attempts, backoff up to ~90 s, per repo task, per poll) treated this deterministic error as transient — so each poll cycle ground through ~10–20 minutes of retries, threw, and started over, forever. It was not actually silent (
Failed to load nanopub <…> to repo 'full': Index 30 out of boundslines were present but sparse), and the liveness age stamp already behaves correctly during failing batches — asks 2 and 3 of the issue were based on wrong incident-time observations.Fix
NanopubLoader:TrustyNanopubUtils.isValidTrustyNanopub(np)is now the first constructor check (~6 ms/np). On mismatch: ERROR log, note in the admin repo,aborted = true— riding the existing abort flow, so the counter advances past the entry. One bad stream entry can no longer stall ingestion; a future resync sails past counter 88951 with a note instead of wedging. Also guards the latent NPE whengetArtifactCodereturns null.JellyNanopubLoader.loadUpdates: a "loader wedged" ERROR with operator guidance once 3 consecutive batch failures pass without counter movement. Log-only by design — a store outage produces the same signature, and count-based skipping there would silently drop real nanopubs.!isSuccess) failures still throw and retry on purpose: truncated stream reads are transient, and skipping would leave holes.Test
PoisonNanopubSkipTestuses the real captured registry stream segment (?afterCounter=88950from registry.petapico.org, 2026-08-30) as a fixture: asserts the poison entry is aborted with the note and returns normally fromload(), and that the valid sibling entry (88952) is not rejected. The abort path runs entirely on the calling thread, which is what makes it testable againstInMemoryTripleStore. All 517 tests pass.Fixes #208.
🤖 Generated with Claude Code