Add initial dataset of invalid blocks with schema, notes, and CI

[deadmanoz]

Summary

Initial dataset for the invalid-blocks side of the split discussed in bitcoin-data/stale-blocks#128: headers and blocks that carry valid PoW but failed a named consensus check. 46 records spanning heights 74638 to 957780, with full blocks for 74638, 477115, 783426, 784121, and 809478. The records come from entries dropped from stale-blocks (bitcoin-data/stale-blocks#134), the 2023 P2P-observed invalid blocks, archived block explorers, and merge-mined recoveries (headers rescued from child-chain commitments on Namecoin, RSK, and others).

data/invalid-blocks.jsonl commits one record per block hash, with the 80-byte header plus core_reject_reason and rule. A nested context object carries the extra evidence a rule needs when the header alone is not enough (BIP34 coinbase height, parent MTP, expected nBits); an observations array records each witness with its source channel and provenance URL.

The admission standard is spelled out in docs/schema.md: PoW against the header's own nBits, a registered rule/reject-string pair, and evidence checked for that rule. Complete .bin files must parse and match their transaction merkle roots and applicable witness commitments; CI checks output overflow, transaction ordering and excessive sigop cost directly. These checks establish the named failure; they do not execute scripts or reconstruct historical chain state. docs/notes.md covers replay behaviour and incident background (replaying the 2023 full blocks with submitblock cannot reproduce their reject strings today, since their violations only surface in ConnectBlock).

The validator uses python-bitcoinlib, pinned to 0.12.2 in requirements.txt, for Bitcoin parsing and serialization. For sigops, CI fetches and verifies previous transactions from public APIs and runs daily after merge to keep the cache warm. Only successful non-PR runs on the default branch save a cache archive when no exact cache exists for that block set; pull requests restore the cache without uploading archives.

Testing

python ci/sanity-check.py --fetch-prevouts passes: 46 records, 46 context objects, 188 observations, 5 block files. It re-derives the header hash, checks PoW against the header's nBits, decodes prev_hash and nTime back out of the header, and enforces sort order and hash uniqueness. All 28 regression tests (with named subtests for related cases) pass, as do workflow lint and whitespace checks. CI calculates sigop cost 80003 for each F2Pool block against the 80000 limit; a public-API smoke run filled two missing cache entries and reproduced both counts.

I verified the rule-to-reject-string table in docs/schema.md against Core master (CheckTransaction, CheckTxInputs, ContextualCheckBlockHeader, ContextualCheckBlock, ConnectBlock), and replayed the four original .bins (74638, 783426, 784121, and 809478) on two mainnet nodes (v30.2 and a master build): 74638 returns bad-txns-vout-toolarge on both; the three 2023 blocks return inconclusive or duplicate depending on whether the node already stores them, which is exactly the behaviour docs/notes.md documents.

All seven entries removed from stale-blocks in bitcoin-data/stale-blocks#134 are covered here.

[deadmanoz]

P.S. I am writing a deeper dive post about these blocks :)

[deadmanoz]

Drafted because I just want to ensure I've got all observations for the merge-mined contributions..

[0xB10C]

The dataset looks Ok. I haven't looked at the CI code in detail. I was wondering if it could be less/simpler, or is needed at all in this capacity, but I also don't mind having it!

[deadmanoz]

The dataset looks Ok. I haven't looked at the CI code in detail. I was wondering if it could be less/simpler, or is needed at all in this capacity, but I also don't mind having it!

I do think we should have as comprehensive evaluation of any blocks that are added, but yeah the tests can for sure be reduced, and probably we can simplify regardless

[deadmanoz]

Switching from stdlib to leveraging python-bitcoinlib, bring LOC down a fair bit

[0xB10C]

Feel free to merge at any point you think this is ready. I think you should have the permissions to do so.