MDEV-40523 Versioned UPDATE on a HEAP table with blobs corrupts the history row

[arcivanov]

MDEV-40523 Versioned UPDATE on a HEAP table with blobs corrupts the history row

A system-versioned UPDATE of a blob column on a HEAP table stored garbage in the history row, and an AFTER UPDATE trigger reading OLD.<blob> saw the same garbage. Both values are durable: the history row is what SELECT ... FOR SYSTEM_TIME ALL returns, and both reach replicas through the row-based binlog image. No ASAN build is needed to reproduce either.

CREATE TABLE t (f TEXT) ENGINE=HEAP WITH SYSTEM VERSIONING CHARACTER SET latin1;
INSERT INTO t VALUES (REPEAT('abcdefgh',8));
UPDATE t SET f = 'foo';
SELECT LENGTH(f), f = REPEAT('abcdefgh',8) AS hist_ok
  FROM t FOR SYSTEM_TIME ALL WHERE f <> 'foo';   -- hist_ok = 0

A non-uniform payload matters: with REPEAT('x',64) the overlapping copy reproduces the same byte at every position and the corruption hides.

How a versioned UPDATE reaches the engine

The row is first updated in place with ha_update_row(old_data, new_data). The SQL layer then calls vers_insert_history_row(), which restores the pre-update row from record[1] into record[0], stamps it with the delete-time and calls ha_write_row().

So the record handed to ha_write_row() is a verbatim copy of the row the engine was just told to overwrite — blob data pointer included.

A versioned DELETE is not affected: TABLE::delete_row() stamps the end field with vers_update_end() and issues a single ha_update_row(). It writes no history row.

Cause

Three individually correct behaviours compose badly.

hp_read_blobs() is zero-copy: it points the record's blob data pointer straight into the HP_BLOCK continuation chain rather than copying. Only a multi-run chain is reassembled, into info->blob_buff.

heap_update() and heap_delete() therefore do not free the old chain — they park it, because the SQL layer keeps reading the pre-update row out of record[1] after ha_update_row() returns. binlog_log_row() builds the before-image from it, and an AFTER UPDATE trigger reads OLD.<blob> from it.

heap_write() redeemed that parking unconditionally, as its first action, before allocating. For the history row that is exactly the wrong moment, since it sources its blob from the chain the update just parked. The free put those records on the delete list, where the allocation immediately below handed them straight back as the history row's own chain — with hp_push_free_block()'s free-list links already scribbled through the payload. Source and destination of the blob copy overlapped, and record[1] was left pointing at reused memory for the rest of the statement.

Fix

The parked chain already holds exactly the bytes the history row needs — it is a verbatim copy of the same record. So instead of freeing it and allocating a duplicate, the new row adopts it:

• hp_flush_unaliased_blob_free() redeems every parked chain except ones the record being written still sources blob data from.
• hp_write_blobs() takes those over: the stored row points at the parked chain and no new chain is written. The pending slot is cleared only once every column has succeeded, so the rollback path can tell an adopted chain from an allocated one and leaves it parked rather than freeing it.

Both record buffers stay valid, because the chain's contents are never disturbed. Adoption also needs no space at all, which matters at max_heap_table_size: the history row previously had to find room for a second copy of a blob that was already resident, and the parked chain was frequently the only reclaimable space. The fix is therefore strictly better than the previous behaviour under memory pressure, not merely correct.

Aliasing is detected by exact pointer equality against the parked chain head. hp_read_blobs() produces zero-copy pointers of exactly two forms — the chain head, or chain + recbuffer — and reassembles a multi-run chain into info->blob_buff, which can never alias, so the two comparisons in hp_blob_sources_chain() are complete and need no walk of the HP_BLOCK tree. Matching is per blob column, since pending_blob_chains[i] is the chain parked for column i, and that slot is NULL for a column the update did not change.

An adopted chain may be longer than the adopting row's blob length: UPDATE t SET f = LEFT(f,6) leaves the shortened value pointing at the original data. That is harmless. hp_read_blobs() picks the chain layout from the chain's own flag byte rather than from the length, so a short read off a long chain still starts at the right offset, and hp_free_run_chain() walks run_rec_count/next_cont rather than the length, so the whole chain is still reclaimed.

REPLACE was never affected: HA_EXTRA_WRITE_CAN_REPLACE makes hp_read_blobs() copy rather than hand out zero-copy pointers. Internal temporary tables never park — they free their chains outright and hp_open() does not allocate the array — so hp_write_blobs() guards on it.

A blob cannot itself be indexed in a HEAP table: ha_heap does not set HA_CAN_INDEX_BLOBS, so both KEY (f(10)) and UNIQUE (f) are rejected at CREATE with ER_BLOB_USED_AS_KEY. There is therefore no versioned blob-as-key combination for adoption to get wrong. Blob key segments exist only for internal temporary tables, which never park a chain and are never versioned.

What triggers the bug, and what reads the corrupted data

Triggering statements are every vers_insert_history_row() caller reaching a HEAP table whose record buffer was filled by a read: single-table UPDATE, multi-table UPDATE (both the on-the-fly and the deferred do_updates() path), INSERT ... ON DUPLICATE KEY UPDATE, and the row-based replication applier. A versioned UPDATE that does not change the blob column is unaffected — heap_update() keeps the chain and parks nothing.

Three consumers then read corrupted data, because all three read it out of record[1] after the history-row write has recycled the chain:

• The history row itself, as returned by SELECT ... FOR SYSTEM_TIME ALL.
• The row-based binlog before-image, so the corruption reaches replicas.
• AFTER UPDATE triggers reading OLD.<blob> — whatever the trigger does with that value, typically writing it to an audit table, stores wrong data, and that write is itself replicated.

The trigger case is the one most easily missed, since LENGTH(OLD.<blob>) is still correct: the length lives in the record buffer and survives, and only the payload has been recycled. A BEFORE UPDATE trigger on the same table reports the correct value, which is what localises the damage to the history-row write happening between the two:

ln   ok_val
640  1     <- BEFORE UPDATE, row 1: correct
640  0     <- AFTER UPDATE,  row 1: corrupt
640  1     <- BEFORE UPDATE, row 2: correct
640  0     <- AFTER UPDATE,  row 2: corrupt

Tests

storage/heap/hp_test_blob_alias-t.c drives the sequence at the heap_write() API for a single-record chain, a zero-copy run and a multi-run chain, checking both the stored row and the caller's buffer. It also pins adoption itself, by the stored row's chain pointer and by the growth of block.last_allocated across the write: exactly one record slot and no chain. The multi-run case is reassembled into info->blob_buff and cannot alias, serving as a control; the test asserts which layout it obtained so the coverage cannot silently degrade.

heap.blob_vers_trigger covers OLD.<blob> in triggers across single-table UPDATE, multiple blob columns, ON DUPLICATE KEY UPDATE and multi-table UPDATE, with BEFORE UPDATE alongside AFTER UPDATE on one table, plus non-versioned HEAP and versioned MyISAM controls.

heap.blob_versioning, heap.blob_vers_odku and heap.blob_vers_multi cover the history row itself for every caller, including a REPLACE control. heap.blob_vers_repl pins the before-image reaching a replica.

heap.blob_versioning additionally covers the case where only one of two blob columns changed, so the history-row write sees a mix of parked and empty slots; a blob shrunk in place with LEFT(), twice, so the second generation shrinks off an already-adopted chain; an indexed table, so the key loop runs while record[1] still holds a zero-copy pointer into the parked chain and a UNIQUE key materializes stored blobs into key_blob_buff during duplicate detection; and the two ER_BLOB_USED_AS_KEY rejections.

Tests were written before the fix and verified to fail against it: failing assertions in the unit test, and wrong OLD.<blob> in every non-control MTR case. The one-of-two-blob-columns case was likewise verified to fail without its guard.

Verification

• HEAP unit tests: 6/6 binaries, including hp_test_blob_alias-t at 62/62.
• versioning,period,heap: 151/151.
• main: 1429/1429.