Restore DataLinq 0.9 allocation efficiency to the 0.8 baseline

[bazer]

Summary

DataLinq 0.9 has substantial transient-allocation regressions relative to the final 0.8 runtime. The goal of this tracker is to restore allocation to the same or a lower level than 0.8 without weakening the correctness boundaries introduced in 0.9.

This is not evidence of a memory leak. Operation telemetry remains stable, and warm relation traversal still allocates 0 B/op. The regression is additive across query planning/binding, provider-neutral row materialization/source loading, mutation snapshots, and smaller execution envelopes.

Compared revisions for the original report:

• Final-0.8 runtime baseline: 8bcfc770, whose runtime and benchmark source is identical to tag 0.8.0
• Original 0.9 comparison: 9f779d9e

Stable benchmark evidence

Scenario	0.8 B/op	Original 0.9 B/op	Increase
Warm primary-key fetch	1,812.48	12,666.88	+10,854.40 / +598.87%
Cold primary-key fetch	6,010.88	19,671.04	+13,660.16 / +227.26%
Cold relation traversal	11,683.84	27,371.52	+15,687.68 / +134.27%
Warm relation traversal	0	0	unchanged
Update employees	14,376.96	37,621.76	+23,244.80 / +161.68%
CRUD workflow batch	53,483.52	101,611.52	+48,128.00 / +89.99%
CRUD workflow small	53,944.32	102,553.60	+48,609.28 / +90.11%
Provider initialization	364,953.60	384,081.92	+19,128.32 / +5.24%
Startup primary-key fetch	70,717.44	93,921.28	+23,203.84 / +32.81%

Entity-query, scalar-query, mutation, materialization, relation-load, and cache telemetry was unchanged for every comparable row.

The retained 0.8 artifact uses legacy schema v2, so it is strong diagnostic evidence rather than a modern release gate. #27 established the current schema-v3 0.8 comparison and regression budgets.

SQL hot-path evidence

The original heavy-profile measurements showed:

Scenario	0.8 B/op	Original 0.9 B/op	Increase
Repeated scalar Any	14,592.00	22,896.64	+8,304.64
Repeated non-PK equality fetch	19,650.56	27,996.16	+8,345.60
Repeated IN predicate fetch	25,108.48	34,344.96	+9,236.48

After #28 and #29, focused query evidence is approximately:

• expression parse / structural template: 9,738.24 B/op
• parse / template / initial bind: 9,799.68 B/op
• invocation bind: 307.20 B/op
• SQL request / capability preparation: 225.28 B/op
• warm LINQ primary-key fetch: 9,605.12 B/op
• generated typed static primary-key Get: 0 KB/op reported on current SQLite file and memory-provider runs

These stages are separate measurements and must not be subtracted literally, but they show that repeated structural planning—not typed row-cache lookup—is now the dominant warm-query target.

Materialization and cold-source evidence

PR #44 completed the main owned-row-buffer transfers in #30:

• canonical decode: 460.80 -> 296.96 B/op
• model materialization: 235.52 -> 143.36 B/op
• combined materialization: 706.56 -> 440.32 B/op
• cold primary-key fetch: 16,609.28 -> 16,250.88 B/op
• cold relation traversal: 24,494.08 -> 23,777.28 B/op
• warm relation traversal remains 0 B/op

The current cold-minus-warm primary-key delta is 6,645.76 B/op versus 4,106.24 B/op for the final-0.8 same-runner baseline. A diagnostic using the legacy direct single-row loader reduced the current delta to approximately 4.30 KB/op, locating roughly 2.2 KB/op in neutral single-row source orchestration and/or general SQL construction. #30 now owns that remaining singular-path work.

Workstreams

Completed foundation work:

• Benchmark: establish a schema-v3 0.8 allocation baseline and regression budgets #27 — Establish a schema-v3 0.8 allocation baseline and regression budgets
• Query planning: precompute structural capability requirements #28 — Precompute structural query capability requirements
• Query binding: eliminate duplicate snapshots and lookup structures #29 — Eliminate duplicate query-binding snapshots and lookup structures
• Mutation: reuse one immutable change snapshot throughout execution #31 — Reuse one immutable mutation snapshot throughout execution

Open allocation work:

• Cold primary-key loading: complete row ownership and add a singular source fast path #30 — Complete row ownership and add a singular neutral-source primary-key path
• Query planning: reuse prepared structural templates for repeated LINQ shapes #45 — Reuse prepared structural templates for repeated LINQ shapes
• Source loading: eliminate batch slicing and request/result snapshot allocations #46 — Eliminate batch slicing and source request/result snapshot allocations
• Materialization: propagate validated canonical primary keys without reconstruction #47 — Propagate validated canonical primary keys without reconstruction
• Query execution: remove transient request and capability-validation allocations #48 — Remove transient query execution and capability-validation allocations
• Binary values: measure ownership transfers and remove redundant buffer clones #49 — Measure binary ownership transfers and remove redundant buffer clones

Suggested implementation order:

1. Cold primary-key loading: complete row ownership and add a singular source fast path #30 singular primary-key source path
2. Query planning: reuse prepared structural templates for repeated LINQ shapes #45 prepared/compiled query templates
3. Materialization: propagate validated canonical primary keys without reconstruction #47 canonical-key propagation
4. Source loading: eliminate batch slicing and request/result snapshot allocations #46 batch source-loading ownership and slicing
5. Query execution: remove transient request and capability-validation allocations #48 execution envelopes
6. Binary values: measure ownership transfers and remove redundant buffer clones #49 binary-buffer work after benchmark attribution

The tracker closes only when the integrated benchmark comparison meets the overall allocation and correctness criteria below. A child issue may close once its own scoped evidence is complete; closing a child does not by itself establish overall 0.8 parity.

Overall acceptance criteria

• Tag 0.8.0 and the candidate are measured with the same current schema-v3 harness, runtime family, provider, profile, filter, and runner class.
• Every comparable allocation target is at or below its final-0.8 baseline; any intentional exception is explicitly measured and justified here rather than silently rebaselined.
• Query, mutation, materialization, relation, and cache telemetry shapes remain unchanged.
• Generated typed warm primary-key lookup remains allocation-free.
• No statistically meaningful latency regression is introduced; warnings caused only by low duration or multimodality are reviewed from persisted artifacts rather than treated as product failures.
• Full unit, compliance, memory-provider, and SQL-provider verification remains green.
• A final benchmark receipt records the contribution of each child issue and the aggregate result.

Correctness guardrails

Allocation work must not weaken:

• query capability validation or backend ownership validation
• captured mutable-array/value snapshot semantics
• canonical provider/model value conversion and nullability checks
• source-result table/key/duplicate/missing-row validation
• mutation drift detection, transaction provenance, commit-uncertainty handling, or cache invalidation
• public detached-copy contracts

Explicit non-goals

• Do not use global string/value interning as a default strategy.
• Do not introduce broad array pooling before ownership-transfer approaches are measured.
• Do not cache captured invocation values or closures.
• Do not pool objects that can be removed from a singular path.
• Do not raise the allocation baseline to current 0.9 behavior.
• Do not adopt spans, pooling, frozen collections, or alternate-key lookup without a specific measured lifetime/lookup problem.

Prepared-query decision

The deferred condition has been met: requirement precomputation and binding optimization did not restore warm-query allocation to the 0.8 budget. #45 now owns explicit prepared/compiled template reuse.

Any later automatic plan cache must be bounded and must key metadata identity, expression structure, specialization shape, projection/result shape, and backend-relevant structure without retaining closures or invocation values.