Restore fresh due command IDs and align cluster oracles with current contracts

Product:
- Due and queue-deadline dispatch derived one stable command ID per hint, so a
  durable PermissionDenied from a paused or revoked creator was replayed on
  every later flush and the schedule never advanced. Each attempt now uses a
  fresh command GUID again, as DueCoordination.md requires; the unused
  DueDispatchCommandIdentity is removed.
- macOS cannot load a PKCS#12 key as an ephemeral key set, so the PostgreSQL
  listener certificate uses the default key set there.

Tests and infrastructure:
- PostgreSQL cluster flows configure KeyLoad:RuntimeSql (the section the
  server reads); the listener never started under KeyLoad:Sql.
- The file-ownership log reader watches the project resource, whose DCP log
  stream uses the suffixed instance name.
- Silo-loss tests wait until Orleans membership reports the survivors (new
  ClusterMembershipReadiness) before reading, and for every node before
  writing; the document session flow kills one of three silos.
- Outbox oracles expect one appended entry per committed batch mutation.
- MCP refusal details use the MCP safe details; required processing retries
  keep the refused command's outcome and complete with a new command ID; the
  queue cancel worker is an administrator; projection consumers are listed in
  key order; SQL uses TOP instead of LIMIT; unified SQL compile refusals carry
  no request ID.
- Limit-only expectations are gone: a large allowlist and a large partition
  query parameter now succeed, and the SQL byte/token/depth budget cases are
  removed (ADR-149).
- Product contributor counts follow the registry (42 unit, 9 cluster).
- Sparse composite source reads run concurrently.
- Integration lanes get a 45-minute deadline and shards 55 minutes; measured
  lanes take 20-25 minutes and five lanes aborted at 25.

Verification: unit and integration test projects and the server build with
-warnaserror; dotnet format verify on changed files; governance PASS. Locally,
one cluster at a time, these pass: AcFilter003, AcPquery001/005, both
ActualMixedProducerBatch cases, TypedEndpointInnerJoin, AcLineage002,
DistinctFiles (22m39s), ExistingScopedPrincipalRevoke, AcDue002And004,
SameNamedQueues, both CurrentRequiredBodyAndHeader cases, NonemptyProfile,
AnnotationsAndPrivateLexicalBounds, PersistedAdministratorProjectionEffects,
AcknowledgedDocumentToken, AcPquery006, AcGsearch006 and
OriginalSelectedQueueDeadline. Int16/Decimal PostgreSQL flows still fail
locally on macOS in the TLS handshake ("bad protocol version"); AcSqlc006
still accepts the original cursor after a cold restart.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Author: KSemenenko

.github/workflows/build-and-tests.yml

@@ -110,7 +110,7 @@ jobs:
     name: Aspire topology ${{ matrix.shard }}
     needs: shards
     runs-on: ubuntu-24.04
-    timeout-minutes: 35
+    timeout-minutes: 55
     strategy:
       fail-fast: false
       matrix: ${{ fromJSON(needs.shards.outputs.integration) }}

docs/ADR/ADR-148-parallel-ci-shards.md

@@ -18,7 +18,7 @@ Build and Tests is a set of parallel jobs driven by one checked-in manifest, `sc
 - **Units:** four shards. Each runs one process per namespace with the assembly-pinned filter `/KeyLoad.UnitTests/<namespace>/*/*` at processor-count parallelism.
 - **Analyzers and process recovery:** one job with filters `/KeyLoad.Analyzers.Tests/*/*/*` and `/KeyLoad.RecoveryTests/*/*/*`.
 - **Aspire topology:** twelve shards of two lanes each.
-  - A lane is its own `dotnet test` process with `--maximum-parallel-tests 1` and a 25-minute deadline, so a runner holds at most two clusters.
+  - A lane is its own `dotnet test` process with `--maximum-parallel-tests 1` and a 45-minute deadline, so a runner holds at most two clusters. Measured lanes take 20–25 minutes, so the deadline sits well above a complete lane.
   - Lanes are typed: one keyed shared cluster, own clusters, other keyed clusters, or one long functional gate method.
   - Every lane process owns its own keyed shared cluster, so a test that kills, restarts or cold-reopens nodes cannot affect another lane.
 - **Verify test partition:** reads every shard's TRX artifact and proves that each test ran exactly once, in its assigned shard and lane.

docs/Features/QueryExecution/NumericParserAdmission.md

@@ -17,9 +17,9 @@ REQ/AC-PARSER-ANNOTATION-001: reject otherwise silently discarded child `Collati
 
 They retain original SQL, complete parse errors, bounded node/parent/child types, offsets and Collation. Actual DLL SHA256 is `1f95f522e68d6232430fb2c00280f16f248bef46b50cb4fbc1ffd555edbf1604`, assembly18.0.0.0. Annotated TOP, numeric unary predicate, ordinal, OFFSET, event generation and typed INSERT were accepted; annotated EXEC was syntax-rejected and has no added guard.
 
-REQ/AC-PARSER-ANNOTATION-002: whole real-store operations prove refusal without partial value, full state/cut/identity unchanged, then ordinary TOP/ordinal/unary/OFFSET/event-source and typed INSERT return independent complete literal rows and terminal cursors through SQL and canonical AST. All original authored private-input variants, malformed/unsupported/token/depth/UTF8 bounds, typed INSERT no-effect refusal, real original receipts/raw outcomes and persisted reader withdrawal/repair remain. Unit receipts verify all five mutation fields including hidden CompositionReferences; public receipt checks cover the four observable JSON fields and complete token. Hidden fields do not cross SDK/MCP JSON.
+REQ/AC-PARSER-ANNOTATION-002: whole real-store operations prove refusal without partial value, full state/cut/identity unchanged, then ordinary TOP/ordinal/unary/OFFSET/event-source and typed INSERT return independent complete literal rows and terminal cursors through SQL and canonical AST. All original authored private-input variants, malformed and unsupported inputs, typed INSERT no-effect refusal, real original receipts/raw outcomes and persisted reader withdrawal/repair remain. Unit receipts verify all five mutation fields including hidden CompositionReferences; public receipt checks cover the four observable JSON fields and complete token. Hidden fields do not cross SDK/MCP JSON.
 
-REQ/AC-PARSER-BOUNDS-003: exactly4096-byte and65536-byte public valid SQL includes astral+CJK comments. UTF8 bytes differ from UTF16 string.Length; one additional ASCII byte must return the existing byte-budget problem, followed by exact original healthy full four-row pages. Direct metadata within the same whole Unit flow checks actual statement UTF16 StartOffset/line/column after a Unicode CRLF comment and malformed token Offset/line/column. diagnostic positions are not public byte offsets; public errors deliberately remain fixed and do not include SQL, parameters or ScriptDOM messages. No new diagnostic payload is introduced.
+REQ/AC-PARSER-BOUNDS-003: exactly4096-byte and65536-byte public valid SQL includes astral+CJK comments. UTF8 bytes differ from UTF16 string.Length; SQL text has no byte, token or depth budget ([ADR-149](../../ADR/ADR-149-no-operation-limits.md)), so each such text returns exact original healthy full four-row pages. Direct metadata within the same whole Unit flow checks actual statement UTF16 StartOffset/line/column after a Unicode CRLF comment and malformed token Offset/line/column. diagnostic positions are not public byte offsets; public errors deliberately remain fixed and do not include SQL, parameters or ScriptDOM messages. No new diagnostic payload is introduced.
 
 REQ/AC-PARSER-LIFETIME-004: genuine pre-cancelled and admitted parser-token cancellation retain the original token and joined work, full raw store conservation and healthy continuation. Reuse the existing checked-token observation; add no production test seam. SDK typed query and SELECT SQL pre-cancellation use Cancelled/read-unavailable; official MCP raw and unified SELECT tasks are awaited and retain their original cancellation token. These before-cancel public checks do not prove mid-parser transport cancellation or hard preemption. Existing SQL EXEC unknown-write outcome contracts remain unchanged.
 

docs/Features/Search/FilteredRetrieval.md

@@ -17,7 +17,7 @@ Persisted authorization remains required even for an empty allowlist.
 |---|---|
 | REQ-FILTER-001: allowlist is an output eligibility constraint | AC-FILTER-001: real-store text, vector and hybrid queries equal an independent exact oracle over the eligible authorized corpus; null/empty/duplicates/unknown IDs and stable ties pass. `FilteredSearchOracleTests` |
 | REQ-FILTER-002: one canonical policy/read cut remains authoritative | AC-FILTER-002: hidden/deleted/stale/reinserted/revoked hits never survive; field/capability denial remains explicit even for zero results. `FilteredSearchPolicyTests` |
-| REQ-FILTER-003: bounded exact fallback produces complete results | AC-FILTER-003: input/work/result limits and cancellation fail with the existing typed failure and no partial success; the next healthy read succeeds; no full JSON/vector retention beyond existing bounds. `FilteredSearchBudgetTests` |
+| REQ-FILTER-003: bounded exact fallback produces complete results | AC-FILTER-003: a large allowlist has no KeyLoad ceiling and returns only eligible hits; the result limit and cancellation fail with the existing typed failure and no partial success; the next healthy read succeeds; no full JSON/vector retention beyond existing bounds. `FilteredSearchBudgetTests` |
 | REQ-FILTER-004: adaptive ANN uses the entire admissible graph | AC-FILTER-004: internal planner uses full-corpus ordinal eligibility, exact small-set, bounded expansion and fully charged fallback; 100/10/1/0.1 percent and correlated cohorts reach recall >=0.95 or fail explicitly. `AdaptiveFilteredPlannerTests`; adaptive planner below |
 | REQ-FILTER-005: public approximation and cluster are separately qualified | AC-FILTER-005: the public approximation proves declared versioned completeness/mode/cost metadata and actual .NET/official MCP cluster results after faults. Planned `FilteredSearchClusterTests`; existing array replies remain complete exact results in the public allowlist |
 

docs/Features/TestInfrastructure.md

@@ -107,7 +107,7 @@ REQ/AC-TEST-015 requires the local-image lifetime helpers to await already compl
 
 Local image cleanup joins in TestSuiteApplication only after actual AppHost stop and producer/runner settlement, under a separate45-second cleanup token, before output settlement and application disposal. Preparation output is included only for explicit local mode. Default GitHub image selection and the full replicated-cluster gates remain mandatory.
 
-REQ/AC-TEST-010 also requires bounded GitHub job time. Build and Tests splits the suites into the sharded jobs described in [CI shards and lanes](#ci-shards-and-lanes): each unit shard and the analyzers-and-recovery job has 30 minutes, each Aspire topology shard 35 minutes with a 25-minute deadline per lane process, and the Build job 30 minutes. No individual suite deadline, required check, artifact or failure predicate is removed. A failed, timed out or cancelled suite, lane or shard remains failed/unqualified, and an unfinished shard cannot qualify the run.
+REQ/AC-TEST-010 also requires bounded GitHub job time. Build and Tests splits the suites into the sharded jobs described in [CI shards and lanes](#ci-shards-and-lanes): each unit shard and the analyzers-and-recovery job has 30 minutes, each Aspire topology shard 55 minutes with a 45-minute deadline per lane process, and the Build job 30 minutes. No individual suite deadline, required check, artifact or failure predicate is removed. A failed, timed out or cancelled suite, lane or shard remains failed/unqualified, and an unfinished shard cannot qualify the run.
 
 AC-TEST-010 requires original per-suite MTP/TRX evidence from every required CI suite. Its Aspire caller explicitly enables `KeyLoadTests:ReportTrx=true`; an omitted report setting, absent report, cancelled suite or failed runner cannot establish qualification. A missing full-suite TRX receipt is an evidence gap, never a pass.
 
@@ -154,7 +154,7 @@ Whole regression: existing ImagesProduceClosedManifestAndRejectTamperingWithoutR
 | Build | governance, website source inventory, static shard partition check, `dotnet build KeyLoad.slnx -warnaserror` | none |
 | Units U1–U4 | the namespaces listed for the shard | one process per namespace, `/KeyLoad.UnitTests/<namespace>/*/*`, `--maximum-parallel-tests` = processor count |
 | Analyzers and process recovery | shards AN then RC | `/KeyLoad.Analyzers.Tests/*/*/*` and `/KeyLoad.RecoveryTests/*/*/*` |
-| Aspire topology I01–I12 | two lanes per runner | each lane is its own `dotnet test` process, `--maximum-parallel-tests 1 --timeout 25m`, class or method alternation under `/KeyLoad.IntegrationTests/*/` |
+| Aspire topology I01–I12 | two lanes per runner | each lane is its own `dotnet test` process, `--maximum-parallel-tests 1 --timeout 45m`, class or method alternation under `/KeyLoad.IntegrationTests/*/` |
 | Verify test partition | every shard's TRX artifact | `ci-test-shards.mjs verify TestResults` |
 
 Every test job runs `dotnet build KeyLoad.slnx -warnaserror` in the default Debug configuration, because tests read compiled outputs of projects their test project does not reference: ProductionSourceManifestOperationTests reads the image of every solution project, and ColdBootstrapSdkTests launches `src/KeyLoad.Cli/bin/Debug/net10.0/KeyLoad.Cli.dll`. Each job then writes TRX reports to `TestResults/<suite>/<shard>/<lane or namespace>` and uploads them as `test-results-<suite>-<shard>` even when tests fail. The filters pin the assembly segment, so the process recovery tests that the unit host also registers run once, in RC.

scripts/Features/TestInfrastructure/ci-test-shards.json

@@ -88,7 +88,7 @@
       "assembly": "KeyLoad.IntegrationTests",
       "partition": "class",
       "parallelism": 1,
-      "timeoutMinutes": 25,
+      "timeoutMinutes": 45,
       "lanesPerRunner": 2,
       "excludedCategories": [
         "HeavyLoad"

src/KeyLoad.Clustering/Messaging/Execution/QueueDeadlineCoordination.cs

@@ -32,7 +32,7 @@ internal static async Task<DueDispatchResult> ExecuteAsync(QueueDeadlineHint hin
         {
             throw Errors.Fail(ErrorCode.Unauthenticated, GrainRoutingProtocol.MissingPrincipal);
         }
-        var commandId = DueDispatchCommandIdentity.ForDeadline(hint);
+        var commandId = Guid.NewGuid();
         var body = OrleansSerialization.Serialize(new CommandRequest(commandId, hint.Lane.Partition, [hint.Mutation]));
         var reply = await dispatch(principal, commandId, body, lifetime.Token).ConfigureAwait(true);
         return new(reply.Error);

src/KeyLoad.Clustering/Messaging/Grains/RecurringDueCoordinatorGrain.cs

@@ -41,7 +41,7 @@ public async Task<DueDispatchResult> ProcessDueAsync(DueWorkHint hint, Cancellat
         }
 
         var creator = principal ?? throw Errors.Fail(ErrorCode.Corruption, DueCoordinatorFields.InvalidHint);
-        var commandId = DueDispatchCommandIdentity.ForDue(hint);
+        var commandId = Guid.NewGuid();
         var mutation = RecurringDueCommand.Create(hint);
         var payload = OrleansSerialization.Serialize(new CommandRequest(commandId, hint.Lane.Partition, [mutation]));
         var result = await DispatchWithOneUncertaintyRetry(creator, commandId, payload, token).ConfigureAwait(true);
@@ -96,7 +96,7 @@ private DueWorkHint DecodeJob(IJobRunContext context)
     private Task<GrainOperationReply> DispatchTimeoutAsync(DueWorkHint hint,
         PrincipalRecord principal, CancellationToken cancellationToken)
     {
-        var commandId = DueDispatchCommandIdentity.ForDue(hint);
+        var commandId = Guid.NewGuid();
         var payload = OrleansSerialization.Serialize(new CommandRequest(commandId, hint.Lane.Partition,
             [new ExpireSaga(hint.Lane, hint.Id, hint.Revision)]));
         return DispatchWithOneUncertaintyRetry(principal, commandId, payload, cancellationToken);

src/KeyLoad.Clustering/Messaging/Identity/DueDispatchCommandIdentity.cs

@@ -9,7 +9,7 @@ internal PostgresServerCertificate(IOptions<PostgresSessionOptions> options)
     {
         var settings = options.Value;
         settings.Validate();
-        Value = X509CertificateLoader.LoadPkcs12FromFile(settings.CertificatePath, null, X509KeyStorageFlags.EphemeralKeySet);
+        Value = X509CertificateLoader.LoadPkcs12FromFile(settings.CertificatePath, null, KeyStorage);
         if (!Value.HasPrivateKey)
         {
             var failures = new List<Exception>
@@ -26,5 +26,11 @@ internal X509Certificate2 Value
         get;
     }
     public void Dispose() => Value.Dispose();
+
+    // macOS cannot hold a PKCS#12 private key as an ephemeral key set; there the key lives in a temporary keychain
+    // that the certificate removes on disposal.
+    private static X509KeyStorageFlags KeyStorage => OperatingSystem.IsMacOS()
+        ? X509KeyStorageFlags.DefaultKeySet
+        : X509KeyStorageFlags.EphemeralKeySet;
     private const string CertificateInvalid = "Execution SQL requires an owned TLS certificate with a private key.";
 }