Skip to content

Commit 9e5e70e

Browse files
committed
Align verification with current storage and startup contracts
Keep BM25 local publication oracles tied to the original physical commit cut, unchanged canonical sequence, exact stored outcome bytes and current admission. Check the current MCP offset and lifetime schemas with exact property sets. Correct the comparison host policy path and reject malformed numeric settings without echoing their values. Repair real child-process quoting, original TRX retention, cleanup failure observation and current native argument expectations. Keep all 884 comparison contract cases. Split the same 142 classes into two bounded native selections and select the compiled benchmark configuration only inside the benchmark preparation job. Record open qualification honestly. Verification on macOS arm64: - Complete 22-project Debug build with -warnaserror: 0 warnings, 0 errors. - Native format checks and repository governance passed. - 395 analyzer, 296 current-contract, 20 support, 19 BM25 and 234 process-recovery cases passed. Five isolated Unit startup cases passed with native parallelism 50. - All five selected SDK/MCP cases passed in each configured three-silo storage mode. Native processes joined; owned containers and volumes were removed. - Comparison contracts: 877/884 passed, 0 skipped. The seven remaining failures require Linux permission behavior or Release BenchmarkDotNet artifacts; no local Release build was run and failures remain recorded as failures. Full Unit CI startup/cancellation, ZoneTree issue 190, coverage and exact-source Linux qualification remain open. Focused local passes do not close those gates.
1 parent 3579172 commit 9e5e70e

29 files changed

Lines changed: 189 additions & 82 deletions

‎.github/workflows/benchmarks.yml‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ jobs:
2626
runtime-artifact-id: ${{ steps.runtime-bundle.outputs.artifact-id }}
2727
env:
2828
GH_TOKEN: ${{ github.token }}
29+
Configuration: Release
2930
KEYLOAD_COMPARISON_JOB_NAME: Prepare comparison runtime
3031
steps:
3132
- name: Download source code
@@ -44,8 +45,10 @@ jobs:
4445
run: dotnet format KeyLoad.slnx --verify-no-changes --no-restore
4546
- name: Check repository rules
4647
run: node scripts/Features/RepositoryGovernance/verify.mjs
47-
- name: Test benchmark contracts
48-
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/(BenchmarkSessionConfigurationTests|BenchmarkSessionContractTests|BenchmarkSessionNativeParserTests|BenchmarkSessionProgressTests|BenchmarkTopologyConfigurationTests|BenchmarkTopologyTargetTests|CentralAppHostOptionsTests|ComparisonCorpusTests|ComparisonExceptionTests|ComparisonExecutionIdentityReferenceTests|ComparisonExecutionIdentityTests|ComparisonGraphOracleTests|ComparisonHostBindingsTests|ComparisonHostCleanupTests|ComparisonHostStartupDiagnosticsTests|ComparisonHostStartupTests|ComparisonParallelismSelectionTests|ComparisonSerializationBufferPolicyTests|ComparisonStatisticsTests|ComparisonTopologyTests|CompiledBenchmarkRuntimeBundleTests|DocumentFinalCopyProofTests|DocumentFunctionalGateTests|DocumentNativeAcquisitionTests|EmbeddedBenchmarkContractTests|EmbeddedBenchmarkGeneratedConsumerTests|HttpTargetArgumentTests|ImmutableComparisonContractTests|IsolatedAggregateFileTests|IsolatedAggregateJsonTests|IsolatedAggregateNodeLifetimeTests|IsolatedCurrentJobCacheTests|IsolatedCurrentJobTests|IsolatedGitHubFailureDiagnosticsTests|IsolatedGitHubFileTests|IsolatedGitHubHeaderTests|IsolatedGitHubMetadataTests|IsolatedGitHubRateTests|IsolatedGitHubTransientTests|IsolatedKeyLoadMutationTests|IsolatedKnownCurrentJobTests|IsolatedMutationCorpusTests|IsolatedMutationOracleTests|IsolatedNeo4jMutationTests|IsolatedRedisMutationTests|KeyLoadTimeSeriesIntensiveReceiptTests|KeyLoadTimeSeriesIntensiveResultTests|KeyLoadTimeSeriesIntensiveSeedTests|KeyLoadTimeSeriesIntensiveTopologyTests|KurrentCleanupStateTests|KurrentGossipVersionTests|KurrentSetupDiagnosticFormatterTests|LargeStreamCleanupPermissionTests|NativeComparisonAdapterPolicyTests|NativeComparisonDiagnosticPolicyTests|NativeComparisonDiagnosticProjectionTests|NativeComparisonMongoPolicyTests|NativeComparisonSerializationPolicyTests|NativeComparisonSerializationProjectionTests|NativeDocumentTopologyKurrentTests|NativeDocumentTopologyMongoTests|NativeDocumentTopologyOpenSearchMembershipTests|NativeDocumentTopologyOpenSearchTests|NativeDocumentTopologyPostgresTests|NativeQuorumTopologyConfigurationTests|NativeQuorumTopologyQdrantTests|NativeQuorumTopologyRabbitTests|NativeSerializationBenchmarkCancellationTests|NativeSerializationBenchmarkCorpusTests|NativeSerializationBenchmarkGeneratedConsumerTests|NativeSerializationBenchmarkManifestTests|NativeSerializationBenchmarkMetadataTests|NativeSerializationBenchmarkOwnershipTests|NativeSerializationGitHubDeadlineTests|NativeSerializationGitHubStepTests|NativeSerializationReportValidatorTests|NativeTargetArgumentTests|NativeWorkloadTestReportRetentionTests|OpenSearchHealthPolicyTests|OutboxFailureDiagnosticClassifierTests|OutboxFailureDiagnosticFormatterTests|PostgresVectorPayloadContractTests|PostgresVectorQueryContractTests|QdrantVectorContractTests|RawStorageBenchmarkTests|RawStorageCrudTests|RawStorageLifetimeTests|RawStorageReportValidatorTests|SampleChunkBenchmarkConsumerTests|ScaleServerCgroupCpuFormatTests|ScaleServerHostEvidenceTests|ScaleServerMemoryCurrentFormatTests|ScaleServerResourceBoundsTests|ScaleServerResourceCancellationTests|ScaleServerResourceEvidenceParserTests|ScaledCorpusReadbackTests|ScaledRawStorageBenchmarkTests|ScaledRawStorageBoundsTests|ScaledRawStorageCorpusTests|ScaledRawStorageExecutionPolicyTests|ScaledRawStorageFixtureLifetimeTests|ScaledRawStorageFixtureTests|ScaledRawStorageOwnerTests|ScaledRawStorageProcessMemoryTests|ScaledRawStorageReadOrderTests|ScaledRawStorageSettingsTests|SessionCohortContractTests|SiteQualificationStartupTests|SourceCheckoutIdentityTests|SourceContextContractTests|StreamCorpusTests|TimeSeriesIntensiveAppendOracleTests|TimeSeriesIntensiveAppendValidationTests|TimeSeriesIntensiveAttemptLedgerTests|TimeSeriesIntensiveAttemptOrderTests|TimeSeriesIntensiveCallScopeTests|TimeSeriesIntensiveCompactEncodingTests|TimeSeriesIntensiveCompletionTests|TimeSeriesIntensiveCorpusTests|TimeSeriesIntensiveDigestTests|TimeSeriesIntensiveExceptionBoundaryTests|TimeSeriesIntensiveFailureMappingTests|TimeSeriesIntensiveNativeFailureFactTests|TimeSeriesIntensiveObservedFailureTests|TimeSeriesIntensiveOracleReferenceTests|TimeSeriesIntensiveOracleTests|TimeSeriesIntensivePreparedExpectationTests|TimeSeriesIntensivePreparedPhaseTests|TimeSeriesIntensiveSummaryTests|TimeSeriesIntensiveTagScopeTests|TimeSeriesIntensiveValidatedDigestTests|TimeSeriesIntensiveValidationTests|TimeSeriesIntensiveWorkerPartitionTests|TimeSeriesIntensiveWriterTests|TimescaleTimeSeriesIntensiveContextTests|TimescaleTimeSeriesIntensiveReceiptTests|TimescaleTimeSeriesIntensiveSessionTests|VectorCorpusContractTests|VectorExecutionCadenceTests|VectorIndexContractTests|VectorProfileContractTests|VectorResponseContractTests)/*' --KeyLoadTests:ResultsDirectory=TestResults/benchmark-contracts --KeyLoadTests:ReportTrx=true
48+
- name: Test benchmark contracts (1 of 2)
49+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/(BenchmarkSessionConfigurationTests|BenchmarkSessionContractTests|BenchmarkSessionNativeParserTests|BenchmarkSessionProgressTests|BenchmarkTopologyConfigurationTests|BenchmarkTopologyTargetTests|CentralAppHostOptionsTests|ComparisonCorpusTests|ComparisonExceptionTests|ComparisonExecutionIdentityReferenceTests|ComparisonExecutionIdentityTests|ComparisonGraphOracleTests|ComparisonHostBindingsTests|ComparisonHostCleanupTests|ComparisonHostStartupDiagnosticsTests|ComparisonHostStartupTests|ComparisonParallelismSelectionTests|ComparisonSerializationBufferPolicyTests|ComparisonStatisticsTests|ComparisonTopologyTests|CompiledBenchmarkRuntimeBundleTests|DocumentFinalCopyProofTests|DocumentFunctionalGateTests|DocumentNativeAcquisitionTests|EmbeddedBenchmarkContractTests|EmbeddedBenchmarkGeneratedConsumerTests|HttpTargetArgumentTests|ImmutableComparisonContractTests|IsolatedAggregateFileTests|IsolatedAggregateJsonTests|IsolatedAggregateNodeLifetimeTests|IsolatedCurrentJobCacheTests|IsolatedCurrentJobTests|IsolatedGitHubFailureDiagnosticsTests|IsolatedGitHubFileTests|IsolatedGitHubHeaderTests|IsolatedGitHubMetadataTests|IsolatedGitHubRateTests|IsolatedGitHubTransientTests|IsolatedKeyLoadMutationTests|IsolatedKnownCurrentJobTests|IsolatedMutationCorpusTests|IsolatedMutationOracleTests|IsolatedNeo4jMutationTests|IsolatedRedisMutationTests|KeyLoadTimeSeriesIntensiveReceiptTests|KeyLoadTimeSeriesIntensiveResultTests|KeyLoadTimeSeriesIntensiveSeedTests|KeyLoadTimeSeriesIntensiveTopologyTests|KurrentCleanupStateTests|KurrentGossipVersionTests|KurrentSetupDiagnosticFormatterTests|LargeStreamCleanupPermissionTests|NativeComparisonAdapterPolicyTests|NativeComparisonDiagnosticPolicyTests|NativeComparisonDiagnosticProjectionTests|NativeComparisonMongoPolicyTests|NativeComparisonSerializationPolicyTests|NativeComparisonSerializationProjectionTests|NativeDocumentTopologyKurrentTests|NativeDocumentTopologyMongoTests|NativeDocumentTopologyOpenSearchMembershipTests|NativeDocumentTopologyOpenSearchTests|NativeDocumentTopologyPostgresTests|NativeQuorumTopologyConfigurationTests|NativeQuorumTopologyQdrantTests|NativeQuorumTopologyRabbitTests|NativeSerializationBenchmarkCancellationTests|NativeSerializationBenchmarkCorpusTests|NativeSerializationBenchmarkGeneratedConsumerTests|NativeSerializationBenchmarkManifestTests)/*' --KeyLoadTests:ResultsDirectory=TestResults/benchmark-contracts/first --KeyLoadTests:ReportTrx=true
50+
- name: Test benchmark contracts (2 of 2)
51+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/(NativeSerializationBenchmarkMetadataTests|NativeSerializationBenchmarkOwnershipTests|NativeSerializationGitHubDeadlineTests|NativeSerializationGitHubStepTests|NativeSerializationReportValidatorTests|NativeTargetArgumentTests|NativeWorkloadTestReportRetentionTests|OpenSearchHealthPolicyTests|OutboxFailureDiagnosticClassifierTests|OutboxFailureDiagnosticFormatterTests|PostgresVectorPayloadContractTests|PostgresVectorQueryContractTests|QdrantVectorContractTests|RawStorageBenchmarkTests|RawStorageCrudTests|RawStorageLifetimeTests|RawStorageReportValidatorTests|SampleChunkBenchmarkConsumerTests|ScaleServerCgroupCpuFormatTests|ScaleServerHostEvidenceTests|ScaleServerMemoryCurrentFormatTests|ScaleServerResourceBoundsTests|ScaleServerResourceCancellationTests|ScaleServerResourceEvidenceParserTests|ScaledCorpusReadbackTests|ScaledRawStorageBenchmarkTests|ScaledRawStorageBoundsTests|ScaledRawStorageCorpusTests|ScaledRawStorageExecutionPolicyTests|ScaledRawStorageFixtureLifetimeTests|ScaledRawStorageFixtureTests|ScaledRawStorageOwnerTests|ScaledRawStorageProcessMemoryTests|ScaledRawStorageReadOrderTests|ScaledRawStorageSettingsTests|SessionCohortContractTests|SiteQualificationStartupTests|SourceCheckoutIdentityTests|SourceContextContractTests|StreamCorpusTests|TimeSeriesIntensiveAppendOracleTests|TimeSeriesIntensiveAppendValidationTests|TimeSeriesIntensiveAttemptLedgerTests|TimeSeriesIntensiveAttemptOrderTests|TimeSeriesIntensiveCallScopeTests|TimeSeriesIntensiveCompactEncodingTests|TimeSeriesIntensiveCompletionTests|TimeSeriesIntensiveCorpusTests|TimeSeriesIntensiveDigestTests|TimeSeriesIntensiveExceptionBoundaryTests|TimeSeriesIntensiveFailureMappingTests|TimeSeriesIntensiveNativeFailureFactTests|TimeSeriesIntensiveObservedFailureTests|TimeSeriesIntensiveOracleReferenceTests|TimeSeriesIntensiveOracleTests|TimeSeriesIntensivePreparedExpectationTests|TimeSeriesIntensivePreparedPhaseTests|TimeSeriesIntensiveSummaryTests|TimeSeriesIntensiveTagScopeTests|TimeSeriesIntensiveValidatedDigestTests|TimeSeriesIntensiveValidationTests|TimeSeriesIntensiveWorkerPartitionTests|TimeSeriesIntensiveWriterTests|TimescaleTimeSeriesIntensiveContextTests|TimescaleTimeSeriesIntensiveReceiptTests|TimescaleTimeSeriesIntensiveSessionTests|VectorCorpusContractTests|VectorExecutionCadenceTests|VectorIndexContractTests|VectorProfileContractTests|VectorResponseContractTests)/*' --KeyLoadTests:ResultsDirectory=TestResults/benchmark-contracts/second --KeyLoadTests:ReportTrx=true
4952
- name: Test canonical benchmark worker selection
5053
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/(BenchmarkSessionContractTests|SessionCohortContractTests|CompiledBenchmarkRuntimeBundleTests)/*' --KeyLoadTests:ResultsDirectory=TestResults/session-contracts --KeyLoadTests:ReportTrx=true
5154
- name: Test prompt termination on Aspire resource failures

‎benchmarks/KeyLoad.ComparisonHost/Configuration/NativeComparisonExecutionRegistration.cs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ namespace KeyLoad.ComparisonHost.Features.BenchmarkComparisons;
99
[ConfigurationBinding]
1010
internal static class NativeComparisonExecutionRegistration
1111
{
12-
internal const string ConfigurationPath = "BenchmarkComparisons/Configuration/native-execution.json";
12+
internal const string ConfigurationPath = "Configuration/native-execution.json";
1313

1414
internal static IOptions<NativeComparisonDiagnosticOptions> ReadDiagnostics(IConfiguration configuration)
1515
{

‎benchmarks/KeyLoad.ComparisonHost/Session/Configuration/BenchmarkSessionHostSettings.cs‎

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -24,14 +24,24 @@ private static BenchmarkSessionHostSettings ReadValidated(IConfiguration configu
2424
throw new InvalidOperationException(Profile);
2525
}
2626
var target = IsolatedHostSettings.Required(configuration, BenchmarkSessionSelection.TargetSetting);
27-
var nodes = int.Parse(IsolatedHostSettings.Required(configuration, BenchmarkSessionSelection.NodeCountSetting),
28-
NumberStyles.None, CultureInfo.InvariantCulture);
27+
if (!int.TryParse(IsolatedHostSettings.Required(configuration, BenchmarkSessionSelection.NodeCountSetting),
28+
NumberStyles.None, CultureInfo.InvariantCulture, out var nodes))
29+
{
30+
throw new InvalidOperationException(Profile);
31+
}
2932
var selection = BenchmarkSessionContract.ResourceSelection(target, nodes);
3033
var revision = configuration[ComparisonHostConstants.SourceRevision];
3134
var identity = ComparisonExecutionIdentity.ReadSession(configuration, revision, selection.Topology);
32-
var job = identity.Provenance is null ? (long?)null
33-
: long.Parse(IsolatedHostSettings.Required(configuration, IsolatedHostConstants.JobId),
34-
NumberStyles.None, CultureInfo.InvariantCulture);
35+
long? job = null;
36+
if (identity.Provenance is not null)
37+
{
38+
if (!long.TryParse(IsolatedHostSettings.Required(configuration, IsolatedHostConstants.JobId),
39+
NumberStyles.None, CultureInfo.InvariantCulture, out var parsedJob))
40+
{
41+
throw new InvalidOperationException(Profile);
42+
}
43+
job = parsedJob;
44+
}
3545
var output = IsolatedHostSettings.Required(configuration, ComparisonHostConstants.Output);
3646
var unsupported = BenchmarkSessionContract.UnsupportedTopologies
3747
.SingleOrDefault(item => item.Target == target && item.NodeCounts.Contains(nodes))?.Reason;

‎docs/Features/Search/Bm25ConfiguredCount.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@
22

33
This native single-owner supplement to [Bm25ConfiguredQuota](Bm25ConfiguredQuota.md) isolates atomic apply count admission. It is not fresh capture count isolation, raw native-WAL inventory, process-kill durability or cluster qualification.
44

5-
- REQ-BM25-COUNT001 → AC-BM25-COUNT001: genuinely capture one-document/one-term phases under the original budget, then join and reopen the SAME canonical ZoneTree owner with only MaxScanRecords reduced to 2 or 1. A second persisted subject's phase succeeds at 2. At 1 it returns BudgetExceeded with the exact Capacity detail, persists the native failed outcome and its canonical applied sequence, and leaves every head key/value byte unchanged.
6-
- REQ-BM25-COUNT002 → AC-BM25-COUNT002: original persisted publisher/subjects, source dependencies and issued phase bytes remain authoritative. Compare full native result, actual token, full retained entry and byte-exact outcome across repeated same-ID replay; inspect complete canonical image with the original independently labelled inspection budget.
5+
- REQ-BM25-COUNT001 → AC-BM25-COUNT001: genuinely capture one-document/one-term phases under the original budget, then join and reopen the SAME canonical ZoneTree owner with only MaxScanRecords reduced to 2 or 1. A second persisted subject's phase succeeds at 2. At 1 it returns BudgetExceeded with the exact Capacity detail, persists the native silo-local failed outcome with CanonicalSequence=0, advances physical store position by one without changing the canonical applied read cut, and leaves every head key/value byte unchanged.
6+
- REQ-BM25-COUNT002 → AC-BM25-COUNT002: original persisted publisher/subjects, source dependencies and issued phase bytes remain authoritative. Reissuing a phase must satisfy the current resource-admission budget; at the reduced 1 or 2 record limit it returns the exact resource-admission BudgetExceeded without effects. Retry the original already-issued envelope without recapturing admission and compare its full native result, actual token, full retained entry and byte-exact outcome across repeated same-ID replay. Fresh persisted authorization still runs on every retry. Inspect the complete canonical image with the original independently labelled inspection budget.
77
- REQ-BM25-COUNT003 → AC-BM25-COUNT003: genuine same-key replacement and subsequent healthy publication work at capacity, preserve exact head count, then join and reopen again, preserving original identity and receipts. The shared RecoveryOwner retains original failure aggregation, canonical storage joins and deletion discipline.
88

99
Automated mapping: Bm25ConfiguredCountTests.OriginalIssuedPublicationHonorsLowerRetainedCountAcrossSameOwnerColdReopen(int), arguments 1 and 2; Bm25ConfiguredCountTrial owns the operation and independent raw family/result/entry oracles. Dependency: the shared Bm25OriginalPhaseRecovery owner with Reopen(DatabaseLimits? configuredLimits = null), lower-only validation before join, original inspection bounds and configured engine factory and original canonical storage identity.

‎docs/Features/Search/Bm25OriginalExpiry.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ Parent contract Bm25CorpusCurrent.md REQ/AC-BM25-CURRENT-005 and its determinist
66

77
REQ-BM25-EXPIRY-001 -> AC-BM25-EXPIRY-001: capture two actual original private45 operations through the existing RecoveryFlow before publishing either. Publish the first with its full genuine CommitReceipt. Retain actual NativePayload and full native entry. Wait using the actual database EvaluationClock and CancellationToken until the later original phase expiry, with the interval bounded by the observed original expiry and existing request lifetime. The existing issuer reissues the same original phase only after the clock has actually reached expiry; independently require new operation.EvaluatedAt >= original OriginalExpiry. The already-admitted first operation still returns its exact original complete result/receipt with full canonical image, position, applied and journal CUT unchanged. The original phase bytes are never reconstructed or altered.
88

9-
REQ-BM25-EXPIRY-002 -> AC-BM25-EXPIRY-002: submit the other originally captured but unadmitted phase after its OriginalExpiry. Require actual BudgetExceeded and exact Bm25CorpusProtocol.Changed detail, null Json/NativeValue. This is a genuine committed failed outcome: last/commit/applied advance by one, the actual entry carries its original full phase body, and the retained family keys/values remain unchanged. Do not assert complete canonical or journal no effects: the outcome and apply records are real admitted effects. Same-ID retry returns the exact complete failed result and leaves full post-failure state/selected entry unchanged.
9+
REQ-BM25-EXPIRY-002 -> AC-BM25-EXPIRY-002: submit the other originally captured but unadmitted phase after its OriginalExpiry. Require actual BudgetExceeded and exact Bm25CorpusProtocol.Changed detail, null Json/NativeValue. This is a genuine silo-local committed failed outcome: physical store position advances by one, the canonical applied read cut remains unchanged, and the local stored outcome has CanonicalSequence=0. The actual entry carries its original full phase body, and the retained family keys/values remain unchanged. Do not assert complete store-image or journal no effects: the local outcome is a real admitted effect. Same-ID retry returns the exact complete failed result and leaves full post-failure state/selected entry unchanged.
1010

1111
REQ-BM25-EXPIRY-003 -> AC-BM25-EXPIRY-003: join and reopen the SAME RecoveryOwner once. Preserve complete canonical state, original identities and journal CUT, both selected actual entries and original phase bytes. Fresh current authorization still applies; repeat the already-successful and durably-failed original operation replays exactly without recapture under either ID. Capture a genuinely fresh-ID phase with expected generation equal to the first actual successful head, publish it, and require literal one-document/one-token statistics plus the original literal healthy Search. Existing owner cleanup retains body, fatal and every join/disposal failure; roots survive failure.
1212

0 commit comments

Comments
 (0)