Skip to content

Commit 1c7445f

Browse files
committed
Replace per-cell benchmarks with shared native database sessions
Replace the 1,898-worker comparison pipeline with two measurement jobs per database, using one live Aspire topology for separate native TUnit cases. Measure the initial million-record load and bounded mixed operation streams without warmup, reseeding or per-case infrastructure restarts. Remove superseded planners, dispatchers, runners, report schemas/readers and their exclusive fixtures. Preserve native adapters, correctness oracles, historical evidence and required functional RF3 qualification. Build and authenticate a common compiled runner bundle before fan-out. Retain original per-case outcomes, native reports, source/image/runtime identities, failed/null results and explicit native capability/topology unavailability. Publication requires a successful, complete authenticated cohort. This is a source checkpoint. The new comparison and site tests, real Linux sessions and complete model-family qualification remain open. Initial session coverage contains seven document cases; it does not qualify the vector, graph, queue, event or time-series families.
1 parent 340e78c commit 1c7445f

663 files changed

Lines changed: 8344 additions & 32867 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# FunctionalHeavyLoadSetup
2+
3+
Prepare the existing KeyLoad functional RF3 million-record ingestion gate with its exact-source images and native runner. Follow root and workflow policies. This action runs no workload and produces no measurements. Preserve the original native admission/workload steps and original result upload as top-level Benchmarks job steps for authenticated provider accounting. No database reset or additional measurement matrix.
Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
name: Functional RF3 heavy-load setup
2+
description: Prepare source-bound images and the existing native functional RF3 runner.
3+
inputs:
4+
bundle-artifact-id:
5+
description: Same-source immutable Docker image bundle
6+
required: true
7+
runs:
8+
using: composite
9+
steps:
10+
- name: Set up .NET
11+
uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68
12+
with:
13+
global-json-file: global.json
14+
- name: Check Docker
15+
shell: bash
16+
run: docker version
17+
- name: Download exact source Docker images
18+
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131
19+
with:
20+
artifact-ids: ${{ inputs.bundle-artifact-id }}
21+
path: ${{ runner.temp }}/keyload-image-bundle
22+
- name: Load and verify Docker images
23+
id: images
24+
shell: bash
25+
run: node scripts/Features/BenchmarkComparisons/import-images.mjs --bundle="$RUNNER_TEMP/keyload-image-bundle"
26+
- name: Configure exact source KeyLoad image
27+
env:
28+
KEYLOAD_SERVER_IMAGE: ${{ steps.images.outputs.server-image }}
29+
shell: bash
30+
run: |
31+
test -n "$KEYLOAD_SERVER_IMAGE"
32+
printf 'KeyLoad__ContainerImages__Server=%s\nKEYLOAD_IMAGE_RECEIPT=%s\n' "$KEYLOAD_SERVER_IMAGE" "$RUNNER_TEMP/keyload-images/image-receipt.json" >> "$GITHUB_ENV"
33+
- name: Restore .NET packages
34+
shell: bash
35+
run: dotnet restore KeyLoad.slnx
36+
- name: Build functional RF3 tests
37+
shell: bash
38+
run: dotnet build tests/KeyLoad.IntegrationTests --no-restore --configuration Release

‎.github/workflows/Features/BenchmarkComparisons/IsolatedCellSetup/AGENTS.md‎

Lines changed: 0 additions & 20 deletions
This file was deleted.

‎.github/workflows/Features/BenchmarkComparisons/IsolatedCellTeardown/AGENTS.md‎

Lines changed: 0 additions & 22 deletions
This file was deleted.

‎.github/workflows/Features/BenchmarkComparisons/QualifySite/action.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -82,12 +82,12 @@ runs:
8282
git -C control rev-parse HEAD > "$EVIDENCE_DIR/control-revision.txt"
8383
closure="scripts/Features/BenchmarkComparisons/site-isolated-dependencies.txt"
8484
cmp "control/$closure" "website/$closure"
85-
[[ "$(wc -l < "control/$closure")" == 84 ]]
85+
node control/scripts/Features/BenchmarkComparisons/session-source-closure.mjs --verify="control/$closure" > "$EVIDENCE_DIR/session-source-closure.json"
8686
LC_ALL=C sort -u "control/$closure" > "$EVIDENCE_DIR/isolated-dependencies.txt"
8787
cmp "control/$closure" "$EVIDENCE_DIR/isolated-dependencies.txt"
8888
sha256sum "website/$closure" >> "$EVIDENCE_DIR/executed-evidence-tools.sha256"
8989
while IFS= read -r relative; do
90-
[[ "$relative" =~ ^(scripts/Features/BenchmarkComparisons/[^/]+\.mjs|site/Features/BenchmarkComparisons/[^/]+\.mjs|site/scripts/build\.mjs|benchmarks/KeyLoad.Comparisons/Features/BenchmarkComparisons/(isolated-contract|Documents/document-contract)\.json)$ ]]
90+
[[ "$relative" =~ ^(scripts/Features/BenchmarkComparisons/[^/]+\.(mjs|ps1)|scripts/Features/CodeQuality/functional-coverage\.(shared|parse|native-merge\.trx)\.ps1|site/Features/BenchmarkComparisons/[^/]+\.mjs|site/scripts/build\.mjs)$ ]]
9191
for root in control website; do
9292
[[ -f "$root/$relative" && ! -L "$root/$relative" ]]
9393
[[ "$(realpath -e "$root/$relative")" == "$GITHUB_WORKSPACE/$root/$relative" ]]
@@ -125,7 +125,7 @@ runs:
125125
dotnet build src/KeyLoad.AppHost/KeyLoad.AppHost.csproj --no-restore --configuration Release
126126
dotnet build tests/KeyLoad.ComparisonTests/KeyLoad.ComparisonTests.csproj --no-restore --configuration Release
127127
node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/SiteQualificationStartupTests/*' --KeyLoadTests:ReportTrx=true "--KeyLoadTests:ResultsDirectory=$EVIDENCE_DIR/startup-tests"
128-
node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/SiteOptionalBenchmarkSelectionTests/*' --KeyLoadTests:ReportTrx=true "--KeyLoadTests:ResultsDirectory=$EVIDENCE_DIR/selection-tests"
128+
node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/SessionCohortContractTests/*' --KeyLoadTests:ReportTrx=true "--KeyLoadTests:ResultsDirectory=$EVIDENCE_DIR/selection-tests"
129129
dotnet build tests/KeyLoad.Analyzers.Tests/KeyLoad.Analyzers.Tests.csproj --no-restore --configuration Release
130130
mkdir -p "$EVIDENCE_DIR/analyzer-coverage"
131131
cp scripts/Features/CodeQuality/site-analyzer-coverage.settings.xml "$EVIDENCE_DIR/analyzer-coverage/coverage.config.xml"
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# SessionContracts
2+
3+
Owns the shared native benchmark preparation checks in `action.yml`. Read the root and workflow policies and ADR-142. Execute only inside Linux Benchmarks; preserve original image, topology, readiness, failure, cancellation and workload assertions. This composite never produces website measurements. Root integrates shared workflow changes.
4+
5+
Every filter must name retained TUnit test classes. The comparison entry requires at least one discovered test; an empty filter is a failure, never a skipped qualification. Shared-session process admission is exercised by `ComparisonHostStartupTests`, `ComparisonHostCleanupTests` and `BenchmarkSessionNativeParserTests`. `BenchmarkSessionProgressTests` uses an explicitly controlled development target to check acknowledgement, cancellation and failed-oracle accounting; it is not native database performance evidence. `DocumentComparisonNativeTests` retains real SurrealDB/HelixDB shared-session correctness and cancellation against small functional corpora.
6+
7+
The native resource-model group retains all current database topology, credential, private-volume and foreign-selection checks. Native shutdown tests retain original primary/cleanup failures and joined resource/log ownership. Keep MongoDB readiness, pinned-image inspection and KeyLoad resource/admission gates independent.
8+
9+
The three removed `IsolatedTimeSeries*ResourceTests` classes belonged exclusively to retired TimeSeries benchmark dispatch. Do not retain their empty filters or substitute pgvector for TimescaleDB. `TimeSeriesIntensivePinnedImageTests`, `TimeSeriesWorkloadTests` and `TimeSeriesPackageVersionTests` remain active. Native Timescale family deployment, `TimeSeriesForeignSchemaCollision` and `TimescaleTimeSeriesIntensiveSchemaAtomicityRegression` remain pending their actual future family integration; pinned image facts do not qualify them.
Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
name: Native benchmark preparation contracts
2+
description: Run the complete native preparation checks once before database sessions.
3+
runs:
4+
using: composite
5+
steps:
6+
- name: Check native KurrentDB discovery settings
7+
shell: bash
8+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/IsolatedKurrentDiscoverySettingsTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/kurrent-discovery
9+
- name: Check exact OpenSearch vector query contract
10+
shell: bash
11+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/OpenSearchVectorQueryTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/opensearch-query
12+
- name: Check exact OpenSearch vector response contract
13+
shell: bash
14+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/OpenSearchVectorResponseTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/opensearch-response
15+
- name: Check TimescaleDB Docker image and startup tools
16+
env:
17+
KEYLOAD_TIMESERIES_IMAGE_FACTS_DIRECTORY: ${{ runner.temp }}/keyload-timeseries-image-facts
18+
shell: bash
19+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/TimeSeriesIntensivePinnedImageTests/*' --KeyLoadTests:ResultsDirectory=TestResults/timeseries-image
20+
- name: Save TimescaleDB image checks
21+
if: always()
22+
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
23+
with:
24+
name: timeseries-native-pinned-image-facts
25+
path: |
26+
${{ runner.temp }}/keyload-timeseries-image-facts/**
27+
**/TestResults/**
28+
artifacts/code-quality/**
29+
if-no-files-found: error
30+
retention-days: 90
31+
- name: Test database cluster resource models
32+
shell: bash
33+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/Isolated*Resource*/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/resource-models
34+
- name: Test native session shutdown and original failure retention
35+
shell: bash
36+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/(IsolatedNativeTeardownSuccessTests|IsolatedNativeTeardownFailureTests|IsolatedNativeLogCaptureConcurrentDisposeTests)/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/session-shutdown --KeyLoadTests:ReportTrx=true
37+
- name: Test MongoDB readiness configuration
38+
shell: bash
39+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/MongoReadinessResourceTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/mongo-resource
40+
- name: Test MongoDB node identity
41+
shell: bash
42+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/MongoReadinessIdentityTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/mongo-identity
43+
- name: Test MongoDB startup failure handling
44+
shell: bash
45+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/MongoReadinessTaskFailureTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/mongo-task-failure
46+
- name: Test KurrentDB storage failure handling
47+
shell: bash
48+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/IsolatedKurrentVolumeRegressionFailureTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/kurrent-volume-failure
49+
- name: Test KeyLoad request admission
50+
shell: bash
51+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/IsolatedKeyLoadAdmissionTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/keyload-admission
52+
- name: Test bounded database log buffers
53+
shell: bash
54+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/ComparisonResourceLogBufferTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/log-buffer
55+
- name: Test live benchmark progress capture
56+
shell: bash
57+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/ComparisonProgress*/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/live-progress
58+
- name: Test shared native session entry and safe failures
59+
shell: bash
60+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/(ComparisonHostStartupTests|ComparisonHostCleanupTests|BenchmarkSessionNativeParserTests)/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/live-entry
61+
- name: Test shared session live acknowledgement and cancellation counters
62+
shell: bash
63+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/BenchmarkSessionProgressTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/live-counters
64+
- name: Test document resource budgets and current native topology
65+
shell: bash
66+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/(DocumentResourceBudgetTests|NativeTopologyAdmissionTests)/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/document-admission --KeyLoadTests:ReportTrx=true
67+
- name: Test controlled native shared-session operations and cancellation
68+
shell: bash
69+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/DocumentComparisonNativeTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/document-native --KeyLoadTests:ReportTrx=true
70+
- name: Test KeyLoad replay admission
71+
shell: bash
72+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/IsolatedKeyLoadReplayAdmissionTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/replay-admission
73+
- name: Test Redis replica error reporting
74+
shell: bash
75+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/RedisReplicaDiagnosticTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/redis-diagnostics
76+
- name: Test MongoDB startup error reporting
77+
shell: bash
78+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/MongoBootstrapDiagnosticTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/mongo-diagnostics
79+
- name: Test KurrentDB cleanup error reporting
80+
shell: bash
81+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/KurrentCleanupDiagnosticTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/kurrent-diagnostics
82+
- name: Test KurrentDB stream ownership
83+
shell: bash
84+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/KurrentStreamOwnershipTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/kurrent-stream-ownership
85+
- name: Test database replay logs
86+
shell: bash
87+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/ComparisonReplayDiagnosticLogTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/replay-log
88+
- name: Test database replay log retention
89+
shell: bash
90+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/ComparisonReplayDiagnosticRetentionTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/replay-retention
91+
- name: Test time-series workload correctness
92+
shell: bash
93+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/TimeSeriesWorkloadTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/timeseries-workload
94+
- name: Check published time-series package version
95+
shell: bash
96+
run: node scripts/Features/TestInfrastructure/run-tests.mjs --KeyLoadTests:Suite=comparison '--KeyLoadTests:Filter=/*/*/TimeSeriesPackageVersionTests/*' --KeyLoadTests:ResultsDirectory=TestResults/comparison-images/timeseries-package
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# SessionSetup
2+
3+
Owns setup for one native database/topology session under ADR-142. Follow root and workflow policies. Authenticate the same-run job, import original source-bound native images and the verified common compiled runtime, and preserve original import receipts. Never restore, build, reset a database, run a workload or manufacture results here. Linux GitHub execution only; scoped image registry shutdown belongs to the existing native lifetime.

0 commit comments

Comments
 (0)