Merge antalya-26.6-20261008 into antalya-26.6 by filimonov · Pull Request #2644 · Altinity/ClickHouse

filimonov · 2026-10-22T22:11:24Z

Merge the temporary branch antalya-26.6-20261008 back into antalya-26.6. It brings these CAS pull requests:

Date a GC mount-token sighting after the read that returned it #2509 GC dates a mount-token sighting after the read that returned it, so a slow walk no longer counts as watch time.
Return the last Indeterminate when a CAS probe pause ends on the deadline #2510 probeSentinel returns the last Indeterminate outcome when its pause ends on the deadline, instead of throwing.
Keep the CAS mount lease renewal alive over the server memory limit #2511 Mount lease renewal survives max_server_memory_usage; MEMORY_LIMIT_EXCEEDED no longer fences the mount.
Accept only a strictly greater epoch when claiming a CAS mount slot #2512 Claiming a mount slot accepts only a strictly greater writer_epoch.
Adopt a CAS blob only when its marker reads Clean #2552 A writer adopts an existing blob only when its .meta marker reads Clean after the HEAD.
Show CAS mount renewal progress in system.cas_mounts #2513 system.cas_mounts shows renewal progress while a renewal retries.
Wait for the lease a CAS mount holder declares before fencing or reclaiming its slot #2516 Observers wait for the lease the slot holder declares before fencing or reclaiming its slot.
Never resend a CAS blob's object-creating S3 request #2555 Object-creating blob requests are sent at most once; the publish loop retries with a fresh envelope.
Renew the CAS mount lease until a definitive answer; an expired lease is a pause #2517 Mount lease renewal retries until a definitive answer; an expired lease pauses writes instead of losing the mount.
CAS: refuse new nested server_root_id, delete S3 staging on abort, actionable member refusals #2566 cas_server_root_id with / is refused, S3 staging is deleted on abort, member refusals say what to do.
Merge antalya-26.6 into antalya-26.6-20261008 #2614 Synced antalya-26.6 into the temporary branch.

Merges cleanly into the current antalya-26.6 (git merge-tree). Merge with a merge commit.

Changelog category (leave one):

Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

CAS GC no longer fences a mount whose token it watched for less than the observation threshold after a slow walk to the slot: the first sighting is dated after the read that returned the token (#2509 by @filimonov)
The CAS sentinel probe returns its last inconclusive outcome instead of throwing when a retry pause ends exactly on the policy deadline (#2510 by @filimonov)
A server memory limit no longer ends the CAS mount lease renewal thread; the conditional PUT of a CAS control-plane write runs on the calling thread with a buffer of the body size (#2511 by @filimonov)
A CAS mount claim accepts only a strictly greater writer_epoch: it no longer refreshes its own pair in the slot or replaces a newer incarnation of the same server, and a zero epoch counter is rejected (#2512 by @filimonov)
Fixes a rare data loss on CAS (content-addressed storage) disks: an insert that reused an existing blob at the moment GC deleted it could commit a reference to the deleted blob, and later reads of that part failed. The writer now uploads the blob again when its freshness marker is missing instead of reusing it. (#2552 by @filimonov)
system.cas_mounts gains renewal_in_progress, renewal_attempts and last_renewal_error, updated while a CAS mount lease renewal runs. (#2513 by @filimonov)
CAS observers (GC fencing, restart and remount reclaim) wait for the lease a mount holder declares in its slot, so a longer cas_mount_lease_ttl_ms on one server is never fenced early by a server with a shorter one; the setting is per server and mixed values are safe. (#2516 by @filimonov)
Fixes a rare data loss on CAS (content-addressed storage) disks backed by S3 stores with content-derived ETags (GCS is unaffected): a retried blob upload could bring back a blob that garbage collection had already scheduled for deletion, and the delayed deletion then removed a blob still in use. Blob uploads no longer resend the request that creates the object; CAS retries with a new upload instead. Under a store outage an insert may now fail with a retryable error after 8 attempts or once the 90-second deadline for starting attempts passes. (#2555 by @filimonov)
A CAS mount lease renewal retries until the object store gives a definitive answer instead of giving up at a deadline. While the lease is too short to admit a write, writes pause and resume under the same epoch once a renewal is confirmed, instead of losing the mount and remounting. (#2517 by @filimonov)
CAS: cas_server_root_id may not contain /. A server whose id contains / (for example from a {shard}/{replica} template) refuses to start the disk and must be recreated under a new id without /; data stored under the old id is not visible under the new one. An aborted transaction deletes its S3 staging object. SYSTEM CAS DROP POOL MEMBER and the owner refusals name the state or command that resolves them. (#2566 by @filimonov)

Documentation entry for user-facing changes

...

CI/CD Options

Exclude tests:

Regression jobs to run:

The GC credited the first sighting of a mount token with a clock sample taken before the LIST of gc/server-roots/ and the slot read. A slow walk then counted as time the token was watched, and a later round could write gc_fenced over a holder whose last renewal still gave it authority. Both callers of computeHeartbeatFloor (the regular round and the rebuild) now pass the observer clock as a function; the sighting is sampled after the read, including the re-decision after a refused fence-out. The stability test keeps its entry sample, and a sample older than the sighting never counts as time held, so the elapsed time cannot wrap. Closes backlog task CAS-342. Related: #2431 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

probeSentinel reissues an Indeterminate probe after a jittered pause and decides before sleeping whether the pause plus one request still fits the policy deadline. The check accepts a pause that ends exactly on the deadline; the loop head then saw the deadline reached and threw "gave up at the policy deadline" although a probe had run and its outcome was known. One pause in about five thousand landed there, which showed as a rare failure of CASSentinelProbe.NativeClassifiesUnmodeledErrorAsIndeterminate. Keep the last probe outcome and return it when the deadline is reached after a probe; the admission gate (a lost fence, no lease budget) still throws at any attempt, and a deadline reached before the first probe still throws. No request is sent after the deadline. Closes backlog task CAS-325. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

The renewal thread allocated under the global memory tracker, so a server near max_server_memory_usage could throw MEMORY_LIMIT_EXCEEDED inside a renewal; the loop's catch-all then fenced the mount and ended the thread for good, and the mount went through a remount under a fresh epoch because of memory pressure that had nothing to do with the object store. Hold LockMemoryExceptionInThread(VariableContext::Global) for the whole renewal iteration: the tracker still counts this thread's allocations but never throws on them. Run the conditional PUT on the calling thread (s3_allow_parallel_part_upload = false for conditional writes), so the guard covers the upload; open the PUT buffer at the body size instead of the default 1 MiB. std::bad_alloc stays terminal. Closes: #2403 Closes backlog task CAS-292. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Recovery seals only the epochs below the live one, so the slot must always hold the newest incarnation of a server_uuid and nothing older may be written under that uuid. claimMount broke this in three places: finding its own (uuid, epoch) in the slot it "refreshed" the body, which let a second process share the incarnation that another process had been allocated; an older allocation that reached the slot after a newer epoch could wait for a death certificate and replace it, and that write would never be sealed; and a next_writer_epoch of 0 (the encoding's absence value) was accepted when the counter was decoded. A claim now writes only above the slot's epoch: a newer own epoch in the slot throws ABORTED without a write; the own pair in the slot is never rewritten (gc_fenced → FencedSelf, otherwise LiveDoubleStart with the holder's body, and the caller takes a fresh epoch); a raced write that meets a newer own epoch fails at once instead of waiting for a certificate that cannot come; a zero next_writer_epoch is rejected. Nothing changes on the store. Closes backlog task CAS-298 (the claim part). Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

While a lease renewal retries, system.cas_mounts showed nothing: an aging last_renewed_at, lifecycle still Live, and the attempt counters moved only after the call ended. An operator could not tell a renewal retrying through an object-store outage from a renewal thread that had died. Add three columns updated as the renewal runs: renewal_in_progress, renewal_attempts (physical PUTs sent by the current call) and last_renewal_error (verb, key and error class of the last refused request; no credentials, no bodies). The request engine gets a RequestObserver with on_write_attempt and on_failure callbacks; the renewal thread keeps a RenewalProgress record under a leaf mutex and counts CASMountRenewalAttempts/Retries per attempt instead of from the result. A local failure of the resolve read is not published. Nothing changes on the store. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…aiming its slot Every observer waited its own cas_mount_lease_ttl_ms before treating an unchanged slot token as a dead holder: the GC floor, the same-server restart and remount reclaim. A holder configured with a longer lease was fenced inside its own right to write when any observer ran with a shorter value, and nothing in the protocol could tell. Observers now wait max(own lease, declared lease) + 5 % + their own poll step, where the declared lease is expires_at_ms - started_at_ms of the body they observe (0 when the stamps are missing, equal, or reversed, so the wait is never shorter than before). The formula stays in one place, mountObservationThresholdMs, and saturates instead of wrapping. The setting is per server: how long this server may write after its last confirmed renewal; every other server waits at least that long. Mixed values are safe in any order. The GC leader logs one WARNING per slot whose declared lease differs from its own; fenced and departed slots are not warned about. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…ng-after-read Date a GC mount-token sighting after the read that returned it

…tinel-boundary Return the last Indeterminate when a CAS probe pause ends on the deadline

…emory-guard Keep the CAS mount lease renewal alive over the server memory limit

`PartWriteTxn::ensureBlobPresent` reads the body with `HEAD`, then the marker with a separate `GET`. GC deletes the marker right after the body, so a writer whose `GET` landed after both deletes found no marker, backfilled `Clean` and adopted a deleted body. An absent marker now proves nothing: the body is republished as if absent and the reconcile creates the marker. `CASMetaAdoptBackfill` is replaced by `CASMetaAbsentRepublish`. A publication that landed but lost its response now costs one more upload. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

The graduation gate's retry stamps the refusing round, later than `entry.condemn_round`, so `deleteConfirmedMeta` bounded by the entry's round left that `Condemned` marker over a deleted body forever. Bound the delete by the round before the deleting one. Removing a marker never lets a writer adopt, since adoption needs a `Clean` marker. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Fix the `newest_round` argument comment that clang-tidy rejects. Seed a `Clean` marker in the two remaining `CASProtocol` tests that relied on adopting a body without one. Correct the user docs, the README and the comments that still said an absent marker permits adoption or that the marker delete stops at the entry's round. `CASMetaAbsentRepublish` counts publication attempts, not uploads. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…im-monotonicity Accept only a strictly greater epoch when claiming a CAS mount slot

…ocol The redelete diagram still named the old bound; the delete job removes markers below the deleting round. A failed delete job can still leave a `Condemned` marker behind, so the text no longer says none outlives its body. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

`copyObject` passed the disk's request settings to `copyS3File` unchanged, so `s3_max_unexpected_write_error_retries_override` bounded the `NO_SUCH_KEY` loop of `writeObject` but not the identical `CopyObject` loop. Apply it to a local copy of the request settings, as `writeObject` does. Add a counting S3 test server and an `S3ObjectStorage` test factory for per-verb wire counts. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

A resent `PutObject`, `CopyObject` or `CompleteMultipartUpload` that the store already applied can add a version on a versioned bucket, emit a second `ObjectCreated` event, or land after a newer write of the key and overwrite it. A writer that cannot accept that sets `WriteSettings::object_storage_single_attempt_object_creation`. - `S3ObjectStorage::writeObject` and `copyObject` then pass a second client to `WriteBufferFromS3` and `copyS3File`: a single-attempt clone with the configured timeouts. Adaptive first-attempt timeouts assume a second attempt, which this client never makes. - Only the creating requests use it. `CreateMultipartUpload`, `UploadPart` and `UploadPartCopy` keep the retrying client. - A copy must use `ObjectStorageCopyMode::NativeOnly`; `copyS3File` throws `LOGICAL_ERROR` otherwise, because the read-write fallback would write through the retrying client. - `getSingleAttemptClient` takes `configured_timeouts` as part of its cache key, so existing single-attempt callers keep their clones unchanged. Writes without the flag are unchanged. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

`Client::CompleteMultipartUpload` turns `NoSuchUpload` plus a successful `HEAD` of the key into success, on the theory that an earlier attempt of the same request completed the upload and its response was lost. A single-attempt client sent the request once, so the object at the key may come from another writer. Return the original error for such a client. Skip the optional GCS self-compose for it too: the compose rewrites the object as a new generation, a second creating request. Clients with the retrying strategy are unchanged. The change also applies to existing single-attempt callers. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Blob publication ran with default `WriteSettings`, so the S3 client and the `NO_SUCH_KEY` loops could resend an identical envelope. On a content-ETag store a resend that lands after GC deleted that incarnation restores its ETag, and a late exact-token delete then removes a body another writer republished. `ObjectStorageBackend::publish` now sets `object_storage_single_attempt_object_creation` and a retry override of 1 for both the streaming upload and the staged copy. `ensureBlobPresent` already retries with a fresh envelope after re-observing the key. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

`SYSTEM CAS DROP POOL MEMBER 'a'` and the staging sweep select objects by key prefix, so a member `a/b` was drained together with `a`. The first claim of an id (owner absent) now refuses an id containing `/`; `validateServerRootId` keeps accepting one so members that already own a nested id keep mounting. The adopt-from-lease decommission fallback goes through the same claim and is refused for a nested id before any object is touched. Tests: `CASServerRootId.NewIdWithSlashIsRefused`, `CASDecommission.LeaseAdoptFallbackRefusesANestedIdWithoutOwner` (red on the base: the owner was created), guard `CASServerRootId.ExistingIdWithSlashStillMounts`. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>

An aborted transaction left its S3 staging object until the next mount of the same `server_root_id`; the comment calling it a re-readable publication source was wrong, nothing reads `pending_blobs` after the cleanup. Commit and abort now delete it alike, best-effort. Incomplete multipart uploads are invisible to object LIST, so the docs state the bucket lifecycle rule `AbortIncompleteMultipartUpload`. Tests: `CASS3Staging.AbortDeletesS3Staging` (red on the base: one object remains), guard `CASS3Staging.AbortAfterStorageShutdownDoesNotThrow`. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The drain lists `<pool_prefix>/staging/<srid>/` as a logical key and the backend applies the local root itself, while the writer mints keys through `physicalKey`. Two guard tests stage a blob through a real transaction, model a lost abort delete, and decommission the member over an EmulatedSingleProcess (root-prefixed) and a Native (verbatim) backend. A third pins that the next mount's sweep reclaims the same object. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The foreign-owner refusal advised a hand delete that the next start refuses over existing data; it now names `SYSTEM CAS DROP POOL MEMBER` for a dead owner and the hand delete last. The retired-owner refusal advised clearing the tombstone, which would let writer epochs repeat; it now says to configure a new `cas_server_root_id`. The `NoWait` refusal said "wait for its lease to lapse", but the claim trusts no wall clock: it now names the terminal states (`terminated`, `fenced`) and how GC reaches `fenced` for a hard-killed member. Tests: `CASServerRootClaim.RefusalsNameTheSupportedRecovery`, `CASDecommission.NoWaitRefusalNamesTheTerminalStates` (red on the base: substrings absent). The substrings the integration test matches on (`alive or contended`, `explicitly decommissioned`) are kept. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Fencing happens only in a round run by the disk's GC leader, after it has seen the mount token unchanged for about one lease TTL plus one renew period. The refusal text and the docs said any live member's next round. They now name the leader and `is_leader` in `system.cas_mounts`. The test also pins `alive or contended` and `state = 'terminated'`. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The three CASServerRootSafety owner-claim expectations now assert code and message, so they pass only through the emptiness check that runs before the nesting rule. The post-shutdown abort test makes the delete throw so it reaches the catch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>

…ifecycle PR Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>

…-only-clean-marker Adopt a CAS blob only when its marker reads Clean

Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Observers wait for the lease a holder declares, so an hour-long `cas_mount_lease_ttl_ms` would block reclaim of a silently dead member for an hour. Clamp the configured value to 600000 ms when the settings load, with one warning naming the configured and effective values. The renewal deadline, the slot's `expires_at_ms` and the observer's own threshold all read the clamped setting. The observer side does not clamp what a holder declares. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The GC leader fences an expired mount lease only at the start of a GC round, and under load a round took 183-245 s. The decommission refusal and the docs now say so. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ers-wait-declared-lease

…-wait-declared-lease Wait for the lease a CAS mount holder declares before fencing or reclaiming its slot

The S3-to-S3 shortcut ignored the write settings: it neither applied the unexpected-write-error retry override nor used the single-attempt creation client, so a lost CopyObject was resent. It now mirrors copyObject, including the NativeOnly requirement. A non-S3 destination is rejected under the flag (NOT_IMPLEMENTED in NativeOnly mode). Docs: the 90-second figure is a deadline for starting attempts, not a wall-clock bound; a started attempt can run longer. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…e-attempt-blob-creation-split Never resend a CAS blob's object-creating S3 request

Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Decommission and the staging sweep select a member's objects by key prefix, so a member `a/b` would be drained by a decommission of `a`. An owner-claim check covered only new ids; a nested id with no ref catalog entry yet still let `a` claim over it. `validateServerRootId` now refuses '/' for every id, so the disk config, `Pool::open` and `SYSTEM CAS DROP POOL MEMBER` all refuse it, and the claim-time check is gone. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The base changed `mountObservationThresholdMs` and `computeHeartbeatFloor` to take the observer and declared leases; the test kept the old signatures and broke the build: https://altinity-build-artifacts.s3.amazonaws.com/json.html?PR=2517&sha=d410298e2c4717738d9c3c8adc055ab99a240b01&name_0=PR&name_1=Build%20%28amd_debug%29 Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ewal-until-definitive Renew the CAS mount lease until a definitive answer; an expired lease is a pause

…fecycle-pr-a CAS: refuse new nested server_root_id, delete S3 staging on abort, actionable member refusals

Brings 16 merges from antalya-26.6, among them #2521 (null type in Iceberg `MODIFY COLUMN`, the `IcebergMetadataGenerator` UBSan red on every PR of this base), #2529 (Alpine upgrade), #2483 (CAS GC janitor time budget) and the version bump to 26.6.8.20002. `docs/en/antalya/cas/configuration.md`: keep both edits, the janitor batch deletes in `cas_gc_io_concurrency` and the blob-publication note in `cas_attempt_timeout_ms`. Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Merge antalya-26.6 into antalya-26.6-20261008

github-actions · 2026-10-22T22:12:29Z

Workflow [PR], commit [97ac101]

mkmkme · 2026-10-24T07:11:11Z

@blau-ai

blau-ai · 2026-10-24T07:16:27Z

CI triage for #2644

Verdict: 0 PR-caused. Every red check is either a process gate or a pre-existing/flaky/infra failure. This PR is a merge of the dated branch antalya-26.6-20261008 back into antalya-26.6, and its diff is entirely CAS metadata-storage + S3ObjectStorage + docs/ProfileEvents — no Processors/, aggregation, or swarm code is touched, which is the key fact for the failures below.

Check	Class	PR-caused?
DCO	missing sign-off (process)	No
Stateless (amd_asan_ubsan, distributed plan, 1/2)	known aggregation crash	No
Stateless (amd_asan_ubsan, cas s3 storage, 1/2)	same crash	No
Stateless (amd_binary, cas s3 storage)	`DROPPED` / infra	No
Regression release swarms	flaky resilience scenario	No (see caveat)
PR	aggregate gate (rolls up the above)	No

1. DCO — process, author action needed

3 commits lack a Signed-off-by line (every other commit in the branch is fine):

bc7f453 Date a GC mount-token sighting after the read that returned it
58debc8 Return the last Indeterminate when a probe pause ends on the deadline
949578b Keep the CAS mount lease renewal alive over the server memory limit

Not a code problem. Fix by adding a DCO remediation commit, or git rebase --signoff + force-push the branch. I can't do this — it needs a force-push to the PR's head branch, which is outside my permissions; the commit author has to.

2 & 3. Two stateless `fail`s — same known aggregation crash, not this PR

Both report Failed: 0 real test failures; they go red only because the post-run fatal-error scan found a server crash:

Logical error: 'GroupingAggregatedTransform pushed bucket 5 twice' (STID: 3994-4c1d)
  src/Processors/Transforms/MergingAggregatedMemoryEfficientTransform.cpp:63
  -> GroupingAggregatedTransform::tryPushTwoLevelData() -> prepare()

Identical STID 3994-4c1d in both suites — one crash, surfaced twice. This is the two-level distributed-aggregation engine, which this PR does not touch. It's a known upstream CI crash: ClickHouse/ClickHouse#115986 "[CI crash] GroupingAggregatedTransform pushData failure" (open, crash-ci, comp-aggregation), same family as the closed ClickHouse#115663 and ClickHouse#116483 ('Bucket N is pushed twice (STID: …)').
Next step: safe to re-run; the crash is pre-existing in the engine and independent of the merge.

4. Stateless (amd_binary, cas s3 storage, parallel) — dropped

Status is DROPPED (never executed — its sibling in the group crashed/aborted the slot), not a test failure. Re-run.

5. Regression release swarms — flaky resilience scenario

1 scenario failed / 1535 (14 skipped), in the node failure feature — the failing scenario is check restart swarm node, assertion raised inside run_long_query (swarms/tests/node_failure.py:108). These node-failure scenarios deliberately restart nodes / overload CPU / fill disk and are among the most timing-sensitive in the regression suite. It exercises distributed query resilience, not the CAS metadata layer.
Caveat / next step: this PR does change S3ObjectStorage single-attempt object creation, so I can't rule it out with 100% confidence from static reasoning alone. But the failing scenario is about node-restart resilience, not object creation, and it's 1/1535 — consistent with a flake. Re-run the swarm job; if check restart swarm node reproduces, pull swarms/.../node_failure/test.log for the exact assertion before digging further.

Bottom line: nothing here points at the CAS/S3 changes. Re-run the two stateless suites, the dropped binary suite, and the swarm job; handle DCO by signing off the three commits above. If a re-run comes back clean on 2–5, only DCO remains and it's purely a sign-off fix.

filimonov and others added 30 commits October 8, 2026 10:27

Merge pull request #2509 from Altinity/fix/antalya-26.6/cas-gc-sighti…

2f34581

…ng-after-read Date a GC mount-token sighting after the read that returned it

Merge pull request #2510 from Altinity/fix/antalya-26.6/cas-probe-sen…

78cad40

…tinel-boundary Return the last Indeterminate when a CAS probe pause ends on the deadline

Merge pull request #2511 from Altinity/fix/antalya-26.6/cas-renewal-m…

d76b59f

…emory-guard Keep the CAS mount lease renewal alive over the server memory limit

Merge pull request #2512 from Altinity/fix/antalya-26.6/cas-mount-cla…

2a034cd

…im-monotonicity Accept only a strictly greater epoch when claiming a CAS mount slot

Document the limits of the nested server_root_id rule

0b224dd

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>

Fix prose and refusal texts found in the final review of the member l…

e749b58

…ifecycle PR Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com>

Merge pull request #2552 from Altinity/fix/antalya-26.6/cas-360-adopt…

fef96d5

…-only-clean-marker Adopt a CAS blob only when its marker reads Clean

Merge antalya-26.6-20261008 into the CAS mount renewal progress branch

76373d2

Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Annotate the CAS mount renewal progress with its mutex

ba17785

Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

filimonov and others added 15 commits October 16, 2026 06:45

Merge branch 'antalya-26.6-20261008' into fix/antalya-26.6/cas-observ…

459dbfd

…ers-wait-declared-lease

Merge pull request #2516 from Altinity/fix/antalya-26.6/cas-observers…

21b7170

…-wait-declared-lease Wait for the lease a CAS mount holder declares before fencing or reclaiming its slot

Merge pull request #2555 from Altinity/fix/antalya-26.6/cas-361-singl…

8fefc27

…e-attempt-blob-creation-split Never resend a CAS blob's object-creating S3 request

Merge antalya-26.6-20261008 into the CAS renewal until-definitive branch

d410298

Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Merge antalya-26.6-20261008 into cas-member-lifecycle-pr-a

47475fe

Document the global '/' rule for cas_server_root_id

23d515b

Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Drop the claim that a server_root_id may contain '/' from the parser

596fced

Signed-off-by: Mikhail Filimonov <mfilimonov@altinity.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Merge pull request #2517 from Altinity/fix/antalya-26.6/cas-mount-ren…

9e54034

…ewal-until-definitive Renew the CAS mount lease until a definitive answer; an expired lease is a pause

Merge pull request #2566 from Altinity/fix/antalya-26.6/cas-member-li…

af662dc

…fecycle-pr-a CAS: refuse new nested server_root_id, delete S3 staging on abort, actionable member refusals

Merge pull request #2614 from Altinity/merge/antalya-26.6-into-20261008

97ac101

Merge antalya-26.6 into antalya-26.6-20261008

alsugiliazova mentioned this pull request Oct 23, 2026

CAS: packed parts #2579

Merged

10 of 30 tasks

mkmkme merged commit c4e9bc7 into antalya-26.6 Oct 24, 2026
549 of 558 checks passed

mkmkme added antalya antalya-26.6 CAS port-antalya PRs to be ported to all new Antalya releases labels Oct 24, 2026

zvonand mentioned this pull request Oct 24, 2026

Port graph for antalya-26.8 #2322

Open

59 tasks done

svb-alt added the antalya-26.6.8.20003 label Oct 27, 2026

blau-ai mentioned this pull request Oct 28, 2026

Antalya-26:6 - iceberg v3: rewriteFiles - Added logic to merge small files into bins #2420

Merged

20 of 30 tasks

k-morozov self-requested a review October 28, 2026 16:06

k-morozov approved these changes Oct 28, 2026

View reviewed changes

mkmkme added the verified Approved for release label Oct 29, 2026

Navigation Menu

Repository navigation

Merge antalya-26.6-20261008 into antalya-26.6 - #2644

Merge antalya-26.6-20261008 into antalya-26.6#2644
mkmkme merged 61 commits into
antalya-26.6from
antalya-26.6-20261008

filimonov commented Oct 22, 2026 •

edited by mkmkme

Loading

Uh oh!

github-actions Bot commented Oct 22, 2026

Uh oh!

mkmkme commented Oct 24, 2026

Uh oh!

blau-ai commented Oct 24, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

filimonov commented Oct 22, 2026 • edited by mkmkme Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog category (leave one):

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Documentation entry for user-facing changes

CI/CD Options

Exclude tests:

Regression jobs to run:

Uh oh!

github-actions Bot commented Oct 22, 2026

Uh oh!

mkmkme commented Oct 24, 2026

Uh oh!

blau-ai commented Oct 24, 2026

CI triage for #2644

1. DCO — process, author action needed

2 & 3. Two stateless fails — same known aggregation crash, not this PR

4. Stateless (amd_binary, cas s3 storage, parallel) — dropped

5. Regression release swarms — flaky resilience scenario

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

filimonov commented Oct 22, 2026 •

edited by mkmkme

Loading

2 & 3. Two stateless `fail`s — same known aggregation crash, not this PR