Skip to content

3.x: Route SELECT at SERIAL/LOCAL_SERIAL consistency like LWT#891

Merged
dkropachev merged 3 commits into
scylladb:scylla-3.xfrom
nikagra:fix-serial-consistency-lwt-routing
May 22, 2026
Merged

3.x: Route SELECT at SERIAL/LOCAL_SERIAL consistency like LWT#891
dkropachev merged 3 commits into
scylladb:scylla-3.xfrom
nikagra:fix-serial-consistency-lwt-routing

Conversation

@nikagra
Copy link
Copy Markdown

@nikagra nikagra commented May 16, 2026

Note

Fixes DRIVER-616. Related to: #886

Problem

Statements with SERIAL or LOCAL_SERIAL set as their main consistency level (via setConsistencyLevel) are serialised through the Paxos/serial path on the server side — the same path used by LWT writes. However, the 3.x driver only applied LWT load-balancing routing (e.g. PRESERVE_REPLICA_ORDER) to statements where isLWT() returned true, which is driven exclusively by Scylla's server-side prepare metadata.

A normal SELECT prepared or executed with LOCAL_SERIAL consistency:

SimpleStatement simpleSelect =
    new SimpleStatement("SELECT * FROM t WHERE pk=?")
        .setConsistencyLevel(ConsistencyLevel.LOCAL_SERIAL);
PreparedStatement ps = session.prepare(simpleSelect);
BoundStatement bs = ps.bind(1);
// bs.isLWT() == false  → routed as REGULAR, not PRESERVE_REPLICA_ORDER

was routed as REGULAR, ignoring the configured LWT routing method entirely.

Changes

  • TokenAwarePolicy: getRequestRouting() now treats statements with SERIAL or LOCAL_SERIAL main consistency as LWT for routing method selection (same as isLWT()). In PreserveReplicaOrderIterator, remote-DC replicas are excluded when the consistency is LOCAL_SERIAL (preserving local-DC semantics).
  • LatencyAwarePolicy: The latency-reordering bypass that already applied to isLWT() statements is extended to cover SERIAL/LOCAL_SERIAL consistency statements.
  • RackAwareRoundRobinPolicy: The rack-prioritization bypass already in place for isLWT() is extended to cover SERIAL/LOCAL_SERIAL consistency statements.

What is NOT changed

  • setSerialConsistencyLevel() (the paxos-phase serial CL option on LWT writes) is intentionally not affected — only the main setConsistencyLevel() triggers LWT routing.
  • isLWT() semantics are unchanged; this is purely additive.

Tests

  • TokenAwarePolicyTest: 7 new unit tests — serial consistency routed as LWT, LOCAL_SERIAL excludes remote replicas, plain SERIAL keeps remote replicas, serial-CL-option does not trigger LWT routing, null routing-method falls back to regular.
  • LatencyAwarePolicyTest: 1 new unit test — serial consistency statements bypass latency reordering.
  • RackAwareRoundRobinPolicyTest: 1 new unit test — serial consistency statements skip rack prioritization.
  • LWTLoadBalancingIT: New CCM integration test — prepares a SimpleStatement with LOCAL_SERIAL/SERIAL consistency, binds and executes it, verifies coordinator is non-null.

All 67 unit tests pass (mvn test -pl driver-core -Dgroups=unit).

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Extends LWT load-balancing behavior to non-LWT statements that use SERIAL/LOCAL_SERIAL as their main consistency level, so that they are routed through the Paxos-aware path (e.g. PRESERVE_REPLICA_ORDER) instead of as REGULAR queries. The change touches three load-balancing policies and adds unit + CCM integration coverage.

Changes:

  • TokenAwarePolicy.getRequestRouting treats serial-CL statements as LWT for routing; PreserveReplicaOrderIterator filters out remote replicas for LOCAL_SERIAL.
  • LatencyAwarePolicy and RackAwareRoundRobinPolicy extend their existing isLWT() bypasses to cover serial-CL statements via a duplicated hasSerialConsistency helper.
  • New unit tests across the three policies and a new LWTLoadBalancingIT CCM integration test.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
driver-core/src/main/java/com/datastax/driver/core/policies/TokenAwarePolicy.java Route serial-CL statements as LWT; restrict to local replicas only for LOCAL_SERIAL.
driver-core/src/main/java/com/datastax/driver/core/policies/LatencyAwarePolicy.java Bypass latency reordering for serial-CL statements.
driver-core/src/main/java/com/datastax/driver/core/policies/RackAwareRoundRobinPolicy.java Skip rack prioritization for serial-CL statements.
driver-core/src/test/java/com/datastax/driver/core/policies/TokenAwarePolicyTest.java New unit tests covering serial routing, LOCAL_SERIAL local-only filtering, and null routing-method fallback.
driver-core/src/test/java/com/datastax/driver/core/policies/LatencyAwarePolicyTest.java New test verifying serial-CL statements are not latency-reordered.
driver-core/src/test/java/com/datastax/driver/core/policies/RackAwareRoundRobinPolicyTest.java New test verifying serial-CL statements skip rack prioritization.
driver-core/src/test/java/com/datastax/driver/core/policies/LWTLoadBalancingIT.java New CCM integration test for SERIAL/LOCAL_SERIAL SELECT routing.

Comment thread driver-core/src/main/java/com/datastax/driver/core/policies/TokenAwarePolicy.java Outdated
Comment thread driver-core/src/main/java/com/datastax/driver/core/policies/TokenAwarePolicy.java Outdated
Fixes: https://scylladb.atlassian.net/browse/DRIVER-616

Statements with SERIAL or LOCAL_SERIAL as their main consistency level
should be routed through the LWT path (PRESERVE_REPLICA_ORDER) in the
same way as statements where isLWT() returns true.

Changes:
- TokenAwarePolicy: treat statements with SERIAL/LOCAL_SERIAL consistency
  as LWT for routing method selection; exclude remote DC replicas from
  PRESERVE_REPLICA_ORDER plans when consistency is LOCAL_SERIAL
- LatencyAwarePolicy: bypass latency-based reordering for SERIAL/LOCAL_SERIAL
  consistency statements, same as for isLWT() statements
- RackAwareRoundRobinPolicy: skip rack prioritization for SERIAL/LOCAL_SERIAL
  consistency statements, same as for isLWT() statements
- Add unit tests in TokenAwarePolicyTest (7 new), LatencyAwarePolicyTest (1),
  RackAwareRoundRobinPolicyTest (1), and integration test LWTLoadBalancingIT
@nikagra nikagra force-pushed the fix-serial-consistency-lwt-routing branch from 78c7343 to 115a445 Compare May 17, 2026 21:48
@nikagra nikagra requested a review from dkropachev May 17, 2026 23:27
Comment thread driver-core/src/main/java/com/datastax/driver/core/Statement.java Outdated
Comment thread driver-core/src/main/java/com/datastax/driver/core/policies/TokenAwarePolicy.java Outdated
nikagra added 2 commits May 20, 2026 16:25
- Remove Statement.hasSerialConsistency() (too specific for public API);
  inline the serial-CL detection in each policy
- Revert LOCAL_SERIAL local-only filtering in PreserveReplicaOrderIterator;
  remote replicas now appear in the tail for both SERIAL and LOCAL_SERIAL
  (alignment with other drivers deferred to a separate task per reviewer)
- Store ConsistencyLevel (not QueryOptions) in LatencyAwarePolicy;
  expose via private isEffectiveConsistencySerial() helper
- Update TokenAwarePolicyTest: replace two separate LOCAL_SERIAL/SERIAL
  remote-replica tests with a single parameterised test covering both CLs
…ggestion

Move null guard for defaultLwtRequestRoutingMethod to the call site in
newQueryPlan, keeping getRequestRouting() focused on routing logic only:
- isLWT() -> return defaultLwtRequestRoutingMethod immediately
- serial effective CL -> return defaultLwtRequestRoutingMethod
- otherwise -> REGULAR
@dkropachev dkropachev merged commit 82e5ec6 into scylladb:scylla-3.x May 22, 2026
11 checks passed
dkropachev pushed a commit that referenced this pull request May 22, 2026
- Remove Statement.hasSerialConsistency() (too specific for public API);
  inline the serial-CL detection in each policy
- Revert LOCAL_SERIAL local-only filtering in PreserveReplicaOrderIterator;
  remote replicas now appear in the tail for both SERIAL and LOCAL_SERIAL
  (alignment with other drivers deferred to a separate task per reviewer)
- Store ConsistencyLevel (not QueryOptions) in LatencyAwarePolicy;
  expose via private isEffectiveConsistencySerial() helper
- Update TokenAwarePolicyTest: replace two separate LOCAL_SERIAL/SERIAL
  remote-replica tests with a single parameterised test covering both CLs
@nikagra nikagra deleted the fix-serial-consistency-lwt-routing branch May 22, 2026 21:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants