feat(upstream): slow start for newly observed upstream nodes

[AlinsRan]

Description

A node that joins an Upstream today takes its full share of traffic immediately. For a JVM that has not JIT-compiled yet, a service with an empty local cache, or one still filling its connection pools, the scale-out meant to add capacity is what drives its latency up or knocks it over. Tuning node weights by hand does not survive autoscaling or rolling releases.

This adds warm_up_conf to the Upstream: a node the gateway observes for the first time takes a reduced share of the traffic and ramps back to its configured weight over slow_start_time_seconds.

{
  "type": "roundrobin",
  "nodes": [{"host": "10.0.0.10", "port": 8080, "weight": 100}],
  "warm_up_conf": {
    "slow_start_time_seconds": 300,
    "min_weight_percent": 1,
    "interval": 1,
    "aggression": 1,
    "startup_grace_period_seconds": 180
  }
}

Without warm_up_conf nothing changes: no state is created, the picker cache key is untouched, and no shared dict is written.

Relation to #12991

#12991 proposed the same feature with the same four core fields and the same weight curve, which this keeps. What it does not keep is where the start of a ramp comes from. There, the Admin API wrote update_time onto each node and persisted it to etcd. That makes the ramp depend on the clock and the honesty of whoever writes the configuration, pollutes the declarative config with runtime state, and does nothing for nodes that never pass through the Admin API - every node from service discovery.

Here the data plane decides on its own. apisix/slow_start.lua compares the node set of each picker build against the one the previous build recorded in a new upstream-slow-start shared dict, and generates every ramp start locally with ngx.now(). Nothing is read from the node configuration, and nothing is written back to etcd.

Lifecycle

• Baseline. The node set an Upstream has when the gateway first builds a picker for it is mature. That covers a cold start, a full resync, and turning warm_up_conf on for an Upstream that is already serving.
• New nodes added afterwards ramp from min_weight_percent over slow_start_time_seconds. A node an active health check keeps out of the picker starts its ramp when it first becomes pickable, not when the configuration first mentioned it.
• Tombstone. A node that leaves the Upstream keeps its state for one window, so coming back within it resumes the ramp. Coming back later, or being kept out of the picker by a health check for longer than a window, starts over - a different process is probably answering on that address.
• Identity is host and port; a domain node is identified by its configured hostname, so weight, priority, metadata, order, and DNS rotation do not restart anything.
• startup_grace_period_seconds keeps nodes that arrive late during a restart mature, so start-up ordering does not manufacture new nodes.

Weight curve

Same as #12991 and Envoy's SlowStartConfig:

elapsed      = clamp(now - ramp_start, 0, slow_start_time_seconds)
time_factor  = max(elapsed, 1) / slow_start_time_seconds
ratio        = max(min_weight_percent / 100, time_factor ^ (1 / aggression))
weight       = floor(configured_weight * ratio)   -- 0 stays 0, otherwise at least 1

Request path

While any node ramps, the picker cache key carries the current interval bucket, so a worker rebuilds the picker once per bucket rather than doing per-node work per request. Once every node is mature the key settles on a stable suffix, and the rebuild that causes is the one that restores the full weights. The hot path does a single shdict:get of an aggregate deadline. A request and its retries keep the same picker.

Consistency across workers

Workers reconcile in parallel and the shared dict has no compare-and-set, so state is never read and then written back blindly:

• Presence and eligibility are tracked separately. Presence comes from the configuration, identical in every worker, and drives the snapshot and the tombstone. Eligibility is each worker's own health view, and only decides when a ramp starts or is interrupted. Tombstoning off the eligible set would let one worker's transient health opinion drop a node the others still serve.
• A ramp start is only created with add, and only replaced through an election - an add on a key naming the start being replaced, whose winner every other worker adopts. Refreshes use expire, which leaves the value alone. This avoids a lock on purpose: create_server_picker can be reached from the balancer phase on a retry, where a sleeping lock is not allowed.

Scope

This first step ramps HTTP roundrobin Upstreams whose nodes share one priority. Anything else is rejected at the Admin API instead of being accepted and silently ignored:

• another balancer type;
• nodes with different priorities - the highest tier is drained first, so a ramp inside one tier cannot hold traffic back;
• interval greater than slow_start_time_seconds;
• a stream route that reaches such an Upstream, directly, through an embedded one, or through a service - plus the reverse check when warm_up_conf is added to an Upstream a stream route already reaches;
• the Upstreams of the traffic-split plugin, which are rebuilt per request and have no stable scope.

These checks run on the configuration entry points only. A configuration written to etcd directly, or embedded in a route or a service, never runs them, and must not take a whole Upstream out of service over a field that only shapes a ramp. At runtime the module therefore fails open: in the stream subsystem, with mixed priorities, or without the shared dict, nodes keep their configured weights and one error is logged per Upstream.

Worth knowing

• A picker rebuild restarts the round-robin cursor, which favours the heaviest node for its first picks. It averages out over a bucket with real traffic; on an Upstream seeing only a handful of requests per interval, a ramping node can get even less than its weight asks for.
• A ramp only moves traffic between nodes. A single-node Upstream, or one whose nodes are all new, still sends every request to them.
• Every APISIX instance ramps from the moment it observed the node; instances are not synchronised.
• upstream-slow-start defaults to 10m. Its size is bounded by the nodes of Upstreams that enable slow start, and eviction fails open: a node whose state is lost keeps its configured weight.

Which issue(s) this PR fixes:

Related to #7992 and #10832, and supersedes #12991.

Checklist

• I have explained the need for this PR and the problem it solves
• I have explained the changes or the new features added to this PR
• I have added tests corresponding to this change
• I have updated the documentation to reflect this change
• I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

Tests

• t/node/upstream-slow-start.t: the weight curve and its clamping; a node added to a running Upstream ramping while the existing one stays at full weight; the ramp ending at the configured weights; an unrelated change not restarting it; the tombstone resuming a ramp; a route-embedded Upstream as its own scope; the picker key moving once per interval and then settling; a node an active health check sidelines keeping its lifecycle; a sidelined node starting its window on first entry; and two workers racing on the same state, replayed deterministically by injecting another worker's write between a read and the write that follows it.
• t/admin/upstream-slow-start.t: field ranges, defaults, every rejection above in both directions, an unreadable etcd reference failing validation rather than the request, declarative validation through /apisix/admin/configs/validate, the data plane keeping an Upstream whose warm_up_conf it cannot honour, and enabling warm_up_conf in a cluster with no stream route at all.