Skip to content

Gantry: 4 - Bound please_pull and pump fan-out #280

@vpatelsj

Description

@vpatelsj

Scope

Bound please_pull and pump fan-out. Do not mix this with peer authorization or repository validation.

Problem

please_pull has no digest cap, discards the dispatch context in the pump path, and has no global pull cap. This allows authorized-but-buggy peers, compromised in-mesh peers, or malformed internal callers to amplify a single request into many background pulls.

Evidence:

  • internal/gantry/coord/coord.go:504
  • cmd/gantry/main.go:1933
  • cmd/gantry/main.go:1957
  • cmd/gantry/main.go:1999

Changes

  • Add explicit config for coord bounds: coord_max_digests_per_request and coord_max_concurrent_pulls.
  • Derive and document defaults. A starting point is 256 digests/request and 16 concurrent pulls, but final values must be justified against manifest child counts, envelope size, node memory/file-descriptor budgets, and DaemonSet resource limits.
  • Cap inbound please_pull digest count in servePleasePull.
  • If len(req.GetDigests()) exceeds the cap, bump OnStreamError and reject the request instead of partially processing it.
  • Add client-side chunking in coord.Client.PleasePull and prefetch/local callers so legitimate internal batches do not exceed the server cap.
  • Respect the dispatch deadline in servePleasePull and StartLocalPull.
  • Use the pump context for the local Has probe in newPullerPump.
  • Add an internal pump outcome instead of overloading booleans.
  • When the global pull semaphore is full, return pumpDeclined and map it to OUTCOME_UNSPECIFIED on the current wire protocol. Do not map it to ALREADY_PULLING.
  • Add a global concurrent-pull semaphore in newPullerPump.

Tests

  • Oversized remote please_pull is rejected and increments stream-error metrics.
  • Client-side PleasePull chunking respects the cap.
  • Cancelled contexts stop remote and local digest loops.
  • Pull semaphore caps concurrent pulls; the N+1 request produces OUTCOME_UNSPECIFIED, not ALREADY_PULLING.
  • Fuzz or property tests cover malformed and oversized PleasePullRequest envelopes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions