Skip to content

grill-perf: workload deadline ceiling (total_ms ≤ 600000) rejects legitimate long-output cells #16

Description

@plotarmordev

Workload::validate rejects total_ms > 600_000 with invalid deadline or buffering limits. A 400-token thinking-on prose cell on a 2×GB10 GLM serve runs ~20 tok/s per stream; with concurrency 2 and DFlash2 deferral one wave already took 112 s, and a long-context decode cell can legitimately exceed 10 min. Suggest raising the cap (or making it a CLI override) and improving the error to say which limit failed. Evidence: run dir decode-main-3584-attempt1-idle-timeout on our kit, idle_timeout after 120 s with Waiting: 2, Deferred: 2 on the server.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions