Skip to content

grill-perf: reported-prefix-hit ineligibility is correct but needs a hint about cache alignment #17

Description

@plotarmordev

On a hybrid-KV vLLM serve (GLM-5.3-Flash, block-aligned prefix cache with a 3584-token hit grid) a reported-prefix-hit workload with ~2K prompts reports cached_prompt_tokens: 0 on every measured turn, so every wave is provider_reported_prefix_cache_miss and the run is completed-with-ineligible-measurements. The verdict is right, but the doc should say that reported hits depend on server block/hit alignment and recommend observe or prompts ≥ the server's alignment for warm-decode cells. Evidence: 12/12 waves ineligible until switching to observe; same server hit 7168 cached tokens on an 8.7K prompt.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions