fix(metrics): extend LLM latency histogram buckets

[y4ho0]

What

• add dedicated bucket boundaries for model-call and end-to-end LLM latency histograms
• preserve the OpenTelemetry SDK boundaries through 10 seconds and extend them to 15 seconds, 30 seconds, 60 seconds, 2 minutes, and 5 minutes
• verify that both Prometheus histograms expose the five-minute bucket

Why

The OpenTelemetry SDK defaults stop at 10 seconds, so slower model calls fall into the +Inf bucket. That makes Prometheus quantiles unable to distinguish a 10-second response from a multi-minute LLM generation.

The dedicated view keeps the existing short-latency resolution while making the long tail observable.

Closes #262

How tested

• cargo fmt --all -- --check
• cargo clippy --workspace --all-targets -- -D warnings
• cargo test -p switchyard-server — 56 tests passed
• cargo test --workspace --exclude switchyard-py — 490 tests and doctests passed

A full workspace run was also attempted. It reached the unchanged switchyard-py cdylib and stopped on local macOS CPython symbol linking; all non-PyO3 workspace tests passed.

Checklist

• Regression coverage added for the changed metrics behavior
• No public API or metric label changes
• Commit signed off per the DCO

Notes for reviewers

The boundaries are fixed to keep this bug fix focused, matching the existing routing-overhead view. Making them configurable can remain a separate change if operators need deployment-specific ranges.

Summary by CodeRabbit

• New Features

  • Improved latency metrics to capture model-call and end-to-end response times across a broader range, including delays up to five minutes.
  • Added minimum and maximum latency measurements for more complete performance monitoring.

• Tests

  • Added coverage confirming five-minute latency buckets are reported correctly.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 71753d45-a26e-43e6-920f-1fd90818fca0

📥 Commits

Reviewing files that changed from the base of the PR and between 48b3b71 and 76bcd19.

📒 Files selected for processing (2)

• crates/switchyard-server/src/metrics.rs
• crates/switchyard-server/tests/server.rs

---

Walkthrough

The metrics provider adds explicit latency buckets through five minutes for model-call and total LLM latency histograms. The view records minimum and maximum values. Server metrics tests verify the new bucket.

Changes

LLM latency metrics

Layer / File(s)	Summary
Configure LLM latency histogram view crates/switchyard-server/src/metrics.rs	Adds extended latency boundaries and registers a view for switchyard.model_call_latency_ms and switchyard.total_latency_ms. The view records minimum and maximum values.
Validate exported latency buckets crates/switchyard-server/tests/server.rs	Checks that both exported histograms include the 300000 millisecond bucket.

Estimated code review effort: 2 (Simple) | ~10 minutes

Poem

I’m a rabbit counting time,
Five-minute hops now fit the line.
Model calls leave clearer tracks,
Total latency answers back.
Min and max sit by the gate,
While tests confirm the bucket’s weight.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the main change: extending LLM latency histogram buckets.
Linked Issues check	✅ Passed	The changes address issue #262 by extending both LLM latency histograms through 300000 ms while preserving routing-overhead behavior.
Out of Scope Changes check	✅ Passed	The changes are limited to the requested histogram configuration and regression coverage for LLM latency metrics.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

---

_{Comment @coderabbitai help to get the list of available commands.}

[ayushag-nv]

@grahamking Can you review this one ?