Research: Model Lifecycle Telemetry as a UX Signal for Local RAG

[AccessiT3ch]

Context

Based on findings from Study 2a, model latency and RAM consumption correlate strongly with retrieval accuracy and generation quality in local RAG environments.

Research Questions

• How can model lifecycle telemetry (RAM drift, TTFT, TPS) be used as a proxy for inference quality?
• Can we define a UX signal for local RAG that predicts failure modes before they manifest in the output?
• What are the thresholds for RAM swap that trigger a forced fallback to a smaller quantized variant?

Acceptance Criteria

• Research doc (D4 format) in docs/research/ synthesizing the correlation between hardware telemetry and RAG accuracy.
• Recommended threshold matrix for model swapping based on local resource constraints.
• Prototype script for real-time telemetry extraction during local inference.