Context
Registry charts publish the mean of a small number of runs (three for registry variations). One outlier can dominate the daily point: a single 294-second run produced a roughly 100-second mean during the incident. Standard deviation and median are already present in the raw data.
This is a methodology decision rather than a one-line display change. Options include:
- plot median;
- retain mean and add error bands/confidence information;
- separate run 0/cold behavior from steady-state runs.
This should compose with the partial-failure work so timeout survivors are not treated as complete samples.
Acceptance criteria
Source
Performance investigation: item 7 — mean-of-three amplifies outliers
Context
Registry charts publish the mean of a small number of runs (three for registry variations). One outlier can dominate the daily point: a single 294-second run produced a roughly 100-second mean during the incident. Standard deviation and median are already present in the raw data.
This is a methodology decision rather than a one-line display change. Options include:
This should compose with the partial-failure work so timeout survivors are not treated as complete samples.
Acceptance criteria
Source
Performance investigation: item 7 — mean-of-three amplifies outliers