Skip to content

Commit afc8ad0

Browse files
committed
chore(kpi): freeze the baseline — 52.6% P@1 macro, labels 12336a824b18
The comparison point every future ranker or content change is measured from. Frozen from a clean tree at ranker 64c0d5f, content e02b754, so it is reproducible by anyone who checks out those two commits and runs `npm run kpi`. P@1 52.6% macro (95% CI [42.7, 62.5]) / 61.6% micro MRR@target 64.5% / 72.5% recall@6 85.9% / 90.4% nDCG@10 63.8% / 74.3% target never returned 3.8%, buried 11-50 3.8%, reference page in top 6 66.7% by source natural 63.1%, seo 81.3%, question 43.5%, intent 100.0%, peer 71.4% 985 headline queries over 42 macro groups, with the 246 `seo` keywords measured and stored but held out of the headline. Read a future delta against this with `npm run kpi -- --compare scripts/search-kpi/data/gold-baseline.json`, which reports McNemar on P@1 and a paired bootstrap CI on MRR@target rather than two means — and which refuses outright if the labels have moved, because a delta across two different label sets is not a result.
1 parent e02b754 commit afc8ad0

1 file changed

Lines changed: 10 additions & 0 deletions

File tree

scripts/search-kpi/data/gold-baseline.json

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,15 @@
11
{
22
"fingerprint": "12336a824b18",
3+
"provenance": {
4+
"ranker": {
5+
"sha": "64c0d5f",
6+
"dirty": false
7+
},
8+
"site": {
9+
"sha": "e02b754",
10+
"dirty": false
11+
}
12+
},
313
"minTopic": 5,
414
"source": null,
515
"macro": {

0 commit comments

Comments
 (0)