From e74212a90933791c0262c85c12b9d973d4c5d4ed Mon Sep 17 00:00:00 2001 From: Alp Date: Mon, 13 Jul 2026 09:33:14 -0700 Subject: [PATCH] Add scoring developer README notes --- README.md | 5 +++++ examples/README.md | 13 +++++++++++++ scoring_engine/README.md | 22 ++++++++++++++++++++++ 3 files changed, 40 insertions(+) create mode 100644 examples/README.md create mode 100644 scoring_engine/README.md diff --git a/README.md b/README.md index ab15e85..600e181 100644 --- a/README.md +++ b/README.md @@ -40,3 +40,8 @@ python -m scoring_engine.cli examples/anonymized_conversation.json The baseline outputs dimension scores, confidence values, evidence keywords, and a rubric version. It must only receive anonymized records. + +## Developer Notes + +- [scoring_engine/README.md](scoring_engine/README.md) explains the scorer package. +- [examples/README.md](examples/README.md) explains the synthetic scoring fixture. diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 0000000..2e25a43 --- /dev/null +++ b/examples/README.md @@ -0,0 +1,13 @@ +# Scoring Examples + +This folder holds synthetic anonymized scoring inputs. + +`anonymized_conversation.json` is shaped like the anonymization service output and can be +used with the CLI: + +```bash +python -m scoring_engine.cli examples/anonymized_conversation.json +``` + +Examples in this repo should never contain raw contributor conversations or identifying +details. diff --git a/scoring_engine/README.md b/scoring_engine/README.md new file mode 100644 index 0000000..b4f9b8b --- /dev/null +++ b/scoring_engine/README.md @@ -0,0 +1,22 @@ +# Scoring Engine Package + +This package scores anonymized conversation records. It must not receive raw submissions. + +## File Map + +- `baseline.py`: transparent keyword baseline and scoring output shape. +- `cli.py`: command-line wrapper around the baseline scorer. +- `__init__.py`: exports `score_record` for tests and future callers. + +## Current Baseline + +The scorer is deterministic and intentionally simple. It checks anonymized message text +for rubric keywords and returns dimension scores, confidence values, evidence keywords, +and `validated_for_claims: false`. + +This is useful for wiring the pipeline together. It is not a validated research model. + +## First Place To Edit + +Add or adjust dimensions in `baseline.py`, then add synthetic tests in +`tests/test_baseline.py`. Keep examples anonymized.