feat(ts-sdk): jurisdiction support for math standards alignment#108
feat(ts-sdk): jurisdiction support for math standards alignment#108adnanrhussain wants to merge 4 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
Adds jurisdiction-aware standards alignment to the TypeScript math evaluator and Knowledge Graph client so callers must explicitly choose which state/adopted framework to use (instead of silently defaulting to Multi-State/CCSS).
Changes:
- Introduces a shared
Jurisdictionenum in the KG client layer and re-exports it for SDK consumers. - Updates math standards alignment evaluator entry points to require
jurisdiction, removesgradefrom the single-questionevaluate()public signature, and threadsacademicSubjectinto KG calls. - Adjusts KG standard lookup behavior to use
limit=1and take the first match; adds framework UUID lookup/caching for non–Multi-State jurisdictions.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| sdks/typescript/tests/unit/knowledge-graph/client.test.ts | Updates KG client behavior expectations for multi-result standard lookups. |
| sdks/typescript/tests/unit/evaluators/math/standards-alignment.test.ts | Updates evaluator tests for new signatures and verifies jurisdiction/subject are passed to KG. |
| sdks/typescript/src/prompts/math/standards-alignment/index.ts | Removes grade as a system-prompt placeholder input. |
| sdks/typescript/src/knowledge-graph/types.ts | Adds Jurisdiction enum (string values matching KG API). |
| sdks/typescript/src/knowledge-graph/index.ts | Re-exports Jurisdiction and option types from KG client. |
| sdks/typescript/src/knowledge-graph/client.ts | Adds jurisdiction/subject-aware options, standard lookup limit=1 behavior, and framework UUID caching for standards-by-grade. |
| sdks/typescript/src/evaluators/math/standards-alignment.ts | Makes jurisdiction required across public entry points, removes grade from evaluate(), and passes subject/jurisdiction to KG. |
| evals/prompts/math/standards-alignment/user.txt | Removes grade from the user prompt template. |
| evals/prompts/math/standards-alignment/system.txt | Removes grade placeholder from system prompt template. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
… Copilot findings
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
| Decision standard: | ||
| * Prefer true when the item clearly elicits the target mathematical work. | ||
| * Prefer false when alignment depends on speculation, related-but-different skills, or only partial overlap. | ||
| You are an expert in K-12 math academic standards. |
There was a problem hiding this comment.
Making this consistent back w/ the source notebook
| @@ -1,9 +1,100 @@ | |||
| Evaluate the assessment question below against each of the {n} learning components listed. Return a JSON object with an "evaluations" array. Each entry must include the "lc_id" field copied exactly from the identifier shown in brackets — this is how we verify the result maps to the right learning component. | |||
There was a problem hiding this comment.
Making this consistent back w/ the source notebook. Minimal changes enable evaluating multiple learning components, vs a single component
…ers from config.json
| lc_id: z.string(), | ||
| reasoning: z.string(), | ||
| aligned: z.boolean(), | ||
| answer: z.enum(['Yes', 'No']), |
There was a problem hiding this comment.
[P2] Been trying to establish a standard output shape for evaluation results. Could it work for this?
evaluators/sdks/python/src/learning_commons_evaluators/schemas/evaluator.py
Lines 194 to 220 in 70362e4
There was a problem hiding this comment.
the client is not generated, since I have some caching, concurrency limits, etc. But I could do a followup and break that apart, and just have the generated faw fetch and the wrappers on top
Summary
Jurisdictionenum (52 values, matching KG API) to the KG client layer so future evaluators can reuse itjurisdictiona mandatory top-level parameter on all evaluator entry points (evaluate,evaluateItems,evaluateQuestionBank,evaluateByGrade) — forces explicit choice, no silent Multi-State defaultacademicSubjectis passed by the caller; math evaluator passes'Mathematics'internally viaKG_SUBJECTlimit=1+ take-first for standard lookups — eliminates deduplication logic and matches API design intentgradefrom the publicevaluate()signature and from system/user prompt templates; grade is still recorded internally when available (viaevaluateItems/evaluateQuestionBank) but isgrade?: stringonStandardAlignmentResult/standards-frameworksTest plan
npm test— 278 pass, 0 failnpm run lint— 0 errorsevaluate(question, statementCode, Jurisdiction.California)passesjurisdiction=CaliforniaandacademicSubject=Mathematicsto the KG search endpointevaluateByGrade(questions, '3', Jurisdiction.California)calls/standards-frameworks?jurisdiction=California&academicSubject=Mathematicsto get the correct framework UUID