Add significant_figures and decimal_places grading parameters - #277
Open
m-messer wants to merge 7 commits into
Open
Add significant_figures and decimal_places grading parameters#277m-messer wants to merge 7 commits into
m-messer wants to merge 7 commits into
Conversation
…ation and documentation
…rameter in evaluation, tests, and documentation
…it systems in evaluation
Resolved conflicts by keeping main's post-merge-base work (bracket matching from #265/#274, atol unit-factor scaling from #272, order operator fix from #275) while layering in the significant-figures feature (sig_figs / sig_figs_tol). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PndQqZ1ZR2QQ1sYH3eNybZ
…(`dp_tol`) parameters in evaluation, with tests and documentation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Custom criteria had no way to grade a numeric response to a fixed number of significant
figures or decimal places — both for correctness and for the precision the response was
actually written to (e.g. rejecting
3.1or3.14159when3.14is asked for). There wasalso no fuzzy variant that derives a numerical tolerance from a sig-fig/decimal-place count
without checking written precision.
This PR merges in #273 (
significant_figures/sig_figsandsignificant_figures_tolerance/sig_figs_tol, still open/unmerged) and adds its decimal-placecounterpart,
decimal_places/dpanddecimal_places_tolerance/dp_tol, reusing the samenumeric-string helpers. The two features are combined in one branch here because
decimal_places builds directly on the sig_figs helpers/integration points, and #273's branch
had drifted behind
main's bracket-matching work (#265/#274) and atol unit-factor fix (#272) —merging it in first and resolving those conflicts made it possible to build decimal_places on a
consistent base rather than duplicating the shared code path across two divergent branches.
Changes
app/utility/expression_utilities.py: significant-figures helpers (round_to_sig_figs,split_numeric_string,count_sig_figs,sig_figs_match,relative_tolerance_from_sig_figs) from Add support for significant figures (sig_figs) parameter #273, plus new decimal-place helpers(
round_to_decimal_places,count_decimal_places,decimal_places_match,absolute_tolerance_from_decimal_places) built on top of them.app/evaluation.py: parsessignificant_figures/sig_figs,significant_figures_tolerance/sig_figs_tol,decimal_places/dpanddecimal_places_tolerance/dp_tol; all four are mutually exclusive with each other and withatol/rtol;sig_figs_tolwritesrtol,dp_tolwritesatol.app/context/symbolic.py,app/context/physical_quantity.py: integratesig_figs/dpatthe point each context decides a direct
response = answernumeric match (before theordinary equality/tolerance logic, since a value that's numerically equal but written to the
wrong precision must still fail).
app/docs/user.md,app/docs/dev.md: user-facing parameter docs and developer notes for allfour parameters, including the caveat that
decimal_placeswithphysical_quantitycomparesat the answer's SI scale (so is intended for same-unit responses), unlike the scale-invariant
significant_figures.app/tests/expression_utilities_test.py,app/tests/symbolic_evaluation_test.py,app/tests/physical_quantity_evaluation_test.pyforevery new helper and both parameters (core cases, aliases, mutual-exclusion, invalid-value
validation, physical-quantity/unit interaction).
Checklist
app/docs.md) if user-facing behaviour changedCloses #
🤖 Generated with Claude Code
https://claude.ai/code/session_01PndQqZ1ZR2QQ1sYH3eNybZ