feat: response areas with evaluation-function parameters, confirmed in the wizard

in2lambda can now carry a response area (input type, reference answer,
evaluation function, and its parameters) from the source document through
to the imported Lambda Feedback JSON, instead of always emitting
"responseAreas": [].

- api: new `ResponseArea` dataclass; `Part.response_areas`;
  `Question.add_response_area()`.
- response_areas/: registry of the 14 documented evaluation functions
  (plus evaluatePython / callLLM) with their response types and parameter
  specs, and per-type `config` defaults. Unknown functions/params are
  passed through untouched.
- json_convert: emits populated `responseAreas` from a new
  `minimal_template_response_area.json`, mirroring the first answer into
  `answerContent` when empty. Parts with none still emit [].
- Markdown filter: a ```lambda-feedback fenced JSON block under a
  ## Solution becomes a ResponseArea; `validation` warns (non-fatally) on
  malformed blocks.
- wizard: the LLM proposes a response area per part from the worked
  solution; `in2lambda wizard` stops on each for [a]ccept / [e]dit /
  [s]kip (edit menu driven by the registry), then writes the confirmed
  block into the markdown. New --yes / --no-response-areas flags; the
  prompt is skipped when not attached to a terminal.

The emitted shape is verified byte-for-byte against real exported
questions (tests/fixtures/response_areas/).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXaWoYhb2yCFwv6xCmK3zT

Author: m-messer

conftest.py

@@ -10,4 +10,5 @@
     collect_ignore = [
         "in2lambda/wizard/extract.py",
         "in2lambda/wizard/run.py",
+        "in2lambda/wizard/confirm.py",
     ]

docs/source/contributing/response-areas.md

@@ -0,0 +1,79 @@
+# Response Areas
+
+A **response area** is where a student answers part of a question and how that
+answer is auto-marked. in2lambda can carry response areas from the source
+document all the way into the imported Lambda Feedback JSON.
+
+## The model
+
+{class}`in2lambda.api.response_area.ResponseArea` holds four things:
+
+| Field | Meaning | Import JSON |
+| --- | --- | --- |
+| `response_type` | input widget - `EXPRESSION`, `NUMBER`, `NUMERIC_UNITS`, `BOOLEAN`, `TEXT`, `ESSAY`, `CODE`, ... | `response.responseInput.responseType` |
+| `answer` | the reference answer, as a string | `response.responseInput.answer` |
+| `evaluation_function` | which evaluation function grades the response | `evaluationFunctionName` |
+| `grade_params` | that function's parameters | `gradeParams` |
+
+`config` (extra `responseInput` settings, e.g. `{"language": "python"}` for
+`CODE`) and `pre_response_text` / `post_response_text` round it out. Everything
+else in the on-import entry - feedback colours, `inputSymbols`, `tests`,
+`cases`, ... - comes from
+`in2lambda/json_convert/minimal_template_response_area.json` and is not something
+a filter sets.
+
+A filter attaches one with
+{meth}`~in2lambda.api.question.Question.add_response_area`:
+
+```python
+from in2lambda.api.response_area import ResponseArea
+
+set.current_question.add_response_area(
+    ResponseArea("EXPRESSION", "pi*d", "compareExpressions", {"rtol": 0.01})
+)
+```
+
+Parts with no response area import exactly as before, with `"responseAreas": []`.
+
+## The evaluation-function registry
+
+{data}`in2lambda.response_areas.EVALUATION_FUNCTIONS` is in2lambda's model of the
+functions documented in the [Lambda Feedback evaluation-function
+reference](https://docs.lambdafeedback.com/teacher/reference/evaluation_functions/).
+Each entry lists the response types it pairs with and its parameters (name,
+type, default, help). It drives what `in2lambda wizard` proposes and what its
+confirmation prompt offers.
+
+`compareExpressions` and `comparePhysicalQuantities` have complete parameter
+lists; the others carry their common parameters and a `TODO` pointing at the
+per-function docs page. Nothing is *enforced* - `json_convert` passes an unknown
+function or parameter straight through, so a newer function still imports, it
+just is not offered in the menu.
+
+## The `lambda-feedback` block
+
+The [`Markdown` filter](../filters/_autosummary/Markdown) reads a fenced code
+block whose info string is `lambda-feedback`, placed under the part or its
+`## Solution`:
+
+````markdown
+## Solution
+
+$C = \pi d$.
+
+```lambda-feedback
+{
+  "responseType": "EXPRESSION",
+  "answer": "pi*d",
+  "evaluationFunction": "compareExpressions",
+  "gradeParams": {"rtol": 0.01}
+}
+```
+````
+
+The body is JSON with the keys `responseType`, `answer`, `evaluationFunction`,
+`gradeParams`, and optionally `config`, `preResponseText`, `postResponseText`.
+`in2lambda wizard` writes these blocks after you confirm its suggestions (see
+[Wizard](../wizard)); you can also hand-write or edit them. Malformed blocks are
+reported by {func}`in2lambda.validation.check_markdown` as a warning - conversion
+still goes ahead, the block is just ignored.

docs/source/index.md

@@ -72,6 +72,7 @@ wizard
 contributing/installation
 contributing/high_level
 contributing/writing-a-filter
+contributing/response-areas
 contributing/documentation
 contributing/new_version
 ```

docs/source/wizard.md

@@ -36,6 +36,36 @@ $ in2lambda wizard problem_sheet.pdf -o draft.md
 sub-questions, and `## Solution` blocks. Any figures found in a PDF are saved
 next to it under `media/`.
 
+### Confirming response areas
+
+For every part that has a worked solution the wizard also proposes a **response
+area** - the input type a student uses and the evaluation function that marks it
+(see [Response areas](contributing/response-areas)). It stops on each one so you
+can check it:
+
+```text
+Question 2, part 1
+  Q: Find the time it takes to reach the ground.
+  Solution: $t = \sqrt{2h/g}$.
+  Proposed: EXPRESSION / compareExpressions answer='sqrt(2*h/g)' params={}
+  Why: The answer is a symbolic expression in h and g.
+  [a]ccept / [e]dit / [s]kip [a]:
+```
+
+`e` walks you through picking the response type, the evaluation function and its
+parameters; `s` drops it. Confirmed response areas are written into `draft.md` as
+```` ```lambda-feedback ```` blocks, which `in2lambda convert ... Markdown` turns
+into real response areas on import.
+
+| Flag | Effect |
+| --- | --- |
+| _(default, in a terminal)_ | prompt for each part |
+| `--yes` / `-y` | keep every proposed response area without prompting |
+| `--no-response-areas` | do not add any response areas |
+
+The prompt is also skipped (all suggestions kept) when the wizard is not attached
+to a terminal, so scripts keep working.
+
 Read through `draft.md`, fix anything the model got wrong, then convert it:
 
 ```bash

in2lambda/api/part.py

@@ -1,6 +1,8 @@
 """A part of a question."""
 
-from dataclasses import dataclass
+from dataclasses import dataclass, field
+
+from in2lambda.api.response_area import ResponseArea
 
 
 @dataclass
@@ -9,3 +11,6 @@ class Part:
 
     text: str = ""
     worked_solution: str = ""
+    # Kept out of repr so the many doctests that assert on Part(...) / Question(...)
+    # output stay valid; response areas are inspected explicitly where they matter.
+    response_areas: list[ResponseArea] = field(default_factory=list, repr=False)

in2lambda/api/question.py

@@ -6,6 +6,7 @@
 import panflute as pf
 
 from in2lambda.api.part import Part
+from in2lambda.api.response_area import ResponseArea
 
 
 @dataclass
@@ -147,3 +148,36 @@ def add_part_text(self, elem: Union[pf.Element, str]) -> None:
             self.parts[self._last_part["text"]].text = elem_text
 
         self._last_part["text"] += 1
+
+    def add_response_area(
+        self, response_area: ResponseArea, *, to_part: int = -1
+    ) -> None:
+        """Attach a response area to one of this question's parts.
+
+        If the question has no parts yet, an empty part is created to hold it,
+        mirroring how :meth:`add_solution` and :meth:`add_part_text` cope with
+        content arriving in an unexpected order.
+
+        Args:
+            response_area: The response area to attach.
+            to_part: Index of the part to attach to (default: the last part).
+
+        Examples:
+            >>> from in2lambda.api.question import Question
+            >>> from in2lambda.api.response_area import ResponseArea
+            >>> question = Question()
+            >>> question.add_part_text("What is 2 + 2?")
+            >>> question.add_response_area(ResponseArea("NUMBER", "4", "isExactEqual"))
+            >>> question.parts[0].response_areas
+            [ResponseArea(response_type='NUMBER', answer='4', \
+evaluation_function='isExactEqual', grade_params={}, config={}, \
+pre_response_text='', post_response_text='')]
+            >>> # Works even before any part text has been added.
+            >>> blank = Question()
+            >>> blank.add_response_area(ResponseArea("BOOLEAN", "True", "compareBoolean"))
+            >>> len(blank.parts)
+            1
+        """
+        if not self.parts:
+            self.parts.append(Part())
+        self.parts[to_part].response_areas.append(response_area)

in2lambda/api/response_area.py

@@ -0,0 +1,38 @@
+"""A response area: where a student answers, and how that answer is marked.
+
+A :class:`ResponseArea` bundles the four things Lambda Feedback needs to
+auto-mark one input on a question part:
+
+* ``response_type`` - the kind of input widget (``EXPRESSION``, ``NUMBER``,
+  ``BOOLEAN``, ``TEXT``, ``ESSAY``, ``CODE``, ``NUMERIC_UNITS``, ...),
+* ``answer`` - the reference answer, as a string,
+* ``evaluation_function`` - which evaluation function grades the response
+  (see :data:`in2lambda.response_areas.EVALUATION_FUNCTIONS`),
+* ``grade_params`` - that function's parameters.
+
+:mod:`in2lambda.json_convert` turns each one into an entry in a part's
+``responseAreas`` array on import.
+"""
+
+from dataclasses import dataclass, field
+
+
+@dataclass
+class ResponseArea:
+    """One markable input on a question part.
+
+    Examples:
+        >>> from in2lambda.api.response_area import ResponseArea
+        >>> ResponseArea("EXPRESSION", "pi*d", "compareExpressions", {"rtol": 0.01})
+        ResponseArea(response_type='EXPRESSION', answer='pi*d', \
+evaluation_function='compareExpressions', grade_params={'rtol': 0.01}, config={}, \
+pre_response_text='', post_response_text='')
+    """
+
+    response_type: str
+    answer: str
+    evaluation_function: str
+    grade_params: dict = field(default_factory=dict)
+    config: dict = field(default_factory=dict)
+    pre_response_text: str = ""
+    post_response_text: str = ""

in2lambda/filters/Markdown/filter.py

@@ -22,15 +22,54 @@
 :mod:`in2lambda.validation` checks it before conversion.
 """
 
+import json
 from typing import Optional
 
 import panflute as pf
 
 from in2lambda.api.part import Part
+from in2lambda.api.response_area import ResponseArea
 from in2lambda.api.set import Set
 from in2lambda.filters.markdown import filter
 
 _SOLUTION_HEADING = "solution"
+_RESPONSE_AREA_CLASS = "lambda-feedback"
+
+
+def _attach_response_area(block_text: str, state: "_State", set: Set) -> None:
+    """Parse a ``lambda-feedback`` fenced block and attach it to the current part.
+
+    Malformed blocks are ignored here - :mod:`in2lambda.validation` reports them
+    to the user before conversion.
+
+    Args:
+        block_text: The raw JSON body of the fenced block.
+        state: The current parser state (its ``part`` receives the response area).
+        set: The Python API the question is being built into.
+    """
+    try:
+        data = json.loads(block_text)
+    except json.JSONDecodeError:
+        return
+    if not isinstance(data, dict):
+        return
+
+    grade_params = data.get("gradeParams")
+    config = data.get("config")
+    response_area = ResponseArea(
+        response_type=str(data.get("responseType", "")),
+        answer=str(data.get("answer", "")),
+        evaluation_function=str(data.get("evaluationFunction", "")),
+        grade_params=grade_params if isinstance(grade_params, dict) else {},
+        config=config if isinstance(config, dict) else {},
+        pre_response_text=str(data.get("preResponseText", "")),
+        post_response_text=str(data.get("postResponseText", "")),
+    )
+
+    if state.part is None:
+        state.part = Part()
+        set.current_question.parts.append(state.part)
+    state.part.response_areas.append(response_area)
 
 
 class _State:
@@ -86,6 +125,15 @@ def pandoc_filter(
         return None
 
     state = _state_for(doc)
+
+    # A ```lambda-feedback fenced block configures a response area for the
+    # current part. Handle it before the generic text handling below, which
+    # would otherwise fold the JSON into the part or solution text.
+    if isinstance(elem, pf.CodeBlock) and _RESPONSE_AREA_CLASS in elem.classes:
+        if not parsing_answers:
+            _attach_response_area(elem.text, state, set)
+        return None
+
     is_heading = isinstance(elem, pf.Header)
     text = pf.stringify(elem).strip()