Fix truncation of evaluation requests larger than 4 KB

[peterbjohnson]

Closes #11

Problem

The pinned lf_toolkit commit (8a687d3, 29 Jul 2026) read at most size bytes from the socket instead of reading to the newline delimiter:

async def read(self, size: int) -> bytes:
    data = b""
    while len(data) < size:          # stops at the 4096-byte hint
        chunk = await self.base.read(1)
        if chunk == b"\n":
            break
        data += chunk
    return data

Any request longer than the 4096-byte read hint arrived truncated. The worker could not decode the partial JSON, never replied, and Shimmy returned a 500 after its 30 second deadline.

Roughly 45 notes was the practical limit. The platform sends response and answer as JSON, so most real pieces exceeded it. This affected the deployed staging function.

Approach

Tests first. evaluation_function/transport_test.py drives lf_toolkit's newline framing directly with a payload either side of the read hint, using an in-memory stand-in for the socket that can return short reads.

Against the previously pinned toolkit the new tests fail exactly at the boundary:

json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 4095 (char 4094)
FAILED test_request_larger_than_read_hint_is_not_truncated
FAILED test_large_request_still_parses_as_json
1 passed

The short-payload control passes throughout, which is why the existing suite never caught this.

These tests live in their own file rather than in evaluation_test.py because they do not exercise this repository's code at all. They assert that the transport underneath us delivers a whole request, which is a deployment contract rather than pipeline logic.

Change

poetry update lf_toolkit, picking up the upstream fix in lambda-feedback/toolkit-python#10 (19 Aug 2026). No source changes in this repo. The three new tests are the regression guard, so a future pin cannot silently reintroduce it.

Verification

Check	Result
New tests, old pin	2 failed, 1 passed
New tests, updated pin	3 passed
Full suite	76 passed
CI lint gate (E9,F63,F7,F82)	0
Container, 200-note request (18.8 KB)	HTTP 200 in 0.12 s

Before the bump that same request returned HTTP 500 after 30 s.

🤖 Generated with Claude Code