refactor(api-proxy): read the image from a buffered call - #49
Merged
Conversation
rickstaa
marked this pull request as ready for review
August 12, 2026 12:07
rickstaa
force-pushed
the
rs/api-proxy-buffered
branch
from
August 12, 2026 12:07
2308397 to
728c4dd
Compare
One bounded call returning a 120 KB image has nothing to stream. stream=True existed only because call_runner's buffered path assumed JSON and raised on image/jpeg. That is fixed upstream, so the async with and aiter_bytes block collapses into result.content. The buffered path bounds the whole request where streaming did not, hence the explicit timeout. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
rickstaa
force-pushed
the
rs/api-proxy-buffered
branch
from
August 12, 2026 12:42
728c4dd to
5bd02dd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Blocked on livepeer/livepeer-python-gateway#51 and #52 landing in
ja/live-runner. The example pins that branch, so nopyproject.tomlchange is needed; mark ready and merge once both are in.One bounded call returning a 120 KB image has nothing to stream.
stream=Truewas there becausecall_runner's buffered path assumed a JSON object and raised onimage/jpeg. With #51 the body arrives inresult.content, so theasync with/aiter_bytes()block collapses to one line.One thing worth knowing: the buffered path bounds the whole request with
call_runner's 5s default, which streaming did not (open_streamsetssock_read=None). Droppingstream=Truewithout a timeout fails after ~5s withfailed to reach endpointand no mention of a timeout, so the call passestimeout=180.The nginx
Acceptpin stays. It is not a workaround for the SDK bug: the runner advertises an image at a fixed price, so the response format is its choice, and the pin still holds for callers that are not this SDK or are on an older pin.Tested
End to end on the paid path against the stacked SDK branch (#51 + #52), with the pin in place: a real Hugging Face call returned a 1024x1024
image/jpeg(56313 bytes) intoresult.content. The temporary SDK pin used for that run was reverted before committing.Closes #47
Unblocked and re-verified 2026-08-12. Both SDK PRs landed on 2026-07-31 and are now in
livepeer-python-gateway'smain, which this repo pins after #66, so nopyproject.tomlchange is needed:call_runnerfor non-JSON responsesRebased onto
mainand re-ran end to end against SDKmain(a0b9129) rather than the stacked branch used for the original test. A real Hugging Face call returned a complete 1024x1024image/jpeg(216829 bytes, valid SOI and EOI markers) intoresult.content.The call took 15 seconds, which is the concrete case for
timeout=180: the buffered path is bounded bycall_runner's 5s default, so without it this exact request fails.