Skip to content

refactor(api-proxy): read the image from a buffered call - #49

Merged
rickstaa merged 1 commit into
mainfrom
rs/api-proxy-buffered
Aug 12, 2026
Merged

refactor(api-proxy): read the image from a buffered call#49
rickstaa merged 1 commit into
mainfrom
rs/api-proxy-buffered

Conversation

@rickstaa

@rickstaa rickstaa commented Jul 30, 2026

Copy link
Copy Markdown
Member

Blocked on livepeer/livepeer-python-gateway#51 and #52 landing in ja/live-runner. The example pins that branch, so no pyproject.toml change is needed; mark ready and merge once both are in.

One bounded call returning a 120 KB image has nothing to stream. stream=True was there because call_runner's buffered path assumed a JSON object and raised on image/jpeg. With #51 the body arrives in result.content, so the async with / aiter_bytes() block collapses to one line.

One thing worth knowing: the buffered path bounds the whole request with call_runner's 5s default, which streaming did not (open_stream sets sock_read=None). Dropping stream=True without a timeout fails after ~5s with failed to reach endpoint and no mention of a timeout, so the call passes timeout=180.

The nginx Accept pin stays. It is not a workaround for the SDK bug: the runner advertises an image at a fixed price, so the response format is its choice, and the pin still holds for callers that are not this SDK or are on an older pin.

Tested

End to end on the paid path against the stacked SDK branch (#51 + #52), with the pin in place: a real Hugging Face call returned a 1024x1024 image/jpeg (56313 bytes) into result.content. The temporary SDK pin used for that run was reverted before committing.

Closes #47


Unblocked and re-verified 2026-08-12. Both SDK PRs landed on 2026-07-31 and are now in livepeer-python-gateway's main, which this repo pins after #66, so no pyproject.toml change is needed:

Rebased onto main and re-ran end to end against SDK main (a0b9129) rather than the stacked branch used for the original test. A real Hugging Face call returned a complete 1024x1024 image/jpeg (216829 bytes, valid SOI and EOI markers) into result.content.

The call took 15 seconds, which is the concrete case for timeout=180: the buffered path is bounded by call_runner's 5s default, so without it this exact request fails.

@rickstaa
rickstaa marked this pull request as ready for review August 12, 2026 12:07
@rickstaa
rickstaa force-pushed the rs/api-proxy-buffered branch from 2308397 to 728c4dd Compare August 12, 2026 12:07
Copilot AI lite review requested due to automatic review settings August 12, 2026 12:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

One bounded call returning a 120 KB image has nothing to stream.
stream=True existed only because call_runner's buffered path assumed
JSON and raised on image/jpeg. That is fixed upstream, so the async with
and aiter_bytes block collapses into result.content. The buffered path
bounds the whole request where streaming did not, hence the explicit
timeout.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@rickstaa
rickstaa force-pushed the rs/api-proxy-buffered branch from 728c4dd to 5bd02dd Compare August 12, 2026 12:42
@rickstaa
rickstaa merged commit a22114d into main Aug 12, 2026
1 check passed
@rickstaa
rickstaa deleted the rs/api-proxy-buffered branch August 12, 2026 12:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

api-proxy: drop stream=True once call_runner returns raw bytes

2 participants