call_runner decides how to hand back a response from its Content-Type alone: JSON is parsed into result.data, anything else arrives unparsed in result.content. A body that is valid JSON but not an object falls between the two and raises instead:
livepeer_gateway.errors.LivepeerGatewayError: Live runner call expected JSON object, got list
Repro
import asyncio
from aiohttp import web
from livepeer_gateway.live_runner import call_runner
async def main():
async def handler(request):
return web.json_response([{"label": "llama", "score": 0.99}])
app = web.Application()
app.router.add_post("/call", handler)
runner = web.AppRunner(app)
await runner.setup()
site = web.TCPSite(runner, "127.0.0.1", 0)
await site.start()
port = site._server.sockets[0].getsockname()[1]
try:
result = await call_runner(f"http://127.0.0.1:{port}/call", payload={"inputs": "x"})
print("data:", result.data, "content:", result.content)
finally:
await runner.cleanup()
asyncio.run(main())
Why it matters
A runner that proxies somebody else's API does not choose its response shape, and a top-level array is a common one. Every Hugging Face inference task except text-to-image answers with one: classification, object detection, embeddings, sentiment. Such a runner is currently unreachable through the SDK.
The workaround available to a proxy runner is to make the response stop claiming to be JSON so it takes the opaque-bytes path. In runner-app-examples#82 that is nginx relabelling a JSON body as text/plain, which is a lie told to route around a type check, and it is the only lever config has, since nginx can pin, inject, and relabel but cannot rewrite a body.
Proposal
Treat non-object JSON the way ndjson and binary are already treated: return the body unparsed in result.content with result.content_type intact, leaving result.data as {}.
Objects keep today's behavior exactly, including session_id extraction, so no working call changes: the only path affected is the one that raises today. content_type still reports application/json, so a caller can tell what it is holding and json.loads it.
The current strictness is deliberate (tests/test_call_runner_raw.py::test_json_array_still_rejected), and it is right for the control-plane calls that read protocol fields out of the body, such as proxy create and trickle channel remove. Those stay strict. It is the pass-through data path where an array is data rather than a malformed reply.
An alternative is to keep data dict-only and expose the parsed value separately (result.json), but that adds API surface for something content already carries.
call_runnerdecides how to hand back a response from itsContent-Typealone: JSON is parsed intoresult.data, anything else arrives unparsed inresult.content. A body that is valid JSON but not an object falls between the two and raises instead:Repro
Why it matters
A runner that proxies somebody else's API does not choose its response shape, and a top-level array is a common one. Every Hugging Face inference task except text-to-image answers with one: classification, object detection, embeddings, sentiment. Such a runner is currently unreachable through the SDK.
The workaround available to a proxy runner is to make the response stop claiming to be JSON so it takes the opaque-bytes path. In runner-app-examples#82 that is nginx relabelling a JSON body as
text/plain, which is a lie told to route around a type check, and it is the only lever config has, since nginx can pin, inject, and relabel but cannot rewrite a body.Proposal
Treat non-object JSON the way ndjson and binary are already treated: return the body unparsed in
result.contentwithresult.content_typeintact, leavingresult.dataas{}.Objects keep today's behavior exactly, including
session_idextraction, so no working call changes: the only path affected is the one that raises today.content_typestill reportsapplication/json, so a caller can tell what it is holding andjson.loadsit.The current strictness is deliberate (
tests/test_call_runner_raw.py::test_json_array_still_rejected), and it is right for the control-plane calls that read protocol fields out of the body, such as proxy create and trickle channel remove. Those stay strict. It is the pass-through data path where an array is data rather than a malformed reply.An alternative is to keep
datadict-only and expose the parsed value separately (result.json), but that adds API surface for somethingcontentalready carries.