Skip to content

call_runner rejects a top-level JSON array, so a runner cannot pass one through #64

Description

@rickstaa

call_runner decides how to hand back a response from its Content-Type alone: JSON is parsed into result.data, anything else arrives unparsed in result.content. A body that is valid JSON but not an object falls between the two and raises instead:

livepeer_gateway.errors.LivepeerGatewayError: Live runner call expected JSON object, got list

Repro

import asyncio
from aiohttp import web
from livepeer_gateway.live_runner import call_runner


async def main():
    async def handler(request):
        return web.json_response([{"label": "llama", "score": 0.99}])

    app = web.Application()
    app.router.add_post("/call", handler)
    runner = web.AppRunner(app)
    await runner.setup()
    site = web.TCPSite(runner, "127.0.0.1", 0)
    await site.start()
    port = site._server.sockets[0].getsockname()[1]
    try:
        result = await call_runner(f"http://127.0.0.1:{port}/call", payload={"inputs": "x"})
        print("data:", result.data, "content:", result.content)
    finally:
        await runner.cleanup()


asyncio.run(main())

Why it matters

A runner that proxies somebody else's API does not choose its response shape, and a top-level array is a common one. Every Hugging Face inference task except text-to-image answers with one: classification, object detection, embeddings, sentiment. Such a runner is currently unreachable through the SDK.

The workaround available to a proxy runner is to make the response stop claiming to be JSON so it takes the opaque-bytes path. In runner-app-examples#82 that is nginx relabelling a JSON body as text/plain, which is a lie told to route around a type check, and it is the only lever config has, since nginx can pin, inject, and relabel but cannot rewrite a body.

Proposal

Treat non-object JSON the way ndjson and binary are already treated: return the body unparsed in result.content with result.content_type intact, leaving result.data as {}.

Objects keep today's behavior exactly, including session_id extraction, so no working call changes: the only path affected is the one that raises today. content_type still reports application/json, so a caller can tell what it is holding and json.loads it.

The current strictness is deliberate (tests/test_call_runner_raw.py::test_json_array_still_rejected), and it is right for the control-plane calls that read protocol fields out of the body, such as proxy create and trickle channel remove. Those stay strict. It is the pass-through data path where an array is data rather than a malformed reply.

An alternative is to keep data dict-only and expose the parsed value separately (result.json), but that adds API surface for something content already carries.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions