feat: Sandbox related enhancement/fixes by danielvallance · Pull Request #8 · unikraft-cloud/python-sdk

danielvallance · 2026-09-23T19:30:34Z

Summary

Adds the sandbox plugin client, idiomatic images and templates clients, and richer
instance operations. It also regenerates the platform and control-plane plumbing from
the current specification and bumps the version to 0.2.0.

Changes

Sandbox plugin. instance.sandbox() runs commands, streams their output, feeds
stdin, sends signals and moves files in chunks. plugin(name) reaches any other
plugin. The plumbing comes from unikraft-cloud-plugin-sandbox-api (PyPI, >=1.1.0,<2).
Images. ukc.images.list() reports what each metro caches. find() and exists()
ask the registry through the control plane.
Templates. ukc.templates.prepare() builds a template from an instance
specification, and get(...).clone() creates instances from it.
Instances. Instance.stop and describe_stop() decode why an instance stopped.
A create whose instance stopped raises InstanceStoppedError, and a lapsed create wait
raises WaitTimeoutError. delete() takes timeout_seconds, missing_ok and
retry_busy. each(tags=...) addresses instances by tag.
Core. The transport is public (request, request_bytes, stream_bytes,
stream, RawResponse). A failed lookup is repeated on the next await.
NotFoundError.absent marks a missing resource, the only not-found that is forgiven.
Generator. The templates render raw payloads and header parameters, keep required
request fields, and refuse specification shapes they cannot render.

Breaking changes

ServiceGroupsApi is now ServicesApi, and UsersApi.add_users is removed, both
following the specification.
A request model missing a required field raises ValidationError at construction.
The private transport methods (_request, _stream, ...) are removed.
or_absent() raises a 404 for a route that is not there, instead of returning None.

Testing

Every commit passes its checks. uv lock --check, ruff, strict mypy and pytest
pass at each of the 36 commits, with 537 tests at the tip.
The unit tests model the real API. Lookup misses use the API's actual answer: a
200 with a per-item not-found code. A fake sandbox plugin reproduces the proxy's 404
while the plugin boots and its 502 or 504 once routed. It also reproduces 408 wait
timeouts, stdin refused to an ended command, and writes that name a directory.
Live runs against a test metro covered:
- Instances: create, get, list and each, by name, UUID and tag; wait, logs,
  metrics and history; tag edits; stop, start and suspend.
- Error paths: a missing image raises InstanceStoppedError with its decoded
  reason. A duplicate name raises AlreadyExistsError, and a lapsed wait raises
  WaitTimeoutError.
- Sandbox commands: exit codes and both output streams, argv, env, cwd and stdin;
  streamed output; a timeout's interrupt (-2), and ExecTimeoutError followed by
  KILL (-9).
- Sandbox files and streams: a 3 MiB round trip; upload into a directory;
  read_to; raw log ranges; byte and audit-event streams stopped early with
  contextlib.aclosing.
- Other resources: images by tag and digest, plus exists(); template prepare,
  clone and delete; volume attach and detach; raw plumbing calls.
- Cleanup: bulk delete by tag, which can safely run again.
The examples run live. quickstart, update, sandbox and templates all
complete.
Packaging works. The built wheel installs with plain pip and pulls in the published
unikraft-cloud-plugin-sandbox-api. That package's code matches a fresh build from
the plugin's specification.
The downstream integration works. The Unikraft integration that depends on this
SDK passes its unit tests and its live tests against this branch.

Copilot

Copilot review overview

🟡 Changes recommended

Timeout enforcement, chunk-size handling, route normalization, and HTTP pool ownership have unresolved correctness and reliability issues.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 7

Open (7)

Give PluginsApi ownership of its HTTP client pool · New
Normalize /v1 suffix in API base URLs · New
Use shared client ownership for SandboxApi resources · New
Guard plugin route resolution in ready() · New
Reject nonpositive chunk sizes before filesystem work · New
Chunk or limit oversized public uploads · New
Recognize localhost as an OCI registry host · New

What changed in this PR

Adds sandbox plugin support and improves reliability around file transfers, command execution, stopped instances, deletion, and image discovery.

Changes:

Adds sandbox command/filesystem APIs with raw-byte streaming and chunked transfers.
Adds stop diagnostics, image lookup/listing, and enhanced instance deletion.
Extends generated clients, public exports, documentation, and tests.

File	Description
`README.md`	Documents stopped-instance errors.
`Makefile`	Adds sandbox API generation.
`templates/resources.tmpl`	Generates raw-byte and header handling.
`src/unikraft_cloud/__init__.py`	Exports new public APIs.
`src/unikraft_cloud/api/__init__.py`	Exposes plugin clients.
`src/unikraft_cloud/api/plugins/__init__.py`	Groups plugin APIs.
`src/unikraft_cloud/api/plugins/_route.py`	Builds plugin routes.
`src/unikraft_cloud/api/plugins/sandbox/__init__.py`	Groups sandbox resources.
`src/unikraft_cloud/api/plugins/sandbox/commands_gen.py`	Adds generated command operations.
`src/unikraft_cloud/api/plugins/sandbox/fs_gen.py`	Adds generated filesystem operations.
`src/unikraft_cloud/api/plugins/sandbox/models_gen.py`	Adds sandbox wire models.
`src/unikraft_cloud/client.py`	Exposes image resources.
`src/unikraft_cloud/core/handle.py`	Supports absence-tolerant handles.
`src/unikraft_cloud/core/http.py`	Adds raw-byte responses and streaming.
`src/unikraft_cloud/plugins/__init__.py`	Resolves instance plugin clients.
`src/unikraft_cloud/plugins/sandbox.py`	Implements sandbox commands and files.
`src/unikraft_cloud/resources/images.py`	Adds image listing and registry lookup.
`src/unikraft_cloud/resources/instances.py`	Enhances create, delete, tags, and plugins.
`src/unikraft_cloud/resources/stop.py`	Decodes instance stop information.
`tests/conftest.py`	Extends asynchronous test transport helpers.
`tests/fake_sandbox.py`	Adds an in-memory sandbox service.
`tests/test_client.py`	Tests instance reliability enhancements.
`tests/test_images.py`	Tests image discovery.
`tests/test_plugins.py`	Tests plugin resolution.
`tests/test_plugins_api.py`	Tests generated plugin plumbing.
`tests/test_sandbox.py`	Tests sandbox workflows and transfers.
`tests/test_stop.py`	Tests stop decoding.
`tests/test_transport.py`	Tests raw-byte transport behavior.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

nderjung · 2026-09-24T05:51:32Z

+@dataclass(frozen=True)
+class ExecResult:
+    """A finished command: its exit code and everything it wrote."""
+
+    uuid: str
+    exit_code: int
+    stdout: bytes
+    stderr: bytes
+    #: Whether the command was interrupted because a timeout elapsed.
+    interrupted: bool = False


Was this file generated? Asking because i think the approach we should have for this should be similar to how we're doing the SDK for plugins compared to the Go and JS SDK. For those, we have tools which generate code, see here:

js/tools/tsplugingen

go/tools/sdkgen

we should have a python/tools/sdkgen

This will make it uniform across all plugins.

Then in the case of the JS SDK (and in the future the Go SDK), it's integrated as a peer dependency package with a nice porcelain layer around the plumbing. See example here for JS: unikraft-cloud/js-sdk#24

@nderjung The _gen.py files are generated and this one isnt

i will refactor the generation to be more like the ones you point to (in this PR its done from within this repo and the output ends up in this repo too)

@nderjung PR for the python/tools/sdkgen tool here: unikraft-cloud/plugin-sdk#13

Copilot

Copilot review overview

🟡 Changes recommended

The local dependency blocks clean CI setup, and plugin lookup recovery and concurrent reads remain incorrect.

Review effort: Balanced
Findings: 1 · 2

Open (3)

CI dependency setup fails without sibling plugin-sdk checkout · New
Failed instance lookups are cached and never retried · New
UUID-pinned instance reads are not memoized concurrently · New

Resolved since last review (7)

Recognize localhost as an OCI registry host
Chunk or limit oversized public uploads
Reject nonpositive chunk sizes before filesystem work
Guard plugin route resolution in ready()
Use shared client ownership for SandboxApi resources
Normalize /v1 suffix in API base URLs
Give PluginsApi ownership of its HTTP client pool

Copilot

Copilot review overview

🟡 Changes recommended

Unresolved endpoint routing, absence classification, and image-reference validation issues affect correctness.

Review effort: Balanced
Findings: 2 · 3

Open (5)

Preserve discovered custom metro endpoint during instance creation · New
CI dependency setup fails without sibling plugin-sdk checkout
Preserve route-level 404s during resource lookup · New
Reject empty digests in image references · New
Failed instance lookups are cached and never retried

Resolved since last review (1)

UUID-pinned instance reads are not memoized concurrently

Copilot

Copilot review overview

🔵 Needs a closer look

Cross-cutting transport, handle, generation, and external plugin changes need human validation alongside the unresolved correctness issues.

Review effort: Balanced
Findings: 1 · 2

Open (3)

Unbounded polling reads can bypass the preparation deadline · New
Missing templates incorrectly trigger MetroFanoutError · New
Warning test swallows the warning it should detect · New

Resolved since last review (5)

Preserve discovered custom metro endpoint during instance creation
CI dependency setup fails without sibling plugin-sdk checkout
Reject empty digests in image references
Preserve route-level 404s during resource lookup
Failed instance lookups are cached and never retried

Copilot

Copilot review overview

🟡 Changes recommended

Previously identified dependency provisioning and template deadline issues remain unresolved.

Review effort: Balanced
Findings: 1

Open (1)

Async generator connection remains open after early exit · New

Resolved since last review (3)

Unbounded polling reads can bypass the preparation deadline
Warning test swallows the warning it should detect
Missing templates incorrectly trigger MetroFanoutError

A request model was detected by its name alone, so a schema that only requests use, such as the image spec a plugin takes, had every field optional when its name lacked Request, and a request missing a field the specification requires failed at the server. A schema that only request bodies reach, directly or through the models they name, is a request model too, as the plugin generator has it; one a response also reaches stays optional. Signed-off-by: Daniel Vallance <daniel@unikraft.com>

A nullable query, path or header parameter rendered an argument the transport cannot send as null: it leaves a None value out and spells a None list item as "None". A nullable response schema was typed as a model the transport cannot decode a null into. Such a parameter or response is refused with a message naming it and what to change in the specification, as the plugin generator does. Signed-off-by: Daniel Vallance <daniel@unikraft.com>

The platform and control-plane clients were rendered from an older specification, so a create could not be given annotations, gpus or type: check_spec refused each as unknown, though the API has accepted all three for some time. Both are regenerated from the published specification: ServiceGroupsApi is ServicesApi, PlatformApi gains the audit API, UsersApi loses add_users, request models keep their required fields, as the README now says, and F401 is on again. Signed-off-by: Daniel Vallance <daniel@unikraft.com>

A plugin serves an API of its own from inside an instance, which the platform proxies under the instance on the metro running it. The SDK had no client for that API and no way to build its route. The plumbing is the unikraft-cloud-plugin-sandbox-api package, which plugin-sdk renders. SandboxApi, reached as ukc.api.plugins.sandbox, sends it through the SDK's transport, and for_instance() points it at one instance; a plugin named v1 keeps its route. Signed-off-by: Daniel Vallance <daniel@unikraft.com>

A handle memoised its lookup as a task and kept it however it ended, so a name not listed yet, or a metro that did not answer, was raised again by every later await. A set kept a failed lookup too. A lookup that failed is made again on the next await, for the handles that only look a resource up, for sets, and for the lookup an operation chained onto such a handle starts from; a create, and the chained operation itself, keep their one outcome. Signed-off-by: Daniel Vallance <daniel@unikraft.com>

Copilot

Copilot review overview

🟡 Changes recommended

Template preparation deadline enforcement and the sandbox fake’s termination outcomes remain correctness concerns.

Review effort: Balanced
Findings: 1

Open (1)

Report the actual terminating signal in fake exit codes · New

Resolved since last review (2)

Propagate route-level 404 errors during missing-ok deletion
Async generator connection remains open after early exit

The SDK had no client for the sandbox plugin. A caller had to find the instance's UUID, build the plugin's route and drive the plumbing by hand to run a command or move a file. sandbox() and plugin() on an instance handle give a Sandbox or a Plugin, which read the instance only when its UUID is not known. Sandbox runs, follows and signals commands and moves files; output is polled, and a timeout interrupts a command, then waits for it. Signed-off-by: Daniel Vallance <daniel@unikraft.com>

collect() gathered a command's output for the result, so a caller that wanted it as it arrived had to poll the command itself, and with it re-implement the interrupt and the grace a timeout gives. A finished command also stayed in the plugin until deleted. An on_output sink is handed each chunk as it is read, an awaitable it returns awaited before the next, while the result still carries everything. forget drops the plugin's record once the command has ended; a command still running is kept. Signed-off-by: Daniel Vallance <daniel@unikraft.com>

The plugin reads a request body of two mebibytes at most, and a whole file came back in one payload. A caller moving anything larger chunked its writes by hand and held a download in memory. write() sends data longer than a chunk in appended pieces and can create the directories above the file; upload_file() streams a local file the same way. stream() and read_to() hand a file over as it arrives, and examples/sandbox.py walks through the client. Signed-off-by: Daniel Vallance <daniel@unikraft.com>

The metro proxy ends a plugin request after 60s. An unbounded wait for a command's end, and any wait longer than that, was cut off with the proxy's 504 page, so a long command could not be waited for or followed to its end. A wait is sent as waits of at most WAIT_SLICE, thirty seconds, until the command ends or the caller's timeout runs out, each given a read timeout that outlasts it; one the proxy drops is retried as a poll is. The README gains the sandbox section. Signed-off-by: Daniel Vallance <daniel@unikraft.com>

The SDK had no idiomatic images client, and images are reported per metro, so an account-wide view meant asking every metro in scope and merging the answers by hand. Images, reached as scope.images, lists every metro in scope and tags each image with its metro; a partial answer raises with what did arrive. A lookup is a filtered listing, as in the CLI, and one metro's view is that metro's client. Signed-off-by: Daniel Vallance <daniel@unikraft.com>

The images client lists what each metro's nodes have cached, which is not whether an image can be pulled: the cache drops images the registry keeps and keeps ones it has dropped. Callers asked the OCI registry directly, with its token exchange, to find out. Images.find and Images.exists ask the control plane, which answers from the registry itself and needs only the SDK's token. A reference may carry a tag, a digest, a registry host or a scheme, and a bare one means the latest tag. Signed-off-by: Daniel Vallance <daniel@unikraft.com>

The API reports a stop as two integers, a reason bitmask and a code whose meaning depends on it. The Go SDK decodes both; this one handed them over raw, so every caller decoded the platform's image-pull failure and the kernel's out-of-memory for itself. Stop, StopReason and the platform and kernel codes port the Go package; Instance.stop reads them off an instance and describe_stop() puts them in words, as the CLI reports a stop, e.g. "kernel crash: out of memory (ENOMEM)". Signed-off-by: Daniel Vallance <daniel@unikraft.com>

A create that waited for its instance to run raised a bare "API reported an error" when the instance stopped instead. The failed item names the instance and nothing else, so the reason survived only on the stopped instance, which every caller had to read. Instances.create reads that instance and raises InstanceStoppedError with its decoded stop; ResponseError carries an item's state. A create that waits stretches its read timeout past that wait, as wait() does, and raises a lapsed wait as WaitTimeoutError. Signed-off-by: Daniel Vallance <daniel@unikraft.com>

A delete answered as soon as it was under way, failed on an instance already gone, and failed at once on one the platform reported busy, as a relay interface is for a while after its instance is deleted. Callers waited and retried by hand. delete() takes timeout_seconds for the API's own wait, missing_ok and retry_busy, which backs off while the API says EBUSY; a wait of -1 leaves the read timeout unbounded. NotFoundError.absent marks a missing resource, and only that not-found is ever forgiven. Signed-off-by: Daniel Vallance <daniel@unikraft.com>

An instance set could be named only by uuid or name, one match per metro. A caller cleaning up everything a job had created had to list by tag and then delete each match itself, or guess at names. each(tags=[...]) locates every instance in scope carrying the tags and hands back the usual set, so `.delete(missing_ok=True)` covers them all. Tags that select nothing are an empty set rather than a failure, so a cleanup can run again. Signed-off-by: Daniel Vallance <daniel@unikraft.com>

Templates had plumbing only. Preparing one meant creating a seed clone, waiting for the template to be listed and deleting the seed by hand, and cloning meant building the create request yourself. Add Templates, reached as scope.templates: prepare makes a template from an instance specification, in the one metro the scope names, and get(...).clone stamps instances out of it; list, each and delete work as for every other resource. examples/templates.py shows it. Signed-off-by: Daniel Vallance <daniel@unikraft.com>

The public surface changed: a dependency on the sandbox plumbing package, which names 0.2.0 as the oldest SDK whose transport it fits, plumbing renamed and removed by the regeneration, and required fields that are now required. The package and `__version__` say 0.2.0, the lock follows, and a changelog records what the release adds, changes and removes. Signed-off-by: Daniel Vallance <daniel@unikraft.com>

The Makefile's help called fmt a format of all sources and misaligned its columns, openapi-gen was passed a variable no template reads, the README linked openapi-gen to the GitHub organisation, and several docstrings and comments were stale. fmt says what it formats, the columns fit the longest target and the dead flag is gone. The README says the Makefile pins openapi-gen, and the stale docstrings and comments are corrected. Signed-off-by: Daniel Vallance <daniel@unikraft.com>

aabedraba

1 comment

aabedraba · 2026-10-06T13:52:17Z

+            image=image,
+            memory_mb=1024,
+            args=["/bin/sh", "-c", "sleep infinity"],
+            plugins=[{"name": "sandbox", "image": "plugins/sandbox:latest", "config": {}}],


what do you think about making this the default so it doesn't need adding?

same for image (debian-slim) and memory (1GB)?

yeah i think that's a nice idea as part of the sandbox porcelain - however i may tackle it in a follow up PR if that's ok

aabedraba · 2026-10-06T13:52:26Z

+            # A timeout interrupts the command; a grace period bounds the wait
+            # for one that ignores the interrupt.
+            try:
+                result = await sb.exec("sleep 300", timeout=2, wait_delay=5)


aabedraba

Approved-by: Abdallah Abedraba abdallah@unikraft.com

danielvallance requested review from nderjung and a balanced review from Copilot and removed request for Copilot September 23, 2026 19:30

Copilot started reviewing on behalf of danielvallance September 23, 2026 19:31 View session

Copilot AI reviewed Sep 23, 2026

View reviewed changes

nderjung reviewed Sep 24, 2026

View reviewed changes

danielvallance force-pushed the danielvallance/unikraft_backend branch 2 times, most recently from 51335a0 to a852550 Compare October 4, 2026 00:15

danielvallance requested a balanced review from Copilot October 4, 2026 00:16

Copilot started reviewing on behalf of danielvallance October 4, 2026 00:17 View session

Copilot AI reviewed Oct 4, 2026

View reviewed changes

Comment thread pyproject.toml Outdated

Comment thread src/unikraft_cloud/plugins/__init__.py

Comment thread src/unikraft_cloud/plugins/__init__.py Outdated

danielvallance force-pushed the danielvallance/unikraft_backend branch from a852550 to ece7669 Compare October 4, 2026 19:04

danielvallance requested a balanced review from Copilot October 4, 2026 19:05

Copilot started reviewing on behalf of danielvallance October 4, 2026 19:06 View session

Copilot AI reviewed Oct 4, 2026

View reviewed changes

Comment thread src/unikraft_cloud/resources/templates.py Outdated

Comment thread src/unikraft_cloud/core/resource.py

Comment thread src/unikraft_cloud/resources/images.py Outdated

danielvallance force-pushed the danielvallance/unikraft_backend branch from ece7669 to 8be4fdc Compare October 5, 2026 11:21

danielvallance requested a balanced review from Copilot October 5, 2026 11:22

Copilot started reviewing on behalf of danielvallance October 5, 2026 11:22 View session

Copilot AI reviewed Oct 5, 2026

View reviewed changes

Comment thread src/unikraft_cloud/resources/templates.py

Comment thread src/unikraft_cloud/resources/templates.py

Comment thread tests/test_plugins.py Outdated

danielvallance force-pushed the danielvallance/unikraft_backend branch from 8be4fdc to 17ddd83 Compare October 5, 2026 12:07

danielvallance requested a balanced review from Copilot October 5, 2026 12:09

Copilot started reviewing on behalf of danielvallance October 5, 2026 12:10 View session

Copilot AI reviewed Oct 5, 2026

View reviewed changes

Comment thread src/unikraft_cloud/core/http.py Outdated

danielvallance marked this pull request as draft October 5, 2026 15:04

danielvallance force-pushed the danielvallance/unikraft_backend branch 2 times, most recently from 48b2831 to 6f752ce Compare October 6, 2026 12:13

danielvallance changed the title ~~Sandbox related enhancement/fixes~~ feat: Sandbox related enhancement/fixes Oct 6, 2026

danielvallance requested a balanced review from Copilot October 6, 2026 12:14

danielvallance added 5 commits October 6, 2026 14:12

danielvallance force-pushed the danielvallance/unikraft_backend branch from 6f752ce to f7eada8 Compare October 6, 2026 13:23

danielvallance requested a balanced review from Copilot October 6, 2026 13:23

Copilot started reviewing on behalf of danielvallance October 6, 2026 13:24 View session

Copilot AI reviewed Oct 6, 2026

View reviewed changes

Comment thread tests/fake_sandbox.py Outdated

danielvallance added 13 commits October 6, 2026 14:33

danielvallance force-pushed the danielvallance/unikraft_backend branch from f7eada8 to c1ee6e5 Compare October 6, 2026 13:39

danielvallance marked this pull request as ready for review October 6, 2026 13:39

danielvallance requested review from aabedraba and nderjung October 6, 2026 13:39

aabedraba reviewed Oct 6, 2026

View reviewed changes

aabedraba approved these changes Oct 6, 2026

View reviewed changes

danielvallance merged commit 4fd7159 into prod-staging Oct 6, 2026
10 checks passed

danielvallance deleted the danielvallance/unikraft_backend branch October 6, 2026 14:17

Conversation

danielvallance commented Sep 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Breaking changes

Testing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nderjung Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danielvallance Sep 29, 2026

Choose a reason for hiding this comment

Uh oh!

danielvallance Oct 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

aabedraba left a comment

Choose a reason for hiding this comment

Uh oh!

aabedraba Oct 6, 2026

Choose a reason for hiding this comment

Uh oh!

danielvallance Oct 6, 2026

Choose a reason for hiding this comment

Uh oh!

aabedraba Oct 6, 2026

Choose a reason for hiding this comment

Uh oh!

aabedraba left a comment

Choose a reason for hiding this comment

danielvallance commented Sep 23, 2026 •

edited

Loading

nderjung Sep 24, 2026 •

edited

Loading