Skip to content

feat: add wait helpers to run all cells - #9545

Open
pjwerneck wants to merge 4 commits into
pjwerneck/validate-peer-setupfrom
pjwerneck/wait-for-helpers
Open

pjwerneck wants to merge 4 commits into
pjwerneck/validate-peer-setupfrom
pjwerneck/wait-for-helpers

Conversation

@pjwerneck

Copy link
Copy Markdown
Collaborator

Summary

Adds wait helpers to the RDS and enclave clients, so a notebook cell waits until the other party is
done. Two demo notebooks can then run all their cells without anyone re-running cells by hand.
Depends on #9537.

Changes

  • wait_until_peered, wait_until_has_dataset and wait_until_has_job on SyftRDSClient and
    SyftEnclaveClient. wait_until_attested on SyftEnclaveClient.
  • wait_until_has_job takes status= and where=, and fails at once on a final status that was
    not asked for. A job name used by more than one submitter or datasite raises ValueError; repeated
    runs of the same job resolve to the newest.
  • Peer waits poll with one Drive request; job and dataset waits sync every 15 s. Default timeout:
    300 s.
  • The 14 enclave demo notebooks use the helpers instead of re-run cells.

Testing

  • 36 new tests on mock Drive, including one that runs the DS, both DOs and the enclave in parallel
    threads, linked only by the waits.
  • The notebooks were not run against real Drive.

Asana task

wait_until_peered, wait_until_has_dataset and wait_until_has_job wait
until the other party is done, so two notebooks run end to end without
re-run cells. wait_until_has_job takes a status and a where= condition,
and fails at once on a final status that was not asked for. The enclave
client adds wait_until_attested. The demo notebooks use the helpers.
@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@pjwerneck
pjwerneck added this pull request to stack #9546 October 1, 2026 18:48
@pjwerneck
pjwerneck requested a review from rasswanth-s October 1, 2026 18:48
@pjwerneck pjwerneck changed the title Pjwerneck/wait for helpers feat: add wait helpers so demo notebooks can run all cells Oct 1, 2026
@pjwerneck pjwerneck changed the title feat: add wait helpers so demo notebooks can run all cells feat: add wait helpers to run all cells Oct 1, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant