Skip to content

Publish a container image and add serve flags for remote use - #560

Merged
pauladkisson merged 12 commits into
mainfrom
containerize
Oct 16, 2026
Merged

pauladkisson merged 12 commits into
mainfrom
containerize

Conversation

@pauladkisson

@pauladkisson pauladkisson commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #358

Details (AI-generated)

Closes #358. The how-to for using the published image is in #569, which stays in draft until the image is on ghcr.

  • Dockerfile on python:3.12-slim with Debian's chromium and chromium-driver for the dashboard's PNG/SVG export. It installs GuPPy from the build context, and its entrypoint is guppy --no-browser --port 5006, so arguments such as --input-root are appended. .dockerignore allowlists only what the install needs.
  • .github/workflows/container-image.yml builds linux/amd64 and runs --version in the image. Pull requests that touch the image's inputs only build it. A published release, or a manual dispatch, also pushes it to ghcr.io/lernerlab/guppy. A release pushes the release tag and latest; a dispatch from main pushes the tag main.
  • guppy --port, --address, and --no-browser, passed through serve_app to pn.serve. Defaults are unchanged: a free port, every interface, and a browser opened. serve_app prints GuPPy is starting. Open the address below in your browser, and press Ctrl+C here to stop GuPPy. and passes verbose=True, so Panel follows it with Launching server at <url> once it is serving, and the image sets PYTHONUNBUFFERED=1 so both lines reach docker run output immediately.

The image is amd64 only. The numcodecs version that zarr<3 resolves to has no Linux arm64 wheel, so an arm64 build fails without a compiler.

Checking the image with a local build

Both walkthroughs build the image from this branch, so nothing has to be published first. The Docker commands use the paths on my machine: the branch checkout at /Users/pauladkisson/Documents/CatalystNeuro/Guppy/worktrees/containerize and the main checkout's /Users/pauladkisson/Documents/CatalystNeuro/Guppy/GuPPy/testing_data. They use the full testing_data/ download (the one the full_data tests read): its folders are the input root, and its output_folder is the output root. testing_data/ is mounted read-only with output_folder mounted writable on top, so GuPPy can only write into output_folder. On an Apple silicon Mac the build and the container run under emulation, so expect the build to take several minutes and the app to be slower than a native install.

Docker

  1. Build the image:

    docker build --platform linux/amd64 -t guppy:local /Users/pauladkisson/Documents/CatalystNeuro/Guppy/worktrees/containerize
  2. Check that it starts and reports the branch's version:

    docker run --rm --platform linux/amd64 guppy:local --version
  3. Start GuPPy on testing_data/:

    docker run --rm --platform linux/amd64 -p 127.0.0.1:5006:5006 -v /Users/pauladkisson/Documents/CatalystNeuro/Guppy/GuPPy/testing_data:/data:ro -v /Users/pauladkisson/Documents/CatalystNeuro/Guppy/GuPPy/testing_data/output_folder:/data/output_folder guppy:local --input-root /data --output-root /data/output_folder

    Once the server is up, the terminal prints:

    GuPPy is starting. Open the address below in your browser, and press Ctrl+C here to stop GuPPy.
    Launching server at http://localhost:5006
    
  4. Open http://localhost:5006. The folder selector should start at /data and list the SampleData_* folders. Reload once to check that a second load also renders.

  5. Open SampleData_csv, select sample_data_csv_1, and run it through the pipeline: label the stores (Sample_Control_Channel as control_region, Sample_Signal_Channel as signal_region, Sample_TTL as ttl), then Read Raw Data, Preprocess, and PSTH. Each step's result page opens in the same browser on port 5006. The new run folder appears under /Users/pauladkisson/Documents/CatalystNeuro/Guppy/GuPPy/testing_data/output_folder/SampleData_csv/.

  6. Open Visualization and save a plot as PNG and as SVG. That checks the Chromium and ChromeDriver inside the image.

  7. Stop the container with Ctrl+C in its terminal. --rm removes it.

Apptainer

Apptainer runs on Linux, usually a shared cluster, and clusters rarely have Docker. So the image is built with Docker on your own machine, saved as an archive, and converted to an Apptainer image on the cluster. The cluster also needs a copy of testing_data/. The cluster-side paths and hosts stay as placeholders: <user>, <login-host>, and <node> are your cluster login, and /path/to/testing_data is where the copy lives there.

  1. On your own machine, build the image as in Docker step 1 and save it:

    docker save guppy:local -o ~/guppy_local.tar
  2. Copy the archive to the cluster:

    scp ~/guppy_local.tar <user>@<login-host>:~/
  3. On the cluster, load Apptainer if your cluster uses modules (check with apptainer --version), and convert the archive:

    apptainer build guppy_local.sif docker-archive://guppy_local.tar

    If the build fails with "disk quota exceeded", /tmp is too small for the unpacked image. Point Apptainer's scratch space at a folder on disk with APPTAINER_TMPDIR=~/apptainer_tmp (create it first) in front of the same command.

  4. Start GuPPy on the node you will use, serving on that node only:

    apptainer run --bind /path/to/testing_data:/testing_data:ro --bind /path/to/testing_data/output_folder:/testing_data/output_folder guppy_local.sif --address 127.0.0.1 --port 5006 --input-root /testing_data --output-root /testing_data/output_folder

    Once the server is up, the terminal prints:

    GuPPy is starting. Open the address below in your browser, and press Ctrl+C here to stop GuPPy.
    Launching server at http://127.0.0.1:5006
    

    If port 5006 is taken on that node, pick another and use it in step 5 too.

  5. On your own machine, forward the port through the login host to that node (hostname on the node prints its name):

    ssh -L 5006:localhost:5006 -J <user>@<login-host> <user>@<node>
  6. Open http://localhost:5006 and repeat Docker steps 4 to 6, with the folder selector starting at /testing_data. The run folder appears under testing_data/output_folder/SampleData_csv/ on the cluster.

  7. Stop GuPPy with Ctrl+C on the node and close the SSH session.

Verification

Built the amd64 image locally from this branch under emulation on an Apple silicon Mac. --version reports 2.0.0b5. Docker steps 1 to 4 work: step 3 prints the startup message and Launching server at http://localhost:5006, and with its mounts the homepage served on 127.0.0.1:5006 lists the SampleData_* folders, and a second load renders with no console errors. Chromium and ChromeDriver 155 are present in the image. Earlier on this branch, with the same image contents, Preprocess ran and opened its result view, Visualization opened, and PNG and SVG export worked inside the container. Mapping host port 8080 to 5006 is refused by Bokeh's websocket origin check. The Apptainer walkthrough was run in an Ubuntu 26.04 arm64 VM under Parallels on an Apple silicon Mac, standing in for both login and compute node: Apptainer from its PPA, qemu-user-binfmt to run the amd64 image (Apptainer warns that the architecture does not match), and the build needed APPTAINER_TMPDIR because /tmp is RAM-backed. GuPPy served with the read-only and writable binds and was reached through ssh -L from the Mac. It works, but very slowly under QEMU emulation. A real cluster with its modules and scheduler is still untested. tests/unit/test_main.py and tests/unit/test_app.py: 22 passed.

🤖 Generated with Claude Code

pauladkisson added a commit that referenced this pull request Oct 14, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

pauladkisson and others added 3 commits October 15, 2026 12:20
Adds a Dockerfile (with Chromium for plot export), a workflow that
builds the image on pull requests touching it and pushes it to
ghcr.io/lernerlab/guppy on each release, and a how-to page for running
it with Apptainer on a cluster or with Docker. guppy gains --port,
--address and --no-browser so the server can be reached through a known
port from another machine.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pages now open the same way locally and in a container, so the section
describing the blocked-tab notification no longer applies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pauladkisson and others added 4 commits October 15, 2026 13:53
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Base automatically changed from single_port to main October 15, 2026 21:27
pauladkisson and others added 3 commits October 16, 2026 08:46
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pauladkisson and others added 2 commits October 16, 2026 13:22
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pauladkisson
pauladkisson marked this pull request as ready for review October 16, 2026 21:14
@pauladkisson
pauladkisson merged commit 99f46e8 into main Oct 16, 2026
20 checks passed
@pauladkisson
pauladkisson deleted the containerize branch October 16, 2026 21:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Containerization

1 participant