Make CellPose run with OME-Zarr

[tischi]

Hi @krokicki,

I found: https://bioimagetools.github.io/nf-module-aggregator/modules/janelia-module-bioformats2raw/

I have a question: Can one also specify a pixel size to this module?

The workflow I'm having in mind is to use CellPose to segment an image and then convert the output of CellPose to an OME-Zarr with proper pixel size, which would be read from the initial input image. Do you have a suggestion how to best implement such a workflow?

My idea was that we maybe would need a module that can extract metadata from an image and save it to a text file. And this metadata text file could be given as an optional input to a module that can convert an image to an OME-Zarr.

[krokicki]

That pipeline is just a wrapper for bioformats2raw. I'm not aware of any way to inject metadata via a parameter. Of course you can do this yourself with a little Python script that runs after Cellpose, but that seems like it might be brittle. I think ideally this should be addressed in Cellpose itself.

[tischi]

I came to the realization that there is a lot of trouble that comes with using the CellPose directly. Wrapping it ourselves in Python would allow us to make it directly read and write OME-Zarr with proper metadata; it also would allow us to directly read data from channel indices>3. I think I will give it a shot to write a CellPose module that can directly deal with OME-Zarr.

[krokicki]

@cgoina did that in our pipeline and might have some ideas.

[cgoina]

@tischi this repo is what is containerized for our cellpose nextflow module

[tischi]

Thanks @cgoina and @krokicki,

I spent a couple of hours today to code a much simpler Cellpose SAM OME-Zarr script. Can you access it? If yes, I'd be happy to get some feedback.

[krokicki]

@tischi Your implementation is very nice in its simplicity, and I like the use of ngff-zarr to handle the OME-Zarr access. I think we should do something like that instead of writing the OME-Zarr metadata by hand.

There are good reasons for the complexity that Cristian implemented, beyond even just dealing with huge images. I think that you may find yourself re-implementing some of the same features in the future. We might also want to consider working together on a Cellpose OME-Zarr module that addresses everyone's use cases.

I asked Claude to compare the two implementations above, and here's what it surfaced as differences (ignoring the obvious distributed computation angle):

Cellpose tuning parameters — This is probably the biggest one. cellpose-tools exposes all the key model.eval() parameters via CLI: diameter, min_size, max_size_fraction, niter, flow_threshold, cellprob_threshold, stitch_threshold, flow3D_smooth, and batch_size. Your script currently passes only do_3D and anisotropy, so everything else uses cellpose defaults. These parameters make a big difference for getting good results across different image types.

Normalization control — cellpose-tools has parameters for lowhigh, percentile, norm3D, sharpen_radius, smooth_radius, tile_norm_blocksize, tile_norm_smooth3D, and invert. Could be useful to expose at least a subset of these.

Preprocessing — cellpose-tools supports a configurable preprocessing pipeline (currently an unsharp mask with voxel-spacing-aware sigma). Not critical, but worth considering if you're working with noisy data.

Model selection — cellpose-tools lets you pick built-in models by name (e.g. cyto3, nuclei, default cpsam) via --model. Your script accepts a custom model file path or falls back to CellposeModel() with no model name — it would be handy to also support the built-in model names.

Multi-channel input — cellpose-tools accepts a list of channel indices, which allows dual-channel input (e.g. cytoplasm + nuclei). Your script currently selects a single channel.

image-label output metadata — cellpose-tools writes the OME-NGFF image-label block with auto-generated RGBA colors per label and a source.image back-reference to the input. This lets viewers link labels back to the source image and render color maps from the metadata.

Voxel spacing override — cellpose-tools has --voxel-spacing and --expansion-factor flags to override or adjust metadata-derived spacing, which can be useful when the metadata is missing or wrong.

[tischi]

Hi @krokicki,

Thank you very much for this analysis!

It will be great to work together on this!

A few thoughts:

Big image data: We will need to decide where the tiling and stitching should happen. Should this be on the Nextflow level or within the CellPose module. I am leaning towards doing this on the Nextflow level, because I think the same logic/code for tiling and stitching of the results could be reused for many segmentation models (see https://github.com/FrancisCrickInstitute/Segment-Flow/blob/master/modules/models/main.nf).

Multi-channel input: I was planning on adding support for this to our script.

CellPose internal model selection: To our best knowledge this is deprecated. When you specify --model it gives you a warning that it's not using it.

CellPose tuning parameters: Yes, many of them are important and should be exposed; for sure the diameter.