2Faculty of Applied Sciences, Ukrainian Catholic University, Lviv, Ukraine
3Department of Electronic Engineering, Micro- and Biomedical Electronics,
Igor Sikorsky Kyiv Polytechnic Institute, Kyiv, Ukraine
4STACC OÜ, Tartu, Estonia
5Better Medicine OÜ, Tartu, Estonia
Overview of QCell. QCell builds on a MaskDINO-style query-based segmentation pipeline, where multi-scale image features and positional embeddings are processed by the encoder and refined by transformer decoder layers with content and DN queries. The proposed modules are shown above: (a) instance recombination decomposes each query into amodal, visible, and occluded sub-representations and recombines them into a refined full-instance query; (b) contrastive query learning uses matched instance queries q̂i by Hungarian matching as anchors, corresponding DN queries d̂+i across all groups as positives and other DN queries as negatives to align queries of the same cell and separate queries of different cells in latent space.
This is the official repository for the paper:
QCell: Recombining and Aligning Cell Queries for Overlapping Instance Segmentation
Yaroslav Prytula, Anton Popov, Dmytro Fishman
British Machine Vision Conference (BMVC), 2026
- Overview
- Installation
- Datasets
- Training
- Inference and Evaluation
- License
- Citation
- Contact
- Acknowledgments
Overlapping-cell instance segmentation is challenging because semi-transparent structures create weak boundaries and mixed visual evidence in overlap regions. QCell introduces two complementary components for global reasoning over overlapping objects:
- Instance recombination decomposes each object query into amodal, visible, and occluded sub-representations, then recombines them into a refined full-instance query.
- Contrastive query alignment uses Hungarian-matched instance queries as anchors, corresponding denoising queries as positives, and denoising queries from other cells as negatives.
QCell improves overlapping-cell separation across multiple microscopy benchmarks, including ISBI 2014, Revvity-25, and the new Organoids dataset.
The development environment uses Python 3.10, PyTorch 2.11, CUDA 12.8, and Detectron2 0.6. Full installation instructions will be finalized with the source-code release.
conda create -n qcell python=3.10 -y
conda activate qcell
pip install torch==2.11.0 torchvision==0.26.0 \
--index-url https://download.pytorch.org/whl/cu128
# Available after the source-code branch is released.
git clone --branch main https://github.com/SlavkoPrytula/QCell.git
cd QCell
pip install -r requirements.txt
pip install 'git+https://github.com/facebookresearch/detectron2.git'
cd maskdino/modeling/pixel_decoder/ops
bash make.sh
cd ../../../..QCell expects COCO-style instance annotations with amodal, visible, and occluded masks. Dataset paths are configured in the registration modules under maskdino/data/datasets/.
Organoids is a new brightfield microscopy benchmark introduced with QCell. It contains 1,186 training images, 1,199 validation images, and 201 test images at a resolution of 540 × 540 pixels. The dataset contains dense, highly overlapping scenes, with up to 223 instances in a single image.
Organoids. Example microscopy images and ground-truth instance annotations from the Organoids dataset.
The Organoids dataset is available upon request.
Revvity-25 is a brightfield microscopy cell instance-segmentation dataset with detailed modal and amodal annotations.
Revvity-25/
├── images/
└── annotations/
├── train.json
└── valid.json
Download the ISBI 2014 Cell Segmentation Challenge dataset and convert its annotations to COCO format.
ISBI2014/
├── isbi_train/
├── isbi_val/
├── isbi_test/
└── annotations/
├── isbi_train.json
├── isbi_val.json
└── isbi_test.json
The QCell configuration combines instance recombination (IR) and contrastive alignment (CA). Example configurations are provided for all three benchmarks:
configs/ISBI2014-AmodalSeg/experiments/ablations/
maskdino_R50_isbi2014_cyto_QCell_IR_CA.yaml
configs/Revvity-AmodalSeg/experiments/ablations/
maskdino_R50_revvity_QCell_IR_CA.yaml
configs/Organoids-AmodalSeg/experiments/ablations/
maskdino_R50_organoids_QCell_IR_CA.yaml
Train on one GPU with:
python train_net.py \
--config-file configs/ISBI2014-AmodalSeg/experiments/ablations/maskdino_R50_isbi2014_cyto_QCell_IR_CA.yaml \
--num-gpus 1For multi-GPU training, change --num-gpus to the number of GPUs available on the machine.
Evaluate a trained checkpoint with:
python train_net.py \
--config-file configs/ISBI2014-AmodalSeg/experiments/ablations/maskdino_R50_isbi2014_cyto_QCell_IR_CA.yaml \
--eval-only \
MODEL.WEIGHTS /path/to/model_best.pthPretrained checkpoint links and benchmark-specific inference commands will be added with the model release.
The project-website content is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License.
The source-code license will be published with the code release.
If you use QCell in your research, please cite:
<TODO>- Yaroslav Prytula: yaroslav.prytula@ut.ee, s.prytula@ucu.edu.ua
- Anton Popov: a.popov@ucu.edu.ua
- Dmytro Fishman: dmytro.fishman@ut.ee
The authors acknowledge the support of the European Union and the Estonian Research Council through project TEM-TA101. Computational resources were provided by the High-Performance Computing Cluster at the University of Tartu 🇪🇪. We thank the Biomedical Computer Vision Lab for its invaluable support. We thank Revvity and the Institut de Recherche en Santé Digestive (IRSD), Inserm UMR 1220, Toulouse, France, for jointly providing the Organoids dataset and supporting its annotation and validation. We express our gratitude to the Armed Forces of Ukraine 🇺🇦 and the bravery of the Ukrainian people for enabling a secure working environment, without which this work would not have been possible.
📄 Paper: coming soon
🤗 Dataset: Revvity-25
⭐ GitHub: SlavkoPrytula/QCell
🌐 Project page: slavkoprytula.github.io/QCell