Cross Paradigm Representation and Alignment Transformer for Image Deraining

🔥🔥🔥 News

• 2025-02-10: This repo is released.

---

Abstract: Transformer-based networks have achieved strong performance in low-level vision tasks like image deraining by utilizing spatial or channel-wise self-attention. However, irregular rain patterns and complex geometric overlaps challenge single-paradigm architectures, necessitating a unified framework to integrate complementary global-local and spatial-channel representations. To address this, we propose a novel Cross Paradigm Representation and Alignment Transformer (CPRAformer). Its core idea is the hierarchical representation and alignment, leveraging the strengths of both paradigms (spatial-channel and global-local) to aid image reconstruction. It bridges the gap within and between paradigms, aligning and coordinating them to enable deep interaction and fusion of features. Specifically, we use two types of self-attention in the Transformer blocks: sparse prompt channel self-attention (SPC-SA) and spatial pixel refinement self-attention (SPR-SA). SPC-SA enhances global channel dependencies through dynamic sparsity, while SPR-SA focuses on spatial rain distribution and fine-grained texture recovery. To address the feature misalignment and knowledge differences between them, we introduce the Adaptive Alignment Frequency Module (AAFM), which aligns and interacts with features in a two-stage progressive manner, enabling adaptive guidance and complementarity. This reduces the information gap within and between paradigms. Through this unified cross-paradigm dynamic interaction framework, we achieve the extraction of the most valuable interactive fusion information from the two paradigms. Extensive experiments demonstrate that our model achieves state-of-the-art performance on eight benchmark datasets and further validates CPRAformer's robustness in other image restoration tasks and downstream applications.

---

⚙️ Dependencies

• Python 3.8
• PyTorch 1.9.0
• NVIDIA GPU + CUDA

# Clone the github repo and go to the default directory 'DAIT'.
git clone https://github.com/zs1314/DAIT.git
conda create -n DAIT python=3.8
conda activate DAIT
pip install matplotlib scikit-image opencv-python yacs joblib natsort h5py tqdm

Install warmup scheduler

cd pytorch-gradual-warmup-lr; python setup.py install; cd ..

⚒️ TODO

• Release code

🔗 Contents

1. Datasets
2. Training
3. Testing
4. Results
5. Citation
6. Contact
7. Acknowledgements

---

🖨️ Datasets

Used training and testing sets can be downloaded as follows:

Training Set	Testing Set	Visual Results
Rain13K[complete training dataset: Google Drive / Baidu Disk]	Test100 + Rain100H + Rain100L + Test2800 + Test1200 [complete testing dataset: Google Drive / Baidu Disk]	Google Drive / Baidu Disk

Download training and testing datasets and put them into the corresponding folders of Datasets/. See Datasets for the detail of the directory structure.

🔧 Training

• Download training (Rain13K, already processed) and testing (Test100 + Rain100H + Rain100L + Test2800 + Test1200 , already processed) datasets, place them in Datasets/.

• Run the following scripts. The training configuration is in training.yml.

  python train.py

• The training experiment is in checkpoints/.

🔨 Testing

• Download testing (Test100 + Rain100H + Rain100L + Test2800 + Test1200) datasets, place them in Datasets/.

• Run the following scripts. The testing configuration is in test.py.

    python test.py

• The output is in results/.

• To reproduce PSNR/SSIM scores of the paper, run (in matlab):

    evaluate_PSNR_SSIM.m 

🔎 Results

We achieve state-of-the-art performance. Detailed results can be found in the paper.

Quantitative Comparison (click to expand)

• results in Table 1 of the main paper

• results in Table 2 of the main paper

• results in Table 2 of the supplementary material

• results in Table 1 of the supplementary material

• results in Table 5 of the supplementary material

Visual Comparison (click to expand)

• results in Figure 5 of the supplementary material

• results in Figure 6 of the main paper

• results in Figure 4 of the supplementary material

• results in Figure 7 of the supplementary material

• results in Figure 9 of the supplementary material

• results in Figure 2 of the supplementary material

• results in Figure 1 of the supplementary material

• results in Figure 11 of the supplementary material

📂 Contact

Should you have any question, please contact shunzou.njau@gmail.com

📎 Citation

We kindly request that you cite our work if you utilize the code or reference our findings in your research:

@inproceedings{zou2025cpraformer,
  title={Cross Paradigm Representation and Alignment Transformer for Image Deraining},
  author={Zou, Shun and Zou, Yi and Li, Juncheng and Gao, Guangwei and Qi, Guojun},
  booktitle={ACM MM},
  year={2025}
}

💡 Acknowledgements

This code is built on MPRNet.