Skip to content

Add RAVEN benchmark support with RavenDPL (--n_values 3|4) - #3

Open
jucamohedano wants to merge 16 commits into
unitn-sml:mainfrom
jucamohedano:raven
Open

Add RAVEN benchmark support with RavenDPL (--n_values 3|4)#3
jucamohedano wants to merge 16 commits into
unitn-sml:mainfrom
jucamohedano:raven

Conversation

@jucamohedano

@jucamohedano jucamohedano commented Aug 18, 2026

Copy link
Copy Markdown

Add support for the RAVEN dataset (Zhang et al., 2019, "RAVEN: A Dataset for Relational and Analogical Visual Reasoning"), an analog of John C. Raven's 1938 Raven's Progressive Matrices for structural, relational, and analogical visual reasoning. Each problem is a matrix of grayscale panels, and the model selects the candidate that completes it. The README.md documents the task and links to the data-generation instructions below.

Added

  • rss/backbones/raven_encoder.py, shared panel encoder (RavenMLP).
  • rss/datasets/raven.py, dataset registration, loaders, labels, backbone wiring.
  • rss/datasets/utils/raven_creation.py, loads .npz panels and .xml concept metadata.
  • rss/models/ravendpl.py, RavenDPL factorized DeepProbLog model.
  • rss/models/utils/utils_problog.py, RAVEN logic matrices.
  • rss/utils/{checkpoint,dpl_loss,losses,metrics,train}.py, RAVEN training, eval, diagnostics, checkpoints.
  • rss/utils/args.py, adds --n_values (restricted to 3 or 4, default 3).
  • README.md + .github/raven-example.png, task docs and attribution.

Existing code is unaffected. The addition is purely additive. New model/dataset registrations, args choices with backward-compatible defaults, and the behavior changes in metrics.py/train.py are gated behind args.dataset == "raven"; checkpoint.py/train.py route artifacts through --output_dir (default ".", so existing paths are preserved). The branch sits cleanly on top of upstream main with no divergence.

Datasets. RAVEN panel data is generated and is not stored in this repo, using the reduced-dataset generator at jucamohedano/RAVEN (a fork of WellyZhang/RAVEN):

  • branch raven-3x3x3 (commit 65c6ba9), for the three-value dataset;
  • branch raven-4x4x4 (commit 8fd5eb8), for the four-value dataset.

From the chosen branch, generate the splits with its Python 2.7 environment:

python src/dataset/main.py --num-samples 5000 --save-dir <output>/RAVEN-3x3x3

The defaults (--val 2 --test 2 --seed 1234) produce 3000 train / 1000 val / 1000 test. Use --save-dir <output>/RAVEN-4x4x4 on the raven-4x4x4 branch for the four-value dataset, then place the generated .npz panels and matching .xml concept labels under rss/data, expected at:

rss/data/RAVEN-3x3x3/center_single/
  RAVEN_*_train.npz
  RAVEN_*_train.xml
  RAVEN_*_val.npz
  RAVEN_*_val.xml
  RAVEN_*_test.npz
  RAVEN_*_test.xml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant