Task-Driven Implicit Representations for Automated Design of LiDAR Systems

Project page  ·  Paper

Behari, N., Young, A., Klinghoffer, T., Dave, A., & Raskar, R. Task-Driven Implicit Representations for Automated Design of LiDAR Systems. CVPR 2026.

LiDAR system design, choosing how many sensors to use, where to place them, what field of view they should have, and how to time-gate their detectors, is a complex, largely manual process. We represent LiDAR configurations in a continuous 6D design space (x, y, z, φ, ψ, τ) and learn task-specific implicit densities over this space via flow-based generative modeling. New LiDAR systems are then synthesized by modeling sensors as parametric distributions in 6D and fitting them to the learned density via expectation–maximization, enabling efficient, constraint-aware LiDAR system design.

Workflow

To run an experiment, step through prepare → train → synthesize → evaluate, each driven by the same YAML config:

1. prepare      (if required) build the task dataset
2. train        fit the implicit density
3. synthesize   fit a sensor mixture (EM) to the density
4. evaluate     simulate the design and score it

For example, warehouse detection:

python -m implicitlidar.experiments.warehouse_detection.prepare    --out outputs/data/warehouse
python -m implicitlidar.experiments.warehouse_detection.train      --config implicitlidar/experiments/warehouse_detection/configs/default.yaml
python -m implicitlidar.experiments.warehouse_detection.synthesize --config implicitlidar/experiments/warehouse_detection/configs/default.yaml
python -m implicitlidar.experiments.warehouse_detection.evaluate   --config implicitlidar/experiments/warehouse_detection/configs/default.yaml

Numerical results are written to outputs/runs/<experiment>/results/*.csv.

Install

Python 3.10+ and a CUDA-capable GPU (recommended, falls back to CPU). Everything installs via pip from pyproject.toml:

git clone https://github.com/NikhilBehari/implicitlidar.git
cd implicitlidar
conda create -n implicitlidar python=3.11 -y
conda activate implicitlidar

# Base install: face_scanning, warehouse_detection, real_world
pip install -e .

# Optional extras (install only what you need)
pip install -e ".[robot]"      # robot_tracking      (mujoco, pytorch_kinematics, pybullet)
pip install -e ".[emitter]"    # emitter_design      (mitsuba 3, mitransient)
pip install -e ".[dev]"        # linters, jupyter, pytest

Verify the install:

pytest -q

The KUKA IIWA 14 model (MuJoCo Menagerie, Apache 2.0) and the Mitsuba scene template for the emitter-design experiment are vendored under assets/; see assets/README.md for attribution.

Experiments

Run any experiment's prepare / train / synthesize / evaluate commands, or kick off the full sweep:

bash run_all.sh

Experiment	Task
face_scanning	Smartphone flash-LiDAR design for 3D face mesh reconstruction (Basel Face Model).
robot_tracking	Distributed ceiling-mounted LiDAR design for end-effector tracking (KUKA IIWA in MuJoCo); a paired config toggles the visibility term for the occlusion ablation.
warehouse_detection	Motion-adaptive scanning design for warehouse object detection.
emitter_design	Constraint-aware emitter synthesis for a fixed detector, evaluated by Mitsuba 3 transient rendering.
real_world	Real single-photon LiDAR mesh reconstruction using designs synthesized in face_scanning.

Layout

implicitlidar/
├── implicitlidar/      Python package: core/, scenes/, eval/, utils/, experiments/
├── assets/             KUKA model, Mitsuba scene, README figures
├── tests/              unit tests
├── docs/               project page
├── run_all.sh          end-to-end driver
└── outputs/            datasets and run outputs (gitignored)

Citation

@inproceedings{behari2026implicitlidar,
  title     = {Task-Driven Implicit Representations for Automated Design of LiDAR Systems},
  author    = {Behari, Nikhil and Young, Aaron and Klinghoffer, Tzofi and Dave, Akshat and Raskar, Ramesh},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2026},
}

Acknowledgments and license

This repository builds on the open-source work of normflows, pytorch_volumetric, pytorch_kinematics, MuJoCo Menagerie, Mitsuba 3, and mitransient.

Released under the MIT License.