Skip to content

feat(simulation): scene-package cooking + MuJoCo scene loading#2544

Draft
Nabla7 wants to merge 1 commit into
mainfrom
pim/feat/scene-cooking
Draft

feat(simulation): scene-package cooking + MuJoCo scene loading#2544
Nabla7 wants to merge 1 commit into
mainfrom
pim/feat/scene-cooking

Conversation

@Nabla7

@Nabla7 Nabla7 commented Jun 20, 2026

Copy link
Copy Markdown
Collaborator

Offline cooking of 3D scene assets into scene packages, and runtime loading of cooked packages into MuJoCo.

  • scene_assets/{spec,mesh_scene}: the ScenePackage contract + mesh/USD import. load_scene_package() consumes cooked artifacts at runtime; it does not perform the heavy bake.
  • mujoco/{scene_mesh_to_mjcf,collision_spec}: bake a scene mesh into an MJCF wrapping the robot, deciding each prim as a MuJoCo primitive, convex hull (coacd), or mesh.
  • mujoco/entity_scene: compose cooked entities into the model via MjSpec.
  • scenes/{catalog,office}: named scene packages.
  • experimental/pimsim/scene/*: the cook orchestration (plan, visual GLB, optional Blender bake, collision hulls, sidecar, inspect).

Adds the 'scene' extra (open3d, trimesh, coacd, usd-core); the Blender visual bake is an optional external 'blender' binary, not a wheel. Extends the mujoco MjSpec type stubs.

Self-contained: the closure imports only dimos.utils. The sim-module runtime wiring that drives this lands with the sim refactor.

Problem

Closes DIM-XXX

Solution

How to Test

Contributor License Agreement

  • I have read and approved the CLA.

Offline cooking of 3D scene assets into scene packages, and runtime
loading of cooked packages into MuJoCo.

- scene_assets/{spec,mesh_scene}: the ScenePackage contract + mesh/USD
  import. load_scene_package() consumes cooked artifacts at runtime; it
  does not perform the heavy bake.
- mujoco/{scene_mesh_to_mjcf,collision_spec}: bake a scene mesh into an
  MJCF wrapping the robot, deciding each prim as a MuJoCo primitive,
  convex hull (coacd), or mesh.
- mujoco/entity_scene: compose cooked entities into the model via MjSpec.
- scenes/{catalog,office}: named scene packages.
- experimental/pimsim/scene/*: the cook orchestration (plan, visual GLB,
  optional Blender bake, collision hulls, sidecar, inspect).

Adds the 'scene' extra (open3d, trimesh, coacd, usd-core); the Blender
visual bake is an optional external 'blender' binary, not a wheel.
Extends the mujoco MjSpec type stubs.

Self-contained: the closure imports only dimos.utils. The sim-module
runtime wiring that drives this lands with the sim refactor.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant