feat(robot): add XLeRobot support — omni base, dual arms, head#2493
feat(robot): add XLeRobot support — omni base, dual arms, head#2493a-bissell wants to merge 2 commits into
Conversation
Add full DimOS integration for the XLeRobot mobile manipulator: Hardware driver (connection.py): - Dual Feetech STS3215 servo buses (bus 1: left arm + head, bus 2: right arm + wheels) - Omni-wheel inverse/forward kinematics for 3-wheel holonomic base - Position-mode arm control with safety clamping - Velocity-mode wheel control - USB camera via OpenCV - Calibration loading from file - Graceful handling of missing motors (strict_handshake=False) - macOS + Linux serial port auto-detection DimOS module (xlerobot_module.py): - cmd_vel input (Twist → wheel velocities) - color_image, odom, joint_states outputs - Dead-reckoning odometry from wheel encoders - 20 Hz sensor polling loop - Skills: move_base, stop_base, move_arm, move_head, open/close_gripper, get_joint_positions, home_arms, observe - Joint state recording to disk Blueprints: - xlerobot-basic: connection + camera + Rerun visualization - xlerobot-agentic: basic + MCP server/client for LLM agent control Also includes: - Interactive calibration script (python -m dimos.robot.xlerobot.calibrate) - XLeRobotConfig with full servo ID mapping and PID tuning - XLeRobotSpec protocol for dependency injection - Optional dependency: dimos[xlerobot] installs lerobot[feetech] Requires: pip install lerobot[feetech] (or uv pip install dimos[xlerobot]) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
a-bissell
requested review from
leshy,
mustafab0,
paul-nechifor and
spomichter
as code owners
Greptile SummaryThis PR adds full DimOS integration for the XLeRobot mobile manipulator — a 3-wheel omni-directional base with dual 6-DOF arms and a 2-DOF head (17 DOF total). The implementation addresses the previously noted concurrency issue (per-bus
Confidence Score: 5/5Safe to merge — new hardware integration with no impact on existing modules and both previously flagged concurrency issues resolved. Both previously flagged issues are resolved. The remaining findings are minor quality items that do not affect correctness under default configuration. dimos/robot/xlerobot/xlerobot_module.py — two inline suggestions worth a quick look. Important Files Changed
Reviews (2): Last reviewed commit: "fix(xlerobot): address PR review — seria..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| def _on_cmd_vel(self, twist: Twist) -> None: | ||
| if self._driver and self._driver.connected: | ||
| self._driver.move( | ||
| x=twist.linear.x, | ||
| y=twist.linear.y, | ||
| theta=math.degrees(twist.angular.z), | ||
| ) | ||
|
|
||
| def _sensor_loop(self) -> None: | ||
| """Poll camera, wheel encoders, and joint states at ~20 Hz.""" | ||
| while self._running: | ||
| try: | ||
| if self._driver and self._driver.connected: | ||
| self._update_camera() | ||
| self._update_odom() | ||
| self._update_joint_states() | ||
| time.sleep(0.05) | ||
| except Exception as e: | ||
| logger.debug(f"Sensor loop error: {e}") | ||
| time.sleep(0.1) |
There was a problem hiding this comment.
_sensor_loop runs in a dedicated background thread and calls sync_read on both buses every 50 ms (_update_odom → read_wheel_velocities on _bus2; _update_joint_states → read_joint_positions on both buses). Concurrently, _on_cmd_vel calls driver.move() → _bus2.sync_write("Goal_Velocity", ...), and any @skill invocation (move_arm, open_gripper, etc.) issues additional sync_write calls. Serial-port communication is not re-entrant; two concurrent sync_read/sync_write operations on the same FeetechMotorsBus will interleave bytes, corrupting packets and potentially sending garbage position targets to the arms or wheels.
A threading.Lock per bus (or a single driver-level lock) acquired around every sync_read/sync_write call is needed to serialize access.
| def _wheel_raw_to_body(self, raw: dict[str, int]) -> dict[str, float]: | ||
| """Convert wheel feedback back to body-frame velocities.""" | ||
| wheel_degps = np.array([_raw_to_degps(raw[n]) for n in self._wheel_names]) | ||
| wheel_linear = np.radians(wheel_degps) * self._config.wheel_radius | ||
| body = self._kin_matrix_inv.dot(wheel_linear) | ||
| return {"x_vel": body[0], "y_vel": body[1], "theta_vel": np.degrees(body[2])} |
There was a problem hiding this comment.
_kin_matrix_inv is always 3×3 (constructed in __init__ regardless of how many motors are later pruned). When strict_handshake=False and one or more wheel motors fail to ping, _prune_missing_motors removes them from _wheel_names. _wheel_raw_to_body then builds wheel_degps = np.array([...]) with 1 or 2 elements, and self._kin_matrix_inv.dot(wheel_linear) raises ValueError: shapes (3,3) and (N,) not aligned. This crashes read_wheel_velocities and get_observation — the latter being called unconditionally in _sensor_loop — so the sensor thread dies (silently swallowed at logger.debug) and odometry stops updating entirely.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: eb9387c413
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| data=frame, | ||
| height=frame.shape[0], | ||
| width=frame.shape[1], | ||
| encoding="bgr8", | ||
| ) |
There was a problem hiding this comment.
When the camera returns any frame, these keywords are passed to dimos.msgs.sensor_msgs.Image, but that dataclass only accepts data, format, frame_id, and ts (or Image.from_numpy). This raises TypeError inside _update_camera, which the sensor loop catches and retries forever, so color_image is never published and the observe() skill remains empty for all XLeRobot runs with a working camera.
Useful? React with 👍 / 👎.
| else: | ||
| _vis = _transports_base | ||
|
|
||
| xlerobot_basic = autoconnect( |
There was a problem hiding this comment.
Adding this module-level blueprint requires regenerating dimos/robot/all_blueprints.py; I checked the generated registry and it has no xlerobot-basic or xlerobot-agentic entries. The CLI lookup in get_all_blueprints.py only consults that generated file, so dimos list/dimos run xlerobot-basic will treat the new blueprint as unknown, and the registry-generation test will report the file as stale.
Useful? React with 👍 / 👎.
| xlerobot = [ | ||
| "lerobot[feetech]", | ||
| ] |
There was a problem hiding this comment.
This adds a new xlerobot extra, but the aggregate all extra just below still omits it despite being the path used by the install docs/scripts (uv sync --extra all). In that standard install, the blueprint can import because the lerobot import is caught, but starting the module constructs XLeRobotDriver and immediately raises the "lerobot[feetech] is required" ImportError, so users following the normal full install cannot run the new robot.
Useful? React with 👍 / 👎.
❌ 4 Tests Failed:
View the top 3 failed test(s) by shortest run time
To view more test analytics, go to the Test Analytics Dashboard |
…ra, lint - Serialize all Feetech bus IO behind a reentrant _io_lock (read/write/ping/ connect/disconnect) to prevent interleaved access from sensor + control threads - Guard 3-wheel base math: return zeros / skip instead of crashing the sensor thread on a pruned wheel count - Build head-camera frames via Image.from_numpy(..., ImageFormat.BGR) instead of invalid height/width/encoding kwargs - Wrap calibrate bus calibration in try/finally so both buses disconnect on error - Add xlerobot to the `all` extra in pyproject.toml - CI: drop __init__.py files, use setup_logger() over logging.getLogger, remove section-marker comments, regenerate all_blueprints.py, ruff cleanups Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 participant
Summary
Adds full DimOS integration for the XLeRobot mobile manipulator — a 3-wheel omni-directional base with dual 6-DOF Feetech STS3215 arms and a 2-DOF head (17 DOF total).
connection.py): dual serial bus control, omni-wheel IK/FK, position-mode arms with safety clamping, velocity-mode wheels, USB camera, calibration loading, macOS + Linux serial port auto-detectionxlerobot_module.py): cmd_vel input, color_image/odom/joint_states outputs, 20 Hz sensor loop, dead-reckoning odometry, 9 skills (move_base, stop_base, move_arm, move_head, open/close_gripper, get_joint_positions, home_arms, observe)xlerobot-basic(connection + camera + Rerun viz) andxlerobot-agentic(+ MCP server/client for LLM agent control)python -m dimos.robot.xlerobot.calibrate— interactive calibration for arm/head servosdimos[xlerobot]installslerobot[feetech]Hardware specs
macOS support
Serial ports auto-detect platform:
/dev/cu.usbmodem*on macOS vs/dev/ttyACM*on Linux. ThepSHMTransportworkaround is applied on Darwin (same pattern as existing Go2 blueprints).Test plan
xlerobot-basicandxlerobot-agenticappear inall_blueprintsafter running the generation testdimos[xlerobot]resolveslerobot[feetech]correctly🤖 Generated with Claude Code