Skip to content

fix(ble): reconnect a dropped peripheral at once, forever, and resume a Maimu train - #202

Merged
gbradham merged 2 commits into
mainfrom
fix/ble-fast-reconnect
Oct 20, 2026
Merged

gbradham merged 2 commits into
mainfrom
fix/ble-fast-reconnect

Conversation

@gbradham

@gbradham gbradham commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Summary

In an RTPP run a Maimu's link dropped whenever the mouse left the stimulation zone, and every outage lasted 15�35 s with the light off. The run log showed four separate problems:

  • Slow start, then a give-up. The reconnect loop waited 5 s before its first try, backed off to a minute, and stopped after twelve attempts.
  • A scan that could never succeed. Each attempt first scanned 6�8 s for the address that had just dropped. A peripheral using BLE privacy re-advertises at a new address after every disconnect (Zephyr's adv.c clears BT_DEV_RPA_VALID on each legacy advertising start), so that address was always gone.
  • The train never came back. On reconnect the Maimu always wrote off, so a 1200 s train cut off by a drop stayed off.
  • Stale commands replayed. Zone enter/exit commands sent during the outage blocked the flow behind the reconnect, then replayed in a burst, oldest first.

Changes

BLEDevice (core):

  • Retries straight away, then after 0.5 s, 1 s and every 2 s.
  • MAX_RECONNECT_ATTEMPTS now defaults to None, meaning never give up. A finite value still ends in ERROR.
  • On a reconnect it looks for the advertised name first, but only when that name has been heard on this host. A peripheral whose name never arrives doesn't pay an extra scan per attempt.
  • Per-attempt logging thins out after the first five, and a line reports how long the reconnect took.

MaimuDevice (plugin):

  • Remembers its last command, and the reconnect hook re-sends it: a train for whatever is left of its duration, on as on, and off if nothing was asked or the train has run out. The log says when a train was resumed.
  • Commands sent while the link is down return immediately; only the newest is sent on reconnect.
  • A lock makes sure the reconnect hook never sends an older command over a newer one from the flow.
  • New train(on_ms, off_ms, duration_s) action, declared in ACTION_ARGS_SCHEMA. The node's duty-cycle path uses it instead of a raw write("p�"), so the device knows a train is running.

Docs: the CHANGELOG, docs-site/reference/devices.md and the plugin README are updated, plus the ERROR wording comment in main_window.py.

Follow-up (3a19351): a link that bounces. On the bench, replugging a DK dropped the link again within 0.25 s of a reconnect, and the resend fell back on write()'s retry-once-inside-the-write path. That held the command lock and stalled the flow for 2.4 s, sent a resend computed before the second drop (2.5 s late, overrunning the train), and showed "disconnected" instead of "reconnecting". BLEDevice.write() now takes retry=False, which sends once over the link as it stands. The Maimu uses it for every command and for the resend. The reconnect loop checks the link after its hook and keeps retrying if it died meanwhile, then runs the hook again with the time left at that moment.

Related firmware (outside this repo)

Firmware 0.1.1 for the units goes after the drops themselves: a 2 s supervision timeout instead of the auto-requested 420 ms, +8 dBm TX instead of 0 dBm, and privacy off for a stable address. Units that haven't been updated get everything above except the stable address.

Test plan

  • pytest tests/unit/hal plugins/glider-maimu/tests: 599 passed. New tests cover:
    • the retry never stopping, then reconnecting once the peripheral is back
    • a finite limit still ending in ERROR, and the backoff not overflowing
    • name-first on reconnect, and no extra scan for a name never heard
    • a train resuming with its remaining time (asymmetric and symmetric forms)
    • a train with under a second left coming back off
    • on/off restored as they were
    • commands during an outage not blocking, with the newest winning
    • shutdown forgetting the last command
    • a drop during the reconnect step being retried by the loop
    • a resend after a bounce using the time left at that later moment
    • a command whose write dies with the link being sent on reconnect
  • pytest tests/unit/core tests/unit/gui/shell/test_main_window_shell.py tests/unit/gui/test_camera_panel_rehearsal.py: everything passes except test_flow_clock.py::test_the_timing_thread_wakes_within_a_couple_of_ms. That test also fails on this machine without this change and doesn't touch the changed code (5.4 ms wake against a 2 ms bound).
  • ruff check . and black --check . clean
  • GLIDER launches from the branch and loads the Maimu plugin
  • On the rig: confirm the reconnect time in the log ("reconnected after X s") and that a train resumes after a drop

… a Maimu train

A Maimu worn by a mouse in an RTPP arena dropped its link whenever the
animal left the stimulation zone, and each outage lasted 15-35 s:

- the reconnect loop waited 5 s before its first attempt, backed off to a
  minute, and gave up after twelve attempts into ERROR;
- every attempt first spent a full scan looking for the address that had
  just dropped, but a peripheral using BLE privacy re-advertises at a new
  address after every disconnect, so that scan could never succeed;
- on reconnect the Maimu always wrote 'off', so a 1200 s train cut off by a
  drop never came back;
- zone commands issued during the outage blocked the flow behind the
  reconnect, then replayed in a burst, oldest first.

BLEDevice now retries straight away, then after 0.5, 1 and every 2 s, and
never gives up by default (a finite MAX_RECONNECT_ATTEMPTS still lands in
ERROR). On a reconnect it looks for the advertised name first, when that
name has been heard on this host. Attempt logging thins out after the
first five so an endless retry does not bury the log.

MaimuDevice remembers its last command and its reconnect hook resends it: a
train for whatever is left of its duration, 'on' as 'on', and 'off' if
nothing was asked or the train has run out. Commands sent while the link
is down return immediately and only the newest is sent on reconnect. A
train(on_ms, off_ms, duration_s) action replaces the node's raw
"p<on>,<off>,<s>" write so the device knows a train is running.
…ink that bounces

Unplugging and replugging a DK running the Maimu firmware showed the link
dropping again within 0.25 s of a reconnect (a bouncing USB plug). The
reconnect hook's resend then went through write()'s reconnect-once-and-retry
path while holding the Maimu's command lock:

- the flow's next command waited 2.4 s behind that in-band reconnect;
- the resend had been worked out before the second drop, so 'p10,40,2' went
  out 2.5 s late and ran the light past the end of that train;
- the device read "disconnected" rather than "reconnecting" meanwhile,
  because the loop that should have been retrying had already finished.

BLEDevice.write() takes retry=False for a caller that owns its recovery: it
sends once over the link as it stands and raises ConnectionError without
connecting. The Maimu uses it for every command and for the reconnect
resend; a command whose write dies with the link is remembered and sent by
the reconnect. The reconnect loop now checks the link after its hook and,
if it died meanwhile, carries on retrying and runs the hook again, so the
resend works out the time left at that later moment.
@gbradham
gbradham merged commit 15ee496 into main Oct 20, 2026
4 checks passed
@gbradham
gbradham deleted the fix/ble-fast-reconnect branch October 20, 2026 21:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant