[w180112]

Summary

• etcd-cpp-apiv3 runs each Watcher's wait callback on a detached thread that can outlive EtcdClientImpl. The callbacks captured a raw this, so on SIGTERM teardown a callback arriving after the client was freed wrote into already-released memory, corrupting the glibc heap (malloc_consolidate abort / invalid pointer passed to free).
• Fix: each wait callback now captures a shared guard (shared_ptr<WatchCallbackGuard> with a mutex + alive flag) instead of raw this. stop_watch() clears alive under the lock as its very first step, before any later callback can touch the object. A stale comment in stop_watch() describing the old (incorrect) teardown behavior is also corrected.
• Added an ASan regression test (test_etcd_watch_teardown) on the controller side that overrides pthread_create to delay new watcher threads during teardown, reproducing the race deterministically: 20/20 iterations caught the use-after-free before the fix, 0/20 after. Wired into run_tests.sh and make test.
• Added node_shutdown_lib.sh for e2e: shutdown steps in phases 17/19/33/40 and the harness's final stop now verify the node log ends with bye!, with no apport record and no kernel trap, so a crash during teardown is no longer silently treated as a pass. The verdict is a self-validated predicate (FAILs when evidence can't be read) with 4 new shutdown drills added to validate.

Test plan

• make test — 2408/2408 passed
• make all — production build rebuilt
• Full e2e (./run_e2e_test.sh, no flags) — 198/198 passed, 25/25 validate drills caught
• Repro loop on the fixed build: 50 iterations, 0 crashes (master comparison: 4 crashes in 30 iterations)

🤖 Generated with Claude Code