Repository navigation
feat(daemon): collect only AI coding agent telemetry in Claude Code cloud sessions - #335
Conversation
…aemon `shelltime track` hands each event to the daemon on /tmp/shelltime.sock before reading any config, and the track tests ran it with that path hard-coded. On a machine whose daemon is running, a Claude Code cloud session's or your own, their `cmd1 0`…`cmd1 15` commands went to that daemon's account and created a host for the machine. The fast path now reads its socket from `trackSocketPath`, which the commands TestMain points at a path that doesn't exist. It also clears CLAUDE_CODE_REMOTE*, like the daemon suite, so the tests behave the same inside a cloud session as on CI. The skip for a present default socket is gone: those tests run everywhere now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Um2DevMgViC7Jtb8jaXFF3
…loud sessions A Claude Code cloud environment's VM isn't one of the user's machines, so ShellTime shouldn't record its shell commands or list it as a host; only the Claude Code session itself belongs in the account. New config key `aiCloudOnly` (model.IsAICloudOnly). Unset, it is on where CLAUDE_CODE_REMOTE=true, which cloud environments set and local machines never do, so environments set up with cloud.bash need no change. An explicit value wins either way. In this mode: - The daemon's socket drops track_pre, track_post and sync messages and answers heartbeats with "disabled". The daemon doesn't open the bolt store or start the sync circuit breaker or heartbeat resync. - `shelltime track` without a daemon stores nothing. `sync`, `alias import`, `dotfiles push` and `hooks install` say why and do nothing. - `shelltime doctor` names the mode and its source under Configuration and skips Shell Hooks. - OTEL events, metrics and spans, `cc pr` and session projects go through as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Um2DevMgViC7Jtb8jaXFF3
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
⏱️
|
| Scenario | base | head | Δ | p | |
|---|---|---|---|---|---|
Startup/floor |
206 µs ±13% | 207 µs ±5% | ~ | 1.000 | A/A |
Startup/version |
2.06 ms ±41% | 2.03 ms ±23% | ~ | 1.000 | ✅ |
Track/daemon/pre |
2.29 ms ±29% | 2.24 ms ±10% | ~ | 0.853 | ✅ |
Track/daemon/post |
2.22 ms ±23% | 2.21 ms ±11% | ~ | 0.631 | ✅ |
Track/direct/pre |
2.22 ms ±22% | 2.13 ms ±5% | ~ | 0.315 | ✅ |
Track/direct/post |
4.43 ms ±13% | 4.28 ms ±13% | ~ | 0.684 | ✅ |
Track/direct/post-sync |
5.35 ms ±28% | 5.44 ms ±17% | ~ | 0.684 | ✅ |
Binary size: 19.2 MiB → 19.2 MiB (+0.0%).
CPU, tail latency and memory
Per exec, base → head (Δ when significant).
| Scenario | p95 wall | user CPU | sys CPU | peak RSS |
|---|---|---|---|---|
Startup/floor |
245 µs → 250 µs | 180 µs → 177 µs | 6 µs → 7 µs | 9.0 MiB → 8.9 MiB |
Startup/version |
2.33 ms → 2.46 ms | 1.51 ms → 1.47 ms | 614 µs → 624 µs | 15.5 MiB → 15.8 MiB (+1.59%) |
Track/daemon/pre |
3.14 ms → 2.53 ms | 1.59 ms → 1.61 ms | 737 µs → 644 µs | 16.5 MiB → 16.6 MiB (+0.73%) |
Track/daemon/post |
3.27 ms → 2.51 ms | 1.58 ms → 1.56 ms | 754 µs → 737 µs | 16.4 MiB → 16.4 MiB |
Track/direct/pre |
2.92 ms → 2.30 ms | 1.61 ms → 1.55 ms | 636 µs → 644 µs | 16.4 MiB → 16.5 MiB (+0.74%) |
Track/direct/post |
5.63 ms → 5.46 ms | 2.93 ms → 2.93 ms | 1.74 ms → 1.88 ms | 17.6 MiB → 17.7 MiB (+0.54%) |
Track/direct/post-sync |
6.33 ms → 6.60 ms | 3.89 ms → 3.81 ms | 1.86 ms → 1.95 ms | 19.5 MiB → 19.5 MiB |
benchstat
goos: linux
goarch: amd64
pkg: github.com/malamtime/cli/perf
cpu: Intel(R) Xeon(R) Processor
│ base │ head │
│ sec/op │ sec/op vs base │
Startup/floor-2 206.2µ ± 13% 207.3µ ± 5% ~ (p=1.000 n=10)
Startup/version-2 2.056m ± 41% 2.031m ± 23% ~ (p=1.000 n=10)
Track/daemon/pre-2 2.292m ± 29% 2.239m ± 10% ~ (p=0.853 n=10)
Track/daemon/post-2 2.222m ± 23% 2.208m ± 11% ~ (p=0.631 n=10)
Track/direct/pre-2 2.221m ± 22% 2.130m ± 5% ~ (p=0.315 n=10)
Track/direct/post-2 4.426m ± 13% 4.285m ± 13% ~ (p=0.684 n=10)
Track/direct/post-sync-2 5.352m ± 28% 5.436m ± 17% ~ (p=0.684 n=10)
geomean 1.966m 1.940m -1.36%
│ base │ head │
│ p50-sec/op │ p50-sec/op vs base │
Startup/floor-2 198.4µ ± 11% 194.2µ ± 6% ~ (p=0.631 n=10)
Startup/version-2 2.017m ± 22% 1.998m ± 7% ~ (p=0.912 n=10)
Track/daemon/pre-2 2.147m ± 15% 2.143m ± 6% ~ (p=0.912 n=10)
Track/daemon/post-2 2.090m ± 21% 2.164m ± 8% ~ (p=1.000 n=10)
Track/direct/pre-2 2.120m ± 19% 2.071m ± 5% ~ (p=0.853 n=10)
Track/direct/post-2 4.183m ± 10% 4.126m ± 8% ~ (p=0.853 n=10)
Track/direct/post-sync-2 5.208m ± 17% 5.252m ± 10% ~ (p=0.971 n=10)
geomean 1.880m 1.873m -0.37%
│ base │ head │
│ p95-sec/op │ p95-sec/op vs base │
Startup/floor-2 245.1µ ± 10% 250.2µ ± 14% ~ (p=0.971 n=10)
Startup/version-2 2.334m ± 86% 2.460m ± 73% ~ (p=0.971 n=10)
Track/daemon/pre-2 3.138m ± 52% 2.526m ± 71% ~ (p=0.579 n=10)
Track/daemon/post-2 3.266m ± 29% 2.509m ± 57% ~ (p=0.684 n=10)
Track/direct/pre-2 2.917m ± 76% 2.305m ± 48% ~ (p=0.052 n=10)
Track/direct/post-2 5.633m ± 43% 5.457m ± 41% ~ (p=0.739 n=10)
Track/direct/post-sync-2 6.331m ± 66% 6.600m ± 53% ~ (p=0.315 n=10)
geomean 2.500m 2.284m -8.65%
│ base │ head │
│ peak-rss-B │ peak-rss-B vs base │
Startup/floor-2 8.989Mi ± 1% 8.948Mi ± 1% ~ (p=0.148 n=10)
Startup/version-2 15.52Mi ± 0% 15.77Mi ± 0% +1.59% (p=0.000 n=10)
Track/daemon/pre-2 16.52Mi ± 0% 16.64Mi ± 0% +0.73% (p=0.000 n=10)
Track/daemon/post-2 16.40Mi ± 0% 16.39Mi ± 0% ~ (p=0.402 n=10)
Track/direct/pre-2 16.39Mi ± 0% 16.51Mi ± 0% +0.74% (p=0.000 n=10)
Track/direct/post-2 17.56Mi ± 0% 17.65Mi ± 0% +0.54% (p=0.000 n=10)
Track/direct/post-sync-2 19.50Mi ± 0% 19.47Mi ± 0% ~ (p=0.255 n=10)
geomean 15.47Mi 15.53Mi +0.42%
│ base │ head │
│ sys-sec/op │ sys-sec/op vs base │
Startup/floor-2 6.360µ ± 25% 6.945µ ± 72% ~ (p=0.912 n=10)
Startup/version-2 613.7µ ± 88% 624.0µ ± 51% ~ (p=0.579 n=10)
Track/daemon/pre-2 736.9µ ± 35% 644.1µ ± 47% ~ (p=0.436 n=10)
Track/daemon/post-2 754.1µ ± 43% 737.4µ ± 21% ~ (p=1.000 n=10)
Track/direct/pre-2 636.4µ ± 57% 644.5µ ± 21% ~ (p=0.739 n=10)
Track/direct/post-2 1.737m ± 18% 1.882m ± 33% ~ (p=0.436 n=10)
Track/direct/post-sync-2 1.862m ± 42% 1.950m ± 43% ~ (p=0.315 n=10)
geomean 461.6µ 467.3µ +1.24%
│ base │ head │
│ user-sec/op │ user-sec/op vs base │
Startup/floor-2 180.1µ ± 8% 177.4µ ± 6% ~ (p=0.631 n=10)
Startup/version-2 1.515m ± 18% 1.472m ± 11% ~ (p=0.796 n=10)
Track/daemon/pre-2 1.591m ± 19% 1.613m ± 15% ~ (p=0.796 n=10)
Track/daemon/post-2 1.580m ± 10% 1.560m ± 16% ~ (p=0.631 n=10)
Track/direct/pre-2 1.612m ± 18% 1.548m ± 9% ~ (p=0.143 n=10)
Track/direct/post-2 2.934m ± 16% 2.932m ± 22% ~ (p=0.481 n=10)
Track/direct/post-sync-2 3.888m ± 19% 3.809m ± 10% ~ (p=0.631 n=10)
geomean 1.436m 1.415m -1.49%
Δ compares medians; ~ means no significant difference (Mann-Whitney U, p ≥ 0.05). A scenario is flagged when it is more than 10% and more than 250µs slower with p < 0.05. Startup/floor runs true, not shelltime: it is an A/A check of runner noise. Runner: linux/amd64, Intel(R) Xeon(R) Processor. go1.27.1, 10 rounds × 100 execs per scenario and binary.
Codecov Report❌ Patch coverage is
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 1 file with indirect coverage changes 🚀 New features to boost your workflow:
|
☁️ Claude Code cloud · Add Claude Code AI-cloud-only collection modeImplemented an AI-cloud-only CLI mode that collects Claude Code cloud sessions while excluding command history and host/server information.
24.4M tokens: 2.5K input · 77K output · 24.1M cache read · 266.8K cache write 📝 AI summaryAfter first proposing and receiving approval for the design, the CLI was updated with an AI-cloud-only operating mode exposed through configuration and command handling. Changes in Cloud-session behavior was refined in View this session on ShellTime → Updated 2026-10-24 06:28 UTC by ShellTime · totals for the whole session so far · the session page is visible to its owner |
|
/claude review |
|
@claude review |
|
Claude finished @AnnatarHe's task in 24s —— View job Review of PR #335: AI cloud only modeI read the full non-test diff against Verdict: The approach is sound and I found nothing blocking. The gating is placed at the right choke points: the daemon socket, the daemon startup, and the CLI commands that create Host rows. I grepped for the remaining upload paths, Looks good
Minor points and suggestions
Nothing here needs to block merge. Items 2 and 4 are the ones I would fix, or document, before merging. |
Why
A Claude Code cloud environment's VM isn't one of the user's machines. ShellTime should record the Claude Code session, not the VM's shell commands or the VM itself as a host.
It already happened: an account has a host
vm(CLI 0.1.95) holding 123cmd1 0…cmd1 15commands infish. Those came fromcommands/track_test.go. A cloud session ran the CLI's tests, andcommandTrack's fast path handed them to the daemon on/tmp/shelltime.sock, which the SessionStart hook keeps running signed in to the account. The same happens on any machine whose daemon is running.What changes
fix(commands): tests never reach a real daemontrackSocketPath. The newcommands/main_test.gopoints it at a path that doesn't exist and clearsCLAUDE_CODE_REMOTE*, asdaemon/main_test.godoes.feat(daemon): AI cloud only modeaiCloudOnly, read throughmodel.IsAICloudOnly(cfg, getenv). Unset, the mode is on whereCLAUDE_CODE_REMOTE=true, so environments set up withcloud.bashneed no change. An explicittrueorfalsewins.track_pre,track_postandsync, and answers heartbeats withdisabled;main.godoesn't open the bolt store or start the sync circuit breaker or heartbeat resync.trackwithout a daemon stores nothing;sync,alias import,dotfiles pushandhooks install(so step 2 ofinittoo) print why and return nil. Alias import and dotfiles push would create Host rows on the server.doctornames the mode and its source under Configuration and skips Shell Hooks.ccr.session.idfill),cc pr, session projects, the Anthropic and Codex usage syncs, and log cleanup.docs/CONFIG.md(AI Cloud Only) andCLAUDE.md.Testing
go vet ./...;go test -timeout 3m -covermode=atomic ./...: all packages pass. The suite ran in a cloud session while its real daemon was up, and that daemon's log shows no track or sync message from it.IsAICloudOnly(table);auth's default config never writes the key;trackstores nothing;sync,alias importanddotfiles pushsend nothing, with something to upload seeded;hooks install, includingaiCloudOnly: falsewinning in the cloud;doctor.CLAUDE_CODE_REMOTE=true, only/api/v1/cc/session-pull-requestsarrived, and the circuit breaker and heartbeat resync didn't start;/api/v1/trackas before;doctor --offlineprint the messages above.Rollout
cloud.bashdownloads the latest release whenever an environment rebuilds its snapshot (on setup changes, and about weekly), so cloud environments get this without edits. Data already uploaded (thevmhost and its test commands) isn't touched.Generated by Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need.