Skip to content

feat(daemon): collect only AI coding agent telemetry in Claude Code cloud sessions - #335

Merged
AnnatarHe merged 2 commits into
mainfrom
claude/ai-cloud-only-mode
Oct 23, 2026
Merged

AnnatarHe merged 2 commits into
mainfrom
claude/ai-cloud-only-mode

Conversation

@AnnatarHe

@AnnatarHe AnnatarHe commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Why

A Claude Code cloud environment's VM isn't one of the user's machines. ShellTime should record the Claude Code session, not the VM's shell commands or the VM itself as a host.

It already happened: an account has a host vm (CLI 0.1.95) holding 123 cmd1 0…cmd1 15 commands in fish. Those came from commands/track_test.go. A cloud session ran the CLI's tests, and commandTrack's fast path handed them to the daemon on /tmp/shelltime.sock, which the SessionStart hook keeps running signed in to the account. The same happens on any machine whose daemon is running.

What changes

fix(commands): tests never reach a real daemon

  • The fast-path socket is now trackSocketPath. The new commands/main_test.go points it at a path that doesn't exist and clears CLAUDE_CODE_REMOTE*, as daemon/main_test.go does.
  • The track tests' skip for a present default socket is gone, so those tests run everywhere.

feat(daemon): AI cloud only mode

  • New config key aiCloudOnly, read through model.IsAICloudOnly(cfg, getenv). Unset, the mode is on where CLAUDE_CODE_REMOTE=true, so environments set up with cloud.bash need no change. An explicit true or false wins.
  • Daemon:
    • the socket drops track_pre, track_post and sync, and answers heartbeats with disabled;
    • main.go doesn't open the bolt store or start the sync circuit breaker or heartbeat resync.
  • CLI:
    • track without a daemon stores nothing;
    • sync, alias import, dotfiles push and hooks install (so step 2 of init too) print why and return nil. Alias import and dotfiles push would create Host rows on the server.
  • doctor names the mode and its source under Configuration and skips Shell Hooks.
  • Unchanged: OTEL events, metrics and spans (including the ccr.session.id fill), cc pr, session projects, the Anthropic and Codex usage syncs, and log cleanup.
  • Docs: docs/CONFIG.md (AI Cloud Only) and CLAUDE.md.

Testing

  • go vet ./...; go test -timeout 3m -covermode=atomic ./...: all packages pass. The suite ran in a cloud session while its real daemon was up, and that daemon's log shows no track or sync message from it.
  • New tests:
    • IsAICloudOnly (table);
    • YAML and local override, and auth's default config never writes the key;
    • the socket drops commands, syncs and heartbeats while session pull requests still go through, both from the config and from the environment;
    • track stores nothing;
    • sync, alias import and dotfiles push send nothing, with something to upload seeded;
    • hooks install, including aiCloudOnly: false winning in the cloud;
    • doctor.
  • Mutation checks: with the gates removed, the socket, track, upload and hooks tests fail.
  • End to end with the built binaries (temp HOME, own socket and port, local API recorder):
    • with CLAUDE_CODE_REMOTE=true, only /api/v1/cc/session-pull-requests arrived, and the circuit breaker and heartbeat resync didn't start;
    • without it, a sync reached /api/v1/track as before;
    • the CLI commands and doctor --offline print the messages above.

Rollout

cloud.bash downloads the latest release whenever an environment rebuilds its snapshot (on setup changes, and about weekly), so cloud environments get this without edits. Data already uploaded (the vm host and its test commands) isn't touched.


Generated by Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

…aemon

`shelltime track` hands each event to the daemon on /tmp/shelltime.sock before
reading any config, and the track tests ran it with that path hard-coded. On a
machine whose daemon is running, a Claude Code cloud session's or your own,
their `cmd1 0`…`cmd1 15` commands went to that daemon's account and created a
host for the machine.

The fast path now reads its socket from `trackSocketPath`, which the commands
TestMain points at a path that doesn't exist. It also clears
CLAUDE_CODE_REMOTE*, like the daemon suite, so the tests behave the same inside
a cloud session as on CI. The skip for a present default socket is gone: those
tests run everywhere now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Um2DevMgViC7Jtb8jaXFF3
…loud sessions

A Claude Code cloud environment's VM isn't one of the user's machines, so
ShellTime shouldn't record its shell commands or list it as a host; only the
Claude Code session itself belongs in the account.

New config key `aiCloudOnly` (model.IsAICloudOnly). Unset, it is on where
CLAUDE_CODE_REMOTE=true, which cloud environments set and local machines never
do, so environments set up with cloud.bash need no change. An explicit value
wins either way.

In this mode:
- The daemon's socket drops track_pre, track_post and sync messages and
  answers heartbeats with "disabled". The daemon doesn't open the bolt store or
  start the sync circuit breaker or heartbeat resync.
- `shelltime track` without a daemon stores nothing. `sync`, `alias import`,
  `dotfiles push` and `hooks install` say why and do nothing.
- `shelltime doctor` names the mode and its source under Configuration and
  skips Shell Hooks.
- OTEL events, metrics and spans, `cc pr` and session projects go through as
  before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Um2DevMgViC7Jtb8jaXFF3
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-23T06:27:16.840534Z
We couldn’t complete this review. Comment @codex review to try again.
dade274 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@github-actions

Copy link
Copy Markdown
Contributor

⏱️ shelltime track performance

✅ No significant change in shelltime track latency

Comparing main @ babae9d → #335 @ dade274. Each scenario execs the binary the way the shell hooks do; values are the median of 10 interleaved rounds. Lower is better.

Scenario base head Δ p
Startup/floor 206 µs ±13% 207 µs ±5% ~ 1.000 A/A
Startup/version 2.06 ms ±41% 2.03 ms ±23% ~ 1.000 ✅
Track/daemon/pre 2.29 ms ±29% 2.24 ms ±10% ~ 0.853 ✅
Track/daemon/post 2.22 ms ±23% 2.21 ms ±11% ~ 0.631 ✅
Track/direct/pre 2.22 ms ±22% 2.13 ms ±5% ~ 0.315 ✅
Track/direct/post 4.43 ms ±13% 4.28 ms ±13% ~ 0.684 ✅
Track/direct/post-sync 5.35 ms ±28% 5.44 ms ±17% ~ 0.684 ✅

Binary size: 19.2 MiB → 19.2 MiB (+0.0%).

CPU, tail latency and memory

Per exec, base → head (Δ when significant).

Scenario p95 wall user CPU sys CPU peak RSS
Startup/floor 245 µs → 250 µs 180 µs → 177 µs 6 µs → 7 µs 9.0 MiB → 8.9 MiB
Startup/version 2.33 ms → 2.46 ms 1.51 ms → 1.47 ms 614 µs → 624 µs 15.5 MiB → 15.8 MiB (+1.59%)
Track/daemon/pre 3.14 ms → 2.53 ms 1.59 ms → 1.61 ms 737 µs → 644 µs 16.5 MiB → 16.6 MiB (+0.73%)
Track/daemon/post 3.27 ms → 2.51 ms 1.58 ms → 1.56 ms 754 µs → 737 µs 16.4 MiB → 16.4 MiB
Track/direct/pre 2.92 ms → 2.30 ms 1.61 ms → 1.55 ms 636 µs → 644 µs 16.4 MiB → 16.5 MiB (+0.74%)
Track/direct/post 5.63 ms → 5.46 ms 2.93 ms → 2.93 ms 1.74 ms → 1.88 ms 17.6 MiB → 17.7 MiB (+0.54%)
Track/direct/post-sync 6.33 ms → 6.60 ms 3.89 ms → 3.81 ms 1.86 ms → 1.95 ms 19.5 MiB → 19.5 MiB
benchstat
goos: linux
goarch: amd64
pkg: github.com/malamtime/cli/perf
cpu: Intel(R) Xeon(R) Processor
                         │     base     │                head                 │
                         │    sec/op    │    sec/op     vs base               │
Startup/floor-2            206.2µ ± 13%   207.3µ ±  5%       ~ (p=1.000 n=10)
Startup/version-2          2.056m ± 41%   2.031m ± 23%       ~ (p=1.000 n=10)
Track/daemon/pre-2         2.292m ± 29%   2.239m ± 10%       ~ (p=0.853 n=10)
Track/daemon/post-2        2.222m ± 23%   2.208m ± 11%       ~ (p=0.631 n=10)
Track/direct/pre-2         2.221m ± 22%   2.130m ±  5%       ~ (p=0.315 n=10)
Track/direct/post-2        4.426m ± 13%   4.285m ± 13%       ~ (p=0.684 n=10)
Track/direct/post-sync-2   5.352m ± 28%   5.436m ± 17%       ~ (p=0.684 n=10)
geomean                    1.966m         1.940m        -1.36%

                         │     base     │                head                 │
                         │  p50-sec/op  │  p50-sec/op   vs base               │
Startup/floor-2            198.4µ ± 11%   194.2µ ±  6%       ~ (p=0.631 n=10)
Startup/version-2          2.017m ± 22%   1.998m ±  7%       ~ (p=0.912 n=10)
Track/daemon/pre-2         2.147m ± 15%   2.143m ±  6%       ~ (p=0.912 n=10)
Track/daemon/post-2        2.090m ± 21%   2.164m ±  8%       ~ (p=1.000 n=10)
Track/direct/pre-2         2.120m ± 19%   2.071m ±  5%       ~ (p=0.853 n=10)
Track/direct/post-2        4.183m ± 10%   4.126m ±  8%       ~ (p=0.853 n=10)
Track/direct/post-sync-2   5.208m ± 17%   5.252m ± 10%       ~ (p=0.971 n=10)
geomean                    1.880m         1.873m        -0.37%

                         │     base     │                head                 │
                         │  p95-sec/op  │  p95-sec/op   vs base               │
Startup/floor-2            245.1µ ± 10%   250.2µ ± 14%       ~ (p=0.971 n=10)
Startup/version-2          2.334m ± 86%   2.460m ± 73%       ~ (p=0.971 n=10)
Track/daemon/pre-2         3.138m ± 52%   2.526m ± 71%       ~ (p=0.579 n=10)
Track/daemon/post-2        3.266m ± 29%   2.509m ± 57%       ~ (p=0.684 n=10)
Track/direct/pre-2         2.917m ± 76%   2.305m ± 48%       ~ (p=0.052 n=10)
Track/direct/post-2        5.633m ± 43%   5.457m ± 41%       ~ (p=0.739 n=10)
Track/direct/post-sync-2   6.331m ± 66%   6.600m ± 53%       ~ (p=0.315 n=10)
geomean                    2.500m         2.284m        -8.65%

                         │     base     │                head                 │
                         │  peak-rss-B  │  peak-rss-B   vs base               │
Startup/floor-2            8.989Mi ± 1%   8.948Mi ± 1%       ~ (p=0.148 n=10)
Startup/version-2          15.52Mi ± 0%   15.77Mi ± 0%  +1.59% (p=0.000 n=10)
Track/daemon/pre-2         16.52Mi ± 0%   16.64Mi ± 0%  +0.73% (p=0.000 n=10)
Track/daemon/post-2        16.40Mi ± 0%   16.39Mi ± 0%       ~ (p=0.402 n=10)
Track/direct/pre-2         16.39Mi ± 0%   16.51Mi ± 0%  +0.74% (p=0.000 n=10)
Track/direct/post-2        17.56Mi ± 0%   17.65Mi ± 0%  +0.54% (p=0.000 n=10)
Track/direct/post-sync-2   19.50Mi ± 0%   19.47Mi ± 0%       ~ (p=0.255 n=10)
geomean                    15.47Mi        15.53Mi       +0.42%

                         │     base     │                head                 │
                         │  sys-sec/op  │  sys-sec/op   vs base               │
Startup/floor-2            6.360µ ± 25%   6.945µ ± 72%       ~ (p=0.912 n=10)
Startup/version-2          613.7µ ± 88%   624.0µ ± 51%       ~ (p=0.579 n=10)
Track/daemon/pre-2         736.9µ ± 35%   644.1µ ± 47%       ~ (p=0.436 n=10)
Track/daemon/post-2        754.1µ ± 43%   737.4µ ± 21%       ~ (p=1.000 n=10)
Track/direct/pre-2         636.4µ ± 57%   644.5µ ± 21%       ~ (p=0.739 n=10)
Track/direct/post-2        1.737m ± 18%   1.882m ± 33%       ~ (p=0.436 n=10)
Track/direct/post-sync-2   1.862m ± 42%   1.950m ± 43%       ~ (p=0.315 n=10)
geomean                    461.6µ         467.3µ        +1.24%

                         │     base     │                head                 │
                         │ user-sec/op  │ user-sec/op   vs base               │
Startup/floor-2            180.1µ ±  8%   177.4µ ±  6%       ~ (p=0.631 n=10)
Startup/version-2          1.515m ± 18%   1.472m ± 11%       ~ (p=0.796 n=10)
Track/daemon/pre-2         1.591m ± 19%   1.613m ± 15%       ~ (p=0.796 n=10)
Track/daemon/post-2        1.580m ± 10%   1.560m ± 16%       ~ (p=0.631 n=10)
Track/direct/pre-2         1.612m ± 18%   1.548m ±  9%       ~ (p=0.143 n=10)
Track/direct/post-2        2.934m ± 16%   2.932m ± 22%       ~ (p=0.481 n=10)
Track/direct/post-sync-2   3.888m ± 19%   3.809m ± 10%       ~ (p=0.631 n=10)
geomean                    1.436m         1.415m        -1.49%

Δ compares medians; ~ means no significant difference (Mann-Whitney U, p ≥ 0.05). A scenario is flagged when it is more than 10% and more than 250µs slower with p < 0.05. Startup/floor runs true, not shelltime: it is an A/A check of runner noise. Runner: linux/amd64, Intel(R) Xeon(R) Processor. go1.27.1, 10 rounds × 100 execs per scenario and binary.

@codecov

codecov Bot commented Oct 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.65217% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
daemon/socket.go 71.42% 2 Missing ⚠️
Flag Coverage Δ
unittests 86.40% <95.65%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
commands/ai_cloud_only.go 100.00% <100.00%> (ø)
commands/alias.go 88.60% <100.00%> (+0.29%) ⬆️
commands/doctor_checks.go 95.31% <100.00%> (+0.10%) ⬆️
commands/dotfiles_push.go 95.00% <100.00%> (+0.17%) ⬆️
commands/hooks.install.go 95.45% <100.00%> (+0.45%) ⬆️
commands/sync.go 93.33% <100.00%> (+1.02%) ⬆️
commands/track.go 87.50% <100.00%> (+0.37%) ⬆️
model/aicode_cloud_session.go 100.00% <100.00%> (ø)
model/config.go 96.40% <100.00%> (+0.05%) ⬆️
daemon/socket.go 85.43% <71.42%> (+1.01%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@shelltime

shelltime Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

☁️ Claude Code cloud · Add Claude Code AI-cloud-only collection mode

Implemented an AI-cloud-only CLI mode that collects Claude Code cloud sessions while excluding command history and host/server information.

💰 Cost 🔢 Tokens ⏱️ Duration 🧠 Models 💬 Prompts ✏️ Lines
$8.51 24.4M 25m (17m active) claude-opus-5-5, claude-haiku-5-5 2 +385 / −1

24.4M tokens: 2.5K input · 77K output · 24.1M cache read · 266.8K cache write

📝 AI summary

After first proposing and receiving approval for the design, the CLI was updated with an AI-cloud-only operating mode exposed through configuration and command handling. Changes in model/types.go, model/config.go, and commands/ai_cloud_only.go define and apply the mode so Claude Code cloud data can be collected without treating cloud sessions as user-owned hosts or servers and without persisting commands.

Cloud-session behavior was refined in model/aicode_cloud_session.go, with coverage added in model/aicode_cloud_session_test.go. Additional integration and regression tests were added to daemon/socket_extra_test.go and commands/main_test.go to verify mode propagation and ensure command and host information are excluded while cloud session collection remains functional.

View this session on ShellTime →

Updated 2026-10-24 06:28 UTC by ShellTime · totals for the whole session so far · the session page is visible to its owner

@AnnatarHe

Copy link
Copy Markdown
Contributor Author

/claude review

@AnnatarHe

Copy link
Copy Markdown
Contributor Author

@claude review

@claude

claude Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @AnnatarHe's task in 24s —— View job


Review of PR #335: AI cloud only mode

I read the full non-test diff against origin/main. I did not run the build or the tests, and I did not read the test files. This is a read-through only.

Verdict: The approach is sound and I found nothing blocking. The gating is placed at the right choke points: the daemon socket, the daemon startup, and the CLI commands that create Host rows. I grepped for the remaining upload paths, SendLocalDataToServer and SendHeartbeatsToServer. They are reached only from the daemon handlers, the heartbeat resync, and trySyncLocalToServer. The socket gate, the startup gate and the sync/track gates cover all of them.

Looks good

  • IsAICloudOnly is simple. An explicit true or false wins, and it falls back to CLAUDE_CODE_REMOTE=true otherwise. It takes getenv, so it is easy to test.
  • trackSocketPath plus commands/main_test.go fixes the root cause of the vm host with cmd1 N commands.
  • mergeConfig handles the local override, so config.local.yaml can flip the mode.
  • socket.go merges the sync and track cases into one branch without changing what they do. Error logs now include messageType.

Minor points and suggestions

  1. daemon/socket.go (heartbeat branch)
    • When the mode is on, the log line still says "codeTracking is disabled". It adds an aiCloudOnly attribute, but a separate message would be clearer when someone debugs a cloud session.
    • Codecov flags 2 uncovered lines in this file. It is probably the marshal and publish error paths, which were already uncovered before this PR.
  2. daemon/socket.go (mode is fixed at construction)
    • aiCloudOnly is read once in NewSocketHandler, from the config and os.Getenv.
    • If the daemon reloads its config, or CLAUDE_CODE_REMOTE differs between the daemon's environment and the CLI's, the daemon and the CLI could disagree. For example, the daemon is started by the SessionStart hook and does not see the variable, but the CLI does. The CLI side then skips storing, and the daemon side would accept the commands. A one-line note in docs/CONFIG.md that the daemon's own environment decides would help.
  3. cmd/daemon/main.go (CodeTracking)
    • CodeTracking stays enabled in the config, and the heartbeat resync is skipped silently. The startup log already says the mode is on, which is enough. A slog.Info saying the heartbeat resync was skipped would make that explicit.
  4. commands/ai_cloud_only.go (configForAICloudOnly)
    • It builds a new ConfigService from $HOME instead of using the injected one. That is why hooks install can run before sign-in, but it bypasses dependency injection. The other commands use the injected service. A comment saying so would help, or use the injected service with a fallback.
    • It also swallows config read errors. Malformed YAML will be treated as "no config", so a user with aiCloudOnly: false and a typo gets the environment default. That is probably acceptable, but a slog.Debug of the error would help troubleshooting.
  5. commands/track.go
    • The gate runs after the config read, so a cloud session with no daemon still pays the config read on every command. Latency is irrelevant here because the perf report shows no change.
    • Also, track returns nil before the ShouldExcludeCommand check, which is the right order.
  6. commands/doctor_checks.go
    • The message "aiCloudOnly is on in your config" only appears when the key is set. Consider handling aiCloudOnly: true outside the cloud explicitly. It works, because cfg.AICloudOnly != nil and the mode is on. The wording is fine.
  7. Gaps that are not covered
    • shelltime auth, init and the other commands that call the API (for example cc statusline, ls) are left alone. That matches the PR description. I only grepped for the upload paths above and did not check every other command that talks to the API.
    • Data already uploaded (the vm host) is untouched, as noted in the PR.

Nothing here needs to block merge. Items 2 and 4 are the ones I would fix, or document, before merging.

@AnnatarHe
AnnatarHe merged commit c54f47f into main Oct 23, 2026
5 checks passed
@AnnatarHe
AnnatarHe deleted the claude/ai-cloud-only-mode branch October 23, 2026 09:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants