fix(heartbeat): deliver every heartbeat and count only the user's activity

CLAUDE.md

@@ -24,14 +24,17 @@ Neovim plugin for coding activity tracking, written in Lua. Communicates with Sh
 ```
 lua/shelltime/
   init.lua          -- Entry point: setup(), enable/disable, status, flush
-  config.lua        -- YAML config loading from ~/.shelltime/config.yaml
+  config.lua        -- Config: the CLI's files (config.yaml > .yml > .toml, plus config.local.*)
   heartbeat.lua     -- Autocmd event handling, heartbeat creation, debouncing
   sender.lua        -- Timer-based batching, socket communication
   socket.lua        -- Unix socket client (vim.loop/libuv)
+  offline.lua       -- Leaves undelivered heartbeats in the daemon's resend file on exit
   utils/
     git.lua         -- Git branch detection
     system.lua      -- OS/hostname info
     yaml.lua        -- Minimal YAML parser
+    toml.lua        -- Minimal TOML parser
+    language.lua    -- Filetype -> VS Code language id
 
 plugin/shelltime.lua  -- Auto-load, user commands (:ShellTimeStatus, etc.)
 ```
@@ -55,7 +58,7 @@ plugin/shelltime.lua  -- Auto-load, user commands (:ShellTimeStatus, etc.)
 
 ## Key Constants
 
-- Config path: `~/.shelltime/config.yaml`
+- Config files: `~/.shelltime/config.{yaml,yml,toml}` + `config.local.*` (or `setup({ config = path })`)
 - Socket path: `/tmp/shelltime.sock`
 - Heartbeat interval: 120000ms (2 min)
 - Debounce interval: 30000ms (30 sec)

README.md

@@ -12,13 +12,13 @@ A Neovim plugin for automatic coding activity tracking. Works with the ShellTime
 ## Features
 
 - **Automatic Time Tracking** - Passively monitors coding activity
-- **Language Detection** - Uses the buffer's filetype, falling back to the file extension
+- **Language Detection** - Uses the buffer's filetype, falling back to the file extension, and reports it as the VS Code language id (`sh` → `shellscript`, `text` → `plaintext`, …) so a language adds up across editors
 - **Project Analytics** - Tracks time per project/workspace
 - **Git Integration** - Records activity by git branch
 - **Debug Detection** - Marks activity as `debugging` while an nvim-dap session is active
-- **Debouncing & Batching** - At most one heartbeat per file every 30s (saves always count), sent in batches every 2 minutes
-- **Offline Support** - Keeps heartbeats in memory and retries them when the daemon is unavailable
-- **Flush on Exit** - Sends pending heartbeats when Neovim quits, so short sessions aren't lost
+- **Debouncing & Batching** - At most one heartbeat per file every 30s, sent in batches every 2 minutes
+- **Offline Support** - Keeps heartbeats in memory and retries them when the daemon is unavailable or has code tracking turned off
+- **Flush on Exit** - Sends pending heartbeats when Neovim quits, and leaves what the daemon can't take for it to send later, so short sessions aren't lost
 - **CLI Update Check** - Warns once per session when your ShellTime CLI is out of date
 
 ## Prerequisites
@@ -61,13 +61,12 @@ Or in TOML format (`~/.shelltime/config.toml`):
 enabled = true
 ```
 
-Configs created by `shelltime init` already include this; older configs may not. The daemon ignores editor heartbeats unless it is `true`, and it reads the setting at startup, so restart the daemon after changing it (for example with `shelltime daemon reinstall`).
+Configs created by `shelltime init` already include this; older configs may not. The daemon ignores editor heartbeats unless it is `true`. Recent daemons pick the change up right away; older ones read it only at startup, so restart them after changing it (for example with `shelltime daemon reinstall`).
 
 ## Requirements
 
 - Neovim >= 0.10.0 (Neovim only; classic Vim isn't supported)
 - ShellTime daemon running (see [Prerequisites](#prerequisites))
-- Git (optional, for branch tracking)
 - `curl` (optional, for the CLI update check)
 
 ## Installation
@@ -117,19 +116,19 @@ Calling `setup()` is required: loading the plugin only registers its commands. W
 
 ## Configuration
 
-The plugin reads settings from `~/.shelltime/config.yaml` by default:
+The plugin reads the same files as the ShellTime CLI: the first of `~/.shelltime/config.yaml`, `config.yml` and `config.toml`, with the first `config.local.yaml`, `config.local.yml` or `config.local.toml` merged over it.
 
 ```lua
--- Default setup (uses ~/.shelltime/config.yaml)
+-- Default setup (reads the CLI's config files)
 require("shelltime").setup()
 
--- Custom config path
+-- Read one file instead (YAML or TOML)
 require("shelltime").setup({
   config = "/path/to/your/config.yaml",
 })
 ```
 
-`config` is the only `setup()` option. The plugin reads just this one YAML file: not `config.toml`, `config.yml` or `config.local.*`. If the file is missing, it uses the defaults below. The file is re-read whenever it changes, so most settings apply without restarting Neovim (`heartbeatInterval` needs a restart).
+`config` is the only `setup()` option. If there is no config file, the plugin uses the defaults below. The files are checked for changes every few seconds, so most settings apply without restarting Neovim, `codeTracking.enabled` included (`heartbeatInterval` needs a restart).
 
 ### Config File Options
 
@@ -150,7 +149,7 @@ require("shelltime").setup({
 | `:ShellTimeStatus` | Show daemon connection status (version, uptime, platform) and pending heartbeats |
 | `:ShellTimeFlush` | Manually flush pending heartbeats to daemon |
 | `:ShellTimeEnable` | Enable tracking for this session (needs `setup()`; doesn't override `codeTracking.enabled: false`) |
-| `:ShellTimeDisable` | Flush pending heartbeats and disable tracking for this session |
+| `:ShellTimeDisable` | Send pending heartbeats (what the daemon doesn't take goes to its resend file, as on exit) and disable tracking for this session |
 
 ## How It Works
 
@@ -163,14 +162,16 @@ The plugin monitors these Neovim events:
 | `BufWritePost` | Saving a file |
 | `CursorMoved` / `CursorMovedI` | Moving cursor |
 
+Commands that pass through many buffers (`:bufdo`, `:argdo`, `:cfdo`, session restore) count only the buffer that is current when they finish, and the text change of a reload from disk (the file changed outside after a `git checkout`, or a formatter or agent rewrote it) isn't counted as editing, while an undo back to the saved state is. Saves with `:wa` are attributed to each buffer written, at its own cursor.
+
 Only regular files are tracked. Special buffers (help, terminal, quickfix, etc.), unnamed buffers, files inside `.git/`, and non-`file://` URLs such as `oil://` or `fugitive://` are skipped.
 
 Heartbeats are:
 - **Deduplicated**: Buffer and cursor events at the same file and cursor position as the last event are skipped (edits always count)
-- **Debounced**: Max 1 heartbeat per file per 30 seconds (except saves)
-- **Batched**: Sent to the daemon over its Unix socket every 2 minutes
-- **Flushed on exit**: Pending heartbeats are sent on `VimLeavePre` (Neovim waits up to 1.5s)
-- **Queued**: Kept in memory and retried on the next flush if the daemon is unavailable (up to 5,000; oldest dropped first). They are not written to disk, so they are lost if Neovim exits while the daemon is down
+- **Debounced**: Max 1 heartbeat per file per 30 seconds. Saves count once per window too, since auto-save plugins write after every change: the last save inside the window is kept, at its own time and cursor, and sent with the file's next heartbeat or the next flush, so the save before `:wq` always counts
+- **Batched**: Sent to the daemon over its Unix socket every 2 minutes, at most 500 per message
+- **Flushed on exit**: Pending heartbeats are sent on `VimLeavePre`, and Neovim waits up to 1.5s for them and for any send already in progress. What the daemon doesn't take, or doesn't acknowledge in time, goes to `~/.shelltime/coding-heartbeat.data.log`, which the daemon resends once it runs
+- **Queued**: Kept in memory and retried on the next flush if the daemon is unavailable or has code tracking turned off (up to 5,000; oldest dropped first)
 
 ### Data Tracked
 
@@ -182,11 +183,11 @@ Each heartbeat includes:
 - **System info**: Hostname, OS, OS version
 - **Activity**: Timestamp, whether it was a save event, category (`coding` or `debugging`)
 
-The project root is the nearest parent directory containing `.git`, `package.json`, `Cargo.toml`, `go.mod`, `pyproject.toml`, `setup.py`, `Makefile`, `CMakeLists.txt`, `.project` or `.root` (otherwise the file's directory). The project name is its last two path segments, e.g. `code/my-app`.
+The project root is the git repository the file is in (as in VS Code and JetBrains, so a monorepo's packages count toward one project). Outside a repository it is the nearest parent directory containing `package.json`, `Cargo.toml`, `go.mod`, `pyproject.toml`, `setup.py`, `Makefile`, `CMakeLists.txt`, `.project` or `.root`, otherwise the file's directory. The project name is its last two path segments, e.g. `code/my-app`. The branch is read from the repository's `HEAD`, without running `git`, except in a reftable repository, whose `HEAD` is a stub: there it comes from `git rev-parse --abbrev-ref HEAD`, asked at most every 30 seconds per repository.
 
 ### CLI Update Check
 
-On startup, if the daemon is reachable, the plugin asks `<apiEndpoint>/api/v1/cli/version-check` (via `curl`) whether the daemon's CLI version is the latest. If not, it shows a warning once per session with the update command (`curl -sSL <webEndpoint>/i | bash`) and copies that command to the `+` register when a clipboard is available. The check is skipped when `apiEndpoint` or `webEndpoint` is not set.
+On startup, if the daemon is reachable, the plugin asks `<apiEndpoint>/api/v1/cli/version-check` (via `curl`) whether the daemon's CLI version is the latest. If not, it shows a warning once per session with the update command (`curl -sSL <webEndpoint>/i | bash`). The check is skipped when `apiEndpoint` or `webEndpoint` is not set.
 
 ## Troubleshooting
 
@@ -218,7 +219,7 @@ On startup, if the daemon is reachable, the plugin asks `<apiEndpoint>/api/v1/cl
 Run `:ShellTimeStatus` to check:
 - If "Disconnected", ensure daemon is running and `socketPath` matches the daemon's socket
 - If pending heartbeats > 0, try `:ShellTimeFlush`
-- If "Connected" and flushes succeed but no activity shows up, check that `codeTracking.enabled` is `true` for the daemon and restart it. The daemon drops heartbeats otherwise, and the plugin can't tell
+- If it warns that code tracking is off in the daemon config, set `codeTracking.enabled` to `true` (and restart an older daemon). The daemon drops heartbeats until then; the plugin keeps them queued
 
 ## License
 

lua/shelltime/config.lua

@@ -1,6 +1,7 @@
 -- Configuration loading for shelltime
 
 local yaml = require('shelltime.utils.yaml')
+local toml = require('shelltime.utils.toml')
 
 local M = {}
 
@@ -18,9 +19,20 @@ local defaults = {
 -- Default config file path
 local default_config_path = '~/.shelltime/config.yaml'
 
+-- Without a `config` option the plugin reads what the ShellTime CLI reads
+-- (cli/model/config.go): the first of these base files that exists, with the
+-- first local file merged over it
+local config_dir = '~/.shelltime'
+local base_files = { 'config.yaml', 'config.yml', 'config.toml' }
+local local_files = { 'config.local.yaml', 'config.local.yml', 'config.local.toml' }
+
+-- How often get_config() looks at the files again; it runs on every cursor move
+local RECHECK_INTERVAL_MS = 5000
+
 -- Cached config
 local cached_config = nil
-local cached_mtime = nil
+local cached_signature = nil
+local last_check = nil
 local config_path = nil
 local test_mode = false
 
@@ -41,17 +53,56 @@ end
 local function get_mtime(path)
   local stat = vim.loop.fs_stat(path)
   if stat then
-    return stat.mtime.sec
+    return stat.mtime.sec * 1e9 + stat.mtime.nsec
+  end
+  return nil
+end
+
+--- First of the given names in the config directory that exists
+---@param names string[] File names
+---@return string|nil Expanded path
+local function first_existing(names)
+  local dir = expand_path(config_dir)
+  for _, name in ipairs(names) do
+    local path = dir .. '/' .. name
+    if vim.loop.fs_stat(path) then
+      return path
+    end
   end
   return nil
 end
 
---- Load configuration from YAML file
----@param path string Config file path
+--- The files to read, base first
+---@return string[] Expanded paths
+local function config_files()
+  if config_path then
+    return { expand_path(config_path) }
+  end
+  local base = first_existing(base_files)
+  if not base then
+    return {}
+  end
+  local local_file = first_existing(local_files)
+  return local_file and { base, local_file } or { base }
+end
+
+--- Paths and modification times of the files, to tell when they change
+---@param files string[] Expanded paths
+---@return string
+local function signature_of(files)
+  local parts = {}
+  for _, path in ipairs(files) do
+    table.insert(parts, path .. ':' .. tostring(get_mtime(path)))
+  end
+  return table.concat(parts, '|')
+end
+
+--- Load configuration from a YAML or TOML file
+---@param path string Expanded config file path
 ---@return table Configuration table
 local function load_config_file(path)
-  local expanded = expand_path(path)
-  local file_config, err = yaml.load_file(expanded)
+  local parser = path:match('%.toml$') and toml or yaml
+  local file_config, err = parser.load_file(path)
 
   if err then
     if defaults.debug then
@@ -63,6 +114,24 @@ local function load_config_file(path)
   return file_config or {}
 end
 
+--- Merge a local override file over the base file, as the CLI's mergeConfig:
+--- set values replace the base's, empty strings don't, and codeTracking merges
+--- field by field (a local file that only sets its token keeps tracking on)
+---@param base table Base file config
+---@param override table Local file config
+---@return table Merged file config
+local function merge_file_configs(base, override)
+  local merged = vim.tbl_extend('force', {}, base)
+  for key, value in pairs(override) do
+    if key == 'codeTracking' and type(value) == 'table' and type(merged[key]) == 'table' then
+      merged[key] = merge_file_configs(merged[key], value)
+    elseif value ~= '' then
+      merged[key] = value
+    end
+  end
+  return merged
+end
+
 --- Merge file config with defaults
 ---@param file_config table Config from file
 ---@return table Merged config
@@ -92,13 +161,15 @@ local function merge_config(file_config)
     config.debounce_interval = file_config.debounceInterval
   end
 
-  -- API and web endpoints for version check
-  if file_config.apiEndpoint then
-    config.api_endpoint = file_config.apiEndpoint
+  -- API and web endpoints for version check (TOML spells them APIEndpoint and WebEndpoint)
+  local api_endpoint = file_config.apiEndpoint or file_config.APIEndpoint
+  if api_endpoint then
+    config.api_endpoint = api_endpoint
   end
 
-  if file_config.webEndpoint then
-    config.web_endpoint = file_config.webEndpoint
+  local web_endpoint = file_config.webEndpoint or file_config.WebEndpoint
+  if web_endpoint then
+    config.web_endpoint = web_endpoint
   end
 
   return config
@@ -108,11 +179,12 @@ end
 ---@param opts table|nil Setup options
 function M.setup(opts)
   opts = opts or {}
-  config_path = opts.config or default_config_path
+  config_path = opts.config
 
   -- Force reload
   cached_config = nil
-  cached_mtime = nil
+  cached_signature = nil
+  last_check = nil
   test_mode = false
 end
 
@@ -124,18 +196,26 @@ function M.get_config()
     return cached_config
   end
 
-  local path = expand_path(config_path or default_config_path)
-  local mtime = get_mtime(path)
+  -- Look at the files at most every few seconds
+  local now = vim.loop.hrtime() / 1e6
+  if cached_config and last_check and now - last_check < RECHECK_INTERVAL_MS then
+    return cached_config
+  end
+  last_check = now
 
-  -- Use cache if valid
-  if cached_config and mtime and mtime == cached_mtime then
+  local files = config_files()
+  local signature = signature_of(files)
+  if cached_config and signature == cached_signature then
     return cached_config
   end
 
   -- Load and cache
-  local file_config = load_config_file(config_path or default_config_path)
+  local file_config = {}
+  for _, path in ipairs(files) do
+    file_config = merge_file_configs(file_config, load_config_file(path))
+  end
   cached_config = merge_config(file_config)
-  cached_mtime = mtime
+  cached_signature = signature
 
   return cached_config
 end
@@ -154,10 +234,19 @@ function M.is_enabled()
   return M.get('enabled') == true
 end
 
---- Get config file path
+--- Get config file path: the one passed to setup(), else the base file the
+--- CLI's discovery picks (config.yaml when there is none yet)
 ---@return string Config path
 function M.get_config_path()
-  return config_path or default_config_path
+  if config_path then
+    return config_path
+  end
+  for _, name in ipairs(base_files) do
+    if vim.loop.fs_stat(expand_path(config_dir .. '/' .. name)) then
+      return config_dir .. '/' .. name
+    end
+  end
+  return default_config_path
 end
 
 --- Set config values directly (for testing only)