Conversation
The symbol menu and hover suppress themselves on comments and string literals. Answering that needed a second parser; the diff viewer had already made the same call while painting the token the user clicked. Vendor three patches that surface it. `@pierre/diffs` asks Shiki for the TextMate token class and stamps it on the span it already annotates with `data-char`. Shiki drops that class twice on the way through a dual-theme render -- once merging the per-theme tokenizations, once flattening the variants into CSS variables -- so both rebuilds now carry it. A viewer rendering one theme keeps `type` on its own and hides both gaps, which is why the guard test renders two. `targetOf` reads the attribute off the clicked span, so `openSymbolMenu` and `requestHover` answer synchronously instead of awaiting a parse. The patches are a local stand-in until this lands upstream; tree-sitter still runs, and still owns keywords, which `StandardTokenType` cannot express. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8ro9sQLfwxHnmbyDDBmJ3
Adding patch-package changed package-lock.json, so the pinned hash no longer matched and `nix build` failed. Value taken from the mismatch the nix job reported, as the comment above it prescribes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8ro9sQLfwxHnmbyDDBmJ3
Adding patch-package gave the package a `prepare` script. The lockfile job installs nothing, so npm ran that script against a patch-package binary that was not there and exited 127 before it ever compared lockfiles. The job already documents that it needs no install; say the same to npm. Regenerate package-lock.json on Node 24 as `engines` requires. The lockfile committed with the patch-package dependency came from Node 22, whose npm keeps an optional peer entry this npm drops -- exactly the drift the job's own error message warns about. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8ro9sQLfwxHnmbyDDBmJ3
Regenerating package-lock.json on Node 24 dropped an optional peer entry, which moved the hash again. Value taken from the mismatch the nix job reported. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8ro9sQLfwxHnmbyDDBmJ3
Resolve the flake.nix npmDepsHash conflict: both sides moved it, and the merged lockfile matches neither, so recompute it against the merge result. Rebase package-lock.json on main's and re-add only the patch-package delta, so the merge does not carry over libc metadata deletions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8ro9sQLfwxHnmbyDDBmJ3
main collapsed `targetOf`'s word fallback now that `clientX` is required; keep that shape and attach `tokenType` to the single return. Add `tokenType: null` to main's new ReviewPane target fixture. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8ro9sQLfwxHnmbyDDBmJ3
0.1.6 release plus the search revamp. Both sides had changed `npmDepsHash`; the merged lockfile matches neither, so recompute it against the merged tree (verified with a full `nix build`). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8ro9sQLfwxHnmbyDDBmJ3
Font bundling moved the dependency set, so recompute `npmDepsHash` against the merged tree (verified with a full `nix build`). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8ro9sQLfwxHnmbyDDBmJ3
The gh-to-token refactor adds Octokit deps, so both sides inserted entries at the same alphabetical spots in the lockfile. Rebase the lockfile on main's and regenerate, leaving only the patch-package additions, then recompute `npmDepsHash` (verified with a full `nix build`). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8ro9sQLfwxHnmbyDDBmJ3
0.1.7 release plus HistoryNav. Recompute `npmDepsHash` against the merged tree (verified with a full `nix build`). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P8ro9sQLfwxHnmbyDDBmJ3
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Carry Shiki's TextMate token classification through the dual-theme render pipeline and expose it on rendered token spans, enabling the symbol menu to skip language-server queries for comments and string literals.
Summary
The syntax highlighter (Shiki) classifies tokens by type (comment, string, regex, etc.) during tokenization, but this information was lost during the dual-theme rendering process. This change preserves and exposes that classification via a
data-token-typeattribute on rendered spans, allowing the LSP client to avoid unnecessary server queries for tokens that cannot contain symbols.Key changes
Patches to Shiki and @pierre/diffs: Three vendored patches preserve the
typefield through the multi-theme token merging pipeline and expose it as adata-token-typeHTML attribute on token spans.@shikijs/primitive: Preservetypewhen merging tokens across themes@shikijs/core: Includetypein flattened token output@pierre/diffs: Map token type todata-token-typeattribute and requestincludeExplanation: "tokenType"from ShikiNew
tokenTypeAt()helper (src/client/lsp/target.ts): Synchronously read the highlighter's token classification from a rendered span element. Validates against known types (comment, string, regex).Updated
TokenTargetinterface: Add optionaltokenTypefield to carry the classification through the click handler.Symbol menu optimization (
src/client/store.ts): Skip language-server queries when the token is classified as a comment or string literal—these cannot contain symbols, so no parse is needed.Test coverage (
test/client/tokenType.test.ts): Verify that the patches work end-to-end across multiple languages and that the helper correctly reads and validates the attribute.Build integration (
package.json): Addpreparescript to apply patches on install.Implementation notes
The solution is synchronous: the diff viewer classifies tokens at render time, so a click needs no additional parse. The three patches are minimal and focused on preserving an existing Shiki field through the rendering pipeline rather than adding new classification logic.
https://claude.ai/code/session_01P8ro9sQLfwxHnmbyDDBmJ3