Skip to content

docs: document state-machine tokenizer architecture#13

Merged
imrim12 merged 2 commits into
mainfrom
claude/docs-fsm-tokenizer
May 1, 2026
Merged

docs: document state-machine tokenizer architecture#13
imrim12 merged 2 commits into
mainfrom
claude/docs-fsm-tokenizer

refactor(parser): collapse to a single Tokenizer, drop regex variant

75d094f
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar