Y::Lexical: native Lexical/Lexxy HTML rendering — full Lexxy schema, byte-for-byte by jpcamara · Pull Request #41 · jpcamara/yrby

jpcamara · 2026-07-03T02:24:47Z

Expands the Lexical work to everything a Lexxy Rails editor can produce, rendered to HTML natively from the CRDT bytes — no Node process, no headless editor, no JSDOM.

The tiptap-php precedent, one level deeper

ueberdosis/tiptap-php renders ProseMirror JSON to HTML in pure PHP — official, schema-pinned, accepted ecosystem practice. This does the same for Lexical/Lexxy, but from the collab (Yjs) structure itself rather than the editor's JSON. Nothing else in the y-crdt ecosystem renders editor HTML natively (survey in the preceding discussion — every existing path bounces through a headless JS editor).

Ground truth, not guesswork

The reference pair was captured from one live Lexxy editor session (agent-browser driving the lexxy-realtime test server):

fixtures/lexxy_full.bin — the synced Yjs state of a document exercising every node type
fixtures/lexxy_full.html — the same editor's own value at that moment: Lexical's $generateHtmlFromNodes + Lexxy's custom text export + DOMPurify sanitize — the exact HTML a Lexxy form submits to Rails

read_html reproduces it byte for byte, asserted in both Rust and Ruby. The format-run algorithm replicates Lexxy's own exportTextNodeDOM (read from their source): inner strong/em, outer code/mark/sub/sup,  only when bold+italic collide, s/u wraps, spans unwrapped — so every format combination is deterministic, not just the captured ones.

Coverage — the full Lexxy 0.9.x node set

Family	Canonical output
Headings h1–h6, paragraphs	plain tags; empty `<p><br></p>`; Lexxy's provisional paragraph exports nothing
Text formats (all bits + combos)	`strong/em/s/u/code/sub/sup/mark`, Lexxy's exact nesting
Links/autolinks	`href` + `title` (target/rel stripped, like Lexxy's sanitize)
Lists: bullet/numbered/check/nested	canonical `li` attribute order (`aria-checked`, `value`, `lexxy-nested-listitem`)
Quotes, code blocks	`blockquote`; `pre data-language` (Lexxy's `early_escape_code`), tokens flatten, tabs `<span>\t</span>`, breaks `<br>`
Horizontal rule	`<hr>` (Lexxy's `horizontal_divider` decorator)
Tables	`figure.lexxy-content__table-wrapper > table > tbody`, `th` with Lexxy's class+background
ActionText attachments	uploads AND mentions/embeds emit canonical `<action-text-attachment>` (Lexxy's attribute order, null-skipping, `presentation="gallery"`) — ready for ActionText re-rendering

Unknown block types degrade to readable paragraphs instead of vanishing.

Also: text extraction now covers attachments

read_xml previously dropped attachments entirely. Now a mention's plainText joins its line ("Mention: @alice done.") and an upload contributes caption/altText/fileName.

Verification

Rust: 35 tests (byte-for-byte fixture render, per-combo format algorithm, escaping, extraction incl. attachments)
Ruby: 102 runs — html_test.rb asserts the byte-for-byte match end-to-end plus one structural probe per node family (a regression names the node)
clippy + rustfmt + rubocop clean

Notes for review

Schema pin: Lexxy 0.9.x. The storage model and every mapping decision are documented in html.rs's header. A Lexxy upgrade that adds nodes degrades gracefully (readable paragraphs) and extends via new match arms + a recaptured fixture.
ProseMirror HTML is explicitly out of scope (docs say so); read_xml/read_text remain the PM path.
Version left at 0.3.1 / CHANGELOG under [Unreleased] — Live Y::Map handles: read and edit shared maps from Ruby #33 (live-maps) currently claims 0.4.0; whichever merges first takes it.

🤖 Generated with Claude Code

`Y::Lexical.new(doc).to_html` renders a Lexxy (Lexical) document to HTML directly from the yrs collab structure -- no Node process, headless editor, or JSDOM. Prior art: ueberdosis/tiptap-php does this for ProseMirror JSON in pure PHP; this is the same schema-pinned approach one level deeper, on the collab bytes themselves. Nothing else in the y-crdt ecosystem renders editor HTML natively. API design: the schema knowledge lives in a class NAMED for the schema -- Y::Lexical wraps a doc (cheap Arc clone, reads live state); Doc itself stays schema-agnostic. to_html defaults to Lexical's standard "root" fragment. A root that isn't Lexical-shaped (no __type-stamped children, e.g. a ProseMirror document) returns nil -- never a lossy rendering of a foreign schema. Within a Lexical doc, an individual unknown node type still degrades to a readable paragraph rather than vanishing. Ground truth, not guesswork: the reference pair (fixtures/lexxy_full.{bin,html}) was captured from ONE live Lexxy editor session -- the .bin is the synced Yjs state, the .html is the editor's own `value` (Lexical's $generateHtmlFromNodes + Lexxy's custom text export + DOMPurify sanitize; the exact HTML a Lexxy form submits to Rails). The renderer reproduces it BYTE FOR BYTE, asserted in both Rust and Ruby tests. Coverage (the full Lexxy 0.9.x node set): headings h1-h6; every text format bit and combination (Lexxy's exact export algorithm: inner strong/em, outer code/mark/sub/sup, only when bold+italic collide, s/u wraps, spans unwrapped); links (href+title; target/rel stripped like Lexxy's sanitize); bullet/numbered/check/nested lists with canonical li attribute order; quotes; code blocks with data-language (Lexxy's early_escape_code type; highlight tokens flatten); tabs (\t) and soft breaks ( ); horizontal rules; tables with header cells (figure wrapper, th class + background, matching Lexxy); ActionText attachments -- uploads and mentions/embeds emit canonical <action-text-attachment> elements with Lexxy's attribute order and null-skipping, ready for ActionText re-rendering; empty paragraph vs Lexxy's provisional-paragraph-exports-nothing distinction. Also: text extraction (read_xml) now includes attachment text -- a mention's plainText joins its line; an upload contributes caption/altText/fileName. Attachments previously vanished from extracted text. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

jpcamara · 2026-07-03T02:49:32Z

API reshaped per review feedback — Doc#read_html was dishonest naming: the renderer is entirely Lexical/Lexxy-specific, and a generic method name on the schema-agnostic Doc misrepresented that (worse, a ProseMirror doc silently rendered to an empty string).

Now:

Y::Lexical.new(doc).to_html — the schema knowledge lives in a class named for it, taking a document (holds a cheap Arc clone; reads live state). to_html defaults to Lexical's standard "root" fragment.
Module renamed html.rs → lexical_html.rs.
Shape detection: a root that isn't Lexical-shaped (no __type-stamped children — e.g. ProseMirror) returns nil, never junk. Regression-tested (a_prosemirror_shaped_root_is_refused_not_mangled).
Doc keeps only the schema-agnostic reads (read_text/read_xml/read_map).

Everything re-verified: Rust 36 / Ruby 103, byte-for-byte fixture match through the new API, clippy+fmt+rubocop clean. (Bonus: writing the refusal test initially reproduced the read_text deadlock class in test code — get_or_insert_xml_fragment inside a live read txn — caught by the 60s hang and fixed with a comment referencing the lesson.)

…mmer Y::Lexical was thread-safe by construction (one transaction per to_html, opened inside nogvl; the renderer only ever touches that transaction, so no nested-transaction path exists) — but not ENFORCED by the project's own conventions. Now it is: - RbLexical joins assert_thread_safe(): if it ever loses Send+Sync, the build fails instead of shipping a thread-unsafe class. - thread_safety_test.rb hammers to_html from readers while writers apply updates to the same doc (different root, full write-lock contention), asserting rendered content is never lost or torn — and hanging under the CI timeout if a future edit reintroduces a nested transaction in the render path (the read_text deadlock class). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

A second ground-truth pair (lexxy_torture.bin/.html) built to abuse nesting, captured from one live Lexxy session via the same inject-and-capture method as lexxy_full: blocks inside table cells (list, quote, code block, mention), header states 2 and 3 (header column / corner cells — all non-zero states render <th>), five levels of mixed bullet/number/check lists carrying links, inline code and mentions at depth, a titled link inside a heading wrapping formatted runs, adjacent single-character format runs including the full bold|italic|strike|underline|code stack, sub/sup combined with strike/underline, tabs, blank lines and emoji inside a code block, adjacent dividers, full and all-null upload attachments (Lexical normalizes null alt/caption to empty strings), CJK/RTL/emoji text, a pre-escaped-looking '&' trap, and whitespace-only paragraphs. Y::Lexical reproduced Lexxy's sanitized export byte-for-byte with no renderer changes. Two findings pinned by the fixture: Lexxy's own export drops colSpan/rowSpan entirely (so not emitting them is parity, not a gap), and null attachment fields reach the collab doc as empty strings, not nulls. Rust: byte-for-byte + named structural probes per nesting family. Ruby: byte-for-byte through the public API + text extraction sanity on the same doc. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

render_block recursed once per nesting level (list/listitem/table/cell, and links through render_inline). The recursion ran in nogvl on the calling thread's native stack, so a deeply nested document overflowed it: ~40k levels on the main thread's 8MB stack, but only ~8k on a ~1MB Puma/ActionCable worker thread — where it's a hard SIGSEGV, not a catchable SystemStackError (Ruby's guard page only converts the overflow on the main thread). A hostile client can sync a doc with a few thousand nested list items and crash the worker rendering it; once this is extracted to a standalone crate, a pure-Rust caller segfaults unconditionally. Block traversal now runs on an explicit heap work-stack (Open pushes a node's children plus a Close for its end tag; leaves render in place), so depth costs memory, not stack frames. A MAX_BLOCK_DEPTH cap (1024, ~128x the deepest real fixture) bounds the work a pathological doc can demand: past it the subtree is dropped but its enclosing tags still close, keeping the output well formed. Inline links still recurse (their body is inline-only, never blocks) but carry their own MAX_INLINE_DEPTH cap, degrading to flat text past it. Output is byte-for-byte unchanged on both fixtures. New tests render 20k-deep nesting on a 512KiB thread without overflow, confirm the cap truncates while keeping tags balanced, and cover a past-cap link chain. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

- to_html now raises ArgumentError on extra args instead of silently ignoring them (arity 0..1). - Document the two parity assumptions the renderer relies on: elem_num assumes normal-magnitude finite numbers (file sizes, dimensions), and render_inline assumes Lexxy's one-metadata-map-per-run invariant. - Comment cleanup: drop editorializing openers and the repeated 'one level deeper' phrasing, trim the two exhaustive test-fixture inventories (the fixtures and named probes already carry the specifics), and fix a stale 'html.rs' reference (now lexical_html.rs). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

A deep pass over the comments in the Lexical work for three recurring tics beyond the codebase's normal dense-why style: - repeated 'exactly/exact' intensifier on 'matches Lexxy' (3x → the one anchoring claim); - echoed rationale for the nesting caps, stated at the constant and re-explained at push_block_children ('pathological'/'demand' twice); - a cluster of punchy negative closers ('not a stack overflow', 'can't overflow on a crafted link chain', 'never a lossy rendering', 'never a crash') that read uniform because they were written in one sitting. Also de-stacked render()'s triple antithesis (rather/rather/instead-of) into one real contrast plus plain statements, and unknotted the thread-safety test's 'this hammer both proves...and would hang'. No behavior change; em-dash/why-comment house style kept to match the rest of the crate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The CHANGELOG, README, and module doc read as machine-written: stacked clauses, 'X rather than Y' antithesis, precious abstractions ('the schema knowledge lives in a class named for it'). Rewritten as short declarative sentences. No content change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

A real Lexxy editor — the demo's lexxy page in headless Chrome via agent-browser — builds a document covering every node type over the live collab stack, and Y::Lexical must reproduce the editor's own value (the sanitized HTML a Lexxy form submits to Rails) byte for byte from the doc bytes, piped through frontend/render_check.rb. The gem's fixture tests pin parity with the editor version they were captured from; this catches serializer drift when @37signals/lexxy is bumped. It runs in the existing Puma e2e step, where Chrome and agent-browser are already set up. Why not a DOM emulator like the Tiptap parity test: Lexxy's value getter is DOMPurify.sanitize($generateHtmlFromNodes(...)) with custom hooks, and under emulated DOMs both halves distort — happy-dom's DOMPurify silently drops heading elements, and the export carries theme classes and dir attributes the browser pipeline strips. A parity reference needs the real engine. The lexxy page's window.__yrb gains encodeState() (doc state as base64) as the test hook. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

cursor · 2026-07-07T14:23:56Z

Bugbot is not enabled for your account, so this pull request was not reviewed.