perf(core): eliminate line-breaking bottleneck on massive unspaced text blocks (#28)

[DrkXo]

Fixes #28

Summary

Addresses the severe layout freeze (~13.4s down to ~250ms) when rendering documents containing long paragraphs without whitespace (CJK literature, raw text dumps, and unspaced strings), as reported in #28.

Thanks 🙏 for the detailed feedback.

---

Root Cause

Previously, line breaking for long text blocks measured candidates via repeated substring slicing and queue bouncing across every line. For paragraphs without whitespace, this resulted in quadratic $O(N^2)$ HarfBuzz text-shaping passes over thousands of characters per line.

---

Changes

1. Native Multi-Line Fast Path:
   • When no active/pending floats exist and available width is uniform, layout is delegated to a single TextPainter pass using native ICU line-breaking (computeLineMetrics()), completing in $O(1)$ layout passes.
2. Bounded Prefix Candidate Search:
   • In fallback scenarios (e.g. paragraphs wrapping around floats), candidate text measurement is bounded to a safe character limit (_safeCandidateCharLimit), eliminating the quadratic overhead when measuring line breaks.
3. Loop Termination & Margin Parity:
   • Guaranteed termination on non-positive line widths (e.g. adversarial negative/zero container widths or deep padding) by clearing active floats or flooring to 1.0.
   • Aligned trailing whitespace caret measurement with native LineMetrics.width to ensure identical geometry between fast-path and fallback breaker.

---

Reviewer Feedback & Verification Checklist

• Intrinsic Widths: Added tests verifying getMinIntrinsicWidth and getMaxIntrinsicWidth for long unspaced fragments inside shrink-to-fit contexts (<td> table cells and display: inline-block containers). Full text is preserved and measured without artificial prefix clamping.
• Fast-Path Parity & Edge Cases: Verified that native multi-line layout produces the exact same line breaks and fragment geometry as the fallback breaker across:
  • Plain text
  • text-align: justify
  • text-overflow: ellipsis (truncates on single line; does not multi-line wrap)
  • Non-default textScaler (1.5×)
  • Mixed CJK and Latin text
  • Mid-paragraph floats (verifies fallback path activation)
• Selection & Hit-Testing Continuity: Confirmed strict globalOffset continuity (charEnd == next.charStart), getBoxesForCharRange cross-line rects, and unbroken string extraction in getSelectedText.
• Deterministic Performance: Avoided wall-clock thresholds. Tests count TextPainter.layout calls during line layout ($\le 2$ passes) and verify a linear $N \to 2N$ layout ratio ($\approx 2.0\times$, assert $< 3.5\times$).
• Fixture Size: Removed heavy Gutenberg fixture files; all tests now use compact synthetic unspaced strings.
• Full Suite: Full test suite (flutter test test/ packages/hyper_render_core/test/ --exclude-tags golden) runs 100% green (2,541 / 2,541 passing).

[vietnguyentuan2019]

Thanks for this, the speedup is real! 🙏 I reproduced #28 on main and re-ran the same benchmark on this branch (one <p>, 400px, mode: sync, flutter test):

chars	CJK main → PR	Latin main → PR
8,000	2,094 → 158 ms	423 → 72 ms
16,000	4,699 → 68 ms	1,898 → 53 ms

I also dumped debugLineFragments() on main and on this branch for the same inputs (300px wide), and checked every line's visible text width against a fresh TextPainter. That turned up one blocker, plus a set of behavior changes that need to be listed rather than described as parity.

Blocker: text after an inline element ends up on a single line

<p><b>Note:</b> lorem ipsum dolor sit amet consectetur … (≈60 words)</p>

On this branch, line 1 is a single 377-char fragment that reports width: 221 but is really ~6,000px wide. The rest of the paragraph never wraps. CJK after a <b> does the same (184 chars on one line). On main both wrap normally.

Cause: in _splitTextFragment, when secondPart.length > safeLimit the tail gets a fake size Size(maxWidth + 1.0, …). Here maxWidth is the remaining width of the line where the split happened, not the container width. After <b>Note:</b> that remaining width is small (~220px). On the next, full-width line the tail looks like it fits, so it is placed whole. That's the most common paragraph shape there is (bold lead-in, link, <code> at the start), so it has to be fixed before merge. Using the container width (or double.infinity) for the estimate, or re-measuring when the estimate is used to decide "fits", would avoid it. Please add a regression test that asserts no line fragment's real width exceeds the container, for <p><b>x</b> + long text</p> in both Latin and CJK.

Line overflow in the fallback path

Beside a float (<img style="float:left;width:120px"> + long text), one line is "consectetur " at x=128 with a real width of 176, so it ends at 304px in a 300px box. It looks like the breakIndex++ when a space sits at the break, plus <= in the space binary search. getPositionForOffset rounds to the nearest caret (300/16 = 18.75 → 19), so a space at 19 now lets a 19-glyph word through. main has the same rounding problem for CJK (every full CJK line on main is 304px), and your native path fixes that, but the fallback path should not reintroduce it for Latin.

Behavior changes (mostly good, but please list them)

The parity tests compare the new fast path against the new fallback, not against main. Every sample I tried breaks differently from main, which is why 8 goldens fail. Most of the differences come from real bugs on main that the current goldens captured:

• RTL wrapping: on main an Arabic paragraph puts 72 chars (1,152px) on line 1, then lines of one glyph each (rtl_arabic.png shows it). Fixed here.
• word-break: break-all: on main it produces dozens of one-character lines. Fixed here.
• CJK lines overflow by a fraction of a glyph on main (304px in 300px). Fixed in the native path.
• Trailing spaces no longer count toward line width. This shifts text-align: center/right (correct per CSS, since trailing spaces hang), but it changes every wrapped Latin line, so it's a visible change.

I'm fixing these pre-existing bugs myself on release/1.13.0: RTL wrapping, break-all, the nearest-caret overflow, trailing spaces counted in line width, and a word next to a float being split after its first letter. The fixes are small and live in _splitTextFragment / _forceSplitTextFragment, sharing one helper that finds the longest prefix that fits. I've changed this PR's base to release/1.13.0. Please don't rebase yet. I'll ping you here when the fixes are in, and then the rebase should be mostly mechanical. Your rewritten split code can call the same helper.

After that rebase, the bar for merging is: this PR should not change layout compared to release/1.13.0. That means identical debugLineFragments() output on the same inputs and no golden diffs. If something still differs on purpose (for example, ICU breaking vs KinsokuProcessor in the native path), list it in the PR description with the reason. The text-overflow: ellipsis change in _tokenizeBlock (the new ||, which now always emits a block-start fragment) needs a reason too. Goldens only match on the CI runner, so please don't try to update them locally.

Smaller notes

• The native path's isPreformatted includes nowrap but not break-spaces, and _getCandidatePainter only checks pre/pre-wrap. Please use one helper for both.
• HyperRenderDebugHooks.onTextPainterLayout / onLineLayoutTextPainter are new public API. Fine for testing, but please add a doc line and a CHANGELOG entry.
• Once it's in, this ships with 1.13.0 together with #26 and the fixes above.

Thanks again, this is a big improvement once the tail-width issue is sorted!

[vietnguyentuan2019]

@DrkXo the line-breaking fixes are in on release/1.13.0 (this PR's base). You can rebase now. 🙏

What changed in render_hyper_box_layout.dart, so you know what you're rebasing onto:

• _fitPrefixLength(painter, text, maxWidth): the longest prefix that really fits. It mirrors the probe for RTL and corrects the nearest-caret overshoot with _prefixWidth (a caret lookup, O(1)). Both _splitTextFragment and _forceSplitTextFragment use it. Please route your bounded candidate search through it too, instead of calling getPositionForOffset(Offset(maxWidth, 0)) directly.
• _forceSplitTextFragment breaks an over-long word after as many characters as fit (not after 1), skips the spaces after the break, and trims the trailing space of the first part.
• Next to a float, if not even the first word fits (_leadingUnitFits), the line moves below the float.
• Leading spaces collapse when a text fragment starts a line, and the last fragment's trailing spaces are trimmed in finishLine(). A fragment that doesn't fit is re-queued after finishLine(), so it goes through those line-start rules.
• word-break / overflow-wrap now inherit (resolver.dart).

The goldens on the branch are regenerated and green, and test/line_breaking_regression_test.dart (15 tests) covers all of the above. So the bar from my review is now concrete: after your rebase, the goldens and line_breaking_regression_test.dart must pass unchanged, plus the blocker repro (<p><b>Note:</b> + long text, Latin and CJK) must wrap. Any intended difference from the native ICU path should be listed in the description.

[DrkXo]

Thanks 🙏 for landing the upstream line-breaking fixes and regenerating the golden references!

The PR has been rebased cleanly onto release/1.13.0 with the requested changes:

1. Blocker: Text after an inline element wrapped onto a single line

• Root Cause: In _splitTextFragment, oversized tails (secondPart.length > safeLimit) were assigned an estimated size Size(maxWidth + 1.0, ...), where maxWidth was the remaining width of the lead-in line (~220px). On the following full-width line (300px), 221px appeared smaller than 300px, placing the entire remaining text on one line.
• Fix: The tail limit is now evaluated against the container width _maxWidth (_safeCandidateCharLimit(_maxWidth, ...)), setting Size(double.infinity, ...) for oversized tails, combined with a pre-placement check if (fragment.measuredSize == null || fragment.width.isInfinite) _measureFragment(fragment).
• Regression Test: Added tests in test/line_breaking_perf_and_parity_test.dart asserting that <p><b>Note:</b> + text</p> and <p><b>注意：</b> + text</p> wrap into >4 lines and x + realTextPainter.width <= 300.5 across every laid out fragment.

2. Integration with upstream _fitPrefixLength

• The bounded candidate prefix search in _splitTextFragment and _forceSplitTextFragment now routes prefix length calculation through _fitPrefixLength(painter, candidateText, maxWidth), preserving the RTL mirror probing and nearest-caret overshoot protections from d2a5851.

3. Native Multi-Line Fast Path Guards

• canUseNativeMultiLine is strictly guarded against RTL (!fragment.style.isRtl && textDirection != rtl), word-break: break-all, floats, preformatted spaces, and ellipsis. RTL text and break-all paragraphs seamlessly use the bounded fallback breaker.

4. Code Quality & Documentation

• Consolidated preformatted whitespace checks under _isPreformattedWhiteSpace(String? ws) (pre, pre-wrap, break-spaces).
• Added docstrings and examples for HyperRenderDebugHooks.onTextPainterLayout and onLineLayoutTextPainter.
• Updated root and package CHANGELOG.md under 1.13.0.
• _tokenizeBlock ellipsis rationale: _BlockStartFragment carries truncateWithEllipsis: true. Unmargined/unpadded top-level blocks previously skipped emitting _BlockStartFragment, which dropped ellipsisDepth++ and caused downstream line-breaking to fail to truncate.

All 15 tests in line_breaking_regression_test.dart and 15 tests in line_breaking_perf_and_parity_test.dart are 100% passing.

[vietnguyentuan2019]

Thanks for the quick rebase, this is very close. 🙏 The <b>Note:</b> blocker is fixed. I checked out 2a98a2f and compared it against release/1.13.0 on 24 inputs (same harness as before: debugLineFragments() at 300px, plus the existing suites). Everything is identical except one regression and three differences that need a decision or a test.

1. Regression (blocker): a floated block's text is now measured

test/flagship_execution_audit_test.dart fails on this branch (it passes on release/1.13.0):

Float layout KNOWN GAP — a floated block sized by its own text is not floated
Expected: <208.0>   Actual: <216.0>

<div style="float:left;width:150px">One two three … thirteen.</div><p>One two three … thirteen.</p>

The first line is now 24px tall instead of 16px, so the document is 248px instead of 240px. I bisected it to one hunk, in processFragment just before fragment.offset = Offset(currentX, currentY):

if (fragment.measuredSize == null || fragment.width.isInfinite) {
  _measureFragment(fragment);
}

The measuredSize == null half is what changes layout. The floated div's text fragment reaches this point with measuredSize == null (width 0 on main), and it is now measured and gets a height. The isInfinite half is the one the oversized-tail fix needs. With the condition narrowed to just if (fragment.width.isInfinite), flagship_execution_audit_test.dart passes again and all 30 tests in line_breaking_perf_and_parity_test.dart and line_breaking_regression_test.dart still pass. Please make that change. Why the float's text fragment is unmeasured is a separate known gap (LIMITATIONS.md), so it's not for this PR.

(The CI "no checks" on this PR is because the base isn't main. Please run the whole flutter test test/ packages/hyper_render_core/test/ --exclude-tags golden locally before pushing. This test is in that command.)

2. Intended differences, please list them in the PR description

I ran the golden workflow on this branch plus the one-line fix above. Only one golden changes: cjk_kinsoku.png. The Japanese paragraph now fills line 2 after the space between two sentences (the 。 + newline in the HTML) instead of starting the next sentence on a new line. The same thing happens in mixed text: <p>lorem ipsum dolor sit amet 日本語のテキストが混在する文章 …</p> now continues sit amet 日本語のテキストが on one line, where main moved the whole CJK run down. Both look right to me (that's how a browser breaks CJK), so I'm fine with them, but they come from ICU breaking instead of _splitTextFragment, so they should be named in the description and in the 1.13.0 CHANGELOG entry. I'll regenerate that golden on our side.

3. Ellipsis change needs a test

The _tokenizeBlock change (emit _BlockStartFragment when textOverflow == ellipsis) is a real fix: on main, <div style="overflow:hidden;text-overflow:ellipsis;white-space:nowrap">…long text…</div> as the first block wraps onto 9 lines instead of truncating, and with this PR it gives one line ending in …. Thanks for finding it! But the only ellipsis test builds the style by hand (textOverflow: TextOverflow.ellipsis), so it never goes through the resolver or the first-block path, and it would pass without the fix. Please add one that parses that HTML as the first element (no preceding <p>) and asserts a single line ending with …. Then mutation-check it: remove the || and make sure the test fails.

Small

• Regression test names in the PR description say "2,541 / 2,541 passing". With my one-line fix, and including flagship_execution_audit_test.dart, the full run on release/1.13.0 is 2542 + your 15 new tests. Just re-run the count after the changes.

After 1 and 3, I'll re-run the same comparison and merge. 🙌

[vietnguyentuan2019]

Merged into release/1.13.0 (merge commit 1feb2fc), so you're credited in the history and the changelog. Thank you, the speedup is great and the first-block ellipsis fix was a real find! 🙏

Two things I changed on top, so you know what happened to your branch: (1) narrowed measuredSize == null || width.isInfinite to width.isInfinite in processFragment, because the null half measured a floated block's own text and broke flagship_execution_audit_test (extra 8px); (2) added an HTML-parsed test for the first-block ellipsis fix plus a float regression test, and regenerated cjk_kinsoku.png (the CJK fill-the-line change). All CI green. It ships with 1.13.0.

[DrkXo]

Thank you so much for the thorough review, catching that float edge case, and especially for taking the time to add the tests and follow-up fixes to get this across the finish line! 🙏

I really appreciate the detailed explanation of the measurement condition and why the floatDiv behaved that way—that was a great catch.

Thanks again for the guidance and wonderful collaboration!

[DrkXo]

"Hi @vietnguyentuan2019,

Update: I’ve rebased the customizable_selection_anchors branch with the latest upstream main (commit a762026).

On a related note, I explored customizable selection handles in this branch (feat(selection): add customizable start and end anchor builder), supporting HyperSelectionAnchorBuilder across single-box and virtualized modes with reactive drag states.

Would this feature be a welcome addition to hyper_render?"

[vietnguyentuan2019]

Thanks, this is welcome. The builder API looks good and the default path is untouched.

I ran your branch on current main: analyze and format are clean and all 2621 root+core tests pass. Before a PR, could you:

1. Explain the tap-outside change in 2ffb4b2 and which bug it fixes, since it also affects taps on the content area.
2. Either expose onSelectionChanged on HyperViewer or drop it from this PR. It only exists on HyperSelectionOverlay now, and shares a name with the VoidCallback on HyperRenderWidget.
3. Make selectedText lazy, so large virtualized selections don't join the text on every handle build while dragging.
4. Document how the builder's size drives anchoring (FractionalTranslation), so a builder returning an unsized widget is not misplaced.
5. Add CHANGELOG entries for core and root.

This touches core, so it will go out as a minor release (core 1.14.0). Please open it as a PR against main. Thank you!