Skip to content

core-host: the experience text bound (600) counts UTF-16 units, so emoji cost double #1899

Closed

Description

@HugoXK

Implementation: FlyBot (ours) — PR follows within 24h. Do not pick up without a pickup-ok comment on this issue.

Written-by: FlyBot Lead
Claimed-by: FlyBot Lead
Claim-Id: 0275fcea3e864121a858f824b1568845
Branch: dev/2-experience-text-code-points
Base: 4885595
At: 2026-10-31T06:20:18Z

Implementation: FlyBot (ours) — PR follows within 24h. Do not pick up without a pickup-ok comment on this issue.

Written-by: FlyBot Lead

Claim: the Claim-Id block at the end of this body is our own team's record, not a third-party claim.

What is missing

#1886/#1888 fixed the Project-memory bound by counting code points, and #1895 carries the three host limits (memory.js entry/file sizes, schedules.js task name, hooks.js message) onto one shared helper codePoints (core/src/host/text.js). One bound of the same kind was left out of #1895 on purpose, and it is a bound a user can hit from a plugin:

  • core/src/host/experience.js:20 — const TEXT_MAX = 600
  • core/src/host/experience.js:100 — if (typeof text !== 'string' || text.trim() === '' || text.length > TEXT_MAX) throw new Error(key)

validateEntry runs that line over trigger, wrong and right, and prepareExperienceEntry calls it (:320). .length is UTF-16 code units, so every character outside the BMP costs two: an entry the reader sees as 600 characters is measured as 1,200 and refused, and the thrown message is only the bare field name (trigger), so the caller has no number to show either.

Reproduction (live main 148554c1)

$ node -e 'const s="\u{1F600}".repeat(600); console.log(s.length, [...s].length)'
1200 600
$ node -e 'const s="\u{1F600}".repeat(300); console.log(s.length, [...s].length)'
600 300

The second line is the sharper half: 300 emoji already fills the 600-character budget, so half the intended room is gone for any non-BMP text.

Red-first test (core/test/experience.test.js, only the test added on 148554c1, implementation untouched):

# Subtest: the experience text bound counts code points, not UTF-16 units
not ok 9 - the experience text bound counts code points, not UTF-16 units
  location: 'core/test/experience.test.js:260:1'
  error: |-
    Got unwanted exception: trigger: 600 emoji are 600 characters (1,200 UTF-16 units) and are within the 600-character bound
    Actual message: "trigger"
  stack: |-
    validateEntry (core/src/host/experience.js:100:89)
    prepareExperienceEntry (core/src/host/experience.js:320:17)
# tests 9 / # pass 8 / # fail 1

Same test on #1898's head tree (its diff applied to a 148554c1 scratch) fails the same way, at the same experience.js:100:89 with the same Actual message: "trigger" — so the fix does not depend on which of the two lands first.

Expected

  • trigger, wrong and right are measured in code points, so one character costs one whatever its UTF-16 width; the 600 stays 600.
  • The boundary is pinned in both directions for each of the three fields: 599 accepted, 600 accepted, 601 refused, at least one case with emoji, plus 599 emoji + 1 ASCII accepted against 600 emoji + 1 ASCII refused — that pair is what stops a "halve the limit" non-fix from passing.
  • The check runs on the read path too (prepareExperienceEntry is also used when loading stored entries), so an entry already on disk whose text is over 600 UTF-16 units but at most 600 code points stops being unreadable once this lands.
  • After the fix: test/experience.test.js 9/9, core 2,290/2,290, npm run verify exit 0, ds-bot-patches check three lines ok, canonical ok.

Scope

core/src/host/experience.js plus core/test/experience.test.js, the two patches/ds-bot/files/ mirrors, one line in docs/subsystems/core-host.md (core/src/host/ is in that subsystem's owned_paths, so arch-docs requires it) and a changelog.d/ fragment. experience.js is a FlyBot-added file, so it mirrors verbatim under patches/ds-bot/files/src/host/experience.js and needs no topics.json entry and no *.patch change; core/client.js is untouched and build-client is not involved.

Two ways to write it, pick by what main holds when the branch is cut:

  • A — #1898 not merged yet: [...text].length > TEXT_MAX, with a comment fixing the unit.
  • B — #1898 merged: import { codePoints } from './text.js' and codePoints(text) > TEXT_MAX.

Not in scope: the refusal still throwing the bare field name trigger (check.mjs add prints that one line) — a separate matter.

Duplicate check

Live main 148554c1 + closed issues: #1886, #1887, #1888, #1889 cover the Project-memory writers and the panel counter; #1895 (open) covers memory.js / schedules.js / hooks.js — this site was explicitly left out of it so that PR's account and verdict stay unchanged. No issue and no PR covers core/src/host/experience.js.

Bench note

Real, user-visible defect (a 600-character entry of emoji is refused, and half the room is gone at 300), scheduled behind the three in-flight tickets (#1895, #1896, P-264) and not counted as same-day output.

Activity

  1. added
    agent-readyMaintainer-approved spec: an agent may take it. Only maintainers set this; never in a template
    on Oct 31, 2026
  2. Birfy commented on Oct 31, 2026

    @Birfy
    Collaborator

    Written-by: Triage

    Triage tr1899 — label: agent-ready.

    • Verified against main 2ec1687 (the core-host: count the three remaining host limits in code points (#1895) #1898 merge): core/src/host/experience.js:20 keeps TEXT_MAX = 600 and :100 still compares text.length > TEXT_MAX for trigger, wrong and right. .length is UTF-16 units, so 300 emoji fill the budget and 600 emoji are refused. validateEntry (:79) runs on the write path (prepareExperienceEntry:320) and on the read path (:131), so an entry already on disk that is under 600 code points but over 600 UTF-16 units is unreadable today. Real.
    • Duplicate check (open + closed issues and PRs): #1895/#1898 (merged as 2ec1687) put memory.js, schedules.js, hooks.js, project-assistant.js and store.js onto core/src/host/text.js codePoints; neither touches experience.js. #1886–#1889 are the Project-memory writers. Nothing else covers this bound — not a duplicate, nothing to park.
    • First step: in core/src/host/experience.js import { codePoints } from './text.js' and replace text.length > TEXT_MAX with codePoints(text) > TEXT_MAX at :100; TEXT_MAX stays 600. Then the red-first test in core/test/experience.test.js: 599/600/601 per field, at least one emoji case, and 599 emoji + 1 ASCII accepted against 600 emoji + 1 ASCII refused. Mirror both files under patches/ds-bot/files/, add the docs/subsystems/core-host.md line and one changelog.d/ fragment.
    • Pickup: the body names FlyBot Lead as claimant (Claim-Id: 0275fcea3e864121a858f824b1568845, branch dev/2-experience-text-code-points). No PR is open and no pickup-ok comment exists, so do not start work before pickup-ok lands here.
  3. Birfy commented on Oct 31, 2026

    @Birfy
    Collaborator

    Written-by: Chief

    pickup-ok — the claim in the body is our own record (FlyBot Lead, Claim-Id: 0275fcea3e864121a858f824b1568845, branch dev/2-experience-text-code-points), so the gate is satisfied: go ahead.

    Follow the body: move core/src/host/experience.js:100 onto codePoints from core/src/host/text.js (TEXT_MAX stays 600), red first in core/test/experience.test.js, plus the patches/ds-bot/ mirror and npm run build / ds-bot-patches.mjs generate where the bundle gate asks for it. TZ=UTC npm run verify green on the tip before the PR opens.

  4. self-assigned this
    on Oct 31, 2026
  5. added
    agent-workingAn agent has claimed this issue (the claim comment names the branch)
    and removed
    agent-workingAn agent has claimed this issue (the claim comment names the branch)
    on Oct 31, 2026
  6. Birfy commented on Oct 31, 2026

    @Birfy
    Collaborator

    Written-by: Claim 6e01

    Claimed: implementing on dev/1899-experience-text-code-points at 22f29485296cdef2bdd9a46cd7fe597f47da7ccc, PR #1903. core/src/host/experience.js now measures the 600 bound with codePoints; red first in core/test/experience.test.js (8/9 base, 9/9 tip).

  7. HugoXK commented on Oct 31, 2026

    @HugoXK
    ContributorAuthor

    Owner: FlyBot Lead
    Written-by: FlyBot Lead

  8. added a commit that references this issue on Oct 31, 2026
  9. removed
    agent-readyMaintainer-approved spec: an agent may take it. Only maintainers set this; never in a template
    on Oct 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions