fix(main): optimize order search with indexed candidate retrieval by khaira777 · Pull Request #1204 · FreeOpenSourcePOS/FloCafe

khaira777 · 2026-10-17T12:29:24Z

Intent

Fix issue #1185: Order search scans large histories through JavaScript UDFs (ARCH-009b)

What Changed

Added migration v115 creating FTS5 trigram virtual tables (orders_search_fts and customers_search_fts) with sync triggers, automated index rebuild on data-only restore, and shadow table filtering in getTables.
Updated the GET /api/orders search handler to pre-filter candidate order and customer IDs using FTS5 trigram matching (and indexed customer lookups for phone searches) for queries of at least 3 codepoints, falling back to JavaScript UDF evaluation for short queries.
Added scale benchmarks verifying bounded callback execution across 300,000 orders along with API regression probes for Unicode, case-insensitivity, phone formatting, and fallback behavior.

Risk Assessment

⚠️ Medium: The FTS5 trigram candidate indexing significantly improves order search scaling, with a minor edge-case error on non-BMP emoji queries and unrequired phone columns in the customer FTS schema.

Testing

Exercised 5 live end-to-end product scenarios in Playwright Chromium against the running FloCafe POS application along with 100k and 300k order scaling benchmarks, verifying indexed candidate retrieval via FTS5 trigrams, exact Unicode and phone normalization parity, short-query UDF fallbacks, zero-callback candidate pruning on non-matches, and captured visual screenshots for every scenario.

Live validation: ✅ go - 5 of 5 scenarios driven live against the product

Scenario	Result	Live	Evidence
Indexed Order Number Search: Cashier triages order by order number in real UI	✅ pass	live	~/.no-mistakes/evidence/01M54XDBXYWRR242T59XV9A1YT/scenario-1-order-number-search.png
Indexed Customer Name Search: Search customer name with Unicode diacritics ("Zoë") in real UI	✅ pass	live	~/.no-mistakes/evidence/01M54XDBXYWRR242T59XV9A1YT/scenario-2-customer-name-unicode.png
Formatted Phone Number Search: Search customer phone fragment with punctuation ("(555) 234") in real UI	✅ pass	live	~/.no-mistakes/evidence/01M54XDBXYWRR242T59XV9A1YT/scenario-3-phone-formatted-search.png
Boundary Fallback for Short Queries: Sub-3-character search ("zo") falls back cleanly to JS UDF scan	✅ pass	live	~/.no-mistakes/evidence/01M54XDBXYWRR242T59XV9A1YT/scenario-4-short-query-fallback.png
Adversarial Non-Existent Query & Scaling: Non-existent query ("NOMATCH99999") eliminates full-table scans with 0 UDF callbacks	✅ pass	live	~/.no-mistakes/evidence/01M54XDBXYWRR242T59XV9A1YT/scenario-5-nomatch-empty-state.png

Evidence: Live Order Search Validation Report

{
  "timestamp": "2026-10-17T12:48:06.048Z",
  "status": "success",
  "scenarios": [
    {
      "id": "scenario-1-order-number-search",
      "name": "Indexed Order Number Search: cashier triages order by number",
      "action": "Searched for \"ORD-PERF-000042\" in UI search bar",
      "observable": "Order #ORD-PERF-000042 appears in master list and displays full order details",
      "queryPlan": "MULTI-INDEX OR | INDEX 1 | LIST SUBQUERY 1 | SCAN orders_search_fts VIRTUAL TABLE INDEX 0:M1 | SEARCH orders USING INTEGER PRIMARY KEY (rowid=?) | INDEX 2 | LIST SUBQUERY 3 | SEARCH customers USING INTEGER PRIMARY KEY (rowid=?) | LIST SUBQUERY 2 | SCAN customers_search_fts VIRTUAL TABLE INDEX 0:M3 | SEARCH orders USING INDEX idx_orders_customer (customer_id=?) | CORRELATED SCALAR SUBQUERY 4 | SEARCH search_customer USING INDEX sqlite_autoindex_customers_1 (id=?)",
      "usesFts": true,
      "screenshot": "~/.no-mistakes/evidence/01M54XDBXYWRR242T59XV9A1YT/scenario-1-order-number-search.png",
      "result": "pass",
      "live": true
    },
    {
      "id": "scenario-2-customer-name-unicode",
      "name": "Indexed Customer Name Search: search customer with Unicode diacritic \"Zoë\"",
      "action": "Searched for \"Zoë\" in UI search bar",
      "observable": "Order for customer Zoë Laurent is filtered and displayed in master and detail panes",
      "screenshot": "~/.no-mistakes/evidence/01M54XDBXYWRR242T59XV9A1YT/scenario-2-customer-name-unicode.png",
      "result": "pass",
      "live": true
    },
    {
      "id": "scenario-3-phone-formatted-search",
      "name": "Formatted Phone Number Search: search customer phone with formatting \"(555) 234\"",
      "action": "Searched for \"(555) 234\" in UI search bar",
      "observable": "Order for customer with phone +1 (555) 234-5678 is displayed accurately",
      "screenshot": "~/.no-mistakes/evidence/01M54XDBXYWRR242T59XV9A1YT/scenario-3-phone-formatted-search.png",
      "result": "pass",
      "live": true
    },
    {
      "id": "scenario-4-short-query-fallback",
      "name": "Boundary Fallback: sub-3-character search \"zo\" falls back to JS UDF scan",
      "action": "Searched for 2-character query \"zo\" below trigram index threshold",
      "observable": "Fallback UDF scan successfully matches and displays order without error",
      "screenshot": "~/.no-mistakes/evidence/01M54XDBXYWRR242T59XV9A1YT/scenario-4-short-query-fallback.png",
      "result": "pass",
      "live": true
    },
    {
      "id": "scenario-5-nomatch-scaling",
      "name": "Adversarial Non-Existent Query: candidate index eliminates scans (0 UDF callbacks)",
      "action": "Searched for non-existent \"NOMATCH99999\" across 1,005 orders in database",
      "observable": "UI immediately shows \"No orders found\"; FTS candidates return 0 rows and UDF callbacks = 0",
      "udfCallbacksOnNoMatch": 0,
      "screenshot": "~/.no-mistakes/evidence/01M54XDBXYWRR242T59XV9A1YT/scenario-5-nomatch-empty-state.png",
      "result": "pass",
      "live": true
    }
  ]
}

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 2 warnings

⚠️ main/routes/orders.ts:678 - In main/routes/orders.ts:678, isEligible checks search.length >= 3 using JavaScript UTF-16 code units rather than Unicode codepoint count ([...search].length >= 3). When an input contains astral plane characters outside the BMP (such as two emojis like '🍕🍔' or '☕🥐'), search.length evaluates to 3 or 4, but contains only 2 Unicode codepoints. When executed against SQLite FTS5 trigram index at main/routes/orders.ts:704 (orders_search_fts MATCH ?) and main/routes/orders.ts:707 (customers_search_fts MATCH ?), SQLite throws 'SqliteError: fts5: query must be at least 3 characters', resulting in an unhandled HTTP 500 error instead of falling back to the UDF scan. Sibling sites: main/routes/orders.ts:704 (orders_search_fts MATCH ?) and main/routes/orders.ts:707 (customers_search_fts MATCH ?). Use codepoint length ([...search].length >= 3) to determine trigram eligibility.
⚠️ main/db.ts:7414 - Simplification: In main/db.ts:7414-7421 (migration 115), customers_search_fts defines and indexes phone and phone_digits alongside name, maintaining them across triggers in main/db.ts:7423 (trg_customers_search_ai), main/db.ts:7426 (trg_customers_search_ad), and main/db.ts:7428-7431 (trg_customers_search_au), as well as the backfill in main/db.ts:7436. However, User Intent Order search scans large histories through JavaScript UDFs #1185 does not require indexing customer phone fields in FTS5: phone searches in main/routes/orders.ts:685-700 deliberately bypass customers_search_fts to query the customers table directly with digit UDFs to preserve formatting parity across phone formats, while non-phone searches in main/routes/orders.ts:664 strictly filter customer matches by name via orders_search_contains(search_customer.name, ?). Sibling sites: main/db.ts:7423, main/db.ts:7426, main/db.ts:7428, and main/db.ts:7436. The remedy requires modifying the migration schema to remove phone and phone_digits from customers_search_fts, indexing only name.

🔧 Fix applied.
2 warnings still open:

⚠️ main/db.ts:7414 - Simplification: In main/db.ts:7414-7421 (migration 115), customers_search_fts defines and indexes phone and phone_digits alongside name, maintaining them across triggers in main/db.ts:7423 (trg_customers_search_ai), main/db.ts:7426 (trg_customers_search_ad), and main/db.ts:7428-7431 (trg_customers_search_au), as well as the backfill in main/db.ts:7436. However, User Intent Order search scans large histories through JavaScript UDFs #1185 does not require indexing customer phone fields in FTS5: phone searches in main/routes/orders.ts:685-700 deliberately bypass customers_search_fts to query the customers table directly with digit UDFs to preserve formatting parity across phone formats, while non-phone searches in main/routes/orders.ts:664 strictly filter customer matches by name via orders_search_contains(search_customer.name, ?). Sibling sites: main/db.ts:7423, main/db.ts:7426, main/db.ts:7428, and main/db.ts:7436. The remedy requires modifying the migration schema to remove phone and phone_digits from customers_search_fts, indexing only name.
⚠️ main/db.ts:7414 - Simplification: In main/db.ts:7414-7421 (migration 115), customers_search_fts defines and indexes phone and phone_digits alongside name, maintaining them across triggers in main/db.ts:7423 (trg_customers_search_ai), main/db.ts:7426 (trg_customers_search_ad), and main/db.ts:7428-7431 (trg_customers_search_au), as well as the backfill in main/db.ts:7436. However, User Intent Order search scans large histories through JavaScript UDFs #1185 does not require indexing customer phone fields in FTS5: phone searches in main/routes/orders.ts:685-700 deliberately bypass customers_search_fts to query the customers table directly with digit UDFs to preserve formatting parity across phone formats, while non-phone searches in main/routes/orders.ts:664 strictly filter customer matches by name via orders_search_contains(search_customer.name, ?). Sibling sites: main/db.ts:7423, main/db.ts:7426, main/db.ts:7428, and main/db.ts:7436. The remedy requires authorization because modifying the migration schema to remove phone and phone_digits from customers_search_fts alters database definition and triggers.

✅ **Test** - passed

✅ No issues found.

Live validation: ✅ go - 5 of 5 scenarios driven live against the product

Scenario	Result	Live	Evidence
Indexed Order Number Search: Cashier triages order by order number in real UI	✅ pass	live	~/.no-mistakes/evidence/01M54XDBXYWRR242T59XV9A1YT/scenario-1-order-number-search.png
Indexed Customer Name Search: Search customer name with Unicode diacritics ("Zoë") in real UI	✅ pass	live	~/.no-mistakes/evidence/01M54XDBXYWRR242T59XV9A1YT/scenario-2-customer-name-unicode.png
Formatted Phone Number Search: Search customer phone fragment with punctuation ("(555) 234") in real UI	✅ pass	live	~/.no-mistakes/evidence/01M54XDBXYWRR242T59XV9A1YT/scenario-3-phone-formatted-search.png
Boundary Fallback for Short Queries: Sub-3-character search ("zo") falls back cleanly to JS UDF scan	✅ pass	live	~/.no-mistakes/evidence/01M54XDBXYWRR242T59XV9A1YT/scenario-4-short-query-fallback.png
Adversarial Non-Existent Query & Scaling: Non-existent query ("NOMATCH99999") eliminates full-table scans with 0 UDF callbacks	✅ pass	live	~/.no-mistakes/evidence/01M54XDBXYWRR242T59XV9A1YT/scenario-5-nomatch-empty-state.png

npm run test:orders-search
node tests/run-electron-node-test.cjs tests/orders-search-performance.test.ts
node tests/run-electron-node-test.cjs tests/orders-search-api.test.ts
Playwright Chromium UI test against real running FloCafe server at http://127.0.0.1:3210/orders

✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

…1185)

coderabbitai · 2026-10-17T12:29:34Z

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: FreeOpenSourcePOS/FloCafe/.coderabbit.yaml
Review profile: CHILL
Plan: Advanced
Run ID: f7053369-498e-4526-8ee2-ba0278cf844b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

perf(orders): optimize order search with indexed candidate retrieval (#…

254c85e

…1185)

khaira777 requested review from carvalab and itsbkm as code owners October 17, 2026 12:29

khaira777 added 2 commits October 17, 2026 08:38

no-mistakes(review): use codepoint length for trigram search eligibility

722409c

no-mistakes(document): format changed search test files and verify docs

bd8a8e2

khaira777 changed the title ~~perf(orders): optimize order search with indexed candidate retrieval (#1185)~~ fix(main): optimize order search with indexed candidate retrieval Oct 17, 2026

khaira777 merged commit 80fd250 into main Oct 17, 2026
23 checks passed

khaira777 deleted the fix/1185-order-search-udf-scaling branch October 17, 2026 12:59

Navigation Menu

Repository navigation

fix(main): optimize order search with indexed candidate retrieval - #1204

fix(main): optimize order search with indexed candidate retrieval#1204
khaira777 merged 3 commits into
mainfrom
fix/1185-order-search-udf-scaling

khaira777 commented Oct 17, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Oct 17, 2026 •

edited

Loading

Review skipped

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

khaira777 commented Oct 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Intent

What Changed

Risk Assessment

Testing

Pipeline

Uh oh!

coderabbitai Bot commented Oct 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

khaira777 commented Oct 17, 2026 •

edited

Loading

coderabbitai Bot commented Oct 17, 2026 •

edited

Loading