TiFlash: read only StringSizes for LENGTH / CHAR_LENGTH scans

[JaySon-Huang]

Feature Request

Is your feature request related to a problem? Please describe:

When a query needs only string length (LENGTH / CHAR_LENGTH) and not the payload, TiFlash still reads both the chars and StringSizes streams from DMFile.

A typical case is a full-table MAX(CHAR_LENGTH(col)):

SELECT
  MAX(CHAR_LENGTH(c1)) AS c1,
  MAX(CHAR_LENGTH(c2)) AS c2,
  -- ... a dozen or more string columns
  MAX(CHAR_LENGTH(c16)) AS c16
FROM t;

In one observation, TableScan outbound was ~66 GB of strings per node and Projection outbound ~38 GB of Int64. At the compute layer, LENGTH already uses offsets and ignores data; CHAR_LENGTH (utf8mb4) must scan payload to count code points. The IO layer has no “this column needs length only” signal, so TableScan still reads the fat chars stream and saturates cloud-disk bandwidth.

This is not limited to MAX(): any projection or aggregation that depends only on length, not on string contents, over-reads payload.

This request is a follow-up of #11049.

Describe the feature you'd like:

Push “length only, no payload” down to the DMFile reader, and choose substreams from the function and pack contents.

String columns in DMFile are already split into:

• StringSizes: per-row length (~4 B/row)
• chars: payload
• nullmap (if any)

Expected behavior:

1. LENGTH(col)
   Read only StringSizes and the nullmap; skip chars. Byte length comes directly from sizes.

2. CHAR_LENGTH(col) (utf8mb4)
   Still read chars by default to count code points. If a pack is proven all-ASCII (e.g. pack-level all_ascii: no high-bit bytes), then CHAR_LENGTH = LENGTH and that pack can also read sizes only.

3. Push-down interface
   Scan / Projection should declare a per-column read mode, e.g. NeedPayload / NeedByteLength / NeedCharLength. MAX(LENGTH(col)) / MAX(CHAR_LENGTH(col)) in HashAgg should pass that through to TableScan, instead of materializing full string columns and dropping payload in compute.

Expected effect: in the observation above, payload dominates outbound; sizes are ~4 B/row. Reading sizes only can drop disk IO for those columns by an order of magnitude. This is general: LENGTH(col), SUM(LENGTH(col)), MAX(LENGTH(col)), and similar queries all benefit.

Implementation notes:

• CHAR_LENGTH on non-ASCII packs must fall back to reading chars, matching utf8mb4 semantics.
• If all_ascii (or equivalent) is used, store it in pack metadata; missing flag on old files is unknown → fall back to chars.
• Nullable columns still need the nullmap; empty-string vs NULL length semantics stay unchanged.
• Complementary to pack-level max_len skip: skip packs that cannot refresh MAX first, then sizes-only on the rest. This request does not depend on skip; landing it alone still helps full-table LENGTH scans.

Describe alternatives you've considered:

1. Pack-level max_len skip
   For MAX(CHAR_LENGTH) / MAX(LENGTH), prune with pack upper bounds and possibly skip entire packs. It does not help queries that must see every row’s length (SUM(LENGTH(col)), SELECT LENGTH(col)). Combined with this request it pays off most, but it does not replace sizes-only.

2. Rewriting CHAR_LENGTH to LENGTH in SQL
   Close on ASCII data, but TableScan still reads chars today, so only CPU is saved. Without this request, changing SQL does not cut IO.

3. Hidden / STORED generated column of per-row length
   Broader coverage as a regular integer column, but more write amplification and schema cost. TiDB cannot add STORED generated columns via ALTER TABLE. The sizes stream already stores per-row byte length; prefer reading existing StringSizes rather than storing another column.

4. Folding CHAR_LENGTH into HashAgg only
   Reduces Int64 intermediates, not chars reads. Wall time barely changes when IO-bound.

Teachability, Documentation, Adoption, Migration Strategy:

• Users: no SQL change. Queries that use only LENGTH / CHAR_LENGTH and never the payload should automatically read less chars on TiFlash.
• When it applies: a read-path optimization; existing DMFiles need not be rewritten. Pack flags such as all_ascii apply only to files written after the change (or after compaction); old files fall back to reading chars; results stay correct.
• Observability: distinguish sizes vs chars bytes read in TableScan / tiflash_scan, and count packs that skipped chars due to ASCII.
• Compatibility: the reader must fall back safely without all_ascii; it must not treat CHAR_LENGTH of non-ASCII data as byte length.
• Docs: note in TiFlash scan docs that length functions prefer the string sizes stream; utf8mb4 CHAR_LENGTH still reads payload when ASCII cannot be proven.