Skip to content

Release 2.6.0 2026-09-25 - #62

Merged
xexyl merged 7 commits into
xexyl:masterfrom
lcn2:jparse-audit
Sep 26, 2026
Merged

xexyl merged 7 commits into
xexyl:masterfrom
lcn2:jparse-audit

Conversation

@lcn2

@lcn2 lcn2 commented Sep 26, 2026

Copy link
Copy Markdown
Contributor

Performed a major code audit.

Harden parser allocation and length handling across JSON decode and scanner entry points

This PR audits the parser and supporting utilities for high-confidence
memory-safety and logic issues, with emphasis on allocation sizing, decode
paths, scanner length boundaries, and exact-length buffer handling. The
changes target overflow/OOB conditions in public and internal APIs while
preserving existing parser behavior for valid inputs.

  • JSON decode and encode bounds

    • Added checked size_t helpers for additive/multiplicative allocation math.
    • Guarded encoded/decoded length accumulation before allocation and before buffer growth assumptions.
    • Hardened decode buffer allocation against wrapped mlen + 2 sizing.
  • Malformed \u escape handling

    • Changed utf8len() to return a signed length so malformed escapes propagate failure instead of wrapping to SIZE_MAX.
    • Updated JSON decode paths to reject invalid UTF-8 escape length results before folding them into decoded-size calculations.
  • Scanner length boundary

    • Added an explicit INT_MAX guard before passing caller-controlled lengths into flex yy_scan_bytes().
    • Oversized inputs now fail safely instead of truncating through the scanner’s signed-length boundary.
  • Exact-length API correctness

    • Fixed json_conv_bool() and json_conv_null() to copy exactly len bytes and terminate locally.
    • This removes the implicit read of len + 1 bytes from caller-provided slices that are not required to be NUL-terminated.
  • Overflow hardening in utility paths

    • Applied checked allocation sizing to helper paths that duplicate or build derived strings
      (jstrencode, jstrdecode, jsemtblgen, verge).
    • Synced the scanner source change into the checked-in flex reference files used by non-regenerated builds.
  • Focused regression coverage

    • Added a small API-level regression test binary covering:
      • malformed Unicode escapes
      • exact-length boolean/null slices
      • oversized parse_json() length rejection

Example of the exact-length fix:

    item->as_str = calloc(alloc_len, sizeof(*(item->as_str)));
    memcpy(item->as_str, ptr, len);
    item->as_str[len] = '\0';

In theory, the normal execution of this code should be identical to
"Release 2.5.11 2026-08-03", except in extreme or unusual cases.

Updated JPARSE_REPO_VERSION to "2.6.0 2026-09-25".
Updated JPARSE_TOOL_VERSION to "2.1.0 2026-09-25".
Updated JPARSE_LIBRARY_VERSION to "2.5.0 2026-09-25".
Updated JPARSE_UTILS_VERSION to "2.2.0 2026-06-25".

Copilot AI and others added 7 commits September 26, 2026 01:37
Co-authored-by: lcn2 <10429707+lcn2@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…canner entry points

This PR audits the parser and supporting utilities for high-confidence
memory-safety and logic issues, with emphasis on allocation sizing, decode
paths, scanner length boundaries, and exact-length buffer handling. The
changes target overflow/OOB conditions in public and internal APIs while
preserving existing parser behavior for valid inputs.

- **JSON decode and encode bounds**
  - Added checked `size_t` helpers for additive/multiplicative allocation math.
  - Guarded encoded/decoded length accumulation before allocation and before buffer growth assumptions.
  - Hardened decode buffer allocation against wrapped `mlen + 2` sizing.

- **Malformed `\u` escape handling**
  - Changed `utf8len()` to return a signed length so malformed escapes propagate failure instead of wrapping to `SIZE_MAX`.
  - Updated JSON decode paths to reject invalid UTF-8 escape length results before folding them into decoded-size calculations.

- **Scanner length boundary**
  - Added an explicit `INT_MAX` guard before passing caller-controlled lengths into flex `yy_scan_bytes()`.
  - Oversized inputs now fail safely instead of truncating through the scanner’s signed-length boundary.

- **Exact-length API correctness**
  - Fixed `json_conv_bool()` and `json_conv_null()` to copy exactly `len` bytes and terminate locally.
  - This removes the implicit read of `len + 1` bytes from caller-provided slices that are not required to be NUL-terminated.

- **Overflow hardening in utility paths**
  - Applied checked allocation sizing to helper paths that duplicate or build derived strings
    (`jstrencode`, `jstrdecode`, `jsemtblgen`, `verge`).
  - Synced the scanner source change into the checked-in flex reference files used by non-regenerated builds.

- **Focused regression coverage**
  - Added a small API-level regression test binary covering:
    - malformed Unicode escapes
    - exact-length boolean/null slices
    - oversized `parse_json()` length rejection

Example of the exact-length fix:

```c
item->as_str = calloc(alloc_len, sizeof(*(item->as_str)));
memcpy(item->as_str, ptr, len);
item->as_str[len] = '\0';
```
Copilot AI lite review requested due to automatic review settings September 26, 2026 02:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The checked-in parser reference artifact is out of sync, and several shipped component version identifiers remain stale.

Review effort: Lite
Findings: 1 Medium severity · 1 Low severity

Open (2)
What changed in this PR

Release 2.6.0 hardens parser allocation, Unicode decoding, scanner length handling, and exact-length APIs, with focused regression tests.

Changes:

  • Adds checked allocation arithmetic and malformed Unicode handling.
  • Enforces scanner length limits and safe exact-length copying.
  • Adds API regression tests and updates release metadata.
File Reviewed change
version.h Updates core release versions
verge.c Hardens version-array allocation
util.h Adds overflow helpers
test_jparse/​Makefile Builds and runs API tests
test_jparse/​json_api_test.c Adds regression coverage
jstrencode.c Checks duplicate-buffer sizing
jstrdecode.c Checks duplicate-buffer sizing
json_utf8.h Changes UTF-8 length API
json_utf8.c Propagates malformed escape failures
json_parse.c Hardens encoding, decoding, and conversions
jsemtblgen.c Checks generated-name allocation sizing
jparse.y Updates parser error sequencing
jparse.ref.c Adds scanner length validation
jparse.l Adds scanner length validation
CHANGES.md Documents release changes
.gitignore Ignores the new test binary

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread jparse.y
*/
if (scanner == NULL) {
err(61, __func__, "NULL scanner");
err(62, __func__, "NULL scanner");
Comment thread json_utf8.h
#define JPARSE_UTF8_VERSION "2.1.4 2026-06-13" /* format: major.minor YYYY-MM-DD */

extern size_t utf8len(const char *str, int32_t surrogate);
extern ssize_t utf8len(const char *str, int32_t surrogate);
@xexyl
xexyl merged commit 16b2340 into xexyl:master Sep 26, 2026
3 checks passed
@xexyl

xexyl commented Sep 26, 2026

Copy link
Copy Markdown
Owner

Well done @lcn2. Thanks! I do not know what Copilot is talking about and I am afraid that I can't check now but I am much more confident in you than it.

If however you seem to think it has any merit please do address it.

I cannot understand why it thinks versions are 'stale' though so unless I read it wrong (entirely possible atm) I can't fathom that objection.

@xexyl

xexyl commented Sep 27, 2026

Copy link
Copy Markdown
Owner

I now know what Copilot was whining about. I fixed it. If you would please, @lcn2, sync it to mkiocccentry that would be helpful - since you've been making a lot of changes (and I have other things I wish to do too).

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants