docs: document standards compliance and parse() vs operator>> strictness

[youdie006]

This documents the library's RFC 8259 / JSONTestSuite compliance and the parse() vs operator>> strictness difference, so the (good) compliance story is discoverable and citable instead of living only in tests/src/unit-testsuites.cpp.

What

• README.md -> new "Standards compliance" note under "Notes": both nst JSONTestSuite revisions run in CI; all mandatory y_/n_ cases pass through the strict parse() entry point; the deliberate implementation-defined i_ choices are listed (unbounded nesting, silent BOM stripping, noncharacters forwarded, strict rejection of invalid UTF-8 and lone surrogates, out_of_range.406 on numeric overflow); and the parse() vs operator>> nuance is called out.
• docs/mkdocs/docs/features/parsing/index.md -> new "Strictness and trailing data" section documenting that parse() is strict and rejects trailing data, while operator>> follows relaxed iostream semantics (parses one value and leaves the stream positioned after it).

Why

Drive-by comparisons keep claiming the library "does not fully pass JSONTestSuite", usually by feeding "valid document + trailing garbage" through operator>> (which accepts it) rather than parse() (which rejects it). Making the compliance story and the two-API strictness difference discoverable addresses that.

Documentation only; no parser behavior change. The i_ choices and the strictness behavior were verified against the source before writing: out_of_range.406 (docs/mkdocs/docs/home/exceptions.md), BOM handling (api/basic_json/parse.md), noncharacters/surrogates (README "Character encoding"), and the operator>> "parses one value and leaves the stream positioned after it" note (api/operator_gtgt.md).

Verified locally with mkdocs build: the docs site builds cleanly and the new parsing section and its links render without new warnings.

Closes #5290.

Disclosure: prepared with AI assistance; I reviewed the change and verified every claim against the source files cited above.

---

• The changes are described in detail, both the what and why.
• An existing issue is referenced.
• Code coverage - N/A (documentation only; no code changed).
• The documentation is updated.
• make amalgamate - N/A (no source code change).

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b65b09d9f2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Do not describe downloaded test data as vendored

In a source checkout or offline build, these suites are not vendored in this repository. A repo-wide file search finds no nst_json_testsuite data, while cmake/download_test_data.cmake downloads the external json_test_data v3.1.0 archive unless JSON_TestDataDirectory is supplied. This statement contradicts the later test instructions and may lead contributors to expect the conformance tests to run without separately obtaining test data.

Useful? React with 👍 / 👎.

[nlohmann]

cmake/download_test_data.cmake downloads the test suite data from a separate repo (nlohmann/json_test_data) at configure time; it isn't committed/bundled in this repo. "Vendored" implies the latter.

[chatgpt-codex-connector]

Avoid claiming every suite case uses strict parse

The referenced test does not run all mandatory cases through the strict parse() entry point: the original suite's y loop ends with CHECK_NOTHROW(f >> j) at tests/src/unit-testsuites.cpp:583, and cases such as y_string_utf16.json and n_number_then_00.json are commented out. Either exercise every advertised case with parse() or narrow this conformance claim so it accurately describes the coverage.

Useful? React with 👍 / 👎.

[nlohmann]

The original JSONTestSuite's y (must-accept) section in tests/src/unit-testsuites.cpp:583 actually uses operator>>, not parse(). Only the n section and the second/updated suite's y+n sections use strict parse(). y_string_utf16.json is also commented out and not exercised at all.

[nlohmann]

The claim is true for nst_json_testsuite2, which uses json::parse(f) for both y (:1010) and n (:1212).

[chatgpt-codex-connector]

Qualify stream positioning after numeric values

This positioning guarantee is false when the extracted value is a number: the lexer reads one byte past the number, and its internal simulated unget() does not restore that byte to the underlying stream. With the current implementation, extracting from 1true returns 1 but leaves the stream at r, so the next extraction fails rather than reading true; repeated extraction only works in this case when a separator such as whitespace can be consumed. The documentation should not promise arbitrary concatenated values or exact positioning.

Useful? React with 👍 / 👎.

[gregmarr]

This description is correct.

[nlohmann]

Indeed. the lexer's unget() is simulated (lexer.hpp:1437 — "The input is not changed"), and input_stream_adapter::get_character() uses sbumpc() with no sungetc() (input_adapters.hpp:127). So the one byte that terminates a number is eaten. Literals and structural values self-terminate, so they're unaffected.

Please add one caveat sentence (also to operator_gtgt.md), e.g. "A number is only terminated by the character following it, which operator>> consumes; concatenated numbers must therefore be whitespace-separated (1 2 works, 1true does not)."

[nlohmann]

The AI-generated comments are not helpful btw. I will review this by the end of the week.

[nlohmann]

cmake/download_test_data.cmake downloads the test suite data from a separate repo (nlohmann/json_test_data) at configure time; it isn't committed/bundled in this repo. "Vendored" implies the latter.

[nlohmann]

The original JSONTestSuite's y (must-accept) section in tests/src/unit-testsuites.cpp:583 actually uses operator>>, not parse(). Only the n section and the second/updated suite's y+n sections use strict parse(). y_string_utf16.json is also commented out and not exercised at all.

[nlohmann]

The claim is true for nst_json_testsuite2, which uses json::parse(f) for both y (:1010) and n (:1212).

[nlohmann]

Indeed. the lexer's unget() is simulated (lexer.hpp:1437 — "The input is not changed"), and input_stream_adapter::get_character() uses sbumpc() with no sungetc() (input_adapters.hpp:127). So the one byte that terminates a number is eaten. Literals and structural values self-terminate, so they're unaffected.

Please add one caveat sentence (also to operator_gtgt.md), e.g. "A number is only terminated by the character following it, which operator>> consumes; concatenated numbers must therefore be whitespace-separated (1 2 works, 1true does not)."