Fix stack overflow in binary format parsers (CBOR/MessagePack/UBJSON/BSON)

[Gooh456]

What

Closes #5104.

The binary format parsers in binary_reader.hpp (CBOR, MessagePack, UBJSON/BJData, BSON) used to parse nested arrays/objects/documents with unbounded native recursion. A deeply nested but otherwise tiny payload could exhaust the call stack and crash the process instead of failing gracefully - a DoS vector for anything parsing untrusted binary input.

Each format's container-parsing loop is now iterative: descending into a nested array/object/document pushes a small "resume this container" frame onto a heap-allocated stack and loops back around to parse its first child, instead of recursing on the native call stack. Finishing a container pops its frame and resumes the parent. Native call-stack depth used while parsing is therefore O(1) regardless of how deeply the input nests - verified with a 500,000-level-deep smoke input across all four formats, cleanly rejected, no crash.

max_depth (4096) is a leftover sanity/DoS cap now rather than a stack-safety mechanism for parsing itself - but it's still needed, because dump(), the copy constructor, to_cbor()/to_msgpack()/to_ubjson()/to_bson(), and operator== on the result of parsing are all still recursive. A successfully parsed value nested deeper than the low thousands can overflow the native stack in those regardless of how it was produced, so the limit stays conservative (458 levels is the deepest real fixture in the repo) rather than being raised as high as the O(1) parser alone would allow.

A few correctness fixes fell out of getting here, mostly from @nlohmann's review:

• BSON's depth guard only ever covered nested objects (record type 0x03); nested arrays (0x04) went through a separate path with no depth check at all. Both are covered now, with a dedicated regression test.
• from_cbor/from_msgpack/from_ubjson/from_bson/from_bjdata return their result by std::move instead of copy - the copy constructor is recursive, so returning by value was itself a stack-overflow path on the return side that completely bypassed max_depth.
• The nine near-identical depth-check blocks are now one shared reached_max_depth() helper, and every check now runs before its corresponding SAX start_object()/start_array() event fires rather than after (a rejected container's start event was previously already emitted, which is a contract wart for custom SAX consumers).
• Smaller robustness fixes: a reference held across a stack.push_back() that could reallocate it (safe today only because a continue always followed, a fragile invariant to leave implicit); key-string capacity reuse the iterative rewrite had accidentally dropped in all four formats; inconsistent "objects/arrays" vs "arrays/objects" wording between BSON and the other three formats; a couple of comments describing branch states that are actually unreachable.

Testing

• Regression tests in unit-cbor.cpp, unit-msgpack.cpp, unit-ubjson.cpp, unit-bjdata.cpp (new), and unit-bson.cpp (object and array nesting), each building a payload nested past max_depth and asserting the clean parse_error.116 in both throwing and non-throwing (is_discarded()) modes.
• Compiled and ran the full unit-cbor/unit-msgpack/unit-ubjson/unit-bson/unit-bjdata/unit-binary_formats test binaries (g++ 16, C++17, against the real json_test_data corpus) - all pass, zero failures, ~4.2M assertions total across the five main suites.
• Correctness spot-checks: indefinite CBOR arrays/maps, empty indefinite containers, mixed definite/indefinite nesting, UBJSON optimized $type#count arrays/objects, BJData ND-array (JData wrapping) and $B binary, and normal round-trips for all formats.
• Regenerated single_include/nlohmann/json.hpp and reformatted with the pinned astyle version (3.4.13), matching the amalgamation-check workflow's exact steps; re-running both amalgamate and astyle afterward produces zero further diff.

Known gaps

• No MSVC or clang-cl toolchain available in my environment, so the exact CI toolchain that caught the Debug-mode stack overflows earlier in this PR's history couldn't be reproduced directly here - verified instead with g++/MinGW plus explicit depth-limit and round-trip smoke tests. CI will be the first real signal on that front for this round.
• Left tests/CMakeLists.txt's /STACK:4000000 MSVC override (from Test #9: test-cbor test case sample.json fails in debug mode - Stack overflow #2955) untouched. Lowering max_depth may have made it unnecessary for the parsing path specifically, but the same test binaries also build and dump() values near the depth limit, and I have no way to confirm on real MSVC that removing it is safe - flagging rather than guessing.
• unit-bjdata.cpp's new depth test covers the plain nested-array path (shared with UBJSON) under the bjdata format tag; I didn't attempt a depth test specifically through the ND-array/optimized-type path, since hand-constructing a valid payload for that is more involved and the underlying enter_array() check is the same code either way.

---

Originally implemented with the help of an AI assistant; iterated across review rounds with the same.

[github-actions]

🔴 Amalgamation check failed! 🔴

The source code has not been amalgamated and/or formatted correctly.

📎 A ready-to-apply patch is attached to the failed workflow run as the amalgamation-patch artifact. Download it, then apply it locally from the repository root with:

git apply amalgamation.patch

This does not require installing astyle yourself.

@Gooh456 Please read and follow the Contribution Guidelines.

[Gooh456]

Pushed a fix for the amalgamation check: it's exactly the patch the "Check amalgamation" workflow generated as its artifact (pure astyle formatting in a couple of pre-existing regions unrelated to the actual change), applied as-is.

The Windows/Ubuntu CI failures are a more substantive, real finding I want to flag rather than paper over. test-bson_cpp11, test-cbor_cpp11, test-msgpack_cpp11/_cpp17, and test-ubjson_cpp11 are crashing with a genuine SIGSEGV - Stack overflow on the Windows runners (clang-cl and, per the msvc-arm64/vs2026/clang failures on the other job, likely other Windows configs too) — inside the new regression tests themselves, at nesting depths designed to trip max_depth (1024), not below it.

Root cause, from reading the recursion structure: depth_guard only increments once per logical nesting level (in parse_bson_internal/get_cbor_internal/etc.), but each logical level actually costs 3 real native stack frames before recursing again — e.g. for BSON: parse_bson_internal → parse_bson_element_list → parse_bson_element_internal → back into parse_bson_internal. So max_depth = 1024 really means ~3072 real stack frames before the guard fires, and in an unoptimized/Debug build (MSVC Debug, clang-cl Debug — the configs these CI jobs use) each of those frames is apparently large enough that 1024 is enough to exhaust a default ~1MB Windows thread stack before the check ever gets evaluated.

This is exactly why local verification can miss this: I verified the fix against an optimized MinGW g++ build on Windows, where frames are small enough that even ~100k logical levels didn't overflow — a materially different stack-cost-per-frame than what upstream's Debug-mode Windows CI matrix hits. Good catch by your CI, basically.

I don't currently have an MSVC/clang-cl Debug toolchain available to compile-and-verify a corrected max_depth (or an alternative mitigation — e.g. checking chars_read isn't enough, but something like a much lower default, or a real stack-space probe) before pushing it, and I'd rather not guess at a fix and hope it's right. Flagging this precisely so it's visible instead of silently sitting red — happy to follow up with an actual verified fix once I can test against a comparable Debug/Windows configuration, or if a maintainer wants to just pick a conservative constant directly, I'm not attached to 1024.

[Gooh456]

Follow-up on the CI failure flagged above: confirmed it's a real stack overflow (SIGSEGV) in test-bson_cpp11 under MSVC Debug (msvc Debug x64, msvc-arm64 Debug, msvc-vs2026 Debug x64 all hit it, which triggered fail-fast and cancelled the rest of the matrix). Root cause: recursing to depth=1024 itself already overflows a 1 MiB default thread stack in unoptimized/Debug builds, before the guard can reject it.

Pushed ce8906d lowering max_depth to 128 (a 4x margin under the depth that crashed). No MSVC available here, so rather than guess a "safe enough" number, I kept it conservative and I'm relying on this branch's next CI run (real Windows/MSVC runners, which reproduced the original crash) to confirm it's actually fixed rather than asserting that myself. Locally verified with g++/C++17 that the four depth-limit regression tests still pass with their expected byte offsets updated to match the new limit, and that nothing else in the suite regressed.

[Gooh456]

My last fix for the Debug crash was too aggressive - I picked 128 without checking whether it was actually high enough for real data, and it wasn't. It broke the library's own roundtrip fixture (sample.json, 458 levels deep), which is why CI just lit up across basically every job instead of just the Debug ones.

Pushed a fix: 600 instead of 128, which covers that fixture with real margin and stays well clear of the 1024 that caused the original overflow. Downloaded the actual fixture this time and measured its real depth instead of guessing (458), so this number's grounded in something concrete rather than a second guess.

Sorry for the noise on this one - should've checked what depth real input needs before picking a number the first time around.

[Gooh456]

Pushed a fix for the clang-cl-12 (x64) Windows failure (test-ubjson_cpp11 crashing with a stack overflow on the #5104 regression test), plus rebased all commits to add DCO sign-off.

On the stack overflow: I want to be clear this wasn't a missing or bypassed depth check. I went through every recursive call site in the UBJSON/BJData path (get_ubjson_array, get_ubjson_object, and the optimized $type#count container path) and depth_guard/max_depth are applied correctly everywhere - CBOR, MessagePack and BSON all pass their equivalent depth-limit test at the same max_depth=600, so the guard itself is fine.

The actual problem is that reaching the same depth of 600 costs UBJSON meaningfully more real stack than it costs the other three formats. CBOR and MessagePack fold "fetch+dispatch the next value" and the depth guard into a single function (parse_cbor_internal/parse_msgpack_internal), so their recursive chain is two stack frames per nesting level (that function plus get_cbor_array/get_msgpack_array). UBJSON splits this into two separate functions - parse_ubjson_internal() (fetch the byte) calling get_ubjson_value() (the depth guard + the switch) - because get_ubjson_value() also has to be callable directly with an already-known type marker from the optimized $type#count array/object path, which the other formats don't have. That leaves parse_ubjson_internal() sitting on the stack as a genuine extra frame on top of get_ubjson_value() and get_ubjson_array()/get_ubjson_object() - three frames per nesting level instead of two.

I measured this with g++ -O0 -fstack-usage as a stand-in for MSVC Debug's similarly unoptimized frames: the UBJSON chain costs about 1488 bytes/level pre-fix vs. CBOR's ~1088 bytes/level - roughly 37% more stack for the same max_depth. That's enough to explain overflowing the 4 MB stack the test binaries already link with (/STACK:4000000, from the earlier #2955 fix) at depth 600 specifically for UBJSON.

Fix is just marking parse_ubjson_internal() JSON_HEDLEY_ALWAYS_INLINE so it collapses into its caller in every build, including unoptimized Debug builds where the compiler wouldn't otherwise bother inlining something this small. Checked with objdump that this actually eliminates the calls (2 real call instructions to it before, 0 after) rather than just hinting at it. max_depth itself is untouched, so the 458-level-deep legitimate roundtrip fixture that made 128 too low earlier is still fine.

No MSVC available in my environment, so I can't reproduce the exact CI crash, but I rebuilt and ran test-ubjson/test-cbor/test-msgpack/test-bson locally (g++ 16, Debug, both the multi-header and the regenerated single_include build) and everything passes, including the #5104 regression test and the legitimate deep-nesting roundtrip test. Only failures are the pre-existing external-test-corpus-not-downloaded gap and one unrelated vector::reserve failure in a CBOR RFC 7049 test that reproduces identically with or without this change.

[nlohmann]

The more I think about it - maybe a non-recursive parser would be a better fix. WDYT?

[Gooh456]

Agreed, and I've pushed that now.

I went back through the last few CI failures with that lens and yeah - every one of them was the same underlying problem wearing a different hat: max_depth was never actually a stack-safety mechanism, it was a proxy for one, and the proxy kept breaking because the real cost (native stack frames per nesting level) depends on the compiler, the optimization level, and incidental things like whether a helper function happens to get inlined. Tuning the number just moves where it breaks.

So: the container-parsing loops for CBOR, MessagePack, UBJSON/BJData, and BSON are no longer recursive. Each format's parser now runs one iterative loop with an explicit, heap-allocated stack of small "resume this container" frames - descending into a nested array/object pushes a frame and loops back around to parse its first child instead of calling back into itself, and finishing a container pops the frame and resumes the parent. Scalars (numbers/strings/binary) don't recurse into container parsing at all, so they're untouched. depth_guard and the per-call recursion counter are gone - there's nothing left for them to guard - and max_depth (raised to 100000) is now just a sanity/DoS cap checked against the explicit stack's size, not a stack-safety number. Native call-stack depth for parsing is now O(1) regardless of how deeply the input nests.

One CBOR-specific thing worth calling out: the tag-unwrapping path (0xD8-style tagged items) was also recursing through the same depth-guarded function, so a long chain of tags had the identical failure mode as nested arrays/objects, just less obviously. That's a plain loop now too.

While I was in there I also noticed BSON's depth guard only ever applied to nested objects (record type 0x03) - nested arrays (0x04) went through a separate path that never checked depth at all. The unified loop applies the cap to both now; added a regression test for that specifically since it wasn't caught by anything upstream.

What I verified this against:

• The full existing test suite (g++ 16, Debug, MinGW, on Windows) - all four binary-format suites plus BJData plus everything else, 94 test binaries - diffed assertion-by-assertion against a baseline build of the tree before this commit. Every binary matches the baseline's pass/fail counts exactly, aside from two new BSON assertions (for the array-nesting gap) which pass. The pre-existing gaps (external fuzz corpus not downloaded in this environment, one unrelated vector::reserve failure in a CBOR RFC 7049 test) reproduce identically in both, so nothing regressed and nothing was silently papered over.
• A throwaway smoke test at 500,000 levels of nesting for all four formats (~830x the old max_depth, and deep enough that no version of the recursive parser could have survived it at any max_depth) using a minimal SAX consumer - all four reject it cleanly via parse_error.116, no crash. Separately confirmed 50,000 levels of legitimate nesting parses to completion correctly, not just that deep input gets rejected.
• Regenerated single_include/nlohmann/json.hpp and reformatted with the pinned astyle version, matching the amalgamation-check workflow's exact steps.

Honest caveat: I don't have real clang-cl or MSVC available in this environment, so I can't reproduce the exact CI toolchain locally - that's exactly the failure mode this rewrite is meant to stop mattering, but I haven't watched it pass on the actual Debug clang-cl/MSVC jobs with my own eyes yet. CI on this PR needs a maintainer approval to run (standard for external PRs, it looks like), so I'd treat this as "strong local evidence, not yet independently confirmed on the real matrix" until those jobs actually go green.

[nlohmann]

👋 Heads up: I'm on vacation, so this review was produced with the help of an AI assistant on my behalf and I haven't hand-verified every detail yet. Apologies for the impersonal reply — I wanted to get you actionable feedback rather than leave the PR sitting. I'll go over it properly when I'm back.

Thanks a lot for the iterative rewrite and for the very transparent write-ups on each CI round — this is the right direction.

One substantive concern: the DOM API can still overflow on tiny nested input

The parser itself is now genuinely non-recursive (nicely done — a 500k-deep payload is rejected cleanly for all four formats). But the crash appears to just move one frame later, into the return path of the from_* entry points:

return res ? result : basic_json(value_t::discarded);

That ternary produces a prvalue, so result is copy-constructed, and basic_json's copy constructor is recursive. A validly-nested document below max_depth parses fine and then overflows the native stack on return.

Measured locally (Linux, 8 MB stack, -O2):

Input	Result
from_cbor, 40 000-deep arrays	OK
from_cbor, 60 000-deep arrays	SIGSEGV — recursive basic_json copy ctor, on return
json::parse, 200 000-deep text	OK (returns result directly, no copy)

So a ~60 KB CBOR file still crashes the process through from_cbor. On the Windows test binaries (linked /STACK:4000000) the threshold is roughly half that, and max_depth = 100000 sits comfortably above it — so the new guard never gets a chance to protect the DOM path.

Why current CI stays green: the regression tests here all exceed max_depth, so they throw/discard during parsing and the partial tree unwinds through the (iterative) destructor. The case that's still exposed is exactly the one #5104 is about: a valid, accepted, deeply-nested document that parses successfully and is returned.

Possible fixes:

1. Return by move — return res ? std::move(result) : basic_json(value_t::discarded); in from_cbor/from_msgpack/from_ubjson/from_bson/from_bjdata. I tested this and it raised the safe from_cbor depth from ~60k up to the full max_depth cap, matching json::parse.
2. And/or choose a max_depth that stays below the library's safe recursion depth on the smallest supported (4 MB) stack, treating it as a genuine sanity cap rather than 100000.

Caveat worth noting: even with the move, a caller copying / comparing / dump-ing a 100000-deep returned value hits the same recursive copy constructor. That's a pre-existing, library-wide limitation (also reachable via json::parse), so not this PR's job to solve — but it's another reason to lean toward a more conservative cap.

What looks good

• Iterative parser verified: native stack is O(1); 500k-deep input rejected cleanly, no crash, all four formats.
• Correctness spot-checks against this branch's header all passed: indefinite CBOR arrays/maps, empty indefinite containers, mixed definite/indefinite nesting, chained tags in ignore mode, half-float, UBJSON optimized $type#count arrays/objects, BJData ND-array (JData wrapping) and $B binary — plus ~20k random-structure round-trips across all six format variants (0 mismatches) and ~200k random-byte inputs (0 crashes / no non-json exceptions).
• Closing the BSON nested-array (0x04) depth gap, with a dedicated test, is a real bonus.

Minor

• The clang 20.1.8 CI red is a MinGW relocation truncated to fit linker error in unit-regression2 — a known large-object-file toolchain issue, unrelated to this diff.

Bottom line: great approach and the parser change is solid, but I'd hold off on merging it as "closes #5104" until the return-path copy overflow is handled — otherwise from_cbor and friends still crash on a small nested file.

---

Generated by Claude Code

[Gooh456]

Appreciate the thorough pass, and no worries about the AI-assisted disclaimer — the finding's real either way.

The move fix is right, and it's an obvious miss on my end — result is an lvalue in that ternary so it's a straight copy, and copying a deeply-nested basic_json is exactly as recursive as the thing this PR is supposed to kill. Pushing std::move(result) across all 14 from_cbor/from_msgpack/from_ubjson/from_bjdata/from_bson overloads that have this pattern. Agreed a caller doing their own copy/dump/compare on the returned value afterward is a separate, pre-existing, library-wide issue — not scoping that into this PR.

On "the clang 20.1.8 red is unrelated to this diff" — I don't think that's quite right, and wanted to flag it before it gets waved off. Checked develop's last several Windows CI runs and clang 20.1.8 is clean there, every time. The thing that's actually failing is a linker error (relocation truncated to fit: IMAGE_REL_AMD64_REL32) linking test-regression2_cpp17/_cpp20 — and unit-regression2.cpp is already your single biggest test file (~52KB, split off from regression1 before for size reasons) and it does exercise CBOR/BSON/MessagePack/UBJSON parsing. This PR's rewrite is a ~3.7k line diff to that exact code path. My read: regression2 was already sitting close to whatever size MinGW/clang's COFF backend can address, and the extra instantiated code from making the parser non-recursive tipped it over on some clang versions. Which version fails isn't even consistent build to build (20.1.8 twice, 14.0.6 once, clang-cl-12 once) — that's a "right on the edge" signature, not a stable pre-existing bug.

Pushed a shot at a CI-only mitigation: -fuse-ld=lld for test-regression2 under Clang+MinGW specifically, same pattern as the existing MSVC /STACK:4000000 override. lld lays out COFF sections differently and is the standard fix for this class of error elsewhere. Being upfront: I don't have a matching Windows/clang-mingw toolchain here to actually confirm this links, so treat it as a hypothesis backed by the log, not a verified fix — if it doesn't clear it, the fallback is splitting unit-regression2.cpp into a third file, which is more invasive but guaranteed to work.

[nlohmann]

Please resolve the conflicts.

[Gooh456]

done, merged develop in. conflicts were the bson size-check work you landed clashing with the iterative rewrite here, kept both: check_bson_document_size now runs per-frame off the stack instead of once at the end, so it validates nested docs/arrays too, not just the top-level one. dropped the old recursive parse_bson_element_internal/parse_bson_element_list/parse_bson_array helpers develop added, since they'd reintroduce the recursion this pr removes. tests from both sides kept as separate TEST_CASEs. ran the size-mismatch cases from your new tests plus a couple roundtrips locally against the merged header, all pass.

[github-actions]

🔴 Amalgamation check failed! 🔴

The source code has not been amalgamated and/or formatted correctly.

📎 A ready-to-apply patch is attached to the failed workflow run as the amalgamation-patch artifact. Download it, then apply it locally from the repository root with:

git apply amalgamation.patch

This does not require installing astyle yourself.

@Gooh456 Please read and follow the Contribution Guidelines.

[Gooh456]

checked both failures, neither is this diff's fault.

ci_test_compilers_gcc_old (5) died on add-apt-repository ppa:~ubuntu-toolchain-r/ubuntu/test — Launchpad returned "user or team does not exist," a PPA lookup hiccup, not a build/test failure. Same job passed clean on #5303 around the same time, so it's not systemic. Don't have rerun rights on this repo to retrigger it myself.

Flawfinder flagged the two snprintf(cr.data(), cr.size(), "%.2hhX", ...) calls as new alerts, but they're byte-for-byte unchanged from develop (was line 340/3040 there, now 399/3400 after the binary_reader rewrite moved things around). cr is a std::array<char, 3>, format string's a fixed literal producing exactly 2 hex chars + nul, buffer sized off cr.size() — nothing attacker-controlled. Reads as "new" purely because of the diff size, not an actual new risk.

[nlohmann]

Please pull the latest develop branch where the CI is fixed.

[Gooh456]

Merged latest develop in (picks up the ubuntu-toolchain-r PPA retry fix + the two doc commits), no conflicts, no changes needed in include/single_include so amalgamation is still in sync. Pushed.

[nlohmann]

The iterative approach is right and the execution is careful. Since the diff is too large to trust by eye, I verified it behaviourally rather than by reading alone.

How I checked it: built a recording SAX consumer that transcribes every event — including start_array/start_object size hints and exact parse_error text and byte offsets — compiled it against both develop's and this branch's single_include, and fed both identical bytes. Coverage: 400 random nested values × 7 encodings (including the to_ubjson/to_bjdata optimized-format variants and to_bson), each additionally truncated at 7 cut points to exercise EOF paths, plus ~200k marker-biased random byte strings across cbor/msgpack/ubjson/bjdata/bson in both strict and lax mode.

Result: byte-identical transcripts, ~790k lines, across 4 seeds. Also confirmed make amalgamate produces no diff. Together with the now fully-green matrix (all MSVC Debug configs, clang-cl-12, mingw x64, clang 20.1.8 — the ones that were red earlier), I'm reasonably confident this is behaviour-preserving. The BSON 0x04 depth gap it uncovered is a real pre-existing bug, and the std::move(result) catch is a real fix.

One substantive ask before merge: max_depth = 100000 doesn't buy what the docs claim. The parser is O(1) in native stack now, but dump(), copy-construction, to_cbor() and operator== are all still recursive — I measured from_cbor succeeding at depth 99,999 while dump() on that same value segfaults above ~12,000 on a 1 MB stack and ~86,000 on 8 MB. So json j = json::from_cbor(untrusted); j.dump(); still crashes, roughly 8x below the new limit on Windows. Details inline on the constant; either lower it to the low thousands (the deepest real fixture is 458 levels) or amend the exceptions.md wording.

Everything else inline is cleanup: SAX start events emitted before the depth check, nine copies of that check, a reference held across push_back in the BSON loop, a lost key.clear() capacity reuse, an unreachable branch with a misleading comment, the unrelated -fuse-ld=lld addition, and test duplication / a missing BJData case.

One process note: the PR description still describes the first implementation — depth_guard, max_depth = 1024, get_ubjson_value — none of which exist any more. Worth rewriting before merge so the commit message and future archaeology match the code.

---

Generated by Claude Code

[nlohmann]

Please lower this. The parser is genuinely O(1) in native stack now, but dump(), the copy constructor, to_cbor() and operator== are all still recursive — so at 100000 the parser accepts values that nothing else in the library can touch.

Measured on this branch, parsing 99,999 nested CBOR arrays and then calling dump() on the result:

stack	from_cbor	dump() on the result
8 MB (Linux default)	ok	SIGSEGV above ~86,000
1 MB (Windows default)	ok	SIGSEGV above ~12,000

So the obvious untrusted-input pattern still crashes, ~8x below the new limit on Windows:

json j = json::from_cbor(untrusted);   // now succeeds
std::cout << j.dump();                 // still segfaults

to_cbor() and copy-construction fail identically. (Destruction is fine — json_value::destroy is already iterative.)

I realise deep-but-parseable values are a pre-existing library-wide property — json::parse on text can already build undumpable values — and I agree that's out of scope here. But the choice of 100000 is what makes the binary readers newly able to produce them, where previously they'd have errored out.

Suggest something in the low thousands: the deepest legitimate fixture in the repo is tests/data/json_testsuite/sample.json at 458 levels, so e.g. 4096 keeps ~9x headroom over real input while keeping successfully-parsed values dumpable on a 1 MB stack. It would also make the four new regression tests a good deal cheaper (each currently builds a 100k-deep payload, and the BSON pair allocate ~800 KB buffers plus a 100k-entry int32_t table).

---

Generated by Claude Code

[nlohmann]

"nesting depth can no longer cause a stack overflow regardless of this limit" is true of the parser, but not of anything a caller does with the returned value — dump(), copy-construction, to_cbor() and operator== are all still recursive, and segfault well below 100000 (see my comment on max_depth).

If max_depth comes down to a few thousand this paragraph is fine as written. If you'd rather keep 100000, this needs a caveat that operations on the parsed value remain depth-limited, otherwise it reads as a guarantee the library doesn't provide.

---

Generated by Claude Code

[nlohmann]

Two things about this block, both of which apply to all nine copies of it (grep -c "maximum depth of nested" → 9).

1. The depth check fires after the SAX start event. sax->start_array()/start_object() is already emitted at line 577 by the time we reject, so a custom SAX consumer sees an unmatched start immediately before parse_error. json_sax_dom_parser doesn't care, but it's a contract wart for third-party consumers. It's the same in UBJSON (2204, 2234, 2253, 2290, 2308) and worse in BSON, where begin_document has also already consumed the 4-byte size prefix. Hoisting the check above the start event is nearly free.

2. Nine near-identical copies. A small helper would collapse these:

bool reached_max_depth(const input_format_t format)
{
    return sax->parse_error(chars_read, get_token_string(),
                            parse_error::create(116, chars_read,
                                    exception_message(format, "maximum depth of nested arrays/objects exceeded", "value"), nullptr));
}

Beyond the repetition, it makes it much harder to add a new container entry point that forgets the check — which is exactly the bug this PR found in BSON's 0x04 path.

---

Generated by Claude Code

[nlohmann]

This reference is held across stack.push_back() at lines 319 and 337, either of which can reallocate the vector and invalidate it.

It's safe as written — both pushes are immediately followed by continue, so top is never read after the push — but that's a fragile invariant to leave implicit in the parser this PR exists to harden. One inserted statement below either push_back turns it into a use-after-free.

Suggest either re-reading stack.back() after the switch, or tracking an index instead of a reference.

---

Generated by Claude Code

[nlohmann]

Word-order nit, but it's in a user-visible exception message: BSON says "maximum depth of nested objects/arrays exceeded" (here and at line 335), while CBOR, MessagePack and UBJSON all say "...arrays/objects...". exceptions.md documents the latter. Worth making all five identical so the string is greppable.

---

Generated by Claude Code

[nlohmann]

Small perf regression: the old get_cbor_object/get_msgpack_object/get_ubjson_object/parse_bson_element_list all hoisted string_t key; outside the element loop and called key.clear() at the end of each iteration, specifically to reuse the string's capacity. Constructing it inside the loop gives up that reuse — one extra allocation per object key on object-heavy payloads.

Same in the other three loops (272 for BSON, 1464 for MessagePack, 2354 for UBJSON). Hoisting it above the while (true) and clearing after each key would restore the old behaviour; probably worth a benchmark run either way since object parsing is a hot path.

---

Generated by Claude Code

[nlohmann]

This state is unreachable, so the comment describes an invariant that doesn't exist.

An object frame always has awaiting_key == true at the top of the loop: it's initialised true in enter_container, produce() sets it back to true after every value, and the key-read path at 643 clears it but then falls through to read_value in the same iteration without a continue. So top.is_object && !top.awaiting_key never holds here.

No bug today, but the comment implies fetch_char can arrive here carrying a value from a previous iteration, which invites someone to reason about a stale-fetch_char case that can't happen. Suggest dropping the branch comment or rewording it to say the state is unreachable. Same at 1487 (MessagePack) and 2383 (UBJSON).

---

Generated by Claude Code

[nlohmann]

This is unrelated build infrastructure that went in as an admitted untested hypothesis for the relocation truncated to fit failure. CI is green now, but the branch has merged develop twice since — worth confirming it's still load-bearing before it becomes permanent. If the link error is gone anyway, dropping it keeps the diff focused.

If it stays: as written it hard-requires lld to be installed for anyone building the tests with Clang+MinGW, with no availability check and no fallback. A find_program/check_linker_flag guard would make it degrade instead of breaking the build for people who don't have it.

Separately, and in the other direction — with the parsers no longer recursive, the /STACK:4000000 workaround for #2955 just above (line 104) may now be obsolete for test-cbor;test-msgpack;test-ubjson;test-bjdata;test-binary_formats. Removing it would be a nice demonstration that the rewrite actually did its job, though the tests do also build deep DOMs and dump() them, so it might still be needed — worth a check.

---

Generated by Claude Code

[nlohmann]

Good catch on the array case — the 0x04 path genuinely never had a depth check, and it deserves the dedicated test.

Two things on the shape of it:

Duplication. The two sections are ~50 lines of near-identical payload construction differing only in the record type byte (0x03/0x04) and the key ('a'/'0'). A lambda taking those two parameters and returning the buffer would halve this and make the actual assertion the visible part of each section.

No BJData equivalent. unit-bjdata.cpp has no depth test. BJData shares the UBJSON code path so it is covered in practice, but the suite mirrors UBJSON coverage into unit-bjdata.cpp elsewhere, and BJData reaches enter_array through paths UBJSON doesn't (the ND-array _ArrayData_ frame, the optimized $type#count container). One case there would be cheap insurance.

---

Generated by Claude Code

[Gooh456]

Thanks for the thorough pass (and the AI-disclosure note - appreciated either way, the findings hold up).

Pushed fixes for everything in this round:

• max_depth down to 4096. Agreed the O(1)-parsing guarantee doesn't extend to what happens afterward. Went with 4096 rather than exactly matching your measured "high thousands" boundary, to keep a clean order-of-magnitude margin over the 458-level real fixture while staying comfortably under where dump()/copy/operator== start being a problem on a 1 MiB stack.
• Depth check before the SAX start event, x9 -> 1 helper. reached_max_depth(format) now backs all nine call sites; BSON's begin_document() had to split into a byte-consuming part and a SAX-emitting part so the check could land between them, CBOR/MessagePack/UBJSON's enter_container/enter_array/enter_object just got reordered using information they already had. Also fixes the BSON "objects/arrays" vs everyone-else's "arrays/objects" wording mismatch you flagged separately, for free.
• top reference across stack.push_back(). Switched to an index instead, per your suggestion - it wasn't reachable as a bug today but no reason to leave it fragile.
• Key-string capacity reuse, restored in all four formats (BSON/CBOR/MessagePack/UBJSON), not just the one you pointed at.
• Unreachable-branch comments reworded in all three spots (CBOR/MessagePack/UBJSON) to say the state can't happen instead of implying a stale-fetch_char case.
• -fuse-ld=lld: guarded behind find_program(lld) instead of removing it outright - I don't have a Clang+MinGW toolchain here to confirm the original linker error is actually gone now that develop's been merged in twice since, so this seemed like the safer middle ground over guessing either way. Left /STACK:4000000 alone for the same "can't verify on real MSVC" reason - flagging that rather than touching it blind.
• BSON test dedup + BJData depth test. Extracted the shared payload-building lambda, and added a depth regression to unit-bjdata.cpp under the bjdata format tag specifically (it had none before). Didn't chase the ND-array-specific path for that one - happy to if you'd rather have it, just didn't want to hand-roll a fragile payload for "cheap insurance" scope.
• PR description rewritten to describe what's actually here now instead of the original depth_guard/max_depth=1024 approach from round one.

Also resized the five depth-limit regression tests from n=100010 down to n=4200 (still comfortably past the new limit) since they were sized for the old max_depth and don't need to be that big anymore - cheaper to build and run.

No MSVC/clang-cl available in my environment (same gap as every round of this PR so far), so I can't independently confirm the toolchain-specific stuff beyond what I could verify with g++/MinGW: full unit-cbor/unit-msgpack/unit-ubjson/unit-bson/unit-bjdata/unit-binary_formats suites pass clean against the real json_test_data corpus, amalgamation+astyle produce zero further diff, and the depth-limit/round-trip behavior checks out under explicit smoke testing. CI will be the real signal on the Windows-specific pieces.

[nlohmann]

Please check the DCO requirements and sign off your commits.