Initial. - #2
Conversation
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
|
Thank you for this — genuinely. I want to start by acknowledging the amount of work that went into it, and the care That said, I have to decline this, and I'm sorry I'm only saying so after you've The reason is scope, not execution. NeverC is deliberately a C-only compiler and I If you're especially interested in C++ tooling for this kind of work, I'd recommend One request for next time, and I'd genuinely welcome it: please open an issue first And if the underlying need is compiling C++ POCs alongside NeverC-built C, I'd be glad Thanks again for taking the project seriously enough to do this. |
C++20 language support (NeverC ABI v1, no streams)
Summary
This pull request brings ISO C++20 language support to NeverC (latest released C++ standard at the time of this work), excluding iostream / stream facilities by deliberate product policy. The implementation spans frontend (parse / Sema / AST), NeverC-only C++ ABI v1 codegen and layout, a bundled
neverc_cxx_runtime, and a non-stream standard library surface. With the remaining ABI/protocol depth items in this branch completed, C++20-minus-streams is feature-complete for NeverC’s stated scope.Verification (full tree build and conformance tests) is intended to follow merge readiness of this surface; the conformance matrix lives at
docs/roadmap/cpp20-conformance-matrix.md.On NeverC
NeverC has been an unusually good fit for the kind of work many of us already do in security research and systems tooling. The project’s design choices are coherent: C23 as a clear default, an integrated linker, cross-compilation that does not require a scavenger hunt for SDKs, DynCode as a first-class pipeline, and built-in facilities (
string,mimalloc,xorstr,strhash) that show up in real workflows rather than only in marketing bullets. The plugin API and phase model make the compiler feel extensible without demanding that every experiment become a fork of LLVM.We have been using NeverC as a day-to-day compiler for research prototypes, freestanding and hosted targets, and dyncode-oriented builds. The experience is practical: one toolchain, predictable C semantics when you want them, and enough infrastructure that “compile this for the other OS” is not a separate project. Adding C++20 on that base is not a repudiation of NeverC’s C-first identity. It is an acknowledgment that the same people who benefit from NeverC’s C and DynCode story often also need to speak C++ fluently in small, deliberate programs—especially when the artifact under study was written in C++.
This PR is written in that spirit: extend NeverC where the research workflow demands it, keep the ABI and library policy under NeverC’s control, and refuse to drag in the parts of the C++ ecosystem that fight the project’s goals.
Motivation: C++ features for reverse engineering and proof-of-concept work
Heavy C++ applications dominate large parts of the software we reverse: game clients and anti-cheat stacks, browsers and renderers, industrial control and CAD, finance and trading systems, endpoint agents, and a long tail of proprietary services. The binaries are not “C with a class keyword.” They are shaped by constructors and destructors, virtual dispatch, template-generated specializations, RTTI-assisted paths, exception edges, lambdas and
std::function-like erasure, coroutines in newer codebases, and module or unity-build packaging. Understanding those mechanisms in IDA, Binary Ninja, Ghidra, or a custom lifter is necessary but not sufficient. At some point the reverse engineer needs to re-express behavior—to build a POC, a harness, a shim, a fuzzer stub, or a minimal reimplementation of a protocol object graph.That is where a C-only toolchain becomes a tax.
Matching the mental model of the target
When a target type is a polymorphic hierarchy with a non-trivial constructor order, a POC written as raw vtable pointers and manual
thisadjustment is possible, but it is fragile and hard to review. The same POC written with classes, bases, virtual methods, and ctor-initializer lists is closer to how the original authors (and the decompiler’s better days) think about the object. The point is not aesthetic preference. The point is reducing translation error between “what the binary does” and “what the harness does.” Every manual lowering of a C++ feature into C is a chance to get layout, lifetime, or dispatch wrong under time pressure.Layout, ABI, and “close enough to call”
Research POCs often need to:
dynamic_castgates, or type-info comparisons.operator new, exception personality stubs, guard variables).A compiler that can emit a documented NeverC C++ ABI v1—vtable layout, RTTI symbols, mangling, delete/dtor pairing—gives POCs a stable substrate. Full system-libstdc++ / libc++ / MSVC ABI compatibility is intentionally out of scope here; for many RE workflows that is acceptable or even preferable. What matters is repeatable object and call semantics under one toolchain, not dropping a research binary into a distro’s C++ runtime lottery.
Templates, concepts, and the code you wish you could type
Target code is full of templates. POCs do not need every specialization the vendor shipped, but they do need generic glue: thin wrappers, type traits,
optional-shaped holders, span-like views, and constrained helpers so that harness code stays short. C++20 concepts and requires-clauses, even at research depth, make it easier to write POCs that fail early when a reconstructed type does not satisfy an assumed interface. That is debugging time saved when the alternative is a wrong overload silently selected in a sea ofvoid*.Lambdas, range-for, and the speed of writing a harness
Proof-of-concept quality is often limited by how fast a correct experiment can be written. Lambdas with captures, range-based for,
auto, and constexpr evaluation are not about “writing applications.” They are about keeping the harness smaller than the insight. When the interesting part is a packet parser or a state machine recovered from a binary, the compiler should not force the harness into pre-C++11 shape. NeverC already optimizes for LLM- and human-friendly C; C++20 support extends that ergonomics to the subset of problems where C++ is the native language of the target.Exceptions, noexcept, and real control flow
Many C++ codebases use exceptions at API boundaries even when the hot path is noexcept-ish. A POC that must trigger or survive catch blocks needs try/catch, throw, and a personality story. Conversely, a POC that must match a noexcept interface needs the declaration surface. Leaving exceptions entirely unimplemented forces reverse engineers back into ad hoc longjmp simulations that do not match the binary’s landing pads. Supporting the language feature set makes experimental code honest about control flow.
Coroutines and modern targets
Newer targets use C++20 coroutines for async I/O, job systems, and RPC. A research compiler that can parse and lower
co_await/co_return/co_yieldwith a small runtime (__neverc_coro_*) lets POCs explore those control-flow graphs without first rewriting them as explicit state machines—useful when the goal is to validate understanding of the original, not to produce a shipping async stack.Modules and BMI as a research boundary
Modules are part of C++20 whether or not every build system has caught up. A NeverC BMI scaffold (export blob, import read path) keeps the door open for isolated interface units in larger research codebases without requiring the full PCMity of Clang’s ecosystem. For RE-oriented monorepos of harnesses and reconstructed headers, that is a structural feature, not a checkbox.
Why this belongs in NeverC specifically
Putting C++20 into NeverC—rather than telling researchers to keep a second full Clang/MSVC install solely for POCs—preserves the project’s advantages: one driver, one linker story, one cross-compile path, DynCode adjacency, and plugin hooks. The C++ support is NeverC-shaped: own ABI, own runtime, own policy on the standard library. That is the correct trade for security research. We are not trying to replace a platform compiler for shipping applications. We are trying to make NeverC the place where C23, dyncode, and C++ POCs can live in one mental and physical toolchain.
What this PR delivers
Language and frontend
-std=c++20/gnu++20/ related aliases) as the target edition; C++23 is out of scope.this, constructors / destructors / methods, operator function-ids, inheritance (including virtual bases at layout level).CXXCtorInitializer, member/base builders), and ctor prologue emission for member initialization after vptr setup.&&/||/!), concept-id soft satisfaction paths.begin/endwith call initializers; ADL-lite associated-namespace lookup.co_await/co_yield/co_return(CoreturnExpr) with runtime resume helpers.ABI, codegen, and runtime
[offset-to-top, RTTI, vfunc…], mangled_ZTV+ length + name._ZTT) for classes with virtual bases.neverc_cxx_runtime(runtime/cxx): new/delete, cxa_*, RTTI, guards, coroutine helpers, pure virtual, etc.Library policy
type_traits,concepts,span,ranges,optional,memory, containers, concurrency headers, C library bridges, and related surface).NoStreams(see below).Documentation
docs/roadmap/cpp20-conformance-matrix.md.Explicit non-goals
Why streams are out (and should stay out)
This section is longer than a typical “won’t fix” note because the exclusion is easy to misread as incomplete work. It is not incomplete work. Leaving streams out is a feature of this design.
Streams are a different product
C++ iostreams are not “printf with
<<.” They are a parallel I/O architecture: locales, facets, tied streams, formatting state machine, virtualstreambufoverflow/underflow, synchronization with C stdio, and a header dependency graph that pulls an enormous amount of machinery into any translation unit that touches them. A research compiler that aims to stay understandable, embeddable, and cross-compilation-friendly should not accidentally become “also maintain a locale and iostreams stack.” That is a multi-year product line of its own. NeverC already has a clear I/O story for C and for its own builtins. Duplicating iostreams would not make RE POCs better; it would make the toolchain heavier and the failure modes stranger.ABI and shared library gravity
In real toolchains, iostreams are where C++ runtime linkage becomes painful.
libstdc++/libc++version skew, dual-ABI footnotes, iostreams tied to exception and RTTI configuration, and “it linked but crashed instd::ios_base::Init” are familiar failure modes. NeverC’s C++ support is explicitly NeverC ABI v1 with a bundled runtime. The moment we promise iostreams, we invite pressure to match platform library behavior, to ship locale data, and to debug initialization order across hosts. That pressure is hostile to a security-research compiler whose value is control and predictability, not drop-in replacement for a distro toolchain.Hidden global state is the opposite of harness clarity
Stream objects carry formatting flags, width, precision, fill, exception masks, locale, and ties to other streams.
std::coutand friends are global, constructible in surprising orders, and interactive with static initialization. Proof-of-concept code wants explicit buffers, explicit lengths, and explicit sinks—socket send, file write, hex dump, fuzzer callback. Iostreams encourage a style where “print the object” becomes a tangle ofoperator<<overloads and stateful manipulators. That style is fine in some application codebases. It is a poor default for reverse-engineering harnesses, where the reader should see bytes move without consulting the nightstand edition of the locale chapter.Templates, concepts, and compile-time cost without research benefit
Iostream-related headers are among the most expensive includes in C++. They amplify compile times, instantiate large template piles, and create underspecified overload sets around
operator<<. For LLM-assisted and human-authored research code alike, NeverC has emphasized small grammar and deterministic semantics on the C side. Pulling iostreams into the default C++ story would undermine that ethos precisely where feedback loops should be tight. A POC that needs text formatting can use C stdio, NeverCstring, or a tiny purpose-built formatter. It does not needstd::stringstreamto justify C++20 classes and vtables.Security and complexity surface
Stream implementations historically interact with locale, encoding, padding, and user-defined
num_put/num_getfacets. That is a wide surface for complexity and for bugs. Research toolchains should minimize mandatory surface area. Every additional global formatter is another thing to audit, stub, or misconfigure under freestanding and cross targets. DynCode and constrained environments make the problem sharper: you often do not want locale machinery in the image at all.Pedagogical and cultural capture
Iostreams taught a generation that “C++ I/O” means
<<and>>. That pedagogy is sticky. If NeverC ships streams “because C++ has them,” every tutorial and every generated snippet will drag them in, and the project will spend its life explaining performance, binary size, and linkage. By hard-erroring stream headers underNoStreams, we make the policy visible and machine-checked. The compiler is allowed to say: this is C++ for types, objects, generics, and control flow—not a second standard I/O stack.What we expect people to use instead
stringand existing builtins for text and memory disciplineThat combination is enough for serious POC work. It keeps dependencies readable. It keeps freestanding and cross builds sane.
A meandering but sincere conclusion on streams
It is tempting to treat iostreams as mandatory for “real C++.” That temptation comes from identity, not from engineering need. C++ is a language for abstraction over memory and control flow; iostreams are one library’s opinion about formatting and devices, frozen into a design from an era when extending
streambufwas a reasonable way to talk to files. The world moved on: we have memory-mapped I/O, async reactors, structured logging, binary protocols, and security boundaries that do not care aboutstd::ios::hex. Reverse engineers live in that world. They reconstruct packets, notoperator<<forstd::vector<Employee>.There is also a maintenance honesty argument. Implementing “enough iostreams to compile hello world” is a trap. Users will next want file streams, stringstreams, manipulators, wide streams, and customization points. Each step pulls locales and more ABI. The only stable positions are full commitment or principled refusal. This PR takes the second position and documents it.
Finally, exclusion clarifies the success metric of NeverC C++: can a researcher model the objects and control flow of a C++ target and compile a POC quickly? Success is not can we run a cppreference iostreams tutorial unchanged. Different products. We choose the first.
Scope lock (feature-complete definition)
For this effort, “feature-complete” means:
neverc_cxx_runtimeRemaining depth work associated with this completion (construction vtables, full array-new cookies, concept subsumption depth, full BMI decl rehydration, promise protocol detail) is treated as in-scope finish work for this feature branch, not as a separate future language edition.
Acknowledgments
Thanks to the NeverC maintainers and community for a compiler that is opinionated in the right places. This work is an attempt to meet the project on those terms: practical for security research, explicit about tradeoffs, and unwilling to pretend that every historical C++ library facility deserves a forever tax on the toolchain.