Repository navigation
Test the reflection path of type_name on gcc 16, with coherent flags - #107
Merged
steve-downey merged 1 commit intoOct 19, 2026
Merged
Conversation
testing/type_name.hpp has two implementations of the same contract, and
picks the P2996 one only when the compiler and the library both offer
reflection. Only cmake/gcc-16-toolchain.cmake adds -freflection, and nothing
in CI or the presets uses it, so the reflection path was never tested there.
-freflection changes the language, so it is set for the whole build, the
library and the fetched Catch2 included, never per target or per directory:
a header that branches on reflection must see the same answer in every
translation unit, or the program is an ODR violation with no diagnostic
required.
- CI: a gcc 16 / c++26 Debug job passes -DCMAKE_CXX_FLAGS=-freflection.
The reusable workflow's toolchain sets only the per-configuration
flags, so this reaches every compile without displacing anything.
- Presets: gcc-16-debug and gcc-16-release configure through
cmake/gcc-16-toolchain.cmake, which pins g++-16 and C++26 and already
adds the flag. gcc-16-debug is plain Debug: that toolchain does not
implement the infra sanitizer selection.
type_name.test.cpp declares its one check only when the compiler has
reflection (`__cpp_impl_reflection`), and then requires that the
reflection implementation was selected. Falling back is otherwise silent
-- every other test still passes -- which is how a reflection build could
go green while testing the fallback, for instance if the library macro
went unseen. A build that forces an implementation is left alone. Every
other configuration keeps covering the fallback.
steve-downey
force-pushed
the
type-name-reflection-ci
branch
from
October 19, 2026 17:25
3ea45cd to
ba5371b
Compare
steve-downey
changed the title
steve-downey
added a commit
that referenced
this pull request
Oct 20, 2026
…il_compile_test module (#108) ## Summary The 59 negative compile tests were registered through a three-argument `add_fail_test` macro local to `tests/beman/expected/CMakeLists.txt`. This PR turns it into a reusable CMake module, `cmake/beman-xfail-compile.cmake`, written in the shape of `beman-install-library.cmake` so it can move to `bemanproject/infra` unchanged. Which tests exist and what each one expects does not change: every regex ctest receives was checked byte-identical to the one the macro passed. Three commits do the work, each with its own verification; the plan and its decision log are in `docs/plan/extract-xfail-compile.md`. 1. **`beman_add_xfail_compile_test`** replaces the macro. Keyword interface (`SOURCE`, `LINK_LIBRARIES`, `COMPILE_DEFINITIONS`, `TARGET_PREFIX`, `LABELS`, `EXPECT`), same per-test target names, same ctest names. 2. **A driver script** runs the nested build instead of relying on `PASS_REGULAR_EXPRESSION`. It tells apart "the source compiled" from "it failed, but not with the expected diagnostic", neither of which the bare regex could report. Tests get the label `xfail-compile` and a `RESOURCE_LOCK`, so `ctest -j` is safe and `ctest -L xfail-compile` runs just this suite. `$<CONFIGURATION>` becomes `$<CONFIG>`. 3. **The regex lives in the test source.** Each `_fail.test.cpp` carries `// EXPECT: "<regex>"` next to the ill-formed line; the module reads it at configure time, and a source without one fails configure naming the file. 35 files already matched, 15 carried an older subset, 9 had none; all 59 now carry the regex that was proven on gcc, clang, and MSVC. Long alternations are split across several `EXPECT` lines (joined with `|`) because clang-format reflows comments past column 119. ## Demonstration A real test under `ctest -V` now says which diagnostic it matched: ``` 5: -- beman-xfail-compile-driver: beman.expected.tests.unexpected_array_fail failed as expected: .../include/beman/expected/unexpected.hpp:73:25: error: static assertion failed: unexpected<E>: E must not be an array type 1/1 Test #5: unexpected_array_fail ............ Passed 0.42 sec ``` A probe file that compiles cleanly, registered with the same regex: ``` 1/1 Test #64: probe_compiles ...................***Failed 0.67 sec beman-xfail-compile-driver: beman.expected.tests.probe_compiles compiled, but was expected to fail with a diagnostic matching E must not be an array type ``` A probe file with a syntax error, which `WILL_FAIL` would have accepted: ``` 1/1 Test #65: probe_syntax .....................***Failed 0.62 sec beman-xfail-compile-driver: beman.expected.tests.probe_syntax failed to compile, but not for the expected reason. No line matches E must not be an array type Build output: ... ``` Removing the `EXPECT` line from a source: ``` CMake Error at cmake/beman-xfail-compile.cmake:120 (message): beman_add_xfail_compile_test(unexpected_cvref_fail): no '// EXPECT: "<regex>"' line in .../tests/beman/expected/unexpected_cvref_fail.test.cpp; add one or pass EXPECT ``` ## Verification | Check | Result | |---|---| | Regex ctest receives vs the macro's, all 59 | byte-identical | | gcc 16, full suite | 1250/1250 | | clang 23 with libstdc++, full suite | 1249/1249 (before the rebase onto #107) | | `ctest -L xfail-compile -j 8`, gcc 16 | 59/59, about half the serial time | | Touch a test source | CMake re-runs before the build | | pre-commit on changed files | clang-format, gersemi, codespell pass | ## Follow-up The module is proposed to `bemanproject/infra` with identical content. Once it lands there, a small PR here bumps the `infra/` pin, deletes `cmake/beman-xfail-compile*.cmake`, and changes one `include()` path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #88.
tests/beman/expected/testing/type_name.hpphas two implementations of the same contract: P2996 reflection, and a fallback that recovers names fromstd::source_location. It picks reflection only when the compiler and the library both offer it. Onlycmake/gcc-16-toolchain.cmakeadds-freflection, and neither CI nor the presets use that file, so the reflection path was never tested there.Changes
-freflectionchanges the language, so it is set for the whole build, never for one target or directory. The library and the fetched Catch2 get it too. If one translation unit saw reflection and another didn't, a header that branches on it would be defined two ways: an ODR violation that needs no diagnostic.c++26job,Debug.-DCMAKE_CXX_FLAGS=-freflection. The reusable workflow's toolchain sets only the per-configuration flags, so this reaches every compile without displacing anything.gcc-16-debugandgcc-16-release. They configure throughcmake/gcc-16-toolchain.cmake, which pinsg++-16and C++26 and already adds the flag.gcc-16-debugis plain Debug, because that toolchain doesn't implement the infra sanitizer selection.type_name.test.cppdeclares its check only when the compiler has reflection (__cpp_impl_reflection), and then requires that the reflection implementation was selected. Without the check, falling back would be silent: every other test still passes, for example if the library's__cpp_lib_reflectionwent unseen. A build that forces an implementation withBEMAN_EXPECTED_TESTING_TYPE_NAME_USE_REFLECTIONis left alone.Every other configuration keeps covering the fallback.
Tested locally
cmake --workflow --preset gcc-16-debug:-freflectionis on all 197 compile commands, and 1250/1250 tests pass, the reflection check included.infra/cmake/gnu-toolchain.cmake,CMAKE_CXX_STANDARD=26,-DCMAKE_CXX_FLAGS=-freflection): all 133 compiles carry the flag, Catch2's included, and 1250/1250 tests pass.CMakePresets.jsonis a copier-templated file.copier updatekeeps the two new presets, because it applies template changes as a three-way merge. If the template changes the neighbouringgcc-releaseentries, expect a merge conflict there, and keep both sides.