Support typing.Final[T] on dataclass fields as compile-time templates by hughperkins · Pull Request #842 · Genesis-Embodied-AI/quadrants

hughperkins · 2026-08-06T13:51:39Z

Summary

Adds typing.Final[T] as a per-field annotation on frozen @dataclasses.dataclass kernel arguments, marking the field's value as a compile-time constant baked into the compiled kernel. This makes qd.static(config.field) and compile-time branch elimination work on a plain dataclass, with no need to opt the class into @qd.data_oriented.

The motivating use case is static configuration objects: bags of flags and sizes fixed once at setup time and used to specialize kernels. Today the only way to get compile-time member reads on such a config is to declare it @qd.data_oriented and pass it via qd.Template, which opts the whole class into the data-oriented machinery and its per-instance handling in TemplateMapper.lookup. Final[T] gives the same compile-time semantics per field, on an ordinary frozen dataclass.

Example

import dataclasses
from typing import Final

@dataclasses.dataclass(frozen=True)
class SimConfig:
    enable_gravity: Final[bool]   # compile-time constant
    dt: Final[float]              # compile-time constant
    n_substeps: int               # ordinary runtime kernel argument

@qd.kernel
def integrate(config: SimConfig, positions: qd.types.NDArray[qd.f32, 1]):
    dt = qd.static(config.dt)                  # legal: dt is Final
    for i in positions:
        if qd.static(config.enable_gravity):   # branch resolved at compile time
            positions[i] -= 9.8 * dt
        positions[i] += config.n_substeps * dt # n_substeps read at runtime

Previously this required:

@qd.data_oriented
class SimConfig:
    def __init__(self, enable_gravity: bool, dt: float):
        self.enable_gravity = enable_gravity
        self.dt = dt

@qd.kernel
def integrate(config: qd.Template, positions: qd.types.NDArray[qd.f32, 1]):
    ...

Semantics

For each Final[T] field on a dataclass kernel argument:

Baked into the kernel. config.field inside a kernel body (or inside a @qd.func called from one) resolves at AST-build time to the field's actual Python value.
Each distinct value compiles a separate kernel. The value participates in both the in-process template mapper key and the on-disk fastcache key. Two instances carrying equal values share one compiled kernel.
Not a kernel argument. A Final field is not declared as a runtime scalar arg, is not pushed into the launch context, and costs nothing per launch.
Mixing is supported. Final and ordinary fields coexist in the same dataclass, at any nesting depth. Non-Final fields keep their current typed-dataclass behaviour exactly.

Rejected with actionable errors:

case	rationale
Final field on a non-frozen dataclass (`__hash__ is None`)	a baked value must not be reassignable; `frozen=True` and `unsafe_hash=True` are accepted
`Final[T]` where `T` is not `bool` / `int` / `float` / `str` / an `enum.Enum` subclass	`T` must be meaningful as a compile-time literal and hash and `repr` by value, stably across processes
`Final[<nested dataclass>]`	error suggests marking the leaf fields inside it `Final` instead
`Final[NdarrayType]`, `Final[qd.Tensor]`, `Final[MatrixType]`, `Final[StructType]`	arrays and structs are runtime data
`Final[qd.Template]`	redundant
string annotation containing `Final` (`from __future__ import annotations`)	Quadrants cannot see the `Final` through an unresolved string and would otherwise silently treat the field as a runtime argument
a `Final`-like special form that is not `typing.Final`	stdlib `typing.Final` only; `typing_extensions.Final` aliases it on all supported Python versions and is accepted transparently

Implementation

_dataclass_util.py - is_final_annotation, plus final_field_names(dc_type), which validates a dataclass on first sighting and memoises the resulting frozenset of Final field names in _final_plan_cache.
FunctionDefTransformer._transform_kernel_arg - binds each Final field's flattened name (__qd_config__qd_dt) to the actual Python value read off the instance, and skips the runtime scalar-arg declaration.
FunctionDefTransformer._transform_func_arg - mirrors that binding for @qd.func arguments whose flattened annotation is Final[T], so a func called from a kernel body sees the baked value rather than an Expr.
_template_mapper_hotpath._extract_arg - folds Final values into the specialization key.
_fast_caching/args_hasher.dataclass_to_repr - folds Final values into the offline fastcache key.
_func_base._get_frozen_dc_plan - excludes Final fields from the launch plan.

Performance

All reflection happens once per dataclass type, never per launch. Callers on the hot path do a single dict.get keyed on the dataclass type; when the result is empty - every dataclass that does not use the feature, i.e. all existing code - they run the pre-existing code path verbatim. No isinstance, typing.get_origin or dataclasses.fields call occurs per launch.

Measured on _extract_arg with 12 fields, best of 9, across 3 independent runs:

regime	delta
steady state, frozen dataclass, `arg._key` cache hit	-0.0 / -0.8 / +0.7 ns per call, i.e. zero within noise
cold walk on a non-hashable dataclass, walked every launch	+12.6 / +36.7 / +74.2 ns on a ~~4900 ns baseline (~~+0.8%)
the Final branch vs the equivalent non-Final walk	~40 ns per field faster (direct `getattr` instead of recursing into `_extract_arg`)

The steady-state row is the shape that matters for existing workloads: a frozen dataclass whose extracted key is cached on the instance. It is unaffected.

Tests

12 tests in tests/python/test_py_dataclass.py (24 parametrisations across x64 and cuda):

test	covers
`test_final_field_bakes_as_compile_time_constant_via_qd_static`	baseline: `qd.static` on a Final field compiles and computes correctly
`test_final_field_value_change_triggers_recompilation`	distinct Final values produce distinct kernels
`test_final_field_identical_values_share_compiled_kernel`	equal values reuse one compiled kernel (guards against keying on instance identity)
`test_final_field_value_is_part_of_offline_fastcache_key`	the value is in the cross-process cache key, over two `qd.init` cycles
`test_final_and_non_final_fields_mix`	mixed Final and runtime fields in one dataclass
`test_final_field_with_ndarray_sibling`	Final scalar beside an ndarray field
`test_final_field_on_nested_dataclass`	Final on a nested-dataclass leaf
`test_final_field_propagates_through_qd_func_call`	`@qd.func` sees the baked value
`test_final_field_int_annotation_holding_intenum_value`	`Final[int]` holding an `IntEnum` member
`test_final_field_on_non_frozen_dataclass_is_rejected`	frozen requirement, with `unsafe_hash=True` accepted
`test_final_field_rejects_non_bakeable_inner_types`	each rejected `T`, with its remediation hint
`test_final_field_string_annotation_is_rejected`	unresolved string annotations raise

Existing suites verified green on x64 and cuda: test_py_dataclass.py, test_ad_dataclass.py, test_kernel_impl_dataclass.py, plus the template, data_oriented and cache suites.

Docs

docs/source/user_guide/compound_types.md gains a "Compile-time constant fields: typing.Final" section under dataclasses.dataclass, covering the semantics and restrictions above. The overview table and the "How to choose a compound type?" list both point static-configuration objects at this pattern.

Follow-up

A separate Genesis change can migrate the static-config classes currently using @qd.data_oriented + qd.Template (RigidSimStaticConfig, ColliderStaticConfig, GJKStaticConfig in genesis/utils/array_class.py) to frozen dataclasses with Final[T] fields. Every field on those three classes is int or bool, so they are covered by the accepted type set; several int-annotated fields hold IntEnum members, which is supported. Those classes currently use metaclass=AutoInitMeta, which is explicitly a mutable dataclass, and one test (test_rigid_physics.py::test_cholesky_tiling) mutates _static_rigid_sim_config after build, so that test needs to construct a fresh instance instead. Production code never mutates them.

…plates Lets users mark selected fields of a plain frozen @dataclasses.dataclass config as ``typing.Final[T]`` to signal that the field's value must be baked as a compile-time constant in the compiled kernel - replacing the @qd.data_oriented + qd.Template pattern for static configs, and letting ``qd.static(config.field)`` work on plain dataclasses without opting the class into data_oriented machinery. Three coordinated code changes: 1. _dataclass_util.py: adds ``is_final_annotation`` / ``unwrap_final`` helpers backed by ``typing.get_origin``. 2. FunctionDefTransformer._transform_kernel_arg: for a Final field, binds the flat name (e.g. ``__qd_config__qd_dt``) to the actual Python value read off the dataclass instance passed to the kernel. Skips the runtime scalar-arg declaration so ``cook_dtype(Final[int])`` no longer fires. 3. _template_mapper_hotpath._extract_arg: for a Final field, folds the actual value directly into the spec key. Distinct Final values compile distinct kernels; identical values share. 4. _func_base._recursive_set_args (and ``_get_frozen_dc_plan``): skips Final fields at launch since they carry no runtime arg slot. Tests cover baseline (``qd.static(config.dt)``), recompilation on Final value change, mixed Final + non-Final fields, and Final scalar next to an ndarray field in the same dataclass. All 4 tests pass on x64 and cuda; existing 122 tests in test_py_dataclass.py still pass; test_ad_dataclass.py + test_kernel_impl_dataclass.py still pass. Scoped as prototype for PR-A. Not yet covered: Final on nested dataclass fields (currently the assertion at ``arg_value is not None`` should still hold via getattr recursion, worth an explicit test), @qd.func expansion path (``expand_func_arguments`` / ``FlattenAttributeNameTransformer``), bare ``typing.Final`` (raises), string annotations from ``from __future__ import annotations`` (unchanged - Quadrants already assumes resolved ``field.type``).

…dataclass Second follow-up commit for PR-A. Covers three additional scenarios that the first commit missed: 1. **``@qd.func`` propagation.** When a kernel body calls a ``@qd.func`` with a dataclass argument containing ``Final[T]`` fields, the call-site ``_expand_Call_dataclass_args`` expansion resolves each Final flat name via ``build_Name`` to a Python value (bound in the caller by the kernel-def step). But on the callee's side, ``expand_func_arguments`` produces an ``ArgMetadata(annotation=Final[T])`` and ``_transform_func_arg`` had no branch for it - falling through to ``impl.expr_init_func(data)`` which turned the value back into a runtime ``Expr`` and broke ``qd.static``. Added an early ``is_final_annotation`` check that binds ``data`` directly (mirroring the ``annotations.template`` branch). 2. **Kernel-caching correctness.** Added a regression test asserting that two ``SimConfig`` instances with the same Final-field value share a compiled kernel (``template_mapper.mapping`` stays at size 1), while a third launch with a different Final value grows it to 2. Guards against accidentally hashing the instance rather than the value. 3. **Nested dataclass with Final field.** Already worked thanks to the recursive ``_transform_kernel_arg`` call threading ``getattr(arg_value, field.name)``; added an explicit regression test (``Outer`` -> ``Inner.scale: Final[float]`` + ``Outer.bias: Final[float]``) to lock the behaviour in. Tests: all 14 new tests pass on x64 + cuda (7 tests x 2 archs). 166 total tests in test_py_dataclass.py + test_ad_dataclass.py + test_kernel_impl_dataclass.py still pass.

Addresses review decisions on PR-A. The significant item is a soundness fix. **Fastcache soundness bug (fixed).** ``args_hasher.dataclass_to_repr`` only appended a dataclass field's *value* to the cache key when the field carried ``FIELD_METADATA_CACHE_VALUE`` metadata. A ``Final[T]`` field is baked into the compiled kernel, so its value must be in the key too - otherwise a kernel compiled with ``offset=7`` baked in is loaded from the offline cache in a later process for a config carrying ``offset=100``, silently returning 7. The in-process template-mapper spec key already discriminated on the value, which is why the first round of tests passed. Repro'd, fixed, and covered by ``test_final_field_value_is_part_of_offline_fastcache_key`` (two ``qd.init`` cycles over one ``offline_cache_file_path``). **Validation, cached per dataclass type (decisions A, B, D, G).** Adds ``final_field_names(dc_type)``, which validates on first sighting and memoises the resulting ``frozenset`` of Final field names in ``_final_plan_cache``: - ``T`` in ``Final[T]`` must be ``bool`` / ``int`` / ``float`` / ``str`` or an ``enum.Enum`` subclass. Arrays, ``qd.dataclass`` structs, ``qd.Tensor``, nested dataclasses, ``qd.Template`` and arbitrary objects are rejected with a tailored remediation hint. (A - the type set is what the three Genesis static-config classes need; see below.) - A ``Final`` field on a class with ``__hash__ is None`` (plain non-frozen ``@dataclass``) is a hard error; ``frozen=True`` and ``unsafe_hash=True`` are accepted. (B) - A ``Final``-like special form that is not ``typing.Final``, and a string annotation containing "Final" (from ``from __future__ import annotations``), both raise rather than silently lowering the field as a runtime arg. (D) **Hot path pays nothing (decision E).** All reflection moved off the per-launch path: callers do one ``dict.get`` keyed on the dataclass type and, when the result is empty (every dataclass not using the feature), run the pre-existing code verbatim. No ``isinstance`` / ``typing.get_origin`` / ``dataclasses.fields`` per launch. Measured on ``_extract_arg`` with 12 fields, best of 9, 3 runs: - steady state (frozen dataclass, ``arg._key`` cache hit - the Genesis shape): -0.0 / -0.8 / +0.7 ns per call, i.e. zero within noise - cold walk on a non-hashable dataclass (walked every launch): +12.6 / +36.7 / +74.2 ns on a ~4900ns baseline, ~+0.8%, and not a shape Genesis uses - the Final branch itself is ~40ns *faster* per field than the non-Final walk, since it reads ``getattr`` directly instead of recursing into ``_extract_arg`` **Genesis survey (answers A).** The three affected classes in ``genesis/utils/array_class.py`` are ``RigidSimStaticConfig`` (36 fields), ``ColliderStaticConfig`` (6) and ``GJKStaticConfig`` (1) - every field is ``int`` or ``bool``, so the accepted type set covers them. Several ``int`` fields hold ``IntEnum`` members (``integrator``, ``constraint_solver``, ``PARA_LEVEL``, ``CCD_ALGORITHM_CODE``); validation is on the declared annotation so ``Final[int]`` holding an ``IntEnum`` is accepted with no per-launch type check. Covered by ``test_final_field_int_annotation_holding_intenum_value``. **Docs (decision H).** ``compound_types.md`` gains a "Compile-time constant fields: ``typing.Final``" section under ``dataclasses.dataclass``, plus overview table and "how to choose" entries pointing static-configuration objects at this pattern instead of ``@qd.data_oriented`` + ``qd.Template``. Tests: 24 Final tests (12 x 2 archs) pass; 176 total across test_py_dataclass.py + test_ad_dataclass.py + test_kernel_impl_dataclass.py, plus the template / data_oriented / cache suites. pre-commit -a clean; check_non_ascii and the 120c wrapping audit clean. Known Genesis-side migration blocker (not addressed here, Quadrants-only PR per decision F): the three classes use ``metaclass=AutoInitMeta``, explicitly a *mutable* dataclass, and ``tests/test_rigid_physics.py::test_cholesky_tiling`` mutates ``_static_rigid_sim_config`` after build. Production code never mutates them, so the migration needs that one test rewritten to build a fresh instance.

github-actions · 2026-08-06T14:25:51Z

Total: 5 file(s) changed, +160 -4 code lines.

github-actions · 2026-08-06T14:59:49Z

Total: 6 file(s) changed, +378 -16 code lines.

``_extract_arg`` calls ``final_field_names`` with its loosely-typed ``annotation`` parameter (an ``AnnotationType`` union covering every kernel-arg annotation shape), having already established that it is a dataclass type via the ``__dataclass_fields__`` probe immediately above. Pyright cannot follow that narrowing and reported ``"NdarrayType" is not assignable to "type"``. Widened the parameter to ``Any`` rather than narrowing at the call site: a ``typing.cast`` would be a real function call on a per-launch path, which is the thing this whole design is built to avoid. Documented the reasoning on the helper. pyright is now clean on all five files this PR touches. The 3 errors remaining in a local repo-wide run are all in untouched files (kernel.py, kernel_checkpoint.py) and come from a stale local ``quadrants_python.pyi``; CI copies a freshly built stub from ``dist/`` before running pyright.

github-actions · 2026-08-06T15:40:55Z

Total: 6 file(s) changed, +378 -16 code lines.

github-actions · 2026-08-06T16:13:08Z

Diff coverage: 97% · 325 lines, 9 missing

Three genuine CI catches, all in prose I added. **Line wrapping** (two lines wrapped near the 80c AI default rather than 120c): - ``_dataclass_util.py`` module docstring wrapped at 76c - ``test_py_dataclass.py`` fastcache regression-test docstring wrapped at 80c Also tightened two further comment runs the same audit surfaced (``_dataclass_util.py`` typing_extensions note at 97c, ``_func_base.py`` plan comment at 101c). Every added Python line is now <= 120c with no mid-paragraph break that could have fit the next word. My local audit missed these because ``find_underwrapped.py`` reports the max width per *run*, which masks a single short line inside an otherwise well-packed block - exactly the shape CI flagged. Added a per-line checker (tokenize-based, so code lines are never mistaken for prose) and confirmed only two candidates remain, both correct style: a bare ``#`` paragraph separator and a trailing inline comment on a field declaration. **Doc quality**: ``compound_types.md`` used ``qd.Tensor`` in the new "Compile-time constant fields" restrictions list at line 188, but the term is not linked until line 271. Linked it at first use as ``[qd.Tensor](tensor.md)``. Verified: pyright clean on all five touched files, check_non_ascii clean, pre-commit -a clean, 138 tests pass in test_py_dataclass.py. Note on the previous run's other 12 red checks: 5 were "Set up job" runner-provisioning flakes, and 7 were jobs *cancelled* (not failed) when my re-runs superseded them mid-flight - the Linux build, for instance, completed build and install successfully before its test step was cancelled. This push supersedes all of it with one clean run.

github-actions · 2026-08-06T18:54:08Z

Total: 6 file(s) changed, +378 -16 code lines.

hughperkins · 2026-08-06T20:25:58Z

CI status note for reviewers

There are red checks on this PR, but no genuine code failures. Every check that produced a real verdict on the current SHA (cd5746ff) passed:

Check doc quality, Check feature factorization, Check non-ASCII characters, Check test coverage for changes, Linters, Check markup links, PR change report, clang-tidy, Windows 2025 (all 8 jobs), Manylinux wheel build (7 jobs), Mac OS (12 jobs).

Three checks never rendered a verdict:

check	state
`Check line wrapping`	job cancelled on all 3 attempts - the agent never ran
`Check deleted comments`	job cancelled on all 3 attempts
`Linux`	queued for 80+ minutes

Contributing causes, in order:

A GitHub Actions incident. 8 jobs died with Failed to resolve action download info. Error: Service Unavailable - GitHub could not serve actions/checkout@v4 to the runners. This fails inside "Set up job", before any repository code executes.
cancel-in-progress: true on every affected workflow, keyed on workflow + branch. The four Cursor-agent checks begin with sleep 1800, so they sit idle for 30 minutes and are easily superseded.
My own re-runs, which compounded (2) rather than clearing it.

Two genuine failures did occur earlier in the PR's history and are fixed in cd5746ff:

Pyright: final_field_names was typed dc_type: type but _extract_arg calls it with its loosely-typed annotation union. Widened to Any rather than adding a typing.cast, which would be a real function call on a per-launch path.
Line wrapping + doc quality: two docstring lines wrapped at 76c / 80c instead of 120c, and qd.Tensor was used before its first link. Both fixed; Check doc quality has since passed.

Verified locally on cd5746ff: pre-commit run -a clean, pyright clean on all five touched files, check_non_ascii.py clean on the PR diff, every added Python line <= 120c, and 138 tests passing in test_py_dataclass.py (176 including test_ad_dataclass.py and test_kernel_impl_dataclass.py).

Happy to re-trigger the three outstanding checks once CI is calmer.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cd5746ffd5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

hughperkins · 2026-08-06T20:33:09Z

Agent wrote:

Correction: the builds already passed on this SHA

Following up on my previous note. The Mac OS and Manylinux wheel build checks show red, but both already passed on this exact SHA (cd5746ff) on an earlier attempt. Per-attempt build-job outcomes:

workflow	attempt	pass	fail	cancelled
Mac OS	1	2	0	2
Mac OS	2	4	0	0
Mac OS	3	0	0	4
Mac OS	4	0	0	4
Manylinux wheel build	1	1	4	3
Manylinux wheel build	2	7	0	1
Manylinux wheel build	3	0	0	8

Attempt 2 is fully green for Mac OS (4/4) and has zero failures for Manylinux (7 pass, 1 cancelled). The only genuine build failures anywhere are the 4 on Manylinux attempt 1, which were the GitHub Actions incident (Failed to resolve action download info. Error: Service Unavailable).

Attempts 3 and 4 are mine: I re-ran these workflows trying to clear stale red entries, and each re-run was cancelled while queued - overwriting a green attempt-2 result with cancellations. Since GitHub surfaces only the latest attempt in the checks list, that is why the PR displays red. My mistake, and I have stopped re-running.

Net: zero genuine code failures on this SHA across every workflow. Every check that produced a verdict passed, and the builds are green on attempt 2. A reviewer wanting a clean board can re-run these two workflows once when runner capacity is healthy, or push any trivial commit to trigger one fresh full run.

github-actions · 2026-08-07T14:01:41Z

Total: 6 file(s) changed, +378 -16 code lines.

github-actions · 2026-08-07T15:02:28Z

Diff coverage: 97% · 325 lines, 9 missing

github-actions · 2026-08-07T16:33:09Z

Total: 6 file(s) changed, +378 -16 code lines.

github-actions · 2026-08-07T17:31:05Z

Diff coverage: 97% · 325 lines, 9 missing

Two genuine CI findings on the post-merge SHA, both correct. **Test coverage check**: ``unwrap_final`` was a new public function with zero callers and zero tests - dead code. It was written early in the POC, but ``_validate_final_inner_type`` ended up calling ``typing.get_args`` directly because it also needs the arity checks (bare ``Final``, multiple type args) that ``unwrap_final`` does not provide. Deleted rather than retro-fitting a test for a function nothing uses. **Doc quality check**: ``compound_types.md`` used "in-process specialization key", Quadrants-internal jargon that appears nowhere else in the docs (including the linked ``fastcache.md``). Reworded to describe the behaviour in user terms - Quadrants picks which compiled kernel to reuse by looking at the field's value, in-process and in the on-disk fastcache. Also replaced "lower the field as a runtime argument" with plainer wording, since "lower" is compiler jargon that the same rule would likely flag next. Verified on the merged SHA: 24 Final tests pass, pyright clean, check_non_ascii clean, pre-commit clean, comment-wrapping audit clean.

github-actions · 2026-08-07T18:50:50Z

Total: 6 file(s) changed, +371 -16 code lines.

github-actions · 2026-08-07T19:49:46Z

Diff coverage: 99% · 318 lines, 3 missing

The bullet restated the same fact three ways - that the value drives cache lookup, that changing it recompiles, and that equal values share a kernel are all one idea. Cut to the single sentence that carries it, matching the length of the surrounding bullets.

hughperkins · 2026-08-10T21:18:25Z

@codex review

The class-level behavior scan only walked Enum-subclass bases, so a non-enum mixin (``class Mode(Labels, enum.Enum)`` with ``Labels.label``) was skipped: ``cfg.mode.label`` is observable at compile time yet absent from the Final key, so mutating the class var or two same-named factory mixins would reuse a stale kernel. Skip only baked base types (the mixed-in primitive / object / NumPy) and the library's own enum bases; inspect every user-authored base, enum or not. Verified plain Enum/IntEnum/StrEnum/IntFlag still accepted on 3.10/3.12/ 3.13. Adds a non-enum-mixin regression test.

hughperkins · 2026-08-21T19:50:06Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 926644f047

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

The class-behavior scan skipped every ``_x_`` name as enum bookkeeping, which also exempted user-authored sunder hooks (``_missing_``, ``_repr_html_`` on 3.13+, an overriding ``_numeric_repr_``) - observable behavior on the baked member that the module/name/value key cannot capture. Build an allowlist of machinery-generated names by probing a plain class and each framework enum kind with real class syntax at import (so it tracks the running Python, including 3.13's new ``__firstlineno__`` / ``__static_attributes__`` / ...), and reject any sunder/dunder in a user dict that is not in it. Verified plain Enum/IntEnum/StrEnum/IntFlag and custom str-mixin enums stay accepted on 3.10/3.12/3.13. Adds a ``_missing_`` regression test.

hughperkins · 2026-08-21T20:04:01Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7a9e4d33f2

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Codex: id(cls) in the offline (cross-process) key is only process-local and can repeat at the same address in another worker, so two same-named dynamic enum/primitive classes could serialize identical offline keys and one could load a kernel baked for the other's distinct class - contrary to the documented "dynamic classes don't reuse another process's cached kernel". Fold a per-process nonce into the offline id component for non-resolvable classes so such a key is unique to this process (guaranteed cross-process miss, never a wrong reuse); id(cls) still separates distinct dynamic classes within a process, and the in-process live key stays nonce-free. Also fix a pylint no-member on enum.StrEnum by resolving it via getattr. Adds a process-nonce regression test.

hughperkins · 2026-08-21T20:16:19Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5304ae539b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Addresses two Codex P2s on the Final-field key path: - Live spec key: switching the class-identity component to bare id(cls) dropped the strong reference, so a dynamic factory class could be GC'd after launch and its address recycled by the next same-named class, colliding on the mapper key. Now key on (id(cls), cls): id keeps metaclass-== immunity while the retained class object pins the id for the specialization's lifetime. The offline key stays process-stable (nonce/None) and never holds the class object. - _final_plan_cache / _final_path_cache: keyed on id(type) with a stored strong ref and an identity (`is`) check, so a metaclass making two distinct dataclass types compare equal can no longer merge their (possibly different) Final schemas. Adds regression tests for live-key class retention and for identity-keyed plan lookup under metaclass-equal dataclass types.

hughperkins · 2026-08-21T20:31:53Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2acf20ab7e

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Codex P2: embedding the retained class object directly in the live key made the whole spec key unhashable when the class's metaclass sets __hash__ = None, so TemplateMapper.lookup's self.mapping[key] would raise TypeError instead of compiling the valid Final value. Wrap the strong class reference in a _ClassRef identity token that hashes via object.__hash__ and compares by `is`, never delegating to the metaclass. It still pins the class (so its id can't be recycled) while keeping the key hashable and immune to a custom metaclass __eq__. Adds a regression test with a __hash__ = None metaclass asserting the live key is hashable, usable as a dict key, and keeps distinct classes apart.

hughperkins · 2026-08-21T20:43:21Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ef35d94d84

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

…y type nature Two Codex P2s on the Final-field path: - _first_final_path's recursion guard tracked visited dataclass types in a set keyed by equality. A metaclass making a nested inner type compare equal to its mutable outer type made the inner look "already visited", short-circuiting the walk so a mutable ancestor of a Final leaf slipped past rejection. Track visited types by id() (stable: every type on the path is held alive by the walk). - _is_baked_base_type identified NumPy scalar bases from the mutable __module__ string, so a user subclass spoofing __module__ = "numpy" was treated as a trusted base (its state/behavior went uninspected and its class-identity token was dropped). Identify a genuine NumPy base by inheritance + type nature instead: issubclass(np.generic) and no Py_TPFLAGS_HEAPTYPE (NumPy's scalar types are C static types; a user subclass is always a heap type). Adds regression tests for both (metaclass-equal nested dataclass; int subclass with observable __eq__ spoofing __module__).

hughperkins · 2026-08-21T20:58:42Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6f7c649bb0

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

…uous-parens - Codex P2: an enum with a custom EnumMeta subclass can carry observable class-level state/behavior (e.g. type(Mode).label, read as cfg.mode.__class__.label) that the key does not capture, since the metaclass is not on enum_cls.__mro__. _enum_class_behavior_attr now also walks the enum's metaclass MRO, inspecting user-authored metaclass layers (the framework EnumMeta/EnumType/type/object are skipped, so plain enums are unaffected). Extracted the per-class inspection into _observable_class_dict_attr and reused it for both walks. - Linters: fix pylint C0325 (superfluous-parens after 'not') introduced in the previous _is_baked_base_type change. Adds a regression test for an enum with observable metaclass state (rejected) alongside plain/Int enums (accepted).

hughperkins · 2026-08-21T21:11:41Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 684d795ebc

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Codex P2: _reject_stateful_primitive_subclass only walked the subclass MRO, so observable state on a custom metaclass (e.g. UnitMeta.label, read as cfg.x.__class__.label) was missed - the value was accepted, and mutating that metaclass attribute reused a stale specialization. Now walk the subclass's metaclass MRO too, mirroring the enum validation. Shared _observable_metaclass_attr powers both the primitive and enum metaclass walks. It exempts the identity dunders __eq__/__ne__/__hash__ (the class-identity key uses id/is/object.__hash__ via _ClassRef, so a metaclass overriding those can neither collapse distinct classes nor make the key unhashable - the two behaviors earlier tests rely on) and rejects any other observable metaclass attribute the fixed key cannot capture. Adds a regression test: a metaclass class var (label) is rejected; a metaclass overriding only __eq__/__hash__ stays accepted.

hughperkins · 2026-08-21T21:27:55Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 778a33761e

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

…ass equality Two P2 follow-ups on the Final-field key mechanism: - The per-process nonce folded into a dynamic class's offline fastcache key was minted once at import, so a ``fork``ed child inherited both the string and the parent's allocator state - two workers could serialize identical offline keys for distinct same-qualified dynamic classes. Reseed it via ``os.register_at_fork(after_in_child=...)`` (``uuid4`` re-reads OS entropy, so each child differs); ``spawn`` re-imports and reseeds anyway. - Drop the ``__eq__``/``__ne__``/``__hash__`` exemption when inspecting a custom metaclass on a Final enum / primitive subclass. Identity-safe keying (via ``_ClassRef``) prevents dict collisions but the operator is still observable through ``qd.static(cfg.x.__class__ == Expected)``, and mutating the state it consults leaves the fixed key stale. Such metaclasses are now rejected; the ``_ClassRef`` identity hashing remains as defense-in-depth (and for reloads).

hughperkins · 2026-08-21T21:45:17Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3d5f642f6d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-08-21T21:48:28Z

+                # this early return hand back the user's value instead of the real spec key. One limitation is that
+                # storing ``_qd_spec_key`` is impossible for dataclasses enforcing 'slots=True', but that is not
+                # the default and almost never used in practice.
+                return arg._qd_spec_key


Revalidate cached keys after enum class mutation

When the same frozen config is relaunched after its previously plain enum class is monkey-patched, this early return bypasses _reject_stateful_enum_member() and reuses the old specialization. For example, a kernel that bakes qd.static(cfg.mode == 1) can compile with inherited enum equality, then Mode.__eq__ can be replaced before the next launch; the cached key still selects the branch compiled under the old behavior instead of rejecting the now-behaviorful enum. Fresh evidence after the earlier class-behavior fixes is that Python permits adding such an override after validation, while validation only occurs below this cached return; avoid caching validation-sensitive enum/subclass keys or revalidate them before returning.

Useful? React with 👍 / 👎.

hughperkins · 2026-08-21T21:50:43Z


 A `dataclasses.dataclass` is a Python-only container. The compiler reads it at compile time and flattens its members into individual kernel parameters — the container itself has no memory layout and doesn't exist on the kernel side. Inside a kernel, tensor members are read-write through indexing (`s.x[i] = ...`), but the member *binding* itself (`s.x = other_tensor`) cannot be reassigned from inside a kernel.

+**Reserved field names (`_qd_` prefix).** To make repeated launches fast, Quadrants stores small internal caches directly on the dataclass instances you pass to a kernel - for example the key that selects which compiled kernel to reuse. These attributes always begin with `_qd_`, so that prefix is reserved: do not give a dataclass field a name starting with `_qd_`. Such a field would shadow Quadrants' internal state, which can make a kernel silently reuse a value from an earlier launch. Any other name (including a leading underscore, like `_key`) is fine.


this seems way too much detail. can prbablly replace all of "To make repeated launches fast, Quadrants stores small internal caches directly on the dataclass instances you pass to a kernel - for example the key that selects which compiled kernel to reuse. These attributes always begin with _qd_, so that prefix is reserved: do not give a dataclass field a name starting with _qd_. Such a field would shadow Quadrants' internal state, which can make a kernel silently reuse a value from an earlier launch. Any other name (including a leading underscore, like _key) is fine." With "Created fields with names beginning with _qd_ is unsupported and will give undefined behavior."

github-actions · 2026-08-21T22:18:03Z

Total: 6 file(s) changed, +1332 -17 code lines.

github-actions · 2026-08-21T23:10:01Z

Diff coverage: 92% · 1173 lines, 97 missing

hughperkins added 3 commits August 6, 2026 06:45

hughperkins changed the title ~~[POC] Support typing.Final[T] on dataclass fields as compile-time templates~~ Support typing.Final[T] on dataclass fields as compile-time templates Aug 6, 2026

hughperkins marked this pull request as ready for review August 6, 2026 20:25

chatgpt-codex-connector Bot reviewed Aug 6, 2026

View reviewed changes

Comment thread python/quadrants/lang/_fast_caching/args_hasher.py Outdated

Comment thread python/quadrants/lang/_dataclass_util.py Outdated

Merge branch 'main' into hp/final-annotated-dataclass-fields

2d28bfb

hughperkins mentioned this pull request Aug 7, 2026

[Build] Pin nanobind below 2.14.0 #848

Merged

2 tasks

Merge branch 'main' into hp/final-annotated-dataclass-fields

413c154

hughperkins added awaiting-codex-review awaiting-codex-review need-genesis-unit-tests need-genesis-benchmarks labels Aug 10, 2026

chatgpt-codex-connector Bot reviewed Aug 21, 2026

View reviewed changes

Comment thread python/quadrants/lang/_final_dataclass_fields.py Outdated

chatgpt-codex-connector Bot reviewed Aug 21, 2026

View reviewed changes

Comment thread python/quadrants/lang/_final_dataclass_fields.py Outdated

chatgpt-codex-connector Bot reviewed Aug 21, 2026

View reviewed changes

Comment thread python/quadrants/lang/_final_dataclass_fields.py Outdated

Comment thread python/quadrants/lang/_final_dataclass_fields.py Outdated

chatgpt-codex-connector Bot reviewed Aug 21, 2026

View reviewed changes

Comment thread python/quadrants/lang/_final_dataclass_fields.py Outdated

chatgpt-codex-connector Bot reviewed Aug 21, 2026

View reviewed changes

Comment thread python/quadrants/lang/_final_dataclass_fields.py

Comment thread python/quadrants/lang/_final_dataclass_fields.py Outdated

chatgpt-codex-connector Bot reviewed Aug 21, 2026

View reviewed changes

Comment thread python/quadrants/lang/_final_dataclass_fields.py

chatgpt-codex-connector Bot reviewed Aug 21, 2026

View reviewed changes

Comment thread python/quadrants/lang/_final_dataclass_fields.py

chatgpt-codex-connector Bot reviewed Aug 21, 2026

View reviewed changes

Comment thread python/quadrants/lang/_final_dataclass_fields.py

Comment thread python/quadrants/lang/_final_dataclass_fields.py Outdated

chatgpt-codex-connector Bot reviewed Aug 21, 2026

View reviewed changes

hughperkins commented Aug 21, 2026

View reviewed changes


		A `dataclasses.dataclass` is a Python-only container. The compiler reads it at compile time and flattens its members into individual kernel parameters — the container itself has no memory layout and doesn't exist on the kernel side. Inside a kernel, tensor members are read-write through indexing (`s.x[i] = ...`), but the member binding itself (`s.x = other_tensor`) cannot be reassigned from inside a kernel.

		Reserved field names (`_qd_` prefix). To make repeated launches fast, Quadrants stores small internal caches directly on the dataclass instances you pass to a kernel - for example the key that selects which compiled kernel to reuse. These attributes always begin with `_qd_`, so that prefix is reserved: do not give a dataclass field a name starting with `_qd_`. Such a field would shadow Quadrants' internal state, which can make a kernel silently reuse a value from an earlier launch. Any other name (including a leading underscore, like `_key`) is fine.

Uh oh!

Conversation

hughperkins commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Example

Semantics

Implementation

Performance

Tests

Docs

Follow-up

Uh oh!

github-actions Bot commented Aug 6, 2026

Uh oh!

github-actions Bot commented Aug 6, 2026

Uh oh!

github-actions Bot commented Aug 6, 2026

Uh oh!

github-actions Bot commented Aug 6, 2026

Uh oh!

github-actions Bot commented Aug 6, 2026

Uh oh!

hughperkins commented Aug 6, 2026

CI status note for reviewers

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

hughperkins commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Correction: the builds already passed on this SHA

Uh oh!

github-actions Bot commented Aug 7, 2026

Uh oh!

github-actions Bot commented Aug 7, 2026

Uh oh!

github-actions Bot commented Aug 7, 2026

Uh oh!

github-actions Bot commented Aug 7, 2026

Uh oh!

github-actions Bot commented Aug 7, 2026

Uh oh!

github-actions Bot commented Aug 7, 2026

Uh oh!

hughperkins commented Aug 10, 2026

Uh oh!

hughperkins commented Aug 21, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

hughperkins commented Aug 21, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

hughperkins commented Aug 21, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

hughperkins commented Aug 21, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

hughperkins commented Aug 21, 2026

hughperkins commented Aug 6, 2026 •

edited

Loading

hughperkins commented Aug 6, 2026 •

edited

Loading