Skip to content

Latest commit

 

History

History
493 lines (369 loc) · 22 KB

File metadata and controls

493 lines (369 loc) · 22 KB

Codegen

Summary

The binding code generator turns Geode binding files into C++ bindings and Luau type stubs. It is a Python tool in tools/luau_codegen/. This is an overview, not a full reference for every module.

What it produces

The generator reads Broma binding files for one Geometry Dash version and one platform, then writes:

  • C++ binding sources into build/luauapi-gen/src. These expose game classes to Lua and implement the hook functions.
  • Value struct check/push headers into build/luauapi-gen/src/framework/stack/ (Types.generated.hpp and Types.generated.containers.hpp). See Value structs below for split and include order.
  • One Luau type stub at types/geode.d.luau, holding all bound classes, factories, enum names, and the root geode namespace. The types/ folder is created in the repo root during build and is gitignored.
  • Metadata files: a schema, a report, a parity file, and an audit.

The generated C++ is machine written and compiled into the main library with warnings disabled and size optimization. Do not edit it by hand.

How the build calls it

CMakeLists.txt runs python -m luau_codegen (the package in tools/luau_codegen/, with tools/ on PYTHONPATH) as the luauapi_codegen target, and the main library depends on it. Configure runs --list-all-outputs once so CMake knows the binding and type byproducts.

The stamp command runs codegen in one step:

  • Delegate specs are loaded.
  • Trampolines are loaded.
  • Bindings and stubs are emitted.

The codegen platform comes from the build target. See cmake/Codegen.cmake luauapi_set_codegen_platform().

Build target Codegen platform
Windows win
Android arm64-v8a android64
Android armeabi-v7a android32
iOS ios
macOS universal (arm64 + x86_64) mac
macOS arm64 only m1
macOS x86_64 only imac

The GD bindings version comes from mod.json gd.<key>. The key follows the codegen platform. Android ABIs use android. Apple desktop platforms use mac. If the key is missing, CMake falls back to gd.win, then to 2.2081. The Broma tree is geode_bindings/bindings/<version>/. Keep LUAUAPI_BINDINGS_GIT_TAG in CMake aligned with the mod.json GD version.

LUAUAPI_HOST_ONLY=ON skips the Geode SDK and configure-time listing for host-only tooling.

Binary size

Bindings make up most of the binary, so the build minimizes them. A size flag is set on the binding target (-Oz on Clang, -Os on GCC, /O1 on MSVC and clang-cl). On Windows, linker /OPT:REF and /OPT:ICF drop unused code and fold identical functions. These apply to non-debug builds only.

The hook generator

emit/hooks.py emits one C++ function per hookable game method. That function:

  1. Runs the registered before callbacks, pushing self and the arguments to Lua.
  2. Applies an argument override or a skip, if the callback returned one.
  3. Calls the original method, unless it was skipped.
  4. Runs the registered after callbacks, applying a return override if given.

emit/cxx_templates.py emits the shared hook runtime that stores callbacks, sorts them by priority, and calls them. Pre and post hook dispatch also record crash sidecar boundaries before each Lua callback. See Crash sidecar. Invalid overrides are type-checked via pcall. Binding and hook decode share convert/marshalling.py emit_stack_check. A method is hookable only when its address is known on the target platform and its arguments and return can be marshalled. Static methods, constructors, and destructors are skipped.

Inputs that trigger a rebuild

The custom command depends on the following. A change to any reruns codegen:

  • Python sources
  • Broma definitions
  • Extra stubs in tools/luau_codegen/extra_bindings/
  • Geode UI headers (Geode/ui/*.hpp, discovered via parse/geode_sdk.py)
  • Free-function headers listed in model/free_fn_sources.py

Geode SDK scan scope

parse/geode_sdk.py discovers extra bindings from the installed Geode SDK headers:

  • GD and Geode enums (see Geode and GD enums below).
  • UI classes and enums from Geode/ui/*.hpp headers included by UI.hpp.
  • Free functions from the manifest in model/free_fn_sources.py (FREE_FUNCTION_SOURCES).

The manifest is the single source for both the scanner and the generated #include list. test_binding_guards_framework.py FreeFnManifestSyncTests fails on drift.

SDK header Namespaces Name filter
utils/general.hpp geode::utils, geode::utils::clipboard, geode::utils::game, geode::utils::thread, geode::utils::platform all
ui/Popup.hpp geode createQuickPopup only
ui/GeodeUI.hpp geode all
utils/string.hpp geode::utils::string all
utils/random.hpp geode::utils::random all
utils/cocos.hpp geode::cocos all

Only GEODE_DLL declarations are processed, and functions must be marshallable. Only the first free function per name and arity is kept. HTTP, async tasks, and file I/O are not in this manifest. They live in handwritten bindings with matching extra stub files.

Geode and GD enums

parse/geode_sdk.py reads bindings/include/Geode/Enums.hpp and UI enum tables into CodegenContext. emit/bindings/geode_enums.py registers integer constant tables under geode.gd and geode. emit/luau_types/enums.py emits stub types in types/geode.d.luau.

Enums marshal as numbers in both directions (check<int> in, lua_pushnumber out). For script usage and compares, see enums.

Handwritten type-stub fields

Some bindings are written by hand in src/bindings/ and src/framework/. Their type signatures still need to be in types/geode.d.luau.

Two ways to add them:

  • tools/luau_codegen/extra_bindings/*.dluau: appended at the end of the stub. Current files: fs.dluau, gd3d.dluau, hook.dluau, imgui.dluau, json.dluau, keyboard.dluau, loader.dluau, mod.dluau, task.dluau, web.dluau, websocket.dluau. Use this for new globals and for support types the geode namespace references.
  • tools/luau_codegen/emit/luau_types/manual_fields.py: injects fields into a namespace that codegen already emits, such as geode.cocos.

Handwritten namespaces are two copies of the same surface (the C++ registrar and the stub declaration), so they must stay in sync. test_manual_fields_sync.py and test_extra_bindings_sync.py fail CI on drift.

geode.cocos is hybrid:

  • Codegen emits geode::cocos helpers into bindings_free_functions.cpp
  • GeodeCocosBinding.cpp registers only:
    • color helpers
    • ccDrawColor4B
    • hex parsers
  • test_binding_guards_*.py enforces that split

Metadata outputs

After generation, build/luauapi-gen/ holds:

  • schema.json: parsed Broma classes, bound fields, ambiguous overloads, supportedFreeFunctions, unscannedGdEnums, manualFields, and extraBindings
  • report.md: counts, skip reason histograms, intersection stats, linkless classes per platform, SDK scan warnings, unscanned GD enum aliases, and handwritten stub provenance
  • parity.json: cross-platform method support matrix (see Platform parity)
  • audit.md: skip audit grouped by bucket (see Audit buckets below)
  • delegate_specs.py: generated delegate type specs, installed before binding emit
  • value_struct_specs.py: generated value-struct type specs, installed before binding emit

emit/metadata.py writes the schema and report. emit/parity.py writes parity.json. emit/audit.py writes the audit. Run --audit-report-out or --parity-report-out to regenerate a report without a full stamp.

Audit buckets

emit/audit.py groups skipped methods and free functions into buckets. Each bucket has a count, a reason histogram, and up to 25 sample skips.

Bucket id Category
callback_method Broma callback methods
sel_arg cocos2d selector args
std_function_arg std::function / geode::Function args
callback_alias Callback alias args
delegate_arg delegate pointer args without a bound spec
container_arg unsupported gd container args
value_type_arg FMOD / Kazmath value types (see FMOD binding below)
http_async_excluded HTTP / async free functions
other unclassified skips

Supported bindings do not appear in these buckets. See tests/luau_codegen/audit/test_audit.py for bucket rules.

Type classification

convert/type_classification.py maps each C++ arg or return to a TypeInfo kind. Container parsing lives in the same module. Normalization helpers live in convert/type_map.py. Binding, hook, delegate, and stub emit all call classify_arg() or classify_return().

Resolution order in _classify_core():

  1. Geode task handles
  2. Root pointer containers, audited pointer-grid fields, ccCArray views, and recursive supported composites
  3. SeedValue wrappers
  4. Primitives: bool, wide integers as string, numeric, string
  5. Value structs from VALUE_TYPES in model/value_types.py (via convert/type_map.py)
  6. Enums from CodegenContext
  7. Opaque handles from OPAQUE_HANDLE_TYPES
  8. geode::Result<> as boolean or string
  9. Callbacks (std::function, geode::Function) on args only
  10. cocos2d selectors on args only
  11. Delegate pointers on args only, via model/delegate_specs.py
  12. Object pointers last

CALLBACK_ALIASES and CLASS_CALLBACK_ALIASES in type_map.py expand shorthand names before classification. Unsupported types return None, which becomes a skip reason such as unsupported-arg:<type>.

Geode task handles

arc::TaskHandle<T> returns become GeodeTaskHandle<T> in Luau. std::optional<arc::TaskHandle<T>> returns become GeodeTaskHandle<T>?. arc::TaskHandle<void> returns become GeodeTaskHandle<nil>.

Task handle args stay unbound. Luau code can observe returned native tasks but cannot pass native task handles back into generated bindings.

The generated C++ uses GeodeTaskHandleBinding to push returns. The bridge polls native handles from the task tick after TaskScheduler::advance.

Overload resolution

Broma can declare several methods with the same name. The generator groups them by name and input arity (group_supported() in policy/filtering.py). When two overloads share a name and input arity, only one can bind, because Lua dispatches on arity. The generator keeps one only if PREFERRED_OVERLOADS lists its normalized arg signature, otherwise every colliding overload is skipped with ambiguous-overload-arity:<arity>. Ambiguous overloads cause codegen to exit with code 6 when building the emit plan.

One Lua table key per method name. Stubs widen with ...any where overloads disagree. See type stubs.

Luau keyword method names

C++ methods whose names are Luau keywords export as {name}ToLua in stubs and runtime bindings. The rename helper lives in tools/luau_codegen/emit/luau_types/method_types.py (lua_export_name). It applies in emit/luau_types/classes.py, emit/luau_types/factories.py, and emit/bindings/class_file.py.

Hook target IDs still use the original C++ name. Example: hook geode.cocos2d.CCRenderTexture:end/0, call rt:endToLua() from scripts.

If broma already defines {name}ToLua, codegen skips renaming {name} to avoid duplicate keys.

Value structs

Value structs are plain-data C++ structs marshalled as Lua tables in both directions. model/value_types.py is the single registry. Each ValueTypeSpec drives type classification, Luau stub emission, and generated check/push C++. convert/type_map.py re-exports VALUE_TYPES and VALUE_CHECK_CXX_TYPES.

Generated check/push headers

emit/types_binding.py writes two headers under build/luauapi-gen/src/framework/stack/:

Header Contents
Types.generated.hpp Primitive cocos structs and simple opt-in structs (numbers, enums, strings, nested non-deferred value types)
Types.generated.containers.hpp Structs that need container helpers, nullable usertypes, opaque handles, or nested deferred members

Defer rule in types_binding.py:

  • Direct defer when any check or push field kind is container, object_nullable, or opaque_nullable
  • Transitive defer when a nested field references another deferred struct
  • Fixpoint over COCOS_VALUE_STRUCTS order in the combined registry

Production include order:

  • Types.hpp includes Types.generated.hpp before Usertype.hpp
  • ContainerTables.hpp includes Types.generated.containers.hpp after the public container helper declarations
  • Recursive container traits are declared before that include.
  • Their leaf operations are defined after it. Any nested path can then call luax::check<T> and luax::push on deferred types.

Host tests define LUAUAPI_HOST_TESTS and skip both generated headers. Production builds require both generated headers.

Handwritten check/push for UIButtonConfig and SmartPrefabResult stay in Types.hpp (not generated).

Two sources of specs

  1. Builtin specs: hand-authored ValueTypeSpec entries in model/value_types.py for types not present in Broma (cocos types such as CCPoint, RGBColor, BlendFunc, plus UIButtonConfig and SmartPrefabResult, whose check/push are handwritten in src/framework/stack/Types.hpp). The FieldKind set covers primitives, enums, nullable objects/opaques, nested value structs, strings, and containers. The generic auto-C++ path emits a statement body for any member expressible as a FieldDescriptor/PushFieldDescriptor.

  2. Derived specs: Geometry Dash state structs (for example GJGameState, GJShaderState, PulseEffectAction, the Saved* checkpoint records). These are not hand-typed. emit/value_struct_specs.py reads each struct's fields from the parsed Broma root and derives the ValueTypeSpec (member kinds, Luau stub, deps) automatically. Specs write to build/luauapi-gen/value_struct_specs.py and install into model/value_struct_specs.py. The repo copy of model/value_struct_specs.py is a stub (empty VALUE_STRUCT_SPECS). It is not the runtime source. Same pattern as delegate_specs.py.

Opt-in list

model/value_struct_gate.py holds VALUE_STRUCT_OPT_IN: a tuple of class names (leaf-first for nested resolution) that the derived-specs generator binds. To add a struct, append its name. Its layout, stub, deps, and C++ check/push regenerate on the next codegen run. No field-level Python to maintain. No drift when Geode bumps bindings.

emit/luau_types/references.py emits export type blocks from the combined registry. See tests/luau_codegen/typemap/test_value_struct_gate.py, tests/luau_codegen/typemap/test_value_struct_specs.py, and tests/luau_codegen/bindings/test_types_binding.py.

Nullable pointer policy

Return stubs use ? on object and opaque pointers. Arg stubs do not. Runtime rejects nil object args unless allow_nil_object (hook overrides only).

Field and container policy

Class fields are filtered in policy/fields.py by bindable_field().

A field is skipped when any of the following apply:

  • Its type cannot be marshalled
  • It is an array or reference
  • It is a function or string pointer
  • It is listed in INACCESSIBLE_FIELDS
  • Its type cannot be marshalled as a SeedValue* field (see below)

SeedValue* fields (for example geode::SeedValueRSV) bind as Lua number. Classification uses kind seed_value. Getters decode with static_cast<int>. Setters assign with operator=(int), not static_cast<SeedValue>(int). All eight SeedValue variant names share this policy.

Skipped fields appear as -- skipped <name>: <reason> comments in the stub.

bindable_field() uses recursive container policy. cc_c_array_view and direct object-element vector_view fields remain getter-only. Opaque-element vector_view fields bind read-write via table assign (clear plus repopulate through assignOpaqueVectorView). For example, CCMoveCNode.m_groupObjects (gd::vector<GroupCommandObject2*>) accepts moveNode.m_groupObjects = { cmd1, cmd2 }. Object-element vector_view stays getter-only until a retain-aware setter exists.

Composite field setters use assignContainerValue instead of whole-container operator=. This avoids the missing _Rb_tree::_M_move_assign in Geode gnustl on Android. The recursive grammar, pointer-grid exception, assignment behavior, and array cap are documented in Recursive containers. See Pair containers and ccCArray read-only fields for their table shapes.

Out-ref and multi-return

Non-void methods may take out-reference container args (gd::vector<T>&, gd::map<...>&, and similar). When the return type is also bound, codegen emits a multi-value Lua return.

Examples:

  • GJBaseGameLayer:registerSpawnRemap(...) returns (number, { ChanceObject }).
  • FMODAudioEngine:getTweenContainer(...) returns { { first: number, second: number, value: FMODSoundTween } } from a pair-key map out-ref.

Policy lives in policy/containers.py and convert/sel_args.py. Emit tests lock the shape in tests/luau_codegen/marshalling/test_out_ref_policy.py.

FMOD binding

FMOD types split into three buckets:

Bucket Examples Luau shape
Value structs FMODMusic, FMODSound, FMODQueuedEffect, FMODSoundTween Table types from VALUE_STRUCT_OPT_IN
Opaque handles FMODSystem, FMODDSP, FMODChannel, FMODChannelGroup @type-only stub classes
Renamed handle FMOD::Sound* FMODSoundHandle (not FMODSound)

FMODSound is now the value struct name. The opaque sound pointer stub is FMODSoundHandle.

Maps with opaque pointer values (for example gd::unordered_map<int, FMOD::Channel*>) bind normally. Opaque values use the handle stub type, not a value struct table.

Intentionally skipped

Some audit one-offs stay skipped on purpose. Do not re-triage them without a new binding design.

Item Reason
ChallengesPage.updateTimers Broma callback method. GD scheduler hook, not mod-facing API. Hard-skipped in policy/filtering.py when m.is_callback.
DailyLevelPage.updateTimers Same as above.
RewardsPage.updateTimers Same as above.
HardStreak.updateStroke Same as above.
MusicDownloadManager.ProcessHttpRequest Same as above.
MDPopup.void Broma-parsed method (not a field). Scanner artifact with return type geode::Function<...>. Real surface is create and setOnClick with working callback marshalling.

Denylist maintenance

model/denylist.py is hand-maintained with five structures:

  • INACCESSIBLE_METHOD_NAMES
  • INACCESSIBLE_METHODS
  • INACCESSIBLE_CLASSES
  • PREFERRED_OVERLOADS
  • BINDABLE_CONSTRUCTORS

INACCESSIBLE_* entries drop methods and classes from Luau bindings. Put mod-facing rationale in the reference docs, not here.

Entries are conservative. test_denylist.py fails if any entry no longer resolves after a GD or SDK bump.

Constructor binding

Constructors and destructors are rejected in policy/filtering.py. BINDABLE_CONSTRUCTORS is a narrow opt-in when create() is filtered out and Lua has no other way to construct the type. Each listed signature becomes a synthetic static new(...) factory that calls autorelease().

CLI exit codes

Code Meaning
2 Bad arguments, missing required directories, or Python below 3.11
3 No Broma classes found after parsing
4 I/O error reading inputs or writing outputs
5 Unexpected exception during emit
6 Ambiguous overloads remain after building the emit plan

Delegates

Delegate pointer args bind only when a spec exists in delegate_specs.py.

Collection (emit/delegates.py collect()):

  1. parse/broma_delegates.py scans *.bro for *Delegate and *Protocol classes. It keeps virtual methods whose args and returns classify.
  2. Known cocos delegates merge from COCOS_DELEGATES in emit/delegates.py.
  3. Specs write to build/luauapi-gen/delegate_specs.py and install into model/delegate_specs.py.
  4. LuaDelegates.gen.hpp and LuaDelegates.gen.cpp emit C++ trampolines that call a Lua table.

convert/type_classification.py maps each bound delegate pointer to a Lua table type from the spec. Methods listed in SKIP or SKIP_METHODS are omitted.

The main stamp runs delegate collection before bindings. To run only the delegate step:

PYTHONPATH=tools python -m luau_codegen --emit-delegates --bindings <bindings-dir> --out <gen-dir>

See delegates for how scripts use delegate tables.

Related

Source

  • cmake/Codegen.cmake
  • CMakeLists.txt
  • mod.json
  • tools/luau_codegen/cli/main.py
  • tools/luau_codegen/parse/collect.py
  • tools/luau_codegen/parse/geode_sdk.py
  • tools/luau_codegen/parse/broma_delegates.py
  • tools/luau_codegen/model/free_fn_sources.py
  • tools/luau_codegen/model/value_struct_gate.py
  • tools/luau_codegen/emit/types_binding.py
  • tools/luau_codegen/emit/value_struct_specs.py
  • src/framework/stack/Types.hpp
  • src/framework/stack/ContainerTables.hpp
  • tools/luau_codegen/model/delegate_specs.py
  • tools/luau_codegen/model/value_struct_specs.py
  • tools/luau_codegen/convert/type_classification.py
  • tools/luau_codegen/convert/type_map.py
  • tools/luau_codegen/emit/bindings/geode_enums.py
  • tools/luau_codegen/emit/luau_types/enums.py
  • tools/luau_codegen/emit/delegates.py
  • tools/luau_codegen/emit/metadata.py
  • tools/luau_codegen/emit/audit.py
  • tools/luau_codegen/emit/parity.py
  • tools/luau_codegen/policy/filtering.py
  • tools/luau_codegen/policy/fields.py
  • tools/luau_codegen/model/denylist.py
  • tools/luau_codegen/emit/hooks.py
  • tools/luau_codegen/emit/cxx_templates.py
  • tools/luau_codegen/emit/luau_types/
  • tools/luau_codegen/emit/luau_types/method_types.py
  • tools/luau_codegen/extra_bindings/
  • tools/luau_codegen/convert/marshalling.py
  • tools/luau_codegen/emit/bindings/