Skip to content

feat: allow opt-in for new backends - #409

Draft
b-long wants to merge 23 commits into
masterfrom
feature/replacing-pybindgen
Draft

b-long wants to merge 23 commits into
masterfrom
feature/replacing-pybindgen

Conversation

@b-long

@b-long b-long commented Sep 19, 2026 •

Copy link
Copy Markdown
Member

⚠️ work-in-progress ⚠️

Relates-to: #408

To-do List (imperfect / likely to change)

  • Setup machinery for opting into new backends
  • Setup CI test coverage, so that we can exercise the new backends
  • Start with cffi backend
  • Close the cffi gaps and drop the skip list (see "Notes: cffi" below).
  • Start the pybind11 backend (see "Notes: pybind11" below).

Notes: cffi

  • Work through the 6 skipped tests, in order of difficulty:
    • []byte ↔ bytes, natively through cffi.
    • Makefile and pkg mode support.
    • Complex numbers.
    • Go calling back into Python callbacks, which is the hardest. It needs cffi callbacks, and the GIL and thread handling get tricky.
  • Result: cffi passes the full suite. Then we can widen CI to the full 24-job matrix and remove the skip list.

Notes: pybind11

  • The existing Go shim already calls the CPython C API. So pybind11 (or nanobind) only needs to replace the generated C code, meaning the same recorder approach that emits C++ instead of a cffi module. That should be less work than cffi was, because the Go side stays as it is.
  • It needs a C++ compiler and the pybind11 package on all three operating systems, so CI setup is a bit heavier. There are open questions about the ABI and versioning, pybind11 and nanobind differ in Python version support.
  • Result: three comparable backends, but not yet measured.

@b-long b-long mentioned this pull request Sep 19, 2026
Comment thread main_test.go
Comment on lines +475 to +478
// TODO: its Makefile links _simple against a separate simple_go
// shared library with no rpath, so the result only imports with
// that library's directory on the loader's search path.
t.Skip("the pybindgen backend's Makefile output doesn't import as-is")

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix before merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant