From 4d2fe99d0953bc26e8ec54eebb0d35158c7747ae Mon Sep 17 00:00:00 2001 From: Marco Casaroli Date: Tue, 25 Aug 2026 11:21:35 +0200 Subject: [PATCH] Documentation, boards/rp23xx: Describe FDPIC and add a test configuration. Documentation/components/fdpic.rst covers what an FDPIC module is and what it adds over the position independent ELF support already in the tree, how the loader places one, where shared libraries come from and how they are found, which entry points resolve a function descriptor and the rules for adding another, and how to build a module and a library. A comparison table places it against NXFLAT and PIC ELF, and the reference section records the object layout and the relocations. The tools page added earlier in this series now points at it. pimoroni-pico-2-plus:xipfs-fdpic is the configuration the series was tested on: xipfs on the board's QSPI flash, the ELF loader with CONFIG_FDPIC, and apps/examples/fdpicxip with apps/testing/fs/xipfs. CONFIG_DEFAULT_TASK_STACKSIZE is 4096 there rather than the rp23xx default of 2048. Both sides of the loader need it: a module that calls into the firmware's printf family overflows 2048, and with no MPU that is a lockup rather than a diagnostic. CONFIG_ELF_STACKSIZE follows it, and apps/testing/fs/xipfs sizes its own task from it. Assisted-by: Claude Opus 5 (1M context) Signed-off-by: Marco Casaroli --- Documentation/components/fdpic.rst | 420 ++++++++++++++++++ Documentation/components/index.rst | 1 + .../boards/pimoroni-pico-2-plus/index.rst | 15 + .../configs/xipfs-fdpic/defconfig | 66 +++ 4 files changed, 502 insertions(+) create mode 100644 Documentation/components/fdpic.rst create mode 100644 boards/arm/rp23xx/pimoroni-pico-2-plus/configs/xipfs-fdpic/defconfig diff --git a/Documentation/components/fdpic.rst b/Documentation/components/fdpic.rst new file mode 100644 index 0000000000000..1abaca3f3f797 --- /dev/null +++ b/Documentation/components/fdpic.rst @@ -0,0 +1,420 @@ +.. _fdpic: + +============= +FDPIC Modules +============= + +Overview +======== + +An FDPIC module is an ELF shared object whose read-only and writable +segments are placed independently of one another. NuttX uses the +read-only segment where it already lies on the media and never copies +it; only the writable segment is copied to RAM, once per running +instance. A module's code and ``.rodata`` therefore cost no RAM at all, +and several instances of one module share them. + +FDPIC is not a separate binary format and has no loader of its own. An +object announces itself in its OS/ABI byte, +``e_ident[EI_OSABI] == ELFOSABI_ARM_FDPIC`` (65), which ``readelf -h`` +reports as *OS/ABI: ARM FDPIC*, and the ELF loader takes it from there. +Everything else -- ``exec()``, ``posix_spawn()``, ``dlopen()``, the +symbol table -- is the ordinary ELF path. + +What FDPIC adds over the position independent ELF support already in the +tree is a function pointer that carries its own data base. That is what +lets a module be called back on a thread it did not create, and what +lets a module and the libraries it uses hold distinct data bases at the +same time. + +Function descriptors +-------------------- + +Code reaches its own data through a base register -- **r9** on ARM -- +holding the address of that object's GOT. Because code and data are +placed independently, a bare code address is not enough to call a +function: the callee needs its data base too. FDPIC therefore +represents a function pointer as a two word *descriptor*: + +=========== ============================================================== +Word Contents +=========== ============================================================== +``entry`` Code address, including its Thumb bit +``got`` Data base to install in the PIC base register before + branching +=========== ============================================================== + +Building those descriptors is most of what relocation does. Because +each one names its own base, a pointer handed to the base firmware +carries everything needed to call back into the module later, from any +thread. + +A module links against nothing. libc and everything else are undefined +imports, resolved at load time against the globally registered symbols +first, then any shared libraries the module names, then the symbol table +``exec()`` supplied. + +Placement +--------- + +The loader asks the filesystem where the file lies on its media. Two +mechanisms exist and they are not interchangeable: + +* ``XIPFSIOC_PIN`` is for a filesystem that can move a file's blocks. It + returns an address together with a pin that holds the extent still, and + the pin is given back with ``XIPFSIOC_UNPIN`` when the module is + unloaded. :doc:`XIPFS ` is the one in tree. + +* ``FIOC_XIPBASE`` is for a filesystem whose layout never changes, which + has nothing to hold and answers with a bare address. ROMFS and TMPFS + are those. + +The pin is asked for first, because a filesystem that needs one is not +safe without it. The loader asks for a pin only if it can hold one, +which is the flat build, or the pin would stay for ever. + +A filesystem that answers neither is still usable. The loader then +copies the text to RAM, as it does for any other module. The module +loses the shared text and the flash saving, but it runs. + +The writable segment is allocated and copied per instance, and a pool of +function descriptors is reserved behind it for the relocations that ask +the loader to manufacture one. When the task starts, +``up_initial_state()`` installs the object's data base -- ``DT_PLTGOT``, +or the GOT immediately after ``PT_DYNAMIC`` in an object with no +imports -- into the PIC base register. + +Shared libraries +---------------- + +A module may name shared libraries in ``DT_NEEDED``. Each is loaded +during relocation by calling ``dlopen()`` on the name, and the module's +undefined symbols are then bound against that library's exports. This +requires ``CONFIG_LIBC_DLFCN``; without it, a module carrying +``DT_NEEDED`` is refused, because there is no way to bring in what it +asks for. ``CONFIG_LIBC_ELF_MAXNEEDED`` caps how many one module may +name. + +Because ``dlopen()`` does the work, libraries are found the way it finds +them: an absolute path is used as given, and a bare name is searched for +along ``LD_LIBRARY_PATH``, which needs ``CONFIG_LIBC_ENVPATH`` and is +seeded from ``CONFIG_LDPATH_INITIAL``. + +A library lands in the module registry, which holds one instance per +name, so its data is shared by everything that opens it. A module +started with ``exec()`` is different: that path loads a fresh copy each +time, so two running instances of one module have separate data while +sharing one copy of the text in flash. + +Comparison with NXFLAT and PIC ELF +================================== + +All three run position independent code from flash on a target with no +MMU, and all three give several instances of one module a shared +``.text`` with private ``.data``. They differ in what a *pointer* can +express and in what the toolchain has to provide. + +========================= ============== ============== ============= +Property NXFLAT PIC ELF FDPIC +========================= ============== ============== ============= +Format NuttX only ELF ELF +Extra build tools yes none assembler and + linker +Data base per task task object +Shared libraries no no yes +Foreign-thread callback no no yes +Instruction set ARM, Thumb-2 unrestricted Thumb-2 only +========================= ============== ============== ============= + +:ref:`NXFLAT ` is a NuttX-specific format. A module imports +symbols from the base firmware but cannot export any, so shared +libraries are not possible, and the build needs ``mknxflat`` to generate +a thunk, ``ldnxflat`` to link, and one of the ``binfmt/libnxflat`` linker +scripts to place the sections. + +**PIC ELF** needs no extra tools. With ``CONFIG_PIC`` the ELF loader +allocates the writable sections separately and, when the filesystem +answers ``FIOC_XIPBASE``, leaves the read-only ones on the media. Two +limits follow from having one base register per task: a shared object is +loaded as a single allocation, because the distance between its text and +its data is compiled into it, and the data base is installed once per +task, so every object in a task shares one. + +**FDPIC** pays for its descriptors with an ``arm-uclinuxfdpiceabi`` +assembler and linker, and gets back the two things a single register +cannot express. A task or pthread that a module starts inherits the +module's D-Space, so a register would be enough there; a work queue +worker was created at boot and carries no module base, and a descriptor +supplies one, which is how ``SIGEV_THREAD`` notifications reach module +code. + +Requirements +============ + +**An ARM Thumb-2 core.** The boundary is the instruction set, not the +core profile: GCC rejects FDPIC in Thumb-1 mode. + +========================= ========================== ===== +Core Architecture FDPIC +========================= ========================== ===== +Cortex-M3 / M4 / M7 ARMv7-M / ARMv7E-M yes +Cortex-M33 ARMv8-M Mainline yes +Cortex-M0 / M0+ / M23 ARMv6-M / ARMv8-M Baseline no +========================= ========================== ===== + +RISC-V has no FDPIC ABI -- the psABI addendum is an unmerged proposal and +no ``EI_OSABI`` value is assigned -- so a RISC-V target cannot use this. + +**Flash that is memory mapped and executable**, exposed by a filesystem +that answers ``XIPFSIOC_PIN`` or ``FIOC_XIPBASE``. This is what gives +execute in place. Without it the module still loads, but from RAM. + +**An FDPIC assembler and linker.** A stock ``arm-none-eabi`` GCC +compiles correct FDPIC code for both C and C++, but the assembler has to +be in FDPIC mode to accept the relocations that code produces, and only +``arm-uclinuxfdpiceabi`` binutils carry the ``armelf_linux_fdpiceabi`` +emulation the link needs. ``arm-none-eabi-ld``, rather than failing, +marks its output *UNIX - System V*, which the loader will not treat as +FDPIC. + +No distribution packages that target, so build binutils for it -- which +takes about a minute and needs nothing else:: + + configure --target=arm-uclinuxfdpiceabi --prefix=$HOME/fdpic \ + --disable-nls --disable-werror + make && make install + export PATH=$HOME/fdpic/bin:$PATH + +An FDPIC GCC is not needed. + +**The base firmware must reserve r9.** It is not enough for the module +to be well behaved: a firmware routine calling back into module code +arrives with the module's data base in r9 only if the compiler was never +free to allocate that register elsewhere. ``CONFIG_FDPIC`` selects +``CONFIG_PIC``, under which ``arch/arm/src/common/Toolchain.defs`` adds +``--fixed-r9``; see :ref:`nxflat` for why it goes into ``ARCHCFLAGS`` +rather than ``CFLAGS`` and how to check that it arrived. + +Configuration +============= + +``CONFIG_FDPIC`` lives under ``CONFIG_ELF``. A working configuration +also needs a symbol table for modules to import from and a filesystem +that can expose its media:: + + CONFIG_ELF=y + CONFIG_FDPIC=y + CONFIG_LIBC_EXECFUNCS=y + CONFIG_EXECFUNCS_HAVE_SYMTAB=y + CONFIG_EXECFUNCS_SYSTEM_SYMTAB=y + CONFIG_FS_XIPFS=y + +Shared libraries need three more, the last two so that a library can be +named rather than spelled out as an absolute path:: + + CONFIG_LIBC_DLFCN=y + CONFIG_LIBC_ENVPATH=y + CONFIG_LDPATH_INITIAL="/mnt/xipfs" + +``CONFIG_ELF_STACKSIZE`` gives the stack a module runs with. A module +that needs a different one can export an ``nx_stacksize`` symbol, which +the loader prefers when present. + +Building a module +================= + +Select ``CONFIG_FDPIC`` and a module is built by the ordinary in-tree ELF +build: the same ``MODULE = m`` in the same application Makefile as any +other, and the same ``crt0``. Nothing else is needed. + +What the build does differently is add two compiler flags and use a +different linker:: + + arm-none-eabi-gcc -mcpu=cortex-m33 -mthumb -mfdpic -fPIC -Os \ + -fno-builtin -D__NuttX__ -I$NUTTX/include -c mod.c -o mod.o + + arm-uclinuxfdpiceabi-ld -m armelf_linux_fdpiceabi -shared -z now \ + -e _start -T $NUTTX/libs/libc/elf/gnu-elf.ld \ + -o mod $NUTTX/arch/arm/src/crt0.o mod.o + +Only the link needs the FDPIC toolchain. The stock compiler emits correct +FDPIC objects for both C and C++, its assembler included. That linker is in +the NuttX CI image; ``tools/ci/docker/linux/Dockerfile`` shows how it is +built. ``FDPIC_CROSSDEV`` names a different prefix, and the build says so +if it is missing. + +Five flags carry weight: + +* ``-mfdpic`` is stated rather than assumed, so a mis-set toolchain fails + loudly instead of producing a plain ELF the loader will not recognize. + +* ``-fPIC`` is not implied by ``-mfdpic`` on a bare-metal target, and + without it the link emits ``TEXTREL``. Text relocations cannot work + against text executed from read-only flash. + +* ``-shared`` preserves the ``R_ARM_FUNCDESC_VALUE`` relocations for + imported symbols. A PIE link with ``--unresolved-symbols=ignore-all`` + appears to work but degrades every import to ``R_ARM_NONE``, and the + module branches to zero on its first call into the firmware. + +* ``-m armelf_linux_fdpiceabi`` is required: this linker supports several + emulations and will not guess. + +* ``-e _start`` names the entry point. ``crt0.c`` is the module's own + start-up file, the one every other module uses: it walks ``.init_array`` + and then calls ``main``. A shared library is never entered, so it is + linked with ``-e 0`` and without ``crt0``. + +A module links with ``-shared``, so importing something the firmware does +not export links cleanly and fails only on the target, as a bare +``-ENOENT``. Checking the module's undefined symbols against the generated +``libs/libc/exec_symtab.c`` catches that at build time, and is what +``apps/examples/fdpicxip/modules`` does for its own fixtures. + +Building a shared library +------------------------- + +A library is built the same way, with a soname and no entry point, and +the module names it on its link line:: + + arm-uclinuxfdpiceabi-ld -m armelf_linux_fdpiceabi -shared -z now \ + -e 0 -soname libfoo.so -o libfoo.so libfoo.o + + arm-uclinuxfdpiceabi-ld -m armelf_linux_fdpiceabi -shared -z now \ + -e main -o mod.fdpic mod.o libfoo.so + +At run time the library must be reachable under its soname along +``LD_LIBRARY_PATH``. + +Calling back into a module +========================== + +A module's function pointer is the address of a descriptor in its +writable segment. Firmware that stores one and later branches to it +would jump into RAM data, so an entry point that accepts a callback from +a module has to resolve the descriptor first. ``CONFIG_FDPIC`` makes +these do so: + +``qsort``, ``bsearch``, ``pthread_create``, ``signal``/``sigaction``, +``task_create``/``task_create_with_stack``, ``task_spawn``, +``pthread_once``, ``scandir``, and ``mq_notify``/``timer_create`` with +``SIGEV_THREAD``. + +Whether a pointer is a descriptor is decided by reading the PIC base +register: a module's task runs with its data base there, a firmware task +with zero, so a kernel caller is unaffected. + +A new entry point that takes a module callback must resolve it too, under +three rules: + +* **Resolve once, in the innermost common routine.** Resolving twice + treats a code address as a descriptor. ``qsort()`` recurses, so its + public entry resolves and the recursive body does not; ``signal()`` + does not resolve because ``nxsig_action()`` does it for both paths; + ``scandir()`` resolves its filter but not the comparison function it + hands to ``qsort()``. + +* **Exclude sentinel values by hand.** ``fdpic_callback()`` declines to + dereference NULL and nothing else. ``sigaction()`` excludes + ``SIG_IGN``, ``SIG_DFL``, ``SIG_HOLD`` and ``SIG_ERR`` -- the integers + 0, 1, 2 and -1. + +* **A callback on a shared thread needs its base installed.** A + ``SIGEV_THREAD`` notification runs on a work queue worker that carries + no module base, so resolving the entry is not enough. Capture the base + at registration with ``fdpic_base()``, in the module's own context, and + install it around the call with ``fdpic_invoke()``. + +Everywhere else the callback runs in a task that inherited the module's +D-Space, so only the code address needs resolving. + +Limitations +=========== + +**A library's constructors run on the loading task, not on the task that +uses it.** A module that ``exec()`` runs is entered at its own ``crt0``, +which walks ``.init_array`` on the task that runs the module, so its +constructors see that task's group and its own data base. A library is +different: ``dlopen()`` constructs it through ``libelf_insert()``, on +whichever task called the loader, entered through ``fdpic_invoke()`` so +that a global object still reaches its own storage. A library constructor +that reads task-local state, its own pid or its environment, sees that +task's. + +A library named in ``DT_NEEDED`` is constructed before the module that +needs it, because the module's own relocation is what opens it, and +destroyed after, at the last ``dlclose()``. Since the library is one +instance, its constructors run once however many modules name it. + +Destructors are walked from ``DT_FINI_ARRAY`` at unload, in +``libelf_uninit()``, and not from ``crt0``, so they run once whichever way +the object was loaded. + +Reference +========= + +Object layout +------------- + +A linked module already has the layout execute in place needs, with no +linker script:: + + LOAD vaddr 0x00000000 R E .text .rodata .hash .dynsym .dynstr + LOAD vaddr 0x00001244 RW .dynamic .got .data .bss + DYNAMIC DT_PLTGOT -> .got + +``.rodata`` lands in the read-only segment on its own, reached PC +relative or GOT indirect. That matters: in the writable segment it would +be copied to RAM with ``.data``, and most of the saving would evaporate +silently, with everything still working. + +The FDPIC marker is the OS/ABI byte alone. ``e_flags`` reads as an +ordinary ``0x5000000, Version5 EABI``. + +Relocations +----------- + +The static link resolves ``R_ARM_GOT_BREL`` and ``R_ARM_GOTFUNCDESC`` +into the GOT already, so only three types carry work into a linked +module. + +``R_ARM_RELATIVE`` + An address needing its segment's base added. + +``R_ARM_FUNCDESC_VALUE`` + A descriptor the linker has laid out, for the loader to fill in. This + is what a *call* to an imported function produces. When the symbol + resolves to a function in another FDPIC object, both words are copied + from that object's own descriptor, so the callee runs with its own data + base; otherwise the entry is the resolved address and the base is this + object's. + +``R_ARM_FUNCDESC`` + A pointer to a descriptor that does not exist yet, which the loader + manufactures from the pool behind the writable segment. This is what + *taking the address* of a function produces -- a different thing from + calling one, and both can appear for the same symbol. + +Constants, from binutils ``include/elf/arm.h`` and mirrored in +``arch/arm/include/elf.h``: ``R_ARM_GOTFUNCDESC`` 161, +``R_ARM_GOTOFFFUNCDESC`` 162, ``R_ARM_FUNCDESC`` 163, +``R_ARM_FUNCDESC_VALUE`` 164. + +Which table an imported function's descriptor lands in is the linker's +decision: with ``-z now`` imports stay in ``DT_REL``, without it they go +to ``DT_JMPREL``. Both are bound eagerly, so either link works, but the +two are not walked identically. In ``DT_REL`` the word being overwritten +is the addend and is added to the resolved value; in ``DT_JMPREL`` it is +the lazy binding bootstrap and the descriptor is overwritten outright. + +``.rofixup`` is skipped. It is the self-relocation list a *static* +executable's ``crt0`` walks to find its own GOT. A module's ``crt0`` does +not: the loader supplies the data base, in the PIC base register, before +the module is entered. + +Exported functions +------------------ + +A function exported by a module or library is published to ``dlsym()`` +as a descriptor rather than a code address, taken from the same pool, so +that an FDPIC caller can branch through what it gets back. diff --git a/Documentation/components/index.rst b/Documentation/components/index.rst index edd1415ede23d..7c13b4c1b0109 100644 --- a/Documentation/components/index.rst +++ b/Documentation/components/index.rst @@ -15,6 +15,7 @@ case, you can head to the :doc:`reference <../reference/index>`. concurrency/index.rst iterable_sections.rst drivers/index.rst + fdpic.rst nxflat.rst nxgraphics/index.rst paging.rst diff --git a/Documentation/platforms/arm/rp23xx/boards/pimoroni-pico-2-plus/index.rst b/Documentation/platforms/arm/rp23xx/boards/pimoroni-pico-2-plus/index.rst index 8269cc7788c05..6c4045f7ada64 100644 --- a/Documentation/platforms/arm/rp23xx/boards/pimoroni-pico-2-plus/index.rst +++ b/Documentation/platforms/arm/rp23xx/boards/pimoroni-pico-2-plus/index.rst @@ -177,6 +177,21 @@ xipfs XIPFS mounted on the on-board flash, with the ``xipfs`` command and the XIPFS test suite. +xipfs-fdpic +----------- + +Same as ``xipfs``, plus the FDPIC module loader and the +``fdpicxip`` demo, so the ``fdpic`` and ``reject`` sections of the XIPFS +test suite have something to run. The demo carries its modules as +committed byte arrays, so nothing beyond the ordinary ARM toolchain is +needed to build it; rebuilding those from source needs +``arm-uclinuxfdpiceabi`` binutils. See :doc:`/components/fdpic`. + +``CONFIG_ELF_STACKSIZE`` is 4096 here rather than the 2048 the rest of +the board's tasks use. A module that calls into the firmware's printf +family overflows 2048, and with no MPU that is a lockup rather than a +diagnostic. + xipfs-nxflat ------------ diff --git a/boards/arm/rp23xx/pimoroni-pico-2-plus/configs/xipfs-fdpic/defconfig b/boards/arm/rp23xx/pimoroni-pico-2-plus/configs/xipfs-fdpic/defconfig new file mode 100644 index 0000000000000..a06c5c1b82618 --- /dev/null +++ b/boards/arm/rp23xx/pimoroni-pico-2-plus/configs/xipfs-fdpic/defconfig @@ -0,0 +1,66 @@ +# +# This file is autogenerated: PLEASE DO NOT EDIT IT. +# +# You can use "make menuconfig" to make any modifications to the installed .config file. +# You can then do "make savedefconfig" to generate a new defconfig file that includes your +# modifications. +# +# CONFIG_NSH_ARGCAT is not set +# CONFIG_NSH_CMDOPT_HEXDUMP is not set +# CONFIG_NSH_DISABLE_DATE is not set +# CONFIG_NSH_DISABLE_LOSMART is not set +# CONFIG_STANDARD_SERIAL is not set +CONFIG_ARCH="arm" +CONFIG_ARCH_BOARD="pimoroni-pico-2-plus" +CONFIG_ARCH_BOARD_COMMON=y +CONFIG_ARCH_BOARD_PIMORONI_PICO_2_PLUS=y +CONFIG_ARCH_CHIP="rp23xx" +CONFIG_ARCH_CHIP_RP23XX=y +CONFIG_ARCH_RAMVECTORS=y +CONFIG_ARCH_STACKDUMP=y +CONFIG_BINFMT_CONSTRUCTORS=y +CONFIG_BOARDCTL_RESET=y +CONFIG_BOARD_LOOPSPERMSEC=10450 +CONFIG_BUILTIN=y +CONFIG_DEBUG_FULLOPT=y +CONFIG_DEBUG_SYMBOLS=y +CONFIG_DEFAULT_TASK_STACKSIZE=4096 +CONFIG_ELF=y +CONFIG_EXAMPLES_FDPICXIP=y +CONFIG_EXAMPLES_HELLO=y +CONFIG_EXECFUNCS_HAVE_SYMTAB=y +CONFIG_EXECFUNCS_SYSTEM_SYMTAB=y +CONFIG_FDPIC=y +CONFIG_FS_PROCFS=y +CONFIG_FS_PROCFS_REGISTER=y +CONFIG_FS_XIPFS=y +CONFIG_FS_XIPFS_FAULT_INJECT=y +CONFIG_HAVE_CXX=y +CONFIG_INIT_ENTRYPOINT="nsh_main" +CONFIG_INIT_STACKSIZE=16384 +CONFIG_LDPATH_INITIAL="/mnt/xipfs" +CONFIG_LIBC_DLFCN=y +CONFIG_LIBC_ELF_HAVE_SYMTAB=y +CONFIG_LIBC_ENVPATH=y +CONFIG_LIBC_EXECFUNCS=y +CONFIG_MTD=y +CONFIG_NFILE_DESCRIPTORS_PER_BLOCK=6 +CONFIG_NSH_BUILTIN_APPS=y +CONFIG_NSH_READLINE=y +CONFIG_RAM_SIZE=532480 +CONFIG_RAM_START=0x20000000 +CONFIG_READLINE_CMD_HISTORY=y +CONFIG_RP23XX_FLASH_MTD=y +CONFIG_RR_INTERVAL=200 +CONFIG_SCHED_HPWORK=y +CONFIG_SCHED_WAITPID=y +CONFIG_SIG_EVTHREAD=y +CONFIG_START_DAY=9 +CONFIG_START_MONTH=2 +CONFIG_START_YEAR=2021 +CONFIG_SYSLOG_CONSOLE=y +CONFIG_SYSTEM_NSH=y +CONFIG_SYSTEM_XIPFS=y +CONFIG_TESTING_FS_XIPFS=y +CONFIG_TESTING_FS_XIPFS_MTD="/dev/rpflash" +CONFIG_UART0_SERIAL_CONSOLE=y