diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 851c0df7f..b94112a96 100755 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -352,6 +352,9 @@ jobs: target: esp32 - path: 'components/usb_device/example' target: esp32s3 + - path: 'components/usb_device/msc_example' + target: esp32s3 + command: 'IDF_COMPONENT_MANAGER=0 idf.py build' - path: 'components/usb_device/xinput_example' target: esp32s3 command: 'IDF_COMPONENT_MANAGER=0 idf.py build' diff --git a/components/usb_device/CMakeLists.txt b/components/usb_device/CMakeLists.txt index 5b2e4fc9b..8b5d92e3d 100644 --- a/components/usb_device/CMakeLists.txt +++ b/components/usb_device/CMakeLists.txt @@ -3,7 +3,11 @@ idf_component_register( SRC_DIRS "src" # vfs: route_console_to_cdc() registers a VFS device (esp_vfs_register / # esp_vfs_t) to redirect stdout to the CDC interface. - REQUIRES base_component esp_tinyusb vfs + # sdmmc: the public header names sdmmc_card_t (MscMedium::sd_card). + REQUIRES base_component esp_tinyusb sdmmc vfs + # MSC, used only by the source: esp_partition (find the FAT partition), + # wear_levelling (wl_mount), fatfs (esp_vfs_fat_info / ff_diskio_get_drive). + PRIV_REQUIRES esp_partition fatfs wear_levelling ) # X-Input registers a custom TinyUSB application class driver by overriding the diff --git a/components/usb_device/README.md b/components/usb_device/README.md index 42c4dbacb..43ff5c425 100644 --- a/components/usb_device/README.md +++ b/components/usb_device/README.md @@ -27,6 +27,10 @@ Today it can enable, in any combination (subject to the endpoint budget): identity + 0xFF/0xFF/0xFF device class so the host recognizes it). See the [`xinput_example`](xinput_example/). *These are Microsoft's IDs, for emulation / testing of your own device only.* +- An **MSC** (mass storage) function exposing an SD card and/or a FAT partition + in flash as USB drives, shared with the application through an ownership + hand-over (see [Enabling mass storage](#enabling-mass-storage-msc) and the + [`msc_example`](msc_example/)). Interface numbers, endpoint addresses and string indices are allocated *sequentially* as functions are enabled, and the result is checked against the @@ -47,9 +51,9 @@ for back-compatibility. - [Enabling the vendor / WebUSB class](#enabling-the-vendor--webusb-class) - [Enabling the HID class](#enabling-the-hid-class) - [Enabling X-Input (Xbox 360)](#enabling-x-input-xbox-360) + - [Enabling mass storage (MSC)](#enabling-mass-storage-msc) - [Routing the console over CDC](#routing-the-console-over-cdc) - [Endpoint budget (ESP32-S3 USB-OTG)](#endpoint-budget-esp32-s3-usb-otg) - - [Extending with HID / MSC](#extending-with-hid--msc) - [Example](#example) - [Notes](#notes) @@ -62,6 +66,8 @@ for back-compatibility. - **Vendor-specific interface** (class 0xFF): raw bulk IN + bulk OUT byte stream. - **HID interface**: application-supplied report descriptor (built with `hid-rp` in the example) on an interrupt IN endpoint; `write_hid_report()` sends reports. +- **Mass storage (MSC)**: an SD card and/or a wear-levelled FAT flash partition as + USB drives, handed between the application (VFS file access) and the host. - **WebUSB**: BOS + WebUSB URL + MS OS 2.0 descriptors for driverless browser access, with a configurable landing-page URL. - **Console over CDC**: optionally route the ESP console (stdout) to the CDC @@ -139,6 +145,10 @@ Key methods: draining). - `bool is_cdc_connected() const` / `bool is_vendor_connected() const` / `bool is_hid_ready() const`. +- `bool set_msc_owner(size_t lun, MscOwner owner, ...)` — hand an MSC medium to + the application (mounted at its `base_path`) or the host; `msc_owner(lun)`, + `msc_capacity(lun)`, `msc_lun_count()`, `format_msc_medium(lun, ...)` and + `set_msc_event_callback(...)` complete the MSC API. CDC-only preset (`espp::UsbCdc`, unchanged API): `initialize()`, `write()`, `set_receive_callback()`, `is_connected()`. @@ -219,6 +229,88 @@ use **separate endpoint numbers**, and the DMA report buffers are word-aligned, the ESP32-S3 DWC2 requires. See `include/xinput.hpp` for the report/`GamepadState` API and the button/axis layout. +## Enabling mass storage (MSC) + +The MSC function exposes up to **two media** as USB drives: an SD card and/or a +FAT data partition in flash (accessed through wear levelling). It is built on +esp_tinyusb's MSC storage backend, which provides the SCSI handling, so enable it +in sdkconfig (the [`msc_example`](msc_example/) does): + +``` +CONFIG_TINYUSB_MSC_ENABLED=y +# flash media: 4 KiB wear-levelling sectors (one flash erase block) and an MSC +# buffer that holds one +CONFIG_WL_SECTOR_SIZE_4096=y +CONFIG_TINYUSB_MSC_BUFSIZE=4096 +``` + +**Flash media and speed.** esp_tinyusb erases a range before writing it, and NOR +flash erases in 4 KiB blocks. With 4096-byte wear-levelling sectors a host write +costs one erase and one write. With 512-byte sectors each one needs a +read-modify-erase of its block: 4 erases in `CONFIG_WL_SECTOR_MODE_SAFE`, or 1 in +`CONFIG_WL_SECTOR_MODE_PERF`, which loses the whole block if the chip resets +mid-erase. A small file edit from a host can then take seconds. 4 KiB sectors cost +4 KiB of RAM per mounted volume and open file, and every file takes at least 4 KiB +of space; the host sees 4 KiB logical sectors (fine for current macOS, Linux, +Windows). **SD cards bypass all of this**: their controller does its own +erase-block management, so sectors are written directly with no ESP-side wear +levelling. See the [`msc_example` README](msc_example/README.md#flash-write-speed-and-sector-size) +for the full comparison. + +**Ownership.** A medium belongs to one side at a time, so the firmware and a PC +never write the same FAT volume at once: + +- While the **application** owns it, the FAT volume is mounted at the medium's + `base_path` and you use ordinary file APIs (`fopen`, `std::fstream`, + `std::filesystem`). A connected host sees the drive as "no medium". +- While the **host** owns it, `base_path` is unmounted (files you had open there + become invalid) and the PC sees the volume. + +With `MscFunction::auto_handover` (the default) the host takes the media when it +mounts the device, and the application gets them back when the host ejects the +drive or the device is detached. To finish application I/O before any host can +take a medium, set `Config::connect_on_initialize = false` and call `connect()` +when done (the `msc_example` does). The hand-over covers **all media at once**: +esp_tinyusb ignores which drive was ejected, so ejecting either returns both. +Taking a medium from an attached host with `set_msc_owner()` is refused +(`device_or_resource_busy`) because host writes already queued could land under +the application's volume: eject the drive on the host first. Turn it off to decide yourself with +`set_msc_owner(lun, MscOwner::Host / App)` — for example only expose an SD card +while a "USB drive" screen is shown. `msc_owner()`, `msc_capacity()` and +`MscFunction::on_event` (hand-over started / done / failed, format required) +report the state; the event callback runs in the TinyUSB task for host-driven +hand-overs, so act on it from your own task. + +```cpp +espp::UsbDevice::MscMedium card; +card.type = espp::UsbDevice::MscMedium::Type::SdCard; +card.sd_card = sd_card; // an initialized sdmmc_card_t* (SDMMC or SDSPI host) +card.base_path = "/sdcard"; // do NOT also esp_vfs_fat_*_mount() the card yourself + +espp::UsbDevice::MscMedium flash; +flash.type = espp::UsbDevice::MscMedium::Type::FlashPartition; +flash.partition_label = "storage"; // a `data, fat` partition +flash.base_path = "/data"; +flash.volume_label = "MY DATA"; // drive name on the host; needs CONFIG_FATFS_USE_LABEL=y + +espp::UsbDevice::MscFunction msc; +msc.media = {card, flash}; // LUN 0 and LUN 1 +cfg.msc = msc; +``` + +Limits, all from esp_tinyusb's backend: at most one SD card and one flash +partition; SD card media need a target with an SDMMC host peripheral (ESP32-S3 / +-P4), even for an SPI-wired card; the SCSI inquiry strings are esp_tinyusb's +fixed ones. **Formatting** (`format_if_unformatted` / `format_msc_medium()`) runs +on FatFs drive 0 rather than the medium's own drive, so only use it when no other +FAT volume is mounted on the device. A host can only read FAT, so a LittleFS (or +SPIFFS) partition cannot be exposed as a drive — use a FAT partition for storage +you want to share with a PC. + +Destroying the `UsbDevice` releases the media: an application-owned medium's +`base_path` is unmounted (an SD card stays initialized, but you must mount it +again if the application still needs its files). + ## Routing the console over CDC When the native USB port is handed to TinyUSB for a vendor / HID / XInput @@ -271,7 +363,7 @@ consumes: | Vendor / WebUSB | 1 (bulk-IN) | 1 (bulk-OUT) | | HID | 1 (interrupt-IN) | 0 or 1 (optional interrupt-OUT) | | X-Input (Xbox 360)| 1 (interrupt-IN) | 1 (interrupt-OUT) | -| MSC (future) | 1 (bulk-IN) | 1 (bulk-OUT) | +| MSC | 1 (bulk-IN) | 1 (bulk-OUT) | This is why the device is **selectable** ("not all at once"). Combinations that fit comfortably: CDC+Vendor (3 IN / 2 OUT, used by the example), CDC+Vendor+HID, @@ -279,21 +371,10 @@ CDC+Vendor+MSC. Enabling CDC+Vendor+HID+MSC reaches 5 IN endpoints — at the ha limit, not recommended. `initialize()` returns `std::errc::value_too_large` if the IN or OUT budget is exceeded. -## Extending with MSC - -The **HID** function is implemented (see "Enabling the HID class" above). -`espp::UsbDevice::Config` still reserves a `std::optional` slot for an -`MscFunction` as a documented extension point; it is not implemented yet, and -enabling it today makes `initialize()` fail with -`std::errc::function_not_supported`. When implemented it slots into the same -sequential allocator: MSC appends one interface (SCSI + storage -read/write/capacity callbacks) claiming a bulk IN + bulk OUT endpoint, exactly -as HID appends one interface claiming an interrupt-IN endpoint (plus an optional -interrupt-OUT). - ## Example -See `example/` for a full project that wires a **composite CDC + Vendor/WebUSB** +See [`msc_example/`](msc_example/) for a USB drive backed by a flash FAT partition, +[`xinput_example/`](xinput_example/) for an Xbox 360 controller, and `example/` for a full project that wires a **composite CDC + Vendor/WebUSB** `espp::UsbDevice` to the transport-agnostic `espp::OdriveAscii` protocol server. Both interfaces feed the same server (RX from either interface → `process_bytes` → response written back out the same interface), while the log console stays on diff --git a/components/usb_device/idf_component.yml b/components/usb_device/idf_component.yml index 7d952a4ab..2d310f588 100644 --- a/components/usb_device/idf_component.yml +++ b/components/usb_device/idf_component.yml @@ -1,6 +1,6 @@ ## IDF Component Manager Manifest File license: "MIT" -description: "Composable native USB device (esp_tinyusb): CDC-ACM + vendor-specific/WebUSB with configurable VID/PID for ESP-IDF" +description: "Composable native USB device (esp_tinyusb): CDC-ACM, vendor-specific/WebUSB, HID, X-Input and mass storage (MSC) with configurable VID/PID for ESP-IDF" url: "https://github.com/esp-cpp/espp/tree/main/components/usb_device" repository: "git://github.com/esp-cpp/espp.git" maintainers: @@ -9,6 +9,7 @@ documentation: "https://esp-cpp.github.io/espp/buses/usb_cdc.html" examples: - path: example - path: xinput_example + - path: msc_example tags: - cpp - Component @@ -22,11 +23,17 @@ tags: - HID - XInput - Gamepad + - MSC + - Mass-Storage + - SD-Card dependencies: idf: version: '>=5.0' espp/base_component: '>=1.0' - espressif/esp_tinyusb: '>=2.0' + # >= 2.0.1: MSC storage-operation multitask protection, and delete_storage() + # refusing to free a storage with host writes still queued (MSC teardown + # relies on both). + espressif/esp_tinyusb: '>=2.0.1' # The X-Input class driver uses the 5-argument usbd_edpt_xfer(..., is_isr) API # (and the usbd_class_driver_t `xfer_isr` member). TinyUSB 0.21 provides that # 5-argument form; earlier releases (e.g. 0.19) still have the 4-argument diff --git a/components/usb_device/include/usb_device.hpp b/components/usb_device/include/usb_device.hpp index 0240a674c..7fbcb9ed7 100644 --- a/components/usb_device/include/usb_device.hpp +++ b/components/usb_device/include/usb_device.hpp @@ -1,6 +1,7 @@ #pragma once #include +#include #include #include #include @@ -12,8 +13,9 @@ #include #include "base_component.hpp" -#include "tinyusb.h" // for tinyusb_event_t (esp_tinyusb is already a REQUIRES dependency) -#include "xinput.hpp" // X-Input (Xbox 360) gamepad state + descriptor helpers +#include "sd_protocol_types.h" // sdmmc_card_t, for MscMedium::sd_card +#include "tinyusb.h" // for tinyusb_event_t (esp_tinyusb is already a REQUIRES dependency) +#include "xinput.hpp" // X-Input (Xbox 360) gamepad state + descriptor helpers namespace espp { @@ -33,9 +35,12 @@ namespace espp { * enabled, and the device checks the result against the USB-OTG endpoint budget * (reporting an error via `std::error_code` if it is exceeded). * - * The design also leaves room for an **MSC** function to be added later without - * changing the descriptor-building model (see `MscFunction` below and the - * endpoint-budget table in the README). + * It can also enable an **MSC** (mass storage) function exposing up to two + * media -- an SD card and/or a wear-levelled FAT partition in flash -- as USB + * drives. Each medium is owned by one side at a time: the application reads and + * writes files through the VFS at its `base_path`, or the USB host sees the FAT + * volume; ownership moves to the host when it mounts the device and back to the + * application when it ejects or disconnects (see `MscFunction`). * * The VID/PID and manufacturer / product / serial strings are configurable so a * device can advertise its own identifiers (e.g. ODrive-like) on a link that is @@ -208,16 +213,116 @@ class UsbDevice : public BaseComponent { receive_callback_fn on_rumble{nullptr}; }; + /// @brief Which side currently has an MSC medium. A medium belongs to exactly + /// one side: while the host has it, the application's `base_path` is + /// unmounted (open files there become invalid); while the application + /// has it, the host sees the drive as "no medium". + enum class MscOwner : uint8_t { + Host = 0, ///< Exposed to the USB host as a drive. + App, ///< Mounted at MscMedium::base_path for the application (fopen, std::filesystem). + }; + + /// @brief Storage events reported through MscFunction::on_event / + /// set_msc_event_callback(). + enum class MscEvent : uint8_t { + OwnerChangeStarted, ///< A hand-over between application and host is starting. + OwnerChanged, ///< The hand-over completed; `owner` is the new owner. + OwnerChangeFailed, ///< The hand-over failed (e.g. the FAT volume could not be mounted). + FormatRequired, ///< The medium has no FAT filesystem and format_if_unformatted is off. + FormatFailed, ///< Formatting the medium failed. + }; + + /// @brief MSC storage event callback: the medium index (LUN), the event, and + /// the owner at the time of the event: the previous owner for + /// OwnerChangeStarted and OwnerChangeFailed (the side that still has + /// the medium), the new one for OwnerChanged. Runs in the TinyUSB device task for + /// host-driven hand-overs (mount / eject / disconnect) and in the + /// caller's task for set_msc_owner(); keep it short and do not call + /// set_msc_owner() from it. + using msc_event_callback_fn = std::function; + + /** + * @brief One medium exposed by the MSC function (one LUN). + * + * The host only understands FAT, so the medium carries a FAT volume: an SD + * card, or a FAT data partition in flash (accessed through wear levelling). + * esp_tinyusb supports at most one medium of each type. + */ + struct MscMedium { + /// @brief The kind of storage behind this LUN. + enum class Type : uint8_t { + SdCard, ///< An already-initialized SD/MMC card (SDMMC or SDSPI host): `sd_card`. + FlashPartition, ///< A FAT data partition in flash, by label: `partition_label`. + }; + Type type{Type::FlashPartition}; /**< Which storage backs this LUN. */ + /** For Type::SdCard: a caller-owned card initialized with sdmmc_card_init() on + * an SDMMC or SDSPI host. Must outlive the UsbDevice. Do not pass the card + * from esp_vfs_fat_sdmmc_mount() / esp_vfs_fat_sdspi_mount(): the matching + * esp_vfs_fat_sdcard_unmount() frees it. Requires a target with an SDMMC host + * peripheral (e.g. ESP32-S3, ESP32-P4), even when the card is on SPI. */ + sdmmc_card_t *sd_card{nullptr}; + /** For Type::FlashPartition: label of a `data, fat` partition. The device + * mounts wear levelling on it and unmounts it on destruction. */ + std::string partition_label{"storage"}; + /** VFS path where the application sees the files while it owns the medium. + * Must be unique per medium, and must not already be mounted by the app + * (unmount your own esp_vfs_fat mount of the card first). */ + std::string base_path{"/msc"}; + int max_files{5}; /**< Files the application may keep open at once. */ + /** FAT volume label: the name the host shows for the drive (up to 11 + * characters; FAT stores it upper-case). Written at initialize() and after + * format_msc_medium() when it differs from the medium's current label. + * Empty = leave the label alone. Requires CONFIG_FATFS_USE_LABEL=y. */ + std::string volume_label{}; + /** Format the medium as FAT when it is handed to the application and has no + * filesystem. Off by default: an unformatted medium raises + * MscEvent::FormatRequired instead. + * @warning esp_tinyusb formats FatFs drive 0 rather than this medium's own + * drive. Only enable this (or call format_msc_medium()) when no + * other FAT volume is mounted on the device, or it may format that + * volume instead. */ + bool format_if_unformatted{false}; + /** Owner right after initialize(). With auto_handover a host that is (or + * becomes) connected takes the medium when it mounts the device. */ + MscOwner initial_owner{MscOwner::App}; + }; + /** - * @brief (Future) MSC (mass storage) function extension point. Not implemented yet. + * @brief MSC (USB mass storage) function: exposes up to two media as USB drives. + * + * Consumes 1 bulk IN + 1 bulk OUT endpoint. Built on esp_tinyusb's MSC storage + * backend, which provides the SCSI handling, so it requires + * `CONFIG_TINYUSB_MSC_ENABLED=y`; a flash partition additionally needs + * `CONFIG_TINYUSB_MSC_BUFSIZE >= CONFIG_WL_SECTOR_SIZE`. Prefer 4096-byte wear + * levelling sectors with a 4096-byte MSC buffer: each host write is then one + * flash erase + write, while 512-byte sectors need a read-modify-erase of the + * 4 KiB block per sector (slow in the power-safe mode, and + * `CONFIG_WL_SECTOR_MODE_PERF` loses the block on a reset mid-erase). SD cards + * are written directly, with no wear-levelling layer. * - * An MSC function consumes 1 bulk IN + 1 bulk OUT endpoint and requires SCSI + - * storage callbacks (read10 / write10 / inquiry / capacity). Enabling it today - * makes initialize() fail with `std::errc::function_not_supported`. + * Ownership: with `auto_handover` (the default) the media move to the host when + * the host mounts (configures) the device, and back to the application when the + * host ejects a drive or the device is detached. The hand-over is for ALL media + * at once: esp_tinyusb ignores which drive was ejected, so ejecting either one + * returns both to the application (the other drive disappears from the host too). Turn it off to + * decide yourself with set_msc_owner() (e.g. only expose the card while a "USB drive mode" screen + * is shown). Either way, never let the application and the host write the same volume at once -- + * that is what the ownership model prevents. */ struct MscFunction { - std::string interface_name{"espp MSC"}; - // Future: SCSI inquiry strings + read/write/capacity callbacks. + std::string interface_name{"espp MSC"}; /**< MSC interface string descriptor. */ + std::vector media{}; /**< One or two media (LUN 0, LUN 1). */ + bool auto_handover{true}; /**< Host takes all media on mount; the app gets all of them + back on any eject / detach. */ + msc_event_callback_fn on_event{nullptr}; /**< Optional storage event callback. */ + }; + + /// @brief Size of an MSC medium. + struct MscCapacity { + uint32_t sector_count{0}; ///< Number of sectors. + uint32_t sector_size{0}; ///< Bytes per sector. + /// @brief Total size in bytes. + uint64_t bytes() const { return static_cast(sector_count) * sector_size; } }; /** @@ -237,12 +342,16 @@ class UsbDevice : public BaseComponent { uint16_t max_power_ma{100}; /**< bMaxPower in the configuration descriptor, in mA; clamped to 500 and rounded up to the next 2 mA unit. */ bool remote_wakeup{true}; /**< Advertise remote wakeup in the configuration attributes. */ + /** Attach to the bus (enable the D+ pull-up) at the end of initialize(). Set + * false to stay invisible to the host until connect() -- e.g. to finish + * application file I/O on an MSC medium before a host can take it. */ + bool connect_on_initialize{true}; std::optional cdc{}; /**< Enable a CDC-ACM function. */ std::optional vendor{}; /**< Enable a vendor-specific / WebUSB function. */ std::optional hid{}; /**< Enable a HID function. */ std::optional xinput{}; /**< Enable an X-Input (Xbox 360) function. */ - std::optional msc{}; /**< (Future) enable an MSC function. */ + std::optional msc{}; /**< Enable an MSC (mass storage) function. */ espp::Logger::Verbosity log_level{espp::Logger::Verbosity::WARN}; /**< Logger verbosity. */ }; @@ -255,6 +364,9 @@ class UsbDevice : public BaseComponent { /** * @brief Uninstalls the enabled functions and the TinyUSB driver if initialized. + * @note MSC media are released too: an application-owned medium's `base_path` + * is unmounted, flash partitions are unmounted from wear levelling, and + * an SD card is left initialized (the caller owns it) but not mounted. */ ~UsbDevice(); @@ -422,6 +534,66 @@ class UsbDevice : public BaseComponent { /// new input report (no report in flight). bool is_xinput_ready() const; + /** + * @brief Hand an MSC medium to the application or the USB host. + * @param lun Medium index (position in MscFunction::media). + * @param owner New owner. Handing it to the App mounts the FAT volume at the + * medium's `base_path`; handing it to the Host unmounts it there first. + * @param[out] ec Set on failure: MSC not enabled / not initialized + * (`not_connected`), bad index (`invalid_argument`), the medium has no + * FAT filesystem (`no_such_device`, see format_msc_medium()), the host + * is attached and still has the medium (`device_or_resource_busy`, see + * below), or the volume could not be mounted / unmounted (`io_error`). + * @return true if `owner` now has the medium. + * @note Taking a medium from an attached host is refused: esp_tinyusb accepts + * host writes and runs them later, without re-checking ownership, so a + * write already queued could land under the application's mounted FAT + * volume. Have the host eject the drive (auto_handover then returns it), + * or detach / destroy the device, first. Handing a medium to the host is + * always allowed. + * @note Blocks for the mount / unmount. Call it from an application task, not + * from a USB callback. With auto_handover, the next host mount / eject / + * detach still moves the medium automatically -- and a host mount or + * eject that happens during this call races it, so turn auto_handover + * off if the application drives ownership itself. + */ + bool set_msc_owner(size_t lun, MscOwner owner, std::error_code &ec); + + /// @brief Convenience overload of set_msc_owner() that ignores errors. + bool set_msc_owner(size_t lun, MscOwner owner); + + /// @brief Who currently has an MSC medium (nullopt if MSC is not enabled / + /// initialized or the index is out of range). An unformatted medium + /// waiting for format_msc_medium() reports App, with nothing mounted. + std::optional msc_owner(size_t lun) const; + + /// @brief Size of an MSC medium (nullopt if MSC is not enabled / initialized + /// or the index is out of range). + std::optional msc_capacity(size_t lun) const; + + /// @brief Number of MSC media (LUNs); 0 if MSC is not enabled / initialized. + size_t msc_lun_count() const; + + /** + * @brief Create a FAT filesystem on an MSC medium that has none (e.g. after + * MscEvent::FormatRequired), and mount it for the application. + * @param lun Medium index. + * @param[out] ec Set on failure: MSC not enabled / not initialized, bad index, + * the application does not own the medium (`operation_not_permitted`), a + * filesystem already exists (`file_exists`), every FatFs drive slot is in + * use (`device_or_resource_busy`), or formatting failed (`io_error`). + * @return true if the medium was formatted. + * @warning See MscMedium::format_if_unformatted: esp_tinyusb formats FatFs + * drive 0, so only use this when no other FAT volume is mounted. + * @note With auto_handover, the USB connection is dropped for the duration of + * the format (and restored after) so a host attaching mid-format cannot + * take the medium while esp_tinyusb is formatting it. + */ + bool format_msc_medium(size_t lun, std::error_code &ec); + + /// @brief Set or replace the MSC storage event callback (nullptr to detach). + void set_msc_event_callback(const msc_event_callback_fn &cb); + /// @brief Set or replace the CDC receive callback (nullptr to detach). void set_cdc_receive_callback(const receive_callback_fn &cb); @@ -448,6 +620,15 @@ class UsbDevice : public BaseComponent { /// @brief Whether initialize() has completed successfully. bool is_initialized() const; + /// @brief Attach to the bus (enable the D+ pull-up) so a host can enumerate the + /// device. Only needed after Config::connect_on_initialize = false or a + /// disconnect(). @return false if not initialized. + bool connect(); + + /// @brief Detach from the bus (disable the D+ pull-up): the host sees the + /// device unplugged. @return false if not initialized. + bool disconnect(); + /// @brief Whether the CDC function is enabled and a host has asserted DTR. bool is_cdc_connected() const; @@ -509,6 +690,53 @@ class UsbDevice : public BaseComponent { /// function is not enabled). Used by the write path / readiness check. uint8_t xinput_in_endpoint() const; + /// @brief Internal: route an esp_tinyusb MSC storage event (from the TinyUSB + /// task or the task calling set_msc_owner()) to the event callback. + /// @param storage The esp_tinyusb storage handle the event refers to. + /// @param event The translated event. + /// @param owner The owner the event refers to. + void handle_msc_event(const void *storage, MscEvent event, MscOwner owner); + + /// @brief Internal: hand an MSC medium over and confirm the result against the + /// VFS (esp_tinyusb's setter records the requested owner even when the + /// mount / unmount failed, and not every failure raises an event). + bool hand_over_msc(size_t index, MscOwner owner, std::error_code &ec); + + /// @brief Internal: write MscMedium::volume_label to an application-mounted + /// medium if it differs from the current label. + void apply_msc_volume_label(size_t index); + + /// @brief Internal: install the MSC driver and create the storage objects for + /// the configured media (before the TinyUSB driver is installed, so a + /// host that is already connected finds them on its first mount). + bool init_msc(std::error_code &ec); + + /// @brief Internal: tear down the MSC media (storage objects, wear levelling, + /// MSC driver). Safe to call when none were set up. Call it while no + /// TinyUSB task is running (before the driver is installed, or after + /// quiesce_msc_before_uninstall() + tinyusb_driver_uninstall()). A + /// storage object with host writes still queued cannot be deleted; + /// resources behind it are left in place, not freed. + /// @return true if every medium and the MSC driver were released. + bool deinit_msc(); + + /// @brief Internal: pass barriers through the TinyUSB task until everything + /// queued before the call (unplug / auto-hand-over callbacks, deferred + /// MSC writes and the writes they queue) has run. @return false on + /// timeout (the task did not get through its queue). + bool drain_tinyusb_task(); + + /// @brief Internal: undo what a failed tinyusb_msc_format_storage() left + /// registered (VFS path, FatFs mount, diskio drive @p pdrv). + void clean_up_failed_msc_format(size_t index, uint8_t pdrv); + + /// @brief Internal: detach from the host and wait until the TinyUSB task has + /// run everything already queued (deferred MSC writes, a detach / + /// auto-hand-over callback), so the MSC media can be deleted once the + /// task is stopped. Call before tinyusb_driver_uninstall(). @return false + /// if the task did not get through its queue in time. + bool quiesce_msc_before_uninstall(); + /// @brief Internal: the singleton instance handling the global USB callbacks. static UsbDevice *instance(); @@ -527,6 +755,10 @@ class UsbDevice : public BaseComponent { Config config_; std::atomic initialized_{false}; // read from the TinyUSB task via the write paths + // Whether the application wants the device attached (pull-up on): set by + // initialize() / connect() / disconnect(), so internal detaches (formatting) + // restore the caller's choice instead of forcing the device visible. + std::atomic attached_{false}; std::mutex cb_mutex_; receive_callback_fn on_cdc_receive_; @@ -535,6 +767,7 @@ class UsbDevice : public BaseComponent { receive_callback_fn on_hid_receive_; event_callback_fn on_mount_; event_callback_fn on_unmount_; + msc_event_callback_fn on_msc_event_; // Preallocated RX scratch buffers (sized in initialize()) so the TinyUSB-task // RX handlers stay allocation-free (no heap churn on the hot path). diff --git a/components/usb_device/msc_example/CMakeLists.txt b/components/usb_device/msc_example/CMakeLists.txt new file mode 100644 index 000000000..e15a28296 --- /dev/null +++ b/components/usb_device/msc_example/CMakeLists.txt @@ -0,0 +1,37 @@ +# The following lines of boilerplate have to be in your project's CMakeLists +# in this exact order for cmake to work correctly +cmake_minimum_required(VERSION 3.20) + +include($ENV{IDF_PATH}/tools/cmake/project.cmake) + +# add only the component directories that we want to use +set(EXTRA_COMPONENT_DIRS + "../../../components/base_component" + "../../../components/format" + "../../../components/logger" + "../../../components/usb_device" +) + +# With the component manager disabled (IDF_COMPONENT_MANAGER=0, e.g. in CI so the +# build does not need the as-yet unpublished espp/* components in the registry), +# esp_tinyusb/tinyusb are not fetched from the registry; add the vendored +# submodule copies under external/ to the search path. esp_tinyusb's CMakeLists +# adds `tinyusb` to its REQUIRES when the manager is off, so both directories +# must be discoverable. +if(DEFINED ENV{IDF_COMPONENT_MANAGER} AND "$ENV{IDF_COMPONENT_MANAGER}" STREQUAL "0") + list(APPEND EXTRA_COMPONENT_DIRS + "../../../external/esp-usb/device/esp_tinyusb" + "../../../external/tinyusb" + ) +endif() + +set( + COMPONENTS + "main esptool_py base_component format logger usb_device esp_tinyusb fatfs" + CACHE STRING + "List of components to include" + ) + +project(msc_example) + +set(CMAKE_CXX_STANDARD 20) diff --git a/components/usb_device/msc_example/README.md b/components/usb_device/msc_example/README.md new file mode 100644 index 000000000..6067e79a7 --- /dev/null +++ b/components/usb_device/msc_example/README.md @@ -0,0 +1,143 @@ +# USB Mass Storage (MSC) Example + +Exposes a FAT partition in the ESP32-S3's flash as a **USB drive** using +`espp::UsbDevice`'s MSC function, and demonstrates the ownership model that lets +the firmware and a PC share one volume safely: + +- While the **application** owns the medium it reads and writes files through the + VFS at `base_path` (`fopen`, `std::fstream`, `std::filesystem`). A connected + host sees the drive as "no medium". +- When a **host** mounts the device it takes the medium: the application's + `base_path` is unmounted and the PC sees the FAT volume. +- When the host **ejects** the drive (or the cable is unplugged) the medium goes + back to the application, with the host's changes. + +On boot the example updates `boots.txt` (a boot counter) and creates `README.txt` +(only if it is missing, so host edits survive reboots) on the +volume and lists its files. Plug the native USB port into a PC: the drive +appears with those files. Add or edit a file, eject the drive, and the device +logs the updated directory listing. + +## Build & flash + +```sh +idf.py -p flash monitor # console is on UART0 (USB-UART adapter) +``` + +The console is on **UART0**: on the ESP32-S3 USB-Serial-JTAG shares the native +USB port's PHY with USB-OTG, which the mass storage interface takes over. + +The example's `sdkconfig.defaults` enables `CONFIG_TINYUSB_MSC_ENABLED`, uses a +custom `partitions.csv` with a 1 MiB `storage` FAT partition, and selects +**4096-byte wear-levelling sectors** with a matching MSC buffer: + +``` +CONFIG_WL_SECTOR_SIZE_4096=y +CONFIG_TINYUSB_MSC_BUFSIZE=4096 # esp_tinyusb requires >= CONFIG_WL_SECTOR_SIZE +``` + +## Flash write speed and sector size + +NOR flash can only be erased in 4 KiB blocks, and esp_tinyusb erases a range +before writing it. How much work one host write costs depends on the +wear-levelling sector size: + +| `CONFIG_WL_SECTOR_SIZE` | Host sector | Flash work per host write | Reset during an erase | +|---|---|---|---| +| **4096** (this example) | 4 KiB | 1 erase + 1 write | loses only the write in progress | +| 512, `WL_SECTOR_MODE_SAFE` | 512 B | read the 4 KiB block, back it up (erase + write), write a transaction record (erase + write), erase the block, restore the other 7 sectors, clear the record (erase), write -- **4 erases** | safe | +| 512, `WL_SECTOR_MODE_PERF` | 512 B | read the block into RAM, erase, restore, write -- 1 erase | **loses the whole 4 KiB block** | + +A flash erase takes tens of milliseconds, and a host editing even a tiny file +writes many sectors: the data, the FAT (often two copies), the directory entry +and timestamps, plus host metadata such as macOS's `._name` AppleDouble files and +`.fseventsd` logs. With 512-byte Safety-mode sectors that adds up to several +seconds for a ~100-byte edit; with 4096-byte sectors it is a fraction of that. + +Trade-offs of 4096-byte sectors: + +- The host sees a drive with **4 KiB logical sectors**. Current macOS, Linux and + Windows handle that; some old or embedded hosts only accept 512-byte sectors. +- FatFs keeps a sector-sized buffer per mounted volume and per open file, so each + costs **4 KiB of RAM** instead of 512 B (and the MSC buffer is 4 KiB). +- A cluster is at least one sector, so every file, however small, occupies at + least **4 KiB on the volume**; a small partition fits fewer files. +- Changing the sector size changes the on-flash layout: **erase the partition** + after switching either way (below). + +To reduce host metadata writes on macOS, create `.fseventsd/no_log` and +`.metadata_never_index` on the volume, and run +`defaults write com.apple.desktopservices DSDontWriteUSBStores -bool true` to stop +`.DS_Store` files. + +### SD cards avoid all of this + +An SD card has its own controller that does erase-block management and wear +levelling internally, so esp_tinyusb writes its 512-byte sectors straight to the +card (`sdmmc_write_sectors()`): no ESP-side wear-levelling layer, no +read-modify-erase and no sector-size choice to make. Host writes then run at the +card's speed (limited mainly by full-speed USB, about 1 MB/s), and an SD card +holds far more than a flash partition. For storage a PC writes to regularly, +prefer an SD card (see below) and keep flash media for small, rarely changed data. + +### Starting over + +A volume written with a different sector size, or damaged by a reset in +Performance mode (a root directory of unreadable entries, missing files), must be +erased so the example can format it again: + +```sh +idf.py erase-flash flash # or: esptool.py erase_region 0x110000 0x100000 +``` + +## Using an SD card instead + +Initialize the card as usual (SDMMC or SDSPI host) but do **not** mount it with +`esp_vfs_fat_*_mount()` — the MSC function mounts it at `base_path` itself — +then pass the card pointer: + +```cpp +espp::UsbDevice::MscMedium card; +card.type = espp::UsbDevice::MscMedium::Type::SdCard; +card.sd_card = sd_card; // sdmmc_card_t* from sdmmc_card_init() +card.base_path = "/sdcard"; +msc.media = {card}; // or {card, flash} for two drives +``` + +SD card media need a target with an SDMMC host peripheral (ESP32-S3 / -P4), even +when the card is wired to SPI. + +## Example Output + +First boot on an ESP32-S3 (the partition has no filesystem yet, so it is +formatted), then a macOS host mounts the drive, adds two files, and ejects it: + +```console +I (342) main_task: Calling app_main() +[MSC/I][0.342]: Starting USB mass storage example +W (342) tinyusb_msc_storage: Mount failed, trying to format the drive +[MSC/I][1.142]: medium 0 now owned by the app +[UsbDevice/I][1.232]: MSC medium 0: volume label set to 'ESPP MSC' +[UsbDevice/I][1.232]: MSC medium 0: storage (1000 KiB) at '/msc', owned by the application +I (1402) TinyUSB: TinyUSB Driver installed on port 0 +[UsbDevice/I][1.402]: Initialized native USB device (VID=0x1209 PID=0x0d32) cdc=false vendor=false hid=false xinput=false msc=1 +[MSC/I][1.422]: volume: 2000 sectors x 512 bytes = 1000 KiB +[MSC/I][2.152]: boot #1 recorded on the volume +[MSC/I][2.152]: Files on the volume: +[MSC/I][2.152]: boots.txt (2 bytes) +[MSC/I][2.152]: README.txt (104 bytes) +[MSC/I][2.152]: Ready. Connect the native USB port to a PC; eject the drive to hand it back. +[MSC/I][12.732]: medium 0 now owned by the host +[MSC/I][206.992]: medium 0 now owned by the app +[MSC/I][207.462]: Files on the volume: +[MSC/I][207.462]: boots.txt (2 bytes) +[MSC/I][207.462]: README.txt (194 bytes) +[MSC/I][207.462]: .fseventsd/ (0 bytes) +[MSC/I][207.462]: ._README.txt (4096 bytes) +[MSC/I][207.472]: .TemporaryItems/ (0 bytes) +[MSC/I][207.472]: test_item_1.md (15 bytes) +[MSC/I][207.482]: some_other_thing.txt (30 bytes) +``` + +(The dot-files are macOS metadata written by the host. The TinyUSB device +descriptor summary esp_tinyusb prints at install is omitted.) diff --git a/components/usb_device/msc_example/main/CMakeLists.txt b/components/usb_device/msc_example/main/CMakeLists.txt new file mode 100644 index 000000000..c21b74693 --- /dev/null +++ b/components/usb_device/msc_example/main/CMakeLists.txt @@ -0,0 +1,5 @@ +idf_component_register( + SRC_DIRS "." + INCLUDE_DIRS "." + REQUIRES usb_device esp_tinyusb fatfs +) diff --git a/components/usb_device/msc_example/main/msc_example.cpp b/components/usb_device/msc_example/main/msc_example.cpp new file mode 100644 index 000000000..8be0241b0 --- /dev/null +++ b/components/usb_device/msc_example/main/msc_example.cpp @@ -0,0 +1,157 @@ +// USB mass storage (MSC) example. +// +// Exposes a FAT partition in the ESP32-S3's flash as a USB drive using +// espp::UsbDevice's MSC function, and shows the ownership model: the +// application reads and writes files through the VFS while it owns the medium, +// the host gets the drive when it mounts the device, and the application gets +// it back when the host ejects it (or the cable is unplugged). +// +// On boot the app writes a boot counter and a README to the volume. Plug the +// native USB port into a PC: the drive appears with those files. Add or edit +// files, then eject the drive: the app lists what it now sees, including the +// host's changes. An SD card works the same way (see the README). + +#include +#include +#include +#include +#include +#include +#include + +#include "logger.hpp" +#include "usb_device.hpp" + +using namespace std::chrono_literals; +using MscOwner = espp::UsbDevice::MscOwner; +using MscEvent = espp::UsbDevice::MscEvent; + +static constexpr const char *kBasePath = "/msc"; + +static void list_files(espp::Logger &logger) { + std::error_code ec; + size_t garbled = 0; + logger.info("Files on the volume:"); + for (const auto &entry : std::filesystem::directory_iterator(kBasePath, ec)) { + std::error_code size_ec; + const auto size = entry.is_regular_file(size_ec) ? entry.file_size(size_ec) : 0; + // A damaged directory (e.g. an erased flash sector) yields names full of 0xFF + // bytes: print them safely and count them instead of sending raw bytes to the + // console. + std::string name = entry.path().filename().string(); + bool printable = true; + for (auto &c : name) { + if (static_cast(c) < 0x20 || static_cast(c) >= 0x7F) { + c = '?'; + printable = false; + } + } + if (!printable) { + ++garbled; + continue; + } + logger.info(" {}{} ({} bytes)", name, entry.is_directory(size_ec) ? "/" : "", size); + } + if (ec) + logger.error("could not list {}: {}", kBasePath, ec.message()); + if (garbled > 0) + logger.warn("{} directory entries are unreadable: the volume is damaged. Erase the storage " + "partition (or reformat the drive from the host) to start clean.", + garbled); +} + +static void write_boot_files(espp::Logger &logger) { + const std::string counter_path = std::string(kBasePath) + "/boots.txt"; + int boots = 0; + if (std::ifstream in(counter_path); in) + in >> boots; + ++boots; + if (std::ofstream out(counter_path, std::ios::trunc); out) + out << boots << "\n"; + else + logger.error("could not write {}", counter_path); + + // Create the README only once: rewriting it every boot would discard edits the + // host made to it. + const std::string readme_path = std::string(kBasePath) + "/README.txt"; + std::error_code exists_ec; + if (!std::filesystem::exists(readme_path, exists_ec)) { + if (std::ofstream readme(readme_path); readme) { + readme << "Written by the espp usb_device msc_example.\n" + << "Add files here, eject the drive, and the device lists them.\n"; + } + } + logger.info("boot #{} recorded on the volume", boots); +} + +extern "C" void app_main(void) { + espp::Logger logger({.tag = "MSC", .level = espp::Logger::Verbosity::INFO}); + logger.info("Starting USB mass storage example"); + + // Hand-overs happen in the TinyUSB task: only note them here, act on them in + // the main loop. + std::atomic app_regained{false}; + + //! [msc_example] + espp::UsbDevice::Config cfg; + cfg.product = "espp MSC Example"; + cfg.log_level = espp::Logger::Verbosity::INFO; + // Stay invisible to the host until the boot files are written: with + // auto_handover a host that enumerates takes the medium immediately, which + // would unmount /msc in the middle of the application's writes. + cfg.connect_on_initialize = false; + + espp::UsbDevice::MscMedium flash; + flash.type = espp::UsbDevice::MscMedium::Type::FlashPartition; + flash.partition_label = "storage"; // `data, fat` partition in partitions.csv + flash.base_path = kBasePath; + flash.volume_label = "ESPP MSC"; // the name the host shows for the drive + // Safe here: this is the only FAT volume on the device (see the header docs). + flash.format_if_unformatted = true; + flash.initial_owner = MscOwner::App; // write the boot files before a host takes it + + espp::UsbDevice::MscFunction msc; + msc.interface_name = "espp MSC Example"; + msc.media = {flash}; + msc.auto_handover = true; // host takes the drive on mount, app gets it back on eject + msc.on_event = [&](size_t lun, MscEvent event, MscOwner owner) { + if (event == MscEvent::OwnerChanged) { + logger.info("medium {} now owned by the {}", lun, owner == MscOwner::App ? "app" : "host"); + if (owner == MscOwner::App) + app_regained = true; + } else if (event == MscEvent::FormatRequired) { + logger.warn("medium {} has no filesystem", lun); + } + }; + cfg.msc = msc; + + espp::UsbDevice usb(cfg); + std::error_code ec; + if (!usb.initialize(ec)) { + logger.error("Failed to initialize USB device: {}", ec.message()); + return; + } + //! [msc_example] + + if (const auto capacity = usb.msc_capacity(0)) + logger.info("volume: {} sectors x {} bytes = {} KiB", capacity->sector_count, + capacity->sector_size, capacity->bytes() / 1024); + + // initialize() handed the medium to the app, which raised OwnerChanged(App): + // that is not a return from the host, so do not list the files twice. + app_regained = false; + + // Still detached, so no host can take the medium: write to it now, then attach. + if (usb.msc_owner(0) == MscOwner::App) { + write_boot_files(logger); + list_files(logger); + } + usb.connect(); + + logger.info("Ready. Connect the native USB port to a PC; eject the drive to hand it back."); + while (true) { + std::this_thread::sleep_for(500ms); + if (app_regained.exchange(false) && usb.msc_owner(0) == MscOwner::App) + list_files(logger); // shows whatever the host added / changed + } +} diff --git a/components/usb_device/msc_example/partitions.csv b/components/usb_device/msc_example/partitions.csv new file mode 100644 index 000000000..6368f0db8 --- /dev/null +++ b/components/usb_device/msc_example/partitions.csv @@ -0,0 +1,6 @@ +# Name, Type, SubType, Offset, Size, Flags +nvs, data, nvs, 0x9000, 0x6000, +phy_init, data, phy, 0xf000, 0x1000, +factory, app, factory, 0x10000, 1M, +# The FAT volume exposed over USB (wear-levelled; 1 MiB minus WL overhead). +storage, data, fat, , 1M, diff --git a/components/usb_device/msc_example/sdkconfig.defaults b/components/usb_device/msc_example/sdkconfig.defaults new file mode 100644 index 000000000..2882cf185 --- /dev/null +++ b/components/usb_device/msc_example/sdkconfig.defaults @@ -0,0 +1,34 @@ +# This example uses the native USB-OTG peripheral, which is only available on the +# ESP32-S3 (also S2 / P4) -- NOT the classic ESP32. +CONFIG_IDF_TARGET="esp32s3" + +CONFIG_ESP_MAIN_TASK_STACK_SIZE=8192 + +# Console on UART0: on the ESP32-S3 USB-Serial-JTAG shares the native USB port's +# PHY with USB-OTG, which TinyUSB takes over for the mass storage interface. Use a +# USB-UART adapter on UART0 for `idf.py monitor`; USB-Serial-JTAG stays the +# secondary console so early-boot logs are visible before TinyUSB starts. +CONFIG_ESP_CONSOLE_UART_DEFAULT=y +CONFIG_ESP_CONSOLE_SECONDARY_USB_SERIAL_JTAG=y + +# Mass storage only: no CDC. +CONFIG_TINYUSB_CDC_ENABLED=n +CONFIG_TINYUSB_CDC_COUNT=0 +CONFIG_TINYUSB_MSC_ENABLED=y + +# Flash FAT volume: 4096-byte wear-levelling sectors, the size of one flash erase +# block, with an MSC buffer to match (esp_tinyusb requires TINYUSB_MSC_BUFSIZE >= +# the WL sector). Each host write is then one erase + one write, and a reset can +# only lose the write in progress. 512-byte sectors are much slower: every one +# needs a read-modify-erase of its 4 KiB block (4 erases in the power-safe mode), +# so a small file edit from a host takes seconds. See the README for trade-offs. +CONFIG_WL_SECTOR_SIZE_4096=y +CONFIG_TINYUSB_MSC_BUFSIZE=4096 +CONFIG_FATFS_LFN_HEAP=y +# lets MscMedium::volume_label name the drive +CONFIG_FATFS_USE_LABEL=y + +CONFIG_PARTITION_TABLE_CUSTOM=y +CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions.csv" +CONFIG_PARTITION_TABLE_FILENAME="partitions.csv" +CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y diff --git a/components/usb_device/src/usb_device.cpp b/components/usb_device/src/usb_device.cpp index 84c211b4e..3e301353a 100644 --- a/components/usb_device/src/usb_device.cpp +++ b/components/usb_device/src/usb_device.cpp @@ -3,6 +3,7 @@ #include #include #include +#include #include #include @@ -32,6 +33,18 @@ // usbd_app_driver_get_cb). `src/device` is a private include of the tinyusb // component, but `src/` is public, so reach it via the `device/` prefix. #include "device/usbd_pvt.h" +// MSC: esp_tinyusb's storage backend (SCSI callbacks, SD card / wear-levelled +// flash media, VFS hand-over). Compiled in only with CONFIG_TINYUSB_MSC_ENABLED. +#if (CFG_TUD_MSC > 0) +#include "diskio_impl.h" // ff_diskio_get_drive: tell "no free FatFs drive" apart when formatting +#include "diskio_sdmmc.h" // ff_diskio_get_pdrv_card: the FatFs drive of an SD medium +#include "diskio_wl.h" // ff_diskio_get_pdrv_wl: the FatFs drive of a flash medium +#include "esp_partition.h" +#include "ff.h" // f_getlabel / f_setlabel +#include "soc/soc_caps.h" +#include "tinyusb_msc.h" +#include "wear_levelling.h" +#endif #include "xinput.hpp" @@ -61,6 +74,10 @@ struct UsbDevice::Callbacks { static const std::optional &vendor_config(UsbDevice *d) { return d->vendor_config(); } + static void msc_event(UsbDevice *d, const void *storage, UsbDevice::MscEvent e, + UsbDevice::MscOwner o) { + d->handle_msc_event(storage, e, o); + } }; } // namespace espp @@ -185,6 +202,9 @@ constexpr TickType_t kUsbWriteDrainPollTicks = pdMS_TO_TICKS(1) > 0 ? pdMS_TO_TI // fit. constexpr uint8_t kMaxInEndpoints = 5; constexpr uint8_t kMaxOutEndpoints = 5; +// esp_tinyusb's MSC storage backend supports two LUNs, and at most one medium of +// each type (its SD card and wear-levelling media are singletons). +constexpr size_t kMaxMscLuns = 2; // The MS OS 2.0 descriptor set length used below (fixed by the registry-property // payload; identical to TinyUSB's webusb_serial example). @@ -203,6 +223,61 @@ void note_tinyusb_task() { s_tinyusb_task.store(xTaskGetCurrentTaskHandle(), std::memory_order_relaxed); } +#if (CFG_TUD_MSC > 0) +// esp_tinyusb raises MSC storage events both on the TinyUSB task (host mount / +// eject / detach hand-overs) and synchronously in whichever task calls into its +// storage API (set_msc_owner(), format, create / delete). Every task currently +// inside such a synchronous call holds a slot here (several tasks may overlap, +// and one task may nest), so the event trampoline can tell the two apart and +// note the TinyUSB task before running user code (whose write_cdc() / +// write_vendor() must then take the non-blocking path). +constexpr size_t kMaxMscSyncCalls = 8; +std::array, kMaxMscSyncCalls> s_msc_sync_tasks{}; + +struct MscSyncScope { + int slot{-1}; + MscSyncScope() { + const TaskHandle_t self = xTaskGetCurrentTaskHandle(); + for (size_t i = 0; i < kMaxMscSyncCalls; ++i) { + TaskHandle_t expected = nullptr; + if (s_msc_sync_tasks[i].compare_exchange_strong(expected, self)) { + slot = static_cast(i); + break; + } + } + } + ~MscSyncScope() { + if (slot >= 0) + s_msc_sync_tasks[static_cast(slot)].store(nullptr); + } + MscSyncScope(const MscSyncScope &) = delete; + MscSyncScope &operator=(const MscSyncScope &) = delete; +}; + +bool in_msc_sync_call() { + const TaskHandle_t self = xTaskGetCurrentTaskHandle(); + return std::any_of(s_msc_sync_tasks.begin(), s_msc_sync_tasks.end(), + [self](const auto &task) { return task.load() == self; }); +} + +// Barrier on the TinyUSB task: returns once everything queued on it before the +// call has run. The generation counter is static so a barrier that fires after +// its waiter timed out never touches a dead stack frame. +std::atomic s_usb_barrier_done{0}; + +bool wait_for_tinyusb_task(TickType_t timeout_ticks) { + const uint32_t target = s_usb_barrier_done.load() + 1; + usbd_defer_func([](void *) { s_usb_barrier_done.fetch_add(1); }, nullptr, false); + const TickType_t start = xTaskGetTickCount(); + while (s_usb_barrier_done.load() < target) { + if (xTaskGetTickCount() - start >= timeout_ticks) + return false; + vTaskDelay(pdMS_TO_TICKS(5)); + } + return true; +} +#endif + // [[maybe_unused]]: only the CDC/vendor write-drain paths call this, so it is // unused in an X-Input-only build (CFG_TUD_CDC == CFG_TUD_VENDOR == 0). [[maybe_unused]] bool on_tinyusb_task() { @@ -359,6 +434,51 @@ extern "C" usbd_class_driver_t const *usbd_app_driver_get_cb(uint8_t *driver_cou return &s_xinput_class_driver; } +#if (CFG_TUD_MSC > 0) +namespace { +// esp_tinyusb MSC storage event -> UsbDevice. Fires in the TinyUSB task for +// host-driven hand-overs and in the calling task for set_msc_owner() / storage +// creation. Loads the teardown-guarded singleton, like the other trampolines. +// cppcheck-suppress constParameterCallback // signature must match tusb_msc_callback_t +void msc_event_trampoline(tinyusb_msc_storage_handle_t handle, tinyusb_msc_event_t *event, void *) { + if (!in_msc_sync_call()) + note_tinyusb_task(); // a host-driven hand-over: user code runs on the TinyUSB task + auto *dev = s_device.load(); + if (!dev || !event) + return; + using Event = espp::UsbDevice::MscEvent; + Event e = Event::OwnerChangeFailed; + switch (event->id) { + case TINYUSB_MSC_EVENT_MOUNT_START: + e = Event::OwnerChangeStarted; + break; + case TINYUSB_MSC_EVENT_MOUNT_COMPLETE: + e = Event::OwnerChanged; + break; + case TINYUSB_MSC_EVENT_MOUNT_FAILED: + e = Event::OwnerChangeFailed; + break; + case TINYUSB_MSC_EVENT_FORMAT_REQUIRED: + e = Event::FormatRequired; + break; + case TINYUSB_MSC_EVENT_FORMAT_FAILED: + e = Event::FormatFailed; + break; + default: + return; + } + // event->mount_point is the medium's owner at the moment the event is emitted: + // esp_tinyusb emits MOUNT_START before it updates the owner (the previous + // owner) and MOUNT_COMPLETE after (the new owner) -- tinyusb_msc.c + // msc_storage_mount() / msc_storage_unmount(). + const auto owner = event->mount_point == TINYUSB_MSC_STORAGE_MOUNT_APP + ? espp::UsbDevice::MscOwner::App + : espp::UsbDevice::MscOwner::Host; + espp::UsbDevice::Callbacks::msc_event(dev, handle, e, owner); +} +} // namespace +#endif // CFG_TUD_MSC > 0 + namespace espp { // Storage for the descriptors that TinyUSB references by pointer for the lifetime @@ -390,6 +510,29 @@ struct UsbDevice::Impl { // the aligned-down address, prepending the preceding byte to every report // (which shifted our "00 14 .." report by one and made XUSB reject all input). alignas(4) std::array xinput_report{}; + +#if (CFG_TUD_MSC > 0) + // MSC media. A fixed array (never a growing vector): esp_tinyusb keeps a raw + // pointer to each base_path string for the storage object's lifetime. + struct MscLun { + tinyusb_msc_storage_handle_t storage{nullptr}; + wl_handle_t wl{WL_INVALID_HANDLE}; // flash media: our wear-levelling mount + std::string base_path; + // Result of the last hand-over, recorded by the event bridge so + // set_msc_owner() can report it (esp_tinyusb's setter ignores the outcome). + std::atomic last_result{0}; // 0 ok, 1 failed, 2 no filesystem + // Set while hand_over_msc() quietly resets the owner after a failed mount, so + // that internal step is not reported as a user-visible hand-over. + std::atomic reverting{false}; + // The medium has no FAT filesystem (FormatRequired). Unlike last_result it + // survives across calls: esp_tinyusb leaves such a medium application-owned + // with nothing mounted and emits no event when asked for that owner again. + std::atomic no_filesystem{false}; + }; + std::array msc_luns{}; + size_t msc_lun_count{0}; + bool msc_driver_installed{false}; +#endif }; UsbDevice *UsbDevice::instance() { return s_device; } @@ -401,7 +544,8 @@ UsbDevice::UsbDevice(const Config &config) , on_cdc_receive_(config.cdc ? config.cdc->on_receive : nullptr) , on_vendor_receive_(config.vendor ? config.vendor->on_receive : nullptr) , on_xinput_rumble_(config.xinput ? config.xinput->on_rumble : nullptr) - , on_hid_receive_(config.hid ? config.hid->on_receive : nullptr) {} + , on_hid_receive_(config.hid ? config.hid->on_receive : nullptr) + , on_msc_event_(config.msc ? config.msc->on_event : nullptr) {} UsbDevice::~UsbDevice() { #if (CFG_TUD_CDC > 0) @@ -428,8 +572,25 @@ UsbDevice::~UsbDevice() { #if (CFG_TUD_CDC > 0) if (config_.cdc) tinyusb_cdcacm_deinit(kCdcPort); +#endif +#if (CFG_TUD_MSC > 0) + // Detach and let the TinyUSB task finish what is queued (deferred writes, a + // detach / auto-hand-over callback iterating the media) BEFORE stopping it, + // then delete the media with the task gone: nothing can race the deletion. + const bool quiesced = quiesce_msc_before_uninstall(); #endif tinyusb_driver_uninstall(); +#if (CFG_TUD_MSC > 0) + if (!deinit_msc()) { + // A storage object is still mapped (queued writes that never ran, which + // only happens if the task was stuck). Leak impl_ rather than free the + // base_path strings and media esp_tinyusb still points at; no task is left + // to use them. A later UsbDevice cannot install the MSC driver. + logger_.error("MSC media could not be released{}; leaking them", + quiesced ? "" : " (the TinyUSB task did not drain its queue)"); + (void)impl_.release(); + } +#endif initialized_ = false; } } @@ -698,16 +859,82 @@ bool UsbDevice::initialize(std::error_code &ec) { ec = std::make_error_code(std::errc::device_or_resource_busy); return false; } - if (!config_.cdc && !config_.vendor && !config_.hid && !config_.xinput) { - logger_.error("No USB function enabled (enable cdc, vendor, hid and/or xinput)"); + if (!config_.cdc && !config_.vendor && !config_.hid && !config_.xinput && !config_.msc) { + logger_.error("No USB function enabled (enable cdc, vendor, hid, xinput and/or msc)"); ec = std::make_error_code(std::errc::invalid_argument); return false; } if (config_.msc) { - // Reserved extension point; not implemented yet (see README endpoint table). - logger_.error("MSC function is not implemented yet"); +#if (CFG_TUD_MSC == 0) + logger_.error("MSC function requested but CFG_TUD_MSC==0. Set " + "CONFIG_TINYUSB_MSC_ENABLED=y in sdkconfig."); ec = std::make_error_code(std::errc::function_not_supported); return false; +#else + const auto &media = config_.msc->media; + if (media.empty() || media.size() > kMaxMscLuns) { + logger_.error("MSC function needs 1..{} media, got {}", kMaxMscLuns, media.size()); + ec = std::make_error_code(std::errc::invalid_argument); + return false; + } + size_t sd_cards = 0, partitions = 0; + for (size_t i = 0; i < media.size(); ++i) { + const auto &m = media[i]; + if (m.base_path.size() < 2 || m.base_path.front() != '/') { + logger_.error("MSC medium {}: base_path '{}' must be an absolute VFS path like '/msc'", i, + m.base_path); + ec = std::make_error_code(std::errc::invalid_argument); + return false; + } + if (m.volume_label.size() > 11) { + logger_.error("MSC medium {}: volume_label '{}' is longer than FAT's 11 characters", i, + m.volume_label); + ec = std::make_error_code(std::errc::invalid_argument); + return false; + } +#if !FF_USE_LABEL + if (!m.volume_label.empty()) { + logger_.error("MSC medium {}: volume_label needs CONFIG_FATFS_USE_LABEL=y", i); + ec = std::make_error_code(std::errc::function_not_supported); + return false; + } +#endif + if (i > 0 && media[0].base_path == m.base_path) { + logger_.error("MSC media 0 and 1 share base_path '{}'; each needs its own", m.base_path); + ec = std::make_error_code(std::errc::invalid_argument); + return false; + } + if (m.type == MscMedium::Type::SdCard) { + if (!m.sd_card) { + logger_.error("MSC medium {}: type SdCard but sd_card is null", i); + ec = std::make_error_code(std::errc::invalid_argument); + return false; + } +#if SOC_SDMMC_HOST_SUPPORTED + ++sd_cards; +#else + logger_.error("MSC medium {}: SD card media need a target with an SDMMC host " + "(esp_tinyusb's SD backend is not built for this target)", + i); + ec = std::make_error_code(std::errc::function_not_supported); + return false; +#endif + } else { + if (m.partition_label.empty()) { + logger_.error("MSC medium {}: type FlashPartition but partition_label is empty", i); + ec = std::make_error_code(std::errc::invalid_argument); + return false; + } + ++partitions; + } + } + if (sd_cards > 1 || partitions > 1) { + logger_.error("MSC supports at most one SD card and one flash partition (esp_tinyusb's " + "media backends are singletons)"); + ec = std::make_error_code(std::errc::invalid_argument); + return false; + } +#endif } if (config_.cdc) { #if (CFG_TUD_CDC == 0) @@ -842,6 +1069,20 @@ bool UsbDevice::initialize(std::error_code &ec) { impl_->xinput_itf = xinput_itf; } + // MSC: one interface, bulk OUT + bulk IN on one endpoint number. msc_* are only + // consumed by the CFG_TUD_MSC-guarded descriptor branch below. + [[maybe_unused]] uint8_t msc_itf = 0, msc_str = 0, msc_out = 0, msc_in = 0; + if (config_.msc) { + msc_itf = next_itf++; + msc_str = next_str++; + impl_->owned_strings.push_back(config_.msc->interface_name); + const uint8_t m_ep = next_ep++; + msc_out = m_ep; // bulk OUT + msc_in = static_cast(0x80 | m_ep); // bulk IN + in_used++; + out_used++; + } + // --- Endpoint budget check --- if (in_used > kMaxInEndpoints || out_used > kMaxOutEndpoints) { logger_.error("Endpoint budget exceeded: IN={} (max {}), OUT={} (max {})", in_used, @@ -862,7 +1103,8 @@ bool UsbDevice::initialize(std::error_code &ec) { // Xbox 360 controller's identity (VID/PID/bcdDevice) and a 0xFF/0xFF/0xFF // device class so a PC's XUSB driver binds it. Combining XInput with other // functions keeps the normal composite identity (and XUSB will not bind). - const bool xinput_only = config_.xinput && !config_.cdc && !config_.vendor && !config_.hid; + const bool xinput_only = + config_.xinput && !config_.cdc && !config_.vendor && !config_.hid && !config_.msc; impl_->device_desc = tusb_desc_device_t{}; impl_->device_desc.bLength = sizeof(tusb_desc_device_t); impl_->device_desc.bDescriptorType = TUSB_DESC_DEVICE; @@ -917,6 +1159,12 @@ bool UsbDevice::initialize(std::error_code &ec) { itf_count = static_cast(itf_count + 1); total_len = static_cast(total_len + espp::xinput::kInterfaceDescriptorLen); } +#if (CFG_TUD_MSC > 0) + if (config_.msc) { + itf_count = static_cast(itf_count + 1); + total_len = static_cast(total_len + TUD_MSC_DESC_LEN); + } +#endif // Build one configuration descriptor for a given bus speed. Bulk endpoints // are 64 bytes at full speed and 512 at high speed; the HID interrupt @@ -1001,6 +1249,14 @@ bool UsbDevice::initialize(std::error_code &ec) { impl_->xinput_ep_out); append(d.data(), d.size()); } +#if (CFG_TUD_MSC > 0) + if (config_.msc) { + const uint8_t d[] = { + TUD_MSC_DESCRIPTOR(msc_itf, msc_str, msc_out, msc_in, bulk_ep_size), + }; + append(d, sizeof(d)); + } +#endif }; const uint8_t hid_poll_ms = config_.hid ? config_.hid->poll_interval_ms : 0; @@ -1267,13 +1523,30 @@ bool UsbDevice::initialize(std::error_code &ec) { return false; } + // MSC media come up BEFORE the driver: a host that is already connected + // mounts the device as soon as the driver starts, and the hand-over on that + // first mount must find the storage objects. +#if (CFG_TUD_MSC > 0) + if (config_.msc && !init_msc(ec)) { + s_device = nullptr; + return false; + } +#endif + esp_err_t err = tinyusb_driver_install(&tusb_cfg); if (err != ESP_OK) { logger_.error("tinyusb_driver_install failed: {}", esp_err_to_name(err)); s_device = nullptr; + deinit_msc(); ec = std::make_error_code(std::errc::io_error); return false; } + attached_ = config_.connect_on_initialize; + if (!config_.connect_on_initialize) { + // The driver enabled the pull-up; drop it right away. A host needs >= 100 ms + // of attach debounce before it enumerates, so it never sees the device. + tud_disconnect(); + } // --- Initialize the CDC-ACM function (vendor needs no explicit init) --- #if (CFG_TUD_CDC > 0) @@ -1288,7 +1561,18 @@ bool UsbDevice::initialize(std::error_code &ec) { if (err != ESP_OK) { logger_.error("tinyusb_cdcacm_init failed: {}", esp_err_to_name(err)); s_device = nullptr; + // A host may already have enumerated and queued MSC writes: release the + // media while the TinyUSB task can still run them, like the destructor. +#if (CFG_TUD_MSC > 0) + quiesce_msc_before_uninstall(); +#endif tinyusb_driver_uninstall(); +#if (CFG_TUD_MSC > 0) + if (!deinit_msc()) { + (void)impl_.release(); // storage still mapped: keep its strings alive + impl_ = std::make_unique(); + } +#endif ec = std::make_error_code(std::errc::io_error); return false; } @@ -1301,9 +1585,10 @@ bool UsbDevice::initialize(std::error_code &ec) { const uint16_t enum_vid = impl_->device_desc.idVendor; const uint16_t enum_pid = impl_->device_desc.idProduct; logger_.info("Initialized native USB device (VID=0x{:04x} PID=0x{:04x}) cdc={} vendor={} hid={} " - "xinput={}{}", + "xinput={} msc={}{}", enum_vid, enum_pid, config_.cdc.has_value(), config_.vendor.has_value(), - config_.hid.has_value(), config_.xinput.has_value(), webusb ? " webusb" : ""); + config_.hid.has_value(), config_.xinput.has_value(), + config_.msc ? config_.msc->media.size() : 0, webusb ? " webusb" : ""); #if (CFG_TUD_CDC > 0) // Opt-in: route the console to the CDC interface now that TinyUSB owns the USB // port. Best-effort -- a routing failure must not fail initialization (the @@ -1672,6 +1957,20 @@ void UsbDevice::handle_usb_unmount() { bool UsbDevice::is_initialized() const { return initialized_; } +bool UsbDevice::connect() { + if (!initialized_) + return false; + attached_ = true; + return tud_connect(); +} + +bool UsbDevice::disconnect() { + if (!initialized_) + return false; + attached_ = false; + return tud_disconnect(); +} + bool UsbDevice::is_cdc_connected() const { #if (CFG_TUD_CDC > 0) if (!initialized_ || !config_.cdc) @@ -1899,4 +2198,598 @@ bool UsbDevice::is_xinput_ready() const { return tud_mounted() && ep_in != 0 && !usbd_edpt_busy(0, ep_in); } +// --------------------------------------------------------------------------- +// MSC (mass storage). +// --------------------------------------------------------------------------- + +void UsbDevice::apply_msc_volume_label(size_t index) { +#if (CFG_TUD_MSC > 0) && FF_USE_LABEL + const auto &m = config_.msc->media[index]; + if (m.volume_label.empty()) + return; + const auto &lun = impl_->msc_luns[index]; + BYTE pdrv = 0xFF; + if (m.type == MscMedium::Type::FlashPartition) { + pdrv = ff_diskio_get_pdrv_wl(lun.wl); + } else { +#if SOC_SDMMC_HOST_SUPPORTED + pdrv = ff_diskio_get_pdrv_card(m.sd_card); +#endif + } + if (pdrv == 0xFF) { + logger_.warn("MSC medium {}: not mounted; volume label not written", index); + return; + } + const std::string drive = std::to_string(pdrv) + ":"; + std::string wanted = m.volume_label; + for (auto &c : wanted) + c = static_cast(std::toupper(static_cast(c))); // FAT stores it upper-case + char current[34] = {}; // large enough for any FF_LFN_UNICODE encoding of 11 chars + const FRESULT get_res = f_getlabel(drive.c_str(), current, nullptr); + if (get_res == FR_OK && wanted == current) { + logger_.info("MSC medium {}: volume label is '{}'", index, current); + return; // already labelled + } + const FRESULT res = f_setlabel((drive + wanted).c_str()); + if (res == FR_OK) + logger_.info("MSC medium {}: volume label set to '{}'", index, wanted); + else + logger_.warn("MSC medium {}: setting volume label '{}' failed (FRESULT {})", index, wanted, + static_cast(res)); +#else + (void)index; +#endif +} + +bool UsbDevice::init_msc(std::error_code &ec) { +#if (CFG_TUD_MSC > 0) + MscSyncScope sync; // hand-over events raised by this call run in this task + tinyusb_msc_driver_config_t driver_cfg{}; + driver_cfg.user_flags.auto_mount_off = config_.msc->auto_handover ? 0 : 1; + driver_cfg.callback = &msc_event_trampoline; + driver_cfg.callback_arg = nullptr; + esp_err_t err = tinyusb_msc_install_driver(&driver_cfg); + if (err != ESP_OK) { + logger_.error("tinyusb_msc_install_driver failed: {}", esp_err_to_name(err)); + ec = std::make_error_code(std::errc::io_error); + return false; + } + impl_->msc_driver_installed = true; + + const auto &media = config_.msc->media; + for (size_t i = 0; i < media.size(); ++i) { + const auto &m = media[i]; + auto &lun = impl_->msc_luns[i]; + lun.base_path = m.base_path; // esp_tinyusb keeps a pointer to this string + + tinyusb_msc_storage_config_t storage_cfg{}; + // data(), not c_str(): the esp_tinyusb field is a non-const `char *` + storage_cfg.fat_fs.base_path = lun.base_path.data(); + storage_cfg.fat_fs.config.max_files = m.max_files; + storage_cfg.fat_fs.do_not_format = !m.format_if_unformatted; + storage_cfg.fat_fs.format_flags = 0; // FM_ANY + // Always create the medium host-owned and hand it to the application below. + // If esp_tinyusb mounts it during creation and that mount fails, it frees the + // storage object while it is still mapped as a LUN and returns no handle, so + // the stale LUN could never be removed (and SCSI requests would reach freed + // memory). Created host-owned, the handle is ours before anything can fail. + storage_cfg.mount_point = TINYUSB_MSC_STORAGE_MOUNT_USB; + + if (m.type == MscMedium::Type::SdCard) { +#if SOC_SDMMC_HOST_SUPPORTED + storage_cfg.medium.card = m.sd_card; + err = tinyusb_msc_new_storage_sdmmc(&storage_cfg, &lun.storage); +#endif + } else { + const esp_partition_t *partition = esp_partition_find_first( + ESP_PARTITION_TYPE_DATA, ESP_PARTITION_SUBTYPE_DATA_FAT, m.partition_label.c_str()); + if (!partition) { + logger_.error( + "MSC medium {}: no 'data, fat' partition labelled '{}' in the partition table", i, + m.partition_label); + deinit_msc(); + ec = std::make_error_code(std::errc::no_such_device); + return false; + } + err = wl_mount(partition, &lun.wl); + if (err == ESP_OK) { + storage_cfg.medium.wl_handle = lun.wl; + err = tinyusb_msc_new_storage_spiflash(&storage_cfg, &lun.storage); + } else { + logger_.error("MSC medium {}: wear levelling mount of '{}' failed: {}", i, + m.partition_label, esp_err_to_name(err)); + } + } + if (err != ESP_OK || !lun.storage) { + logger_.error("MSC medium {}: creating the storage failed: {}", i, esp_err_to_name(err)); + deinit_msc(); + ec = std::make_error_code(std::errc::io_error); + return false; + } + impl_->msc_lun_count = i + 1; + + // Mount it for the application when it starts there, or briefly to write the + // volume label (safe: the TinyUSB driver is not installed yet, so no host). + if (m.initial_owner == MscOwner::App || !m.volume_label.empty()) { + std::error_code hand_over_ec; + if (hand_over_msc(i, MscOwner::App, hand_over_ec)) { + apply_msc_volume_label(i); + if (m.initial_owner == MscOwner::Host) { + std::error_code back_ec; + if (!hand_over_msc(i, MscOwner::Host, back_ec)) { + logger_.error("MSC medium {}: could not hand it to the host: {}", i, back_ec.message()); + deinit_msc(); + ec = back_ec; + return false; + } + } + } else { + if (hand_over_ec == std::errc::no_such_device) { + if (m.initial_owner == MscOwner::Host) { + // unformatted, meant for the host: give it back (it can format it) + lun.reverting = true; + tinyusb_msc_set_storage_mount_point(lun.storage, TINYUSB_MSC_STORAGE_MOUNT_USB); + lun.reverting = false; + lun.no_filesystem = false; + } + // No FAT filesystem: not fatal. FormatRequired has been reported and an + // app-owned medium stays application-owned so format_msc_medium() can run. + logger_.warn("MSC medium {}: no FAT filesystem yet; format it to use it", i); + } else { + logger_.error("MSC medium {}: could not mount it for the application: {}", i, + hand_over_ec.message()); + deinit_msc(); + ec = hand_over_ec; + return false; + } + } + } + + uint32_t sectors = 0, sector_size = 0; // best-effort, for the log line only + tinyusb_msc_get_storage_capacity(lun.storage, §ors); + tinyusb_msc_get_storage_sector_size(lun.storage, §or_size); + logger_.info("MSC medium {}: {} ({} KiB) at '{}', owned by the {}", i, + m.type == MscMedium::Type::SdCard ? "SD card" : m.partition_label, + static_cast(sectors) * sector_size / 1024, lun.base_path, + m.initial_owner == MscOwner::App ? "application" : "host"); + } + return true; +#else + (void)ec; + return true; +#endif +} + +bool UsbDevice::deinit_msc() { +#if (CFG_TUD_MSC > 0) + MscSyncScope sync; // storage deletion raises its events in this task + bool all_released = true; + for (size_t i = kMaxMscLuns; i-- > 0;) { + auto &lun = impl_->msc_luns[i]; + if (lun.storage) { + // A medium marked application-owned with nothing mounted (unformatted, or a + // failed mount) must be reset to the host first: esp_tinyusb's delete does + // ESP_ERROR_CHECK(msc_storage_unmount()), and that unmount fails with + // ESP_ERR_INVALID_STATE when no drive is registered -- aborting the device. + // The setter records the host owner even though its own unmount fails. + tinyusb_msc_mount_point_t current = TINYUSB_MSC_STORAGE_MOUNT_USB; + tinyusb_msc_get_storage_mount_point(lun.storage, ¤t); + uint64_t total_bytes = 0, free_bytes = 0; + if (current == TINYUSB_MSC_STORAGE_MOUNT_APP && + esp_vfs_fat_info(lun.base_path.c_str(), &total_bytes, &free_bytes) != ESP_OK) { + lun.reverting = true; + tinyusb_msc_set_storage_mount_point(lun.storage, TINYUSB_MSC_STORAGE_MOUNT_USB); + lun.reverting = false; + } + // ESP_ERR_INVALID_STATE: host writes still queued (the task did not drain) + const esp_err_t err = tinyusb_msc_delete_storage(lun.storage); + if (err != ESP_OK) { + // Keep the handle and the medium behind it: the storage object is still + // mapped as a LUN, so unmounting its wear levelling here would leave it + // pointing at an invalid handle. + logger_.error("MSC medium {}: deleting the storage failed ({}); leaving it in place", i, + esp_err_to_name(err)); + all_released = false; + continue; + } + lun.storage = nullptr; + } + if (lun.wl != WL_INVALID_HANDLE) { + wl_unmount(lun.wl); + lun.wl = WL_INVALID_HANDLE; + } + lun.no_filesystem = false; + } + if (!all_released) + return false; // the driver cannot be uninstalled while a LUN is still mapped + impl_->msc_lun_count = 0; + if (impl_->msc_driver_installed) { + const esp_err_t err = tinyusb_msc_uninstall_driver(); + if (err == ESP_OK) + impl_->msc_driver_installed = false; + else + logger_.error("tinyusb_msc_uninstall_driver failed: {}", esp_err_to_name(err)); + } + return true; +#else + return true; +#endif +} + +bool UsbDevice::drain_tinyusb_task() { +#if (CFG_TUD_MSC > 0) + // The first barrier completes every event queued before it; writes those + // events defer are queued behind it and complete by a later barrier. + for (int round = 0; round < 3; ++round) { + if (!wait_for_tinyusb_task(pdMS_TO_TICKS(500))) + return false; + } +#endif + return true; +} + +void UsbDevice::clean_up_failed_msc_format(size_t index, uint8_t pdrv) { +#if (CFG_TUD_MSC > 0) + const auto &lun = impl_->msc_luns[index]; + // Each step is harmless when esp_tinyusb did not get that far. + esp_vfs_fat_unregister_path(lun.base_path.c_str()); + const std::string drive = std::to_string(pdrv) + ":"; + f_mount(nullptr, drive.c_str(), 0); + // Only unregister the drive if the format registered it: the slot we saw as + // free before the format is no longer the first free one. + BYTE first_free = 0xFF; + if (ff_diskio_get_drive(&first_free) != ESP_OK || first_free != pdrv) { + if (config_.msc->media[index].type == MscMedium::Type::FlashPartition) + ff_diskio_clear_pdrv_wl(lun.wl); + ff_diskio_unregister(pdrv); + } +#else + (void)index; + (void)pdrv; +#endif +} + +bool UsbDevice::quiesce_msc_before_uninstall() { +#if (CFG_TUD_MSC > 0) + if (!config_.msc) + return true; + // Stop host traffic, then pass barriers through the TinyUSB task until + // everything already queued has run: the first barrier completes events + // queued before it (including a detach callback that walks the media); writes + // those events defer are queued behind it and complete by a later barrier. + tud_disconnect(); + if (!drain_tinyusb_task()) { + logger_.error("the TinyUSB task did not process its queue; MSC teardown may lose writes"); + return false; + } + return true; +#else + return true; +#endif +} + +void UsbDevice::handle_msc_event(const void *storage, MscEvent event, MscOwner owner) { + size_t lun_index = 0; +#if (CFG_TUD_MSC > 0) + bool found = false; + for (size_t i = 0; i < kMaxMscLuns; ++i) { + if (storage && impl_->msc_luns[i].storage == storage) { + lun_index = i; + found = true; + break; + } + } + if (!found) + return; // not one of ours (or already torn down) + auto &lun = impl_->msc_luns[lun_index]; + if (lun.reverting) + return; // hand_over_msc() resetting the owner after a failed mount + switch (event) { + case MscEvent::OwnerChangeStarted: + break; + case MscEvent::OwnerChanged: + if (owner == MscOwner::App) + lun.no_filesystem = false; // it mounted, so it has one + break; + case MscEvent::OwnerChangeFailed: + lun.last_result = 1; + // `owner` is the side that still has the medium: the attempt was to the other + logger_.warn("MSC medium {}: hand-over to the {} failed (mount / unmount error)", lun_index, + owner == MscOwner::App ? "host" : "application"); + break; + case MscEvent::FormatFailed: + // esp_tinyusb reports no error code with this event; its own log (tag + // "tinyusb_msc_storage") has the FatFs result + lun.last_result = 1; + logger_.warn("MSC medium {}: formatting the FAT filesystem failed", lun_index); + break; + case MscEvent::FormatRequired: + lun.last_result = 2; + lun.no_filesystem = true; + logger_.warn("MSC medium {}: no FAT filesystem (format it, or enable format_if_unformatted)", + lun_index); + break; + } +#else + (void)storage; +#endif + msc_event_callback_fn cb; + { + std::lock_guard lock(cb_mutex_); + cb = on_msc_event_; + } + if (cb) + cb(lun_index, event, owner); // outside the lock: it may take its time / log +} + +bool UsbDevice::hand_over_msc(size_t index, MscOwner owner, std::error_code &ec) { + ec.clear(); +#if (CFG_TUD_MSC > 0) + MscSyncScope sync; // hand-over events raised by this call run in this task + auto &lun = impl_->msc_luns[index]; + const bool to_app = owner == MscOwner::App; + const auto app_mounted = [&lun]() { + uint64_t total_bytes = 0, free_bytes = 0; + return esp_vfs_fat_info(lun.base_path.c_str(), &total_bytes, &free_bytes) == ESP_OK; + }; + + if (to_app) { + tinyusb_msc_mount_point_t current = TINYUSB_MSC_STORAGE_MOUNT_USB; + tinyusb_msc_get_storage_mount_point(lun.storage, ¤t); + if (current == TINYUSB_MSC_STORAGE_MOUNT_APP) { + if (app_mounted()) + return true; // already the application's + if (lun.no_filesystem) { + ec = std::make_error_code(std::errc::no_such_device); // still waiting for a format + return false; + } + // Marked application-owned with nothing mounted (an earlier hand-over + // failed): esp_tinyusb would treat this request as a no-op, so quietly + // reset it to the host (nothing mounted: this only resets the owner) and + // mount it again below. + lun.reverting = true; + tinyusb_msc_set_storage_mount_point(lun.storage, TINYUSB_MSC_STORAGE_MOUNT_USB); + lun.reverting = false; + } + } + + lun.last_result = 0; // the hand-over's events run synchronously in this call + if (tinyusb_msc_set_storage_mount_point(lun.storage, to_app ? TINYUSB_MSC_STORAGE_MOUNT_APP + : TINYUSB_MSC_STORAGE_MOUNT_USB) != + ESP_OK) { + ec = std::make_error_code(std::errc::io_error); + return false; + } + // esp_tinyusb's setter records the requested owner whatever the mount / unmount + // did, and several of its failure paths raise no event, so confirm the result + // against the VFS: the application has the medium exactly when a mounted FAT + // volume answers at its base_path. + if (to_app == app_mounted()) + return true; + + if (to_app) { + if (lun.last_result == 2) { + // No FAT filesystem. Keep the medium application-owned (esp_tinyusb's + // format requires it) so format_msc_medium() can create one. + ec = std::make_error_code(std::errc::no_such_device); + return false; + } + // The mount failed and nothing is mounted, but esp_tinyusb now marks the + // medium application-owned, so the host would be told "no medium" while the + // application has no files either. Give it back to the host: with nothing + // mounted the unmount only resets the owner. + logger_.error("MSC medium {}: mounting it at '{}' failed; it stays with the host", index, + lun.base_path); + lun.reverting = true; + tinyusb_msc_set_storage_mount_point(lun.storage, TINYUSB_MSC_STORAGE_MOUNT_USB); + lun.reverting = false; + ec = std::make_error_code(std::errc::io_error); + return false; + } + + // Handing it to the host, but the application's volume still answers: the host + // must not write under a mounted FAT volume. Remove the VFS registration + // ourselves and check again before reporting success. + logger_.warn("MSC medium {}: '{}' still mounted after the hand-over to the host; unregistering " + "it", + index, lun.base_path); + esp_vfs_fat_unregister_path(lun.base_path.c_str()); + if (!app_mounted()) + return true; + logger_.error("MSC medium {}: could not unmount '{}' for the host", index, lun.base_path); + ec = std::make_error_code(std::errc::io_error); + return false; +#else + (void)index; + (void)owner; + ec = std::make_error_code(std::errc::function_not_supported); + return false; +#endif +} + +bool UsbDevice::set_msc_owner(size_t lun, MscOwner owner, std::error_code &ec) { + ec.clear(); +#if (CFG_TUD_MSC > 0) + if (!initialized_ || !config_.msc) { + ec = std::make_error_code(std::errc::not_connected); + return false; + } + if (lun >= impl_->msc_lun_count) { + ec = std::make_error_code(std::errc::invalid_argument); + return false; + } + if (owner == MscOwner::App && msc_owner(lun) == MscOwner::Host) { + // esp_tinyusb accepts WRITE(10) data and runs the write later on the TinyUSB + // task, without re-checking ownership, so a write the host already queued + // could land after the FAT volume is mounted for the application. + const auto host_attached = [this, lun]() { + if (!tud_mounted()) + return false; + logger_.warn("MSC medium {}: the attached host still has it; eject it on the host (or " + "detach) before handing it to the application", + lun); + return true; + }; + if (host_attached()) { + ec = std::make_error_code(std::errc::device_or_resource_busy); + return false; + } + // Not mounted now -- but tud_mounted() clears before the TinyUSB task runs the + // unplug event and any writes still queued. Let the task get through them + // before mounting the volume here. + if (!drain_tinyusb_task()) { + logger_.error("MSC medium {}: the TinyUSB task did not process its queue", lun); + ec = std::make_error_code(std::errc::timed_out); + return false; + } + if (host_attached()) { // a host enumerated while we waited + ec = std::make_error_code(std::errc::device_or_resource_busy); + return false; + } + } + return hand_over_msc(lun, owner, ec); +#else + (void)lun; + (void)owner; + ec = std::make_error_code(std::errc::function_not_supported); + return false; +#endif +} + +bool UsbDevice::set_msc_owner(size_t lun, MscOwner owner) { + std::error_code ec; + return set_msc_owner(lun, owner, ec); +} + +std::optional UsbDevice::msc_owner(size_t lun) const { +#if (CFG_TUD_MSC > 0) + if (!initialized_ || !config_.msc || lun >= impl_->msc_lun_count) + return std::nullopt; + tinyusb_msc_mount_point_t mount_point = TINYUSB_MSC_STORAGE_MOUNT_USB; + if (tinyusb_msc_get_storage_mount_point(impl_->msc_luns[lun].storage, &mount_point) != ESP_OK) + return std::nullopt; + return mount_point == TINYUSB_MSC_STORAGE_MOUNT_APP ? MscOwner::App : MscOwner::Host; +#else + (void)lun; + return std::nullopt; +#endif +} + +std::optional UsbDevice::msc_capacity(size_t lun) const { +#if (CFG_TUD_MSC > 0) + if (!initialized_ || !config_.msc || lun >= impl_->msc_lun_count) + return std::nullopt; + MscCapacity capacity; + const auto storage = impl_->msc_luns[lun].storage; + if (tinyusb_msc_get_storage_capacity(storage, &capacity.sector_count) != ESP_OK || + tinyusb_msc_get_storage_sector_size(storage, &capacity.sector_size) != ESP_OK) + return std::nullopt; + return capacity; +#else + (void)lun; + return std::nullopt; +#endif +} + +size_t UsbDevice::msc_lun_count() const { +#if (CFG_TUD_MSC > 0) + return initialized_ && config_.msc ? impl_->msc_lun_count : 0; +#else + return 0; +#endif +} + +bool UsbDevice::format_msc_medium(size_t lun, std::error_code &ec) { + ec.clear(); +#if (CFG_TUD_MSC > 0) + if (!initialized_ || !config_.msc) { + ec = std::make_error_code(std::errc::not_connected); + return false; + } + if (lun >= impl_->msc_lun_count) { + ec = std::make_error_code(std::errc::invalid_argument); + return false; + } + if (msc_owner(lun) != MscOwner::App) { + logger_.error("MSC medium {}: hand it to the application before formatting", lun); + ec = std::make_error_code(std::errc::operation_not_permitted); + return false; + } + auto &l = impl_->msc_luns[lun]; + { + uint64_t total_bytes = 0, free_bytes = 0; + if (esp_vfs_fat_info(l.base_path.c_str(), &total_bytes, &free_bytes) == ESP_OK) { + ec = std::make_error_code(std::errc::file_exists); // mounted: it has a filesystem + return false; + } + } + // esp_tinyusb reports "every FatFs drive slot is taken" with the same + // ESP_ERR_NOT_FOUND it uses for "a filesystem already exists". It formats on + // the first free drive, which is this one (re-read right before formatting). + BYTE pdrv = 0xFF; + { + if (ff_diskio_get_drive(&pdrv) != ESP_OK) { + logger_.error("MSC medium {}: no free FatFs drive to format it on (raise " + "CONFIG_FATFS_VOLUME_COUNT or unmount another FAT volume)", + lun); + ec = std::make_error_code(std::errc::device_or_resource_busy); + return false; + } + } + // esp_tinyusb's format does not take the storage lock, and with auto_handover a + // host attaching (or detaching) mid-format would run its mount / unmount on the + // TinyUSB task against the same drive. Drop the connection for the format so no + // attach can happen, let any detach finish first, and reconnect afterwards. + const bool pause_usb = config_.msc->auto_handover; + const bool was_attached = attached_.load(); + if (pause_usb) { + tud_disconnect(); + // tud_mounted() clears before a detach callback runs, so wait on the TinyUSB + // task itself rather than on that flag. + if (!drain_tinyusb_task()) { + if (was_attached) + tud_connect(); + logger_.error("MSC medium {}: the TinyUSB task did not process its queue; not formatting", + lun); + ec = std::make_error_code(std::errc::timed_out); + return false; + } + } + ff_diskio_get_drive(&pdrv); // the drive esp_tinyusb is about to use + const esp_err_t err = [&] { + MscSyncScope sync; + return tinyusb_msc_format_storage(l.storage); + }(); + if (err == ESP_OK) + apply_msc_volume_label(lun); // before the host can attach and see the drive + else + clean_up_failed_msc_format(lun, pdrv); // esp_tinyusb leaves partial registrations + if (pause_usb && was_attached) + tud_connect(); // only if the application had the device attached + switch (err) { + case ESP_OK: + l.no_filesystem = false; + logger_.info("MSC medium {}: formatted and mounted at '{}'", lun, l.base_path); + return true; + case ESP_ERR_NOT_FOUND: // a filesystem is already on the medium + case ESP_ERR_INVALID_STATE: // ...and its VFS path is registered + ec = std::make_error_code(std::errc::file_exists); + return false; + default: + logger_.error("MSC medium {}: format failed: {}", lun, esp_err_to_name(err)); + ec = std::make_error_code(std::errc::io_error); + return false; + } +#else + (void)lun; + ec = std::make_error_code(std::errc::function_not_supported); + return false; +#endif +} + +void UsbDevice::set_msc_event_callback(const msc_event_callback_fn &cb) { + std::lock_guard lock(cb_mutex_); + on_msc_event_ = cb; +} + } // namespace espp diff --git a/doc/Doxyfile b/doc/Doxyfile index 7f048089b..f6cd973ea 100755 --- a/doc/Doxyfile +++ b/doc/Doxyfile @@ -202,6 +202,7 @@ EXAMPLE_PATH = \ $(PROJECT_PATH)/components/twai/example/main/twai_example.cpp \ $(PROJECT_PATH)/components/tt21100/example/main/tt21100_example.cpp \ $(PROJECT_PATH)/components/usb_device/example/main/usb_cdc_example.cpp \ + $(PROJECT_PATH)/components/usb_device/msc_example/main/msc_example.cpp \ $(PROJECT_PATH)/components/usb_device/xinput_example/main/xinput_example.cpp \ $(PROJECT_PATH)/components/usb_host/example/main/usb_host_example.cpp \ $(PROJECT_PATH)/components/wdi/ble_example/main/wdi_ble_example.cpp \ diff --git a/doc/en/buses/msc_example.md b/doc/en/buses/msc_example.md new file mode 100644 index 000000000..6c72f48ef --- /dev/null +++ b/doc/en/buses/msc_example.md @@ -0,0 +1,2 @@ +```{include} ../../../components/usb_device/msc_example/README.md +``` diff --git a/doc/en/buses/usb_cdc.rst b/doc/en/buses/usb_cdc.rst index f78c0416b..adfb66ea6 100644 --- a/doc/en/buses/usb_cdc.rst +++ b/doc/en/buses/usb_cdc.rst @@ -27,12 +27,13 @@ Today it can enable, in any combination (subject to the endpoint budget): Xbox 360 VID/PID and the built-in vendor class also claims interface class 0xFF, **use X-Input as the only enabled function** (Microsoft's IDs, for emulation / testing of your own device only). +- An **MSC** (mass storage) function exposing an SD card and/or a FAT partition in + flash as USB drives, shared with the application through an ownership hand-over. Interface numbers, endpoint addresses and string indices are allocated *sequentially* as functions are enabled, and the result is checked against the USB-OTG endpoint budget (an error is reported via ``std::error_code`` if it is -exceeded). The model is designed so an **MSC** function can be added later -without changing the descriptor-building approach. +exceeded). Because it uses the native USB-OTG peripheral rather than the built-in USB-Serial-JTAG that carries the ESP console, a device can advertise its own USB @@ -52,6 +53,8 @@ Features ``hid-rp`` in the example) and ``write_hid_report()`` - X-Input interface (wired Xbox 360 controller) via a custom application class driver, with ``update_xinput_state()`` and an ``on_rumble`` callback +- Mass storage (MSC): an SD card and/or a wear-levelled FAT flash partition as USB + drives, handed between the application (VFS file access) and the host - Console over CDC: optionally route the ESP console (stdout) to the CDC interface (``CdcFunction::route_console`` or ``route_console_to_cdc()``) so one native USB cable carries the logs alongside a vendor / HID / XInput interface; non-blocking, @@ -173,6 +176,67 @@ interface uses one interrupt-IN (0x81) + one interrupt-OUT endpoint with separat endpoint numbers, and the report DMA buffers are word-aligned as the ESP32-S3 DWC2 requires. +Enabling mass storage (MSC) +--------------------------- + +The MSC function exposes up to **two media** as USB drives: an SD card and/or a +FAT data partition in flash (accessed through wear levelling). It is built on +esp_tinyusb's MSC storage backend, which provides the SCSI handling, so it needs +``CONFIG_TINYUSB_MSC_ENABLED=y``; flash media additionally need +``CONFIG_TINYUSB_MSC_BUFSIZE >= CONFIG_WL_SECTOR_SIZE``. Prefer 4096-byte +wear-levelling sectors with a 4096-byte MSC buffer (the ``msc_example`` does): NOR +flash erases in 4 KiB blocks and esp_tinyusb erases before writing, so a host write +is then one erase and one write. 512-byte sectors need a read-modify-erase of the +block per sector (4 erases in Safety mode; Performance mode loses the whole block +on a reset mid-erase), making small host edits take seconds. 4 KiB sectors cost +4 KiB of RAM per volume / open file and at least 4 KiB of space per file, and the +host sees 4 KiB logical sectors. SD cards avoid the trade-off entirely: the card's +controller manages erase blocks, so sectors are written directly with no ESP-side +wear levelling. + +A medium belongs to one side at a time, so the firmware and a PC never write the +same FAT volume at once. While the **application** owns it, the volume is mounted +at the medium's ``base_path`` and ordinary file APIs work (``fopen``, +``std::fstream``, ``std::filesystem``); a connected host sees "no medium". While the +**host** owns it, ``base_path`` is unmounted and the PC sees the volume. With +``MscFunction::auto_handover`` (the default) the host takes the media when it +mounts the device and the application gets them back when the host ejects the +drive or the device is detached (for all media at once: esp_tinyusb ignores +which drive was ejected). Set ``Config::connect_on_initialize = false`` and call +``connect()`` to finish application I/O before any host can take a medium; turn it off to decide with ``set_msc_owner()``. Taking a +medium from an attached host is refused (``device_or_resource_busy``), since host +writes already queued could land under the application's volume; eject the drive +on the host first. ``msc_owner()``, ``msc_capacity()`` and +``MscFunction::on_event`` report the state (the event callback runs in the +TinyUSB task for host-driven hand-overs). + +.. code-block:: cpp + + espp::UsbDevice::MscMedium card; + card.type = espp::UsbDevice::MscMedium::Type::SdCard; + card.sd_card = sd_card; // an initialized sdmmc_card_t* (SDMMC or SDSPI host) + card.base_path = "/sdcard"; // do not also mount the card with esp_vfs_fat_*_mount() + + espp::UsbDevice::MscMedium flash; + flash.type = espp::UsbDevice::MscMedium::Type::FlashPartition; + flash.partition_label = "storage"; // a `data, fat` partition + flash.base_path = "/data"; + flash.volume_label = "MY DATA"; // drive name on the host; needs CONFIG_FATFS_USE_LABEL=y + + espp::UsbDevice::MscFunction msc; + msc.media = {card, flash}; // LUN 0 and LUN 1 + cfg.msc = msc; + +Limits, all from esp_tinyusb's backend: at most one SD card and one flash +partition; SD card media need a target with an SDMMC host peripheral (ESP32-S3 / +-P4), even for an SPI-wired card; the SCSI inquiry strings are fixed. Formatting +(``format_if_unformatted`` / ``format_msc_medium()``) runs on FatFs drive 0 rather +than the medium's own drive, so only use it when no other FAT volume is mounted. A +host can only read FAT, so LittleFS or SPIFFS partitions cannot be exposed as a +drive. Destroying the ``UsbDevice`` releases the media: an application-owned +medium's ``base_path`` is unmounted (an SD card stays initialized, but must be +mounted again if the application still needs it). + Routing the console over CDC ---------------------------- @@ -227,7 +291,7 @@ OUT endpoints**. Each function consumes: * - X-Input (Xbox 360) - 1 (interrupt-IN) - 1 (interrupt-OUT) - * - MSC (future) + * - MSC - 1 (bulk-IN) - 1 (bulk-OUT) @@ -243,20 +307,6 @@ hard limit and is not recommended. ``espp::UsbDevice`` computes the totals as functions are enabled and returns ``std::errc::value_too_large`` if the IN or OUT budget is exceeded. -Extending with MSC ------------------- - -The **HID** function is implemented (see "Enabling the HID class" above): it -appends one HID interface (application-supplied report descriptor) claiming an -interrupt-IN endpoint, plus an optional interrupt-OUT endpoint. -``espp::UsbDevice::Config`` still reserves a ``std::optional`` slot for an -``MscFunction`` as a documented extension point; it is not implemented yet, and -enabling it today makes ``initialize()`` fail with -``std::errc::function_not_supported``. When implemented it slots into the same -sequential interface / endpoint / string allocator: an MSC function appends one -MSC interface (SCSI + storage read/write/capacity callbacks) claiming a bulk IN + -bulk OUT endpoint. - Notes ----- @@ -281,6 +331,7 @@ Notes usb_cdc_example.md xinput_example.md + msc_example.md .. ---------------------------- API Reference ----------------------------------