Skip to content

[iOS] Split the image processor into its own SwiftPM product instead of bundling it in executorch #22722

Description

@shoumikhin

Problem

The image processor is bundled into the executorch xcframework, so the core runtime
product ends up carrying that helper's Apple framework dependencies.

scripts/build_apple_frameworks.sh merges nine static libraries into a single
executorch.xcframework, and libextension_image.a is one of them. Because it is merged,
a SwiftPM manifest cannot attach anything to that part alone: the smallest thing it can
describe is the whole product. So the frameworks the image processor needs have to be
declared on executorch itself.

That is what #22711 does, and it is the only place the current packaging allows. It reads,
though, as if the ExecuTorch runtime depends on Accelerate and Core Image, which it does
not. Every one of those symbols comes from the image processor:

image_processor_apple.o          8 undefined symbols   (vImage, from Accelerate)
image_processor_apple_gpu.o      8 undefined symbols   (Core Image)

No other object file in the shipped library references either framework.

There is also a size argument. The image processor is about 93 KB of a 967 KB library, so
roughly 10 percent, and every app that only wants Module and Tensor links it anyway.

Proposal

Give the image processor its own product, the way executorch_llm and executorch_dump
already work. Both build a separate xcframework and layer on the runtime rather than being
merged into it:

"executorch_llm": [
  "targets": ["executorch"],
],

An executorch_image product would follow that shape and own its dependencies:

"executorch_image": [
  "frameworks": ["Accelerate", "CoreGraphics", "CoreImage", "CoreVideo", "Foundation"],
  "targets": ["executorch"],
],

executorch then goes back to declaring only c++, and apps that do not process images
stop linking the helper at all.

What it involves

  1. Build a separate executorch_image.xcframework in scripts/build_apple_frameworks.sh,
    holding libextension_image.a.
  2. Add the product above to Package.swift, and remove the framework block from
    executorch. The same change is needed in Package.swift.template on the swiftpm
    branch, which is what prebuilt users actually get.
  3. Move the bindings, ExecuTorchImageProcessor.h/.mm and
    ExecuTorch+ImageProcessor.swift, out of extension/apple into the new product, and
    drop #import "ExecuTorchImageProcessor.h" from the ExecuTorch.h umbrella header.

Step 3 is a breaking change. Today import ExecuTorch gives you the image processor. After
the split it needs a second import, so anyone using it since the bindings landed has to add
a line. Worth calling out in release notes.

Nothing in the runtime depends on extension_image; the only reference is the extension
list in the top-level CMakeLists.txt. So there is no dependency to untangle beyond the
bindings.

Test plan

  • swift package --disable-sandbox dump-package shows executorch with only c++, and
    executorch_image with the five frameworks.
  • Build a consumer that depends on executorch alone and does not use the image processor,
    and confirm it links with no Apple frameworks declared.
  • Build a consumer that depends on executorch_image and uses the image processor, and
    confirm it links and runs on macOS, iOS device and the iOS simulator.
  • Run the iOS framework and demo app CI.

cc @shoumikhin @larryliu0820 @GregoryComer @cbilgin

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

module: build/installIssues related to the cmake and buck2 builds, and to installing ExecuTorchmodule: iosIssues related to iOS code, build, and execution

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions