Ship the quantized kernels as their own library

[shoumikhin]

A quantized model uses smaller numbers than a normal one, so the tensors take less memory.
Running one needs the quantized operator kernels.

The only copy the wheel shipped is the one torch loads to export a model, which a C++ application
cannot use. Such an application links the runtime, loads a quantized model, and the model fails at
run time with a missing operator, which looks like a model problem rather than a packaging one.

Build the quantized kernels as their own shared library and name it as a CMake component, the same
way the other kernel sets are named.

find_package(executorch REQUIRED COMPONENTS kernels_quantized)
target_link_libraries(my_app PRIVATE executorch::runtime
                                     executorch::kernels_quantized)

The wheel now ships lib/libexecutorch_kernels_quantized.so.

Note that the wheel also ships a second copy of these kernels, inside the library torch loads when
you export a model. That copy is built into the plugin rather than resolved from the shared library,
so a process holding both registers the same operators twice, and the runtime treats that as fatal:

Re-registering quantized_decomposed::add.out

This affects only a process that does both, for example an application that embeds a Python
interpreter. A plain C++ application can link the component freely.

Because of that, this is the one component EXECUTORCH_LIBRARIES does not include, so an
application that links whatever the package offers cannot end up in that position without asking.
A consumer that wants the quantized kernels names the component, or on CMake older than 3.28, where
no component targets exist, links EXECUTORCH_QUANTIZED_KERNELS_LIBRARY as well. That variable is
now populated on both CMake routes, so a consumer that adopts the older-CMake recipe and later
upgrades keeps the library on their link line instead of silently losing it.

Built the wheel, installed it into a clean environment, and:

• exported a quantized model and ran it from Python, matching eager PyTorch to within the
  quantization step (measured worst difference 0.0048 against a tolerance of 0.02).
• built a C++ application that links executorch::kernels_quantized, ran the same program, and got
  the same output as Python, byte for byte.
• confirmed the Python extension does not depend on the run-time copy, and that a process holding
  the shipped library and the export plugin aborts in either load order.
• checked every shipped library the same way, to establish that this is the only pair that
  collides: the CPU kernels, the delegate, the thread pool, the profiler and the runtime all
  coexist with both the extension and the export plugin.
• an application linking only EXECUTORCH_LIBRARIES does not depend on the quantized library while
  still depending on the CPU kernels, on CMake 3.28 and on real CMake 3.24. A new check asserts
  this, and it fails on the previous behaviour.
• EXECUTORCH_QUANTIZED_KERNELS_LIBRARY resolves to the shipped library on both the modern-CMake
  route (as the imported target) and the pre-3.28 route (as a file path).
• a missing quantized library now fails the checks instead of skipping them. The preset that builds
  the wheel enables these kernels unconditionally, so their absence is a regression rather than a
  configuration to tolerate, and both the ownership table and the C++ check previously treated it as
  an acceptable state and reported coverage they had not run.

Ran on Linux x86_64 and aarch64.

[shoumikhin]

Stack from ghstack (oldest at bottom):

• Name the Python extension after what it contains #21816
• Ship linkable libraries in the macOS wheel #21771
• Ship the OpenVINO delegate as its own library #21770
• Build and publish CUDA wheels #21668
• Ship the CUDA delegate in the wheel #21645
• -> Ship the quantized kernels as their own library #21642
• Ship a C++ SDK in the wheel #21639
• Split the runtime out of the Python bindings extension #21610

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21642

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ 7 Pending, 2 Unrelated Failures

As of commit e847aaf with merge base 46f9696 ():

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• trunk / test-models-macos-coreml (resnet50) / macos-job (gh) (detected as infra flaky with no log or failing log classifier)

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Apple / build-demo-ios / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 65

This comment was automatically generated by Dr. CI and updates every 15 minutes.