feat(server): add MoE expert-split multi-target foundation

[weicj]

Summary

This PR introduces expert_split, a shared MoE-native substrate for ordered multi-target expert placement and execution.

It defines explicit expert ownership over target lists such as cuda:0, cuda:1, hip:0, cpu, with per-target capacity, backend, and runtime metadata. The goal is to make multi-target routed-expert placement a reusable backend surface instead of continuing to grow it out of one existing route.

qwen35moe is the first backend integrated with this substrate and serves as the reference carrier in this PR.

Why A Separate Path

The new path is worth landing independently because it solves a different problem shape from the existing heterogeneous placement options:

• hot/cold is expert-granular, but it is fundamentally a two-tier budgeted placement model.
• parent/remote keeps the same placement model and only changes where the cold tier is executed.
• layer split is also a heterogeneous placement route, but it is coarse layer ownership. Each shard owns a contiguous or configured layer span, and only activations cross the boundary.
• expert_split is a real multi-target expert ownership model. Each (layer, expert) is assigned to a concrete target from an ordered target list, and runtime dispatch recovers that target-local placement directly.

This is the placement shape needed for large routed-expert deployments that want more than one non-CPU expert target before CPU fallback.

How It Works

The shared substrate is built out of a few abstractions with distinct roles:

• ExpertSplitTargetConfig:
  normalizes the ordered expert target list and per-target capacity policy into one model-agnostic target contract.
• ExpertSplitPlan:
  decides expert ownership over (layer, expert) units. This is the placement decision itself, independent of backend-local tensor loading or dispatch.
• ExpertSplitRuntime:
  turns the plan into execution-neutral lookup state, so the runtime can answer which target owns a given expert and how that ownership is arranged per layer.
• ExpertSplitMaterialization:
  converts the shared ownership plan into the storage-facing view needed by the current primary backend path, including the primary placement and ordered cold union.
• ExpertSplitComputeRuntime:
  converts the same ownership plan into the execution-facing view, including target-local expert maps that routed-expert dispatch can use directly.
• ExpertSplitStateComponents:
  packages those related products into one backend-facing bundle.
• MoeExpertCompute:
  is the common routed-expert execution contract. It allows the backend to keep execution local, attach IPC-backed secondary targets, or compose a multi-target runtime from both.

Model backends and adapters remain responsible for model-specific metadata, tensor binding, and request lifecycle, while reusing the shared target parsing, planning, runtime assembly, and dispatch contracts.

The compatibility boundary stays explicit: the existing single-primary GPU -> CPU route remains intact, and the multi-target compute path activates only when the configured expert target layout actually requires more than one
non-CPU target.

First Integration

qwen35moe is the first backend wired to the shared substrate because it combines:

• an MoE backend with existing hot/cold semantics;
• established comparison lanes across pure CUDA all-hot, pure HIP all-hot, mixed layer-split, parent/remote expert compute, and hot/cold / Spark-style budgeted routes;
• a real large-weight roadmap where ordered multi-target expert ownership is necessary rather than optional.

That makes it a practical comparison environment for hardening expert_split against different heterogeneous placement shapes before broader MoE adapter rollout.

The same substrate is intended to carry future MoE adapters and execution backends without each model family inventing its own target ordering, capacity parsing, and routed-expert dispatch contract.

Validation

Local validation covers the new planner, runtime, and multi-target compute path with dedicated unit coverage.

Remote validation on a mixed CUDA/HIP host confirmed both runtime viability and large-scale startup:

Qwen35MoE runtime viability

• a real qwen35moe request completed end to end on the expert-split path;
• the runtime used explicit ordered expert targets rather than the existing single-primary hot/cold route;
• the run confirms that the substrate is active in a real runtime path.

397B-class Qwen35MoE runtime evidence

On deepreinforce-ai_Ornith-1.0-397B-IQ2_XXS, a 397B-class qwen35moe-family routed-expert model, the same substrate also proved real multi-target startup:

• ordered expert targets launched successfully on cuda:0, hip:0, cpu
• logs confirmed target-local placements such as:
  • cuda:0=5861|hip:0=41|cpu=24818
  • cuda:0=5290|hip:0=41|cpu=25389

Together, these runs show that the substrate is live on both ordinary qwen35moe execution and 397B-class routed-expert startup.