Skip to content

Add io/mmap state machine for tensor-backed model loading #61

@gabewillen

Description

@gabewillen

Summary

Add an io/mmap state machine under src/emel/io that provides a memory-mapped tensor-loading strategy beneath tensor-owned residency management.

This issue should land after the architectural seams in #59 and #60 are in place.

Motivation

Memory mapping is a distinct loading strategy with different behavior than staged reads or explicit copies:

  • low-copy / low-setup residency when the platform and file layout allow it
  • different lifetime and eviction semantics than copy-based loading
  • different failure modes and portability constraints than explicit read-based strategies

It should be modeled as its own strategy actor, not embedded inline into tensor or loader orchestration.

Scope

  • Add src/emel/io/... machine(s) for mmap-backed tensor loading.
  • Define the request/result events needed for tensor-owned loading to request a mapped tensor buffer.
  • Handle mmap-specific success/error behavior explicitly.
  • Integrate with the tensor-to-io boundary introduced by Add emel/io module and tensor-to-io orchestration boundary #60.

Hard Constraints

  • This issue implements the mmap strategy only.
  • Tensor lifecycle/residency semantics remain owned by model/tensor.
  • Platform-specific mapping details must stay behind the I/O abstraction boundary.
  • Do not fold read/copy or staged/chunked logic into this issue.

Acceptance Criteria

  • A dedicated mmap strategy state machine exists under src/emel/io.
  • Tensor-owned load flow can request mmap-backed residency through the io boundary.
  • mmap-specific errors are surfaced explicitly and deterministically.
  • Tests cover supported-path behavior and representative failure handling.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions