feat: add payload offloader support (Approach B)

README.md

@@ -23,6 +23,7 @@ Build resilient, long-running AWS Lambda functions that automatically checkpoint
 - **Replay Safety** – Functions deterministically resume from checkpoints after interruptions
 - **Type Safety** – Full generic type support for step results
 - **Data-Driven Concurrency** – Apply a function across a collection with `map()`, with per-item error isolation and configurable completion criteria
+- **Payload Offloading** – Keep large serialized payloads in durable external storage while checkpoints retain compact references
 
 ## How It Works
 
@@ -50,6 +51,16 @@ Your durable function extends `DurableHandler<I, O>` and implements `handleReque
 </dependency>
 ```
 
+Filesystem payload offloading is available as an optional artifact:
+
+```xml
+<dependency>
+    <groupId>software.amazon.lambda.durable</groupId>
+    <artifactId>aws-durable-execution-sdk-java-extra-filesystem-offloader</artifactId>
+    <version>VERSION</version>
+</dependency>
+```
+
 ### Your First Durable Function
 
 ```java
@@ -111,6 +122,7 @@ See [Deploy Lambda durable functions with Infrastructure as Code](https://docs.a
 **Advanced Topics**
 
 - [<u>Configuration</u>](docs/advanced/configuration.md) - Customize SDK behaviour
+- [<u>Payload Offloading</u>](docs/advanced/configuration.md#payload-offloading) - Store serialized payloads outside checkpoints
 - [<u>Error Handling</u>](docs/advanced/error-handling.md) - SDK exceptions for handling failures
 - [<u>Logging</u>](docs/advanced/logging.md) - How to use DurableLogger
 - [<u>Migrating from 1.x to 2.x</u>](docs/migration-1.x-to-2.x.md) - Upgrade guide for breaking changes since `v1.2.1`

coverage-report/pom.xml

@@ -22,6 +22,11 @@
             <artifactId>aws-durable-execution-sdk-java</artifactId>
             <version>${project.version}</version>
         </dependency>
+        <dependency>
+            <groupId>software.amazon.lambda.durable</groupId>
+            <artifactId>aws-durable-execution-sdk-java-extra-filesystem-offloader</artifactId>
+            <version>${project.version}</version>
+        </dependency>
         <dependency>
             <groupId>software.amazon.lambda.durable</groupId>
             <artifactId>aws-durable-execution-sdk-java-testing</artifactId>

docs/adr/005-filesystem-serdes.md

@@ -1,8 +1,11 @@
 # ADR-005: Payload Offloading for Filesystem Storage
 
-**Status:** Proposed  
+**Status:** Accepted
+
 **Date:** 2026-07-02
 
+**Decision:** Approach B, a dedicated `PayloadOffloader` interface with an optional filesystem implementation.
+
 ## Context
 
 Issue [#463](https://github.com/aws/aws-durable-execution-sdk-java/issues/463) asks for Java parity with the JavaScript SDK's filesystem-backed SerDes. The JavaScript implementation receives a `SerdesContext` containing a stable durable execution ARN and an entity ID, then stores either inline JSON or a file pointer in the checkpoint payload. That context lets the implementation choose a collision-free path for each operation payload.
@@ -407,9 +410,9 @@ This approach gives the SDK one consistent policy for root payloads, operation r
 | Immediate delivery risk | Lower. Builds on existing customization point. | Higher. Requires new API and more runtime integration. |
 | Long-term design risk | Higher. Blurs SerDes semantics and may accumulate storage behavior in serializers. | Lower if offloading grows into a first-class feature, but higher if this remains a one-off filesystem parity feature. |
 
-## AI Recommendation
+## Decision Rationale
 
-**AI recommendation:** Prefer **Approach B: Create a `PayloadOffloader` interface** if the team is willing to treat payload offloading as a first-class Java SDK capability rather than only a JavaScript parity item.
+Approach B was selected because payload offloading is treated as a first-class Java SDK capability rather than only a JavaScript parity item.
 
 Reasoning:
 
@@ -419,7 +422,7 @@ Reasoning:
 - Both approaches use one optional extra package for filesystem-specific code; that is not a differentiator. The package would be either filesystem SerDes or filesystem offloader depending on the chosen approach. The differentiator is that Approach B gives future storage extras such as S3 or DynamoDB offload the same focused core offloader contract instead of encoding storage behavior as more SerDes implementations.
 - SDK-owned envelopes and two-layer caching make replay behavior easier to test and reason about.
 
-The main reason to choose Approach A is schedule and parity: it is smaller and maps directly to the JavaScript feature request. If the team needs to satisfy #463 quickly with minimal public API design, Approach A is a reasonable incremental step, but it should be documented as payload offloading implemented through SerDes rather than as the long-term ideal boundary.
+Approach A remains smaller and maps directly to the JavaScript feature shape, but it would make serialization responsible for external storage and require thread-local context propagation.
 
 ## Other Alternatives Considered
 
@@ -459,7 +462,7 @@ Rejected. The backend request/response envelope is protocol data. User payload c
 
 Positive:
 
-- Both approaches enable filesystem-backed payload storage without changing the existing `SerDes` interface.
+- Filesystem-backed payload storage is available without changing the existing `SerDes` interface.
 - Filesystem-specific functionality stays out of the core SDK artifact.
 - The repository gets a repeatable `aws-durable-execution-sdk-java-extra-xxx` artifact pattern for optional packages.
 - Custom payload implementations get enough context to use external storage safely.
@@ -471,15 +474,12 @@ Negative:
 
 - Adds executor, context, and caching machinery that must stay deterministic.
 - Adds at least one Maven module and published artifact to release and document.
-- Approach A requires thread-local SerDes context because the existing `SerDes` methods do not accept context.
 - Repeated `get()` calls may return the same object instance in one invocation.
 - Filesystem-backed storage introduces operational durability requirements outside the SDK's control.
-- Approach A risks overloading the meaning of SerDes.
 - Approach B requires a larger core SDK design before delivering filesystem parity.
 
 Deferred:
 
-- Choosing whether payload offloading is a first-class SDK concept or a parity feature implemented through SerDes.
 - A fully async Java SerDes or payload pipeline contract.
 - A separate, explicitly dangerous protocol-envelope customization API.
 - File cleanup, retention policies, and lifecycle management for offloaded payloads.

docs/advanced/configuration.md

@@ -17,7 +17,9 @@ public class OrderProcessor extends DurableHandler<Order, OrderResult> {
         return DurableConfig.builder()
             .withLambdaClientBuilder(lambdaClientBuilder)
             .withSerDes(new MyCustomSerDes())           // Custom serialization
+            .withPayloadOffloader(myPayloadOffloader)   // Optional external payload storage
             .withExecutorService(Executors.newFixedThreadPool(10))  // Custom thread pool
+            .withPayloadOffloadExecutorService(payloadIoExecutor)    // Blocking payload I/O
             .withLoggerConfig(LoggerConfig.withReplayLogging())     // Enable replay logs
             .build();
     }
@@ -33,13 +35,86 @@ public class OrderProcessor extends DurableHandler<Order, OrderResult> {
 |-----------------------------|-----------------------------------------|-------------------------------|
 | `withLambdaClientBuilder()` | Custom AWS Lambda client                | Auto-configured Lambda client |
 | `withSerDes()`              | Serializer for step results             | Jackson with default settings |
+| `withPayloadOffloader()`    | External storage for serialized user payloads | Disabled |
 | `withExecutorService()`     | Thread pool for user-defined operations | Cached daemon thread pool     |
+| `withPayloadOffloadExecutorService()` | Thread pool for blocking payload storage I/O | Cached daemon thread pool |
 | `withLoggerConfig()`        | Logger behavior configuration           | Suppress logs during replay   |
 | `withPollingStrategy()`     | Backend polling strategy                | Exponential backoff: 1s base, 2x rate, FULL jitter, 10s max |
 | `withCheckpointDelay()`     | How often the SDK checkpoints updates   | `Duration.ofSeconds(0)` (as soon as possible) |
 
 The `withExecutorService()` option configures the thread pool used for running user-defined operations. Internal SDK coordination (checkpoint batching, polling) runs on an SDK-managed thread pool.
 
+### Payload offloading
+
+`SerDes` remains responsible for converting objects to serialized text. A `PayloadOffloader` runs after serialization
+and decides whether that text remains inline or is stored externally. On replay, the SDK resolves the stored reference
+before passing the serialized text back to `SerDes`.
+
+Add the optional filesystem artifact:
+
+```xml
+<dependency>
+    <groupId>software.amazon.lambda.durable</groupId>
+    <artifactId>aws-durable-execution-sdk-java-extra-filesystem-offloader</artifactId>
+    <version>VERSION</version>
+</dependency>
+```
+
+Configure a durable shared mount:
+
+```java
+import java.nio.file.Path;
+import software.amazon.lambda.durable.extra.filesystem.FileSystemPathEncoding;
+import software.amazon.lambda.durable.extra.filesystem.FileSystemPayloadOffloader;
+import software.amazon.lambda.durable.extra.filesystem.PayloadOffloadMode;
+
+var offloader = FileSystemPayloadOffloader.builder(Path.of("/mnt/efs/durable-payloads"))
+    .storageMode(PayloadOffloadMode.OVERFLOW)
+    .pathEncoding(FileSystemPathEncoding.HASH)
+    .previewGenerator((serialized, context) -> Map.of(
+        "entityId", context.entityId(),
+        "payloadKind", context.payloadKind().name()))
+    .build();
+
+return DurableConfig.builder()
+    .withSerDes(new JacksonSerDes())
+    .withPayloadOffloader(offloader)
+    .build();
+```
+
+`ALWAYS` writes every serialized payload to the filesystem. `OVERFLOW` keeps payloads inline until they approach the
+256 KB checkpoint limit, then stores them externally. `URI` path encoding produces readable directories and file names;
+`HASH` uses fixed-length SHA-256 names and is safer for long or unusual entity identifiers.
+
+The global offloader applies to root input/output, operation results, invoke payloads, callback results,
+wait-for-condition state, child/map/parallel results, and serialized exception data. Operation configuration can
+override it:
+
+```java
+var stepConfig = StepConfig.builder()
+    .payloadOffloader(otherOffloader)
+    .build();
+
+var inlineStepConfig = StepConfig.builder()
+    .payloadOffloader(PayloadOffloader.disabled())
+    .build();
+```
+
+The same `payloadOffloader(...)` option is available on `InvokeConfig`, `CallbackConfig`,
+`RunInChildContextConfig`, `MapConfig`, `ParallelConfig`, `ParallelBranchConfig`, and
+`WaitForConditionConfig`.
+
+The SDK uses a versioned checkpoint envelope and continues to read payloads written by older SDK versions as raw
+serialized text. Within one Lambda invocation, resolved storage data and deserialized objects are cached, so repeated
+`DurableFuture.get()` calls do not repeatedly read the same file.
+
+> **Do not use Lambda `/tmp` for durable payloads.** It is local to one execution environment and might not exist on
+> replay. Use a shared durable filesystem such as EFS. S3 Files can have delayed synchronization and recent writes can
+> be lost if the runtime crashes before the mount flushes; use it only when that durability tradeoff is acceptable.
+
+The SDK does not delete offloaded files. Configure storage lifecycle and retention separately, and keep the mounted
+path accessible to every function environment that may replay or consume the payload.
+
 ### Dynamic plugin loading
 
 Dynamic plugin loading is an opt-in alternative to registering plugins in application code. Put provider JARs on the application class path, then set `DURABLE_EXECUTION_PLUGINS` to an ordered, comma-separated list of provider names:

docs/design.md

@@ -11,6 +11,7 @@ This document explains the internal architecture, threading model, and extension
 ```
 aws-durable-execution-sdk-java/
 ├── sdk/                      # Core SDK - DurableHandler, DurableContext, operations
+├── extra-filesystem-offloader/ # Optional durable filesystem payload storage
 ├── sdk-testing/              # Test utilities for local and cloud testing
 ├── sdk-integration-tests/    # Integration tests using LocalDurableTestRunner
 └── examples/                 # Real-world usage patterns as customers would implement them
@@ -19,6 +20,7 @@ aws-durable-execution-sdk-java/
 | Module | Purpose | Key Classes |
 |--------|---------|-------------|
 | `sdk` | Core runtime - extend `DurableHandler`, use `DurableContext` for durable operations | `DurableHandler`, `DurableContext`, `DurableExecutor`, `ExecutionManager` |
+| `extra-filesystem-offloader` | Optional payload offloader for durable shared filesystems | `FileSystemPayloadOffloader` |
 | `sdk-testing` | Test utilities: `LocalDurableTestRunner` (in-memory, simulates re-invocations and time-skipping) and `CloudDurableTestRunner` (executes against deployed Lambda) | `LocalDurableTestRunner`, `CloudDurableTestRunner`, `LocalMemoryExecutionClient`, `TestResult` |
 | `sdk-integration-tests` | Dogfooding tests - validates the SDK using its own test utilities. Separate module keeps dependencies acyclic: `sdk` → `sdk-testing` → `sdk-integration-tests`. | Test classes only |
 | `examples` | Real-world usage patterns as customers would implement them, with local and cloud tests | Example handlers, `CloudBasedIntegrationTest` |

extra-filesystem-offloader/README.md

@@ -0,0 +1,34 @@
+# Filesystem Payload Offloader
+
+This optional module stores serialized durable execution payloads on a shared filesystem while the durable checkpoint
+contains a compact SDK-owned reference.
+
+```xml
+<dependency>
+    <groupId>software.amazon.lambda.durable</groupId>
+    <artifactId>aws-durable-execution-sdk-java-extra-filesystem-offloader</artifactId>
+    <version>VERSION</version>
+</dependency>
+```
+
+```java
+var offloader = FileSystemPayloadOffloader.builder(Path.of("/mnt/efs/durable-payloads"))
+    .storageMode(PayloadOffloadMode.OVERFLOW)
+    .pathEncoding(FileSystemPathEncoding.HASH)
+    .build();
+
+return DurableConfig.builder()
+    .withPayloadOffloader(offloader)
+    .build();
+```
+
+- `ALWAYS` stores every payload in a file.
+- `OVERFLOW` keeps small payloads inline and offloads values near the 256 KB checkpoint limit.
+- `URI` creates readable paths.
+- `HASH` creates fixed-length SHA-256 path segments.
+
+Do not use Lambda `/tmp`: it is not shared across environments or guaranteed to survive replay. EFS provides a shared
+durable mount. S3 Files can delay synchronization and can lose recent writes if the runtime crashes before a flush, so
+use it only when the application accepts that tradeoff.
+
+The SDK does not delete files. Configure retention and lifecycle management on the backing storage.

extra-filesystem-offloader/pom.xml

@@ -0,0 +1,75 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+
+    <parent>
+        <groupId>software.amazon.lambda.durable</groupId>
+        <artifactId>aws-durable-execution-sdk-java-parent</artifactId>
+        <version>2.1.1-SNAPSHOT</version>
+    </parent>
+
+    <artifactId>aws-durable-execution-sdk-java-extra-filesystem-offloader</artifactId>
+    <packaging>jar</packaging>
+
+    <name>AWS Lambda Durable Execution SDK Filesystem Payload Offloader</name>
+    <description>Filesystem-backed payload offloader for the AWS Lambda Durable Execution SDK for Java</description>
+    <url>https://github.com/aws/aws-durable-execution-sdk-java</url>
+
+    <scm>
+        <connection>scm:git:https://github.com/aws/aws-durable-execution-sdk-java.git</connection>
+        <developerConnection>scm:git:https://github.com/aws/aws-durable-execution-sdk-java.git</developerConnection>
+        <url>https://github.com/aws/aws-durable-execution-sdk-java</url>
+    </scm>
+
+    <dependencies>
+        <dependency>
+            <groupId>software.amazon.lambda.durable</groupId>
+            <artifactId>aws-durable-execution-sdk-java</artifactId>
+            <version>${project.version}</version>
+        </dependency>
+        <dependency>
+            <groupId>org.junit.jupiter</groupId>
+            <artifactId>junit-jupiter</artifactId>
+            <scope>test</scope>
+        </dependency>
+    </dependencies>
+
+    <build>
+        <plugins>
+            <plugin>
+                <groupId>org.apache.maven.plugins</groupId>
+                <artifactId>maven-compiler-plugin</artifactId>
+            </plugin>
+            <plugin>
+                <groupId>org.apache.maven.plugins</groupId>
+                <artifactId>maven-surefire-plugin</artifactId>
+            </plugin>
+            <plugin>
+                <groupId>org.apache.maven.plugins</groupId>
+                <artifactId>maven-source-plugin</artifactId>
+                <executions>
+                    <execution>
+                        <id>attach-sources</id>
+                        <goals>
+                            <goal>jar-no-fork</goal>
+                        </goals>
+                    </execution>
+                </executions>
+            </plugin>
+            <plugin>
+                <groupId>org.apache.maven.plugins</groupId>
+                <artifactId>maven-javadoc-plugin</artifactId>
+                <executions>
+                    <execution>
+                        <id>attach-javadocs</id>
+                        <goals>
+                            <goal>jar</goal>
+                        </goals>
+                    </execution>
+                </executions>
+            </plugin>
+        </plugins>
+    </build>
+</project>

extra-filesystem-offloader/src/main/java/software/amazon/lambda/durable/extra/filesystem/FileSystemPathEncoding.java

@@ -0,0 +1,12 @@
+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+// SPDX-License-Identifier: Apache-2.0
+package software.amazon.lambda.durable.extra.filesystem;
+
+/** Controls how durable execution and entity identifiers are encoded into filesystem paths. */
+public enum FileSystemPathEncoding {
+    /** Use percent-encoded, human-readable path segments. */
+    URI,
+
+    /** Use fixed-length SHA-256 hashes for filesystem-safe path segments. */
+    HASH
+}