Repository navigation
[Feature Request]: Add Scatter-Gather Design Pattern #3577
Description
Name of the Design Pattern
Scatter-Gather Pattern
Category
Concurrency / Integration Patterns
Intent
The Scatter-Gather design pattern is a messaging and integration pattern used to parallelize task execution. It broadcasts (scatters) a single request to multiple recipients or worker services concurrently, and then aggregates (gathers) their responses back into a single response. This significantly reduces overall response time when fetching data from multiple independent sources.
Detailed Explanation
In enterprise architectures, a client request often requires gathering information from multiple downstream services or running multiple independent calculations. Running these calls sequentially can lead to high latency.
The Scatter-Gather pattern addresses this by:
- Scatter: Spawning multiple parallel threads or asynchronous tasks (workers) to call each downstream service or execute task partitions.
- Gather: Waiting for all tasks to finish (or timing out if some are slow), collecting the results, and passing them to an aggregator.
- Aggregate: Merging individual responses into a unified result format returned to the client.
Real-World Example
Consider a travel booking aggregator (like Kayak or Expedia). When a user searches for a flight, the portal scatters the query to multiple airlines' API servers simultaneously. It gathers the prices, sorts them, and displays the complete list to the user.
Conceptual Architecture
Here is a conceptual design for the Scatter-Gather pattern:
classDiagram
direction TB
class TaskSupplier {
<<interface>>
+execute() String
}
class AirlineService {
-String airlineName
-double basePrice
+execute() String
}
class ScatterGatherExecutor {
-ExecutorService executor
+scatterGather(List~TaskSupplier~ tasks, long timeoutMs) List~String~
}
class Aggregator {
+aggregate(List~String~ results) String
}
TaskSupplier <|.. AirlineService
ScatterGatherExecutor --> TaskSupplier : scatters tasks
ScatterGatherExecutor .. Aggregator : passes results to
Activity
bonigo1998 commentedon Aug 27, 2026
Hi, I'd like to work on this issue.
I'm planning to implement the Scatter-Gather pattern using a TaskSupplier interface, example AirlineService implementations, a ScatterGatherExecutor backed by ExecutorService, and an Aggregator to combine the results.
I’ll also add tests and documentation following the repository’s existing pattern structure and contribution guidelines. Please let me know if you recommend any changes to this approach before I begin.
bonigo1998 commentedon Aug 27, 2026
I noticed the existing fanout-fanin pattern lists Scatter-Gather as an alternate name. For #3577, should the new implementation specifically demonstrate the integration-style variant described in the issue—broadcasting one request to multiple independent services, supporting timeout/partial responses, and aggregating their responses—or would you prefer changes to the existing Fan-Out/Fan-In module instead?
sohamcodes-ctrl commentedon Aug 27, 2026
Hi @iluwatar, I’d like to work on this issue as my first open-source contribution.
I’ve reviewed the proposed Scatter-Gather pattern and understand the implementation requirements, including parallel task execution, gathering results, and aggregating them into a unified response.
I’m interested in implementing this following the project’s existing conventions and adding appropriate tests and documentation.
Could you please assign this issue to me? I’d be happy to take ownership of the implementation and submit a PR.
Thank you!
ylcn91 commentedon Sep 3, 2026
Hi @iluwatar, I opened an independent implementation in #3601 (different domain and design from #3590) with a populated, logging App entry point, timeout handling with partial results, and an explicit note on how it differs from fanout-fanin. Happy to align with whichever direction you prefer.
feat: add Scatter-Gather pattern (iluwatar#3577)
feat: add Scatter-Gather pattern (iluwatar#3577)
iluwatar commentedon Sep 7, 2026
@bonigo1998 well spotted and good question. I think we could have separate implementations for Fan-Out/Fan-In (already existing) and Scatter-Gather (this issue).
The patterns are similar, but emphasis is a little bit different