[groupthinking]

The Map/Reduce guide described invocation, thread cancellation, tracking and verification guarantees that the implementation does not provide. This update reconciles the complete guide and module docstrings with current master and adds executable local examples for all five documented usage patterns.

• Document full-keyword dispatch and current built-in signatures. api_health_checker now accepts service/url after Consolidate workflow patterns with bounded correction and recovery #106; data_processor accepts actual data_path/source inputs. Runnable examples use real available data/log protocols and directories rather than silently ignored endpoint arguments or automatically generated placeholder protocols.
• Correct the lifecycle diagram and explain soft timeouts, continued worker activity, shared process state, the initial-map concurrency limit, untracked retries and timeout paths, coroutine-dispatch limitations, and final SQLite/Markdown snapshots.
• Document the complete constructor and exports, semantic-to-mechanical fallback, the inclusive 80% mechanical threshold, returned success:false behavior, compatible retry inputs, actual Slack CLI escalation and the legacy mutation mechanism's lack of sandbox/rollback guarantees.
• Preserve architecture, API, integration, configuration, failure handling and all five pattern categories: minimal, fan-out, heterogeneous tasks, custom gate/reducer and step-by-step execution. A local demonstration supplies real files, a result-aware gate, real persistence and visible local failure; research/classification recipes remain explicitly identified as application extensions.
• Clarify that max_job_history limits only the in-memory self.jobs collection. SQLite rows and STATE.md entries require separate retention; the guide no longer implies that pruning memory deletes persisted history.
• Preserve current master, including Consolidate workflow patterns with bounded correction and recovery #106. The final difference from 05e47c4b40bf5f9838fd1b837571afd283690bca is four files: the guide, module docstrings, one local example and eight focused tests. The orchestrator's executable AST is unchanged after excluding docstrings.

Validation at f896334b8f7b419721e363185f335a337aa5e940

Both hosted Python jobs, 3.10 and 3.11, in the current CI run completed with 143 passed, 3 skipped and 2 existing warnings per job. The complete job logs verify Black formatting, TypeScript validation and the full pytest suite. All eight current checks finished: seven passed and CodeRabbit skipped its review. The prior source d72faa95 also passed four hosted Python jobs with the same test totals.

The follow-up to Copilot's memory-retention finding changes only the guide. A local probe planned and persisted three actual jobs with max_job_history=1: one job remained in memory, while all three SQLite rows and Markdown entries remained. The eight existing documentation behavior tests pass. Runtime code is unchanged by this follow-up.

Independent local review verified all 255 current-master paths were preserved, all 257 candidate/source hashes matched, and the runtime AST remained unchanged. The five-pattern CLI and eight behavior tests passed. A separate probe used directories with 7 and 2 records and checked their actual SQLite/Markdown results, returned-failure/no-retry behavior, local escalation and the mechanical threshold. All 15 Python documentation blocks compile; lint passes on the new example and test files.

The three existing skips require configured quantum execution, a DWAVE token or an accessible MCP server. Repository-wide Flake8 retains advisory baseline diagnostics; this is not a claim that full-repository lint is clean. No live model provider, Slack recipient, production endpoint or deployment was contacted.

This corrects documentation and examples; the disclosed runtime limits remain. In particular, timeout does not terminate a worker thread, semantic-provider failure can fall back to mechanical approval, and legacy mutation is not sandboxed. No merge or production change is included in this review.

[coderabbitai]

Summary by CodeRabbit

• Documentation
  • Added extensive documentation for the orchestrator module including detailed coverage of system architecture, orchestration design patterns, complete API reference with code examples, practical usage scenarios from basic to advanced, guidance for customizing and extending functionality, comprehensive error handling and recovery documentation, state persistence information, memory optimization strategies, and integration guidelines for connecting with other system components.

Walkthrough

Adds docs/orchestrator_mapreduce.md, a 571-line reference document for the orchestrator_mapreduce module. The document covers the public API (__all__), data models (TaskStatus, SubTask, OrchestratedJob), HierarchicalOrchestrator constructor and lifecycle methods, customization contracts, protocol calling conventions via inspect.signature(), state persistence to STATE.md, failure tiers, usage examples, integration points, configuration reference, a lifecycle diagram, and a v1.0.0 changelog entry.

Changes

HierarchicalOrchestrator Documentation

Layer / File(s)	Summary
Module identity, public API, and data models docs/orchestrator_mapreduce.md	Documents module dependencies and __all__ exports alongside TaskStatus states and SubTask/OrchestratedJob field shapes.
Constructor and lifecycle methods docs/orchestrator_mapreduce.md	Documents HierarchicalOrchestrator constructor parameters and the run/plan/map_execute/reduce_verify methods, including bounded concurrency, per-subtask timeouts, retry/mutation loop, escalation, and STATE.md persistence. Includes the lifecycle flow diagram.
Customization contracts and protocol calling conventions docs/orchestrator_mapreduce.md	Documents the verification_gate return contract and reducer interface with worked examples; describes three task() calling conventions resolved via inspect.signature().
Persistence, failure tiers, and memory management docs/orchestrator_mapreduce.md	Documents STATE.md audit append behavior, the three-tier failure strategy (retry/escalate/timeout), and max_job_history pruning during plan().
Usage examples, integration, configuration, and changelog docs/orchestrator_mapreduce.md	Provides minimal through step-by-step usage examples; integration with protocols.loader, agents.mutator, utils.tracker, utils.logger; parameter configuration reference with tuning guidance; and v1.0.0 changelog entry.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• groupthinking/self-correcting-executor#114: Introduces the orchestrator_mapreduce.py implementation that this documentation directly describes, including HierarchicalOrchestrator, SubTask, OrchestratedJob, and the Plan→Map→Reduce lifecycle.

Suggested labels

documentation

Pre-merge checks: 4

Check name	Status	Explanation
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Title check		The title clearly identifies the documentation change and its two main corrections: orchestrator invocation and timeout guarantees.
Description check		The description is comprehensive and directly explains the documentation changes, rationale, scope, testing, validation results, limitations, and runtime impact. It does not use the repository templat…

Full details: Description check

Explanation

The description is comprehensive and directly explains the documentation changes, rationale, scope, testing, validation results, limitations, and runtime impact. It does not use the repository template headings for Title, Type, Checklist, and Sections, but it provides the required information in equivalent prose and is mostly complete.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch docs/orchestrator-mapreduce

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{🗺️ The plan divides, the map fans wide,
Subtasks race with bounded stride,
A gate inspects each reduced result—
Retry, mutate, or humans consult.
STATE.md holds the audit trail,
In quantum steps, no task shall fail! ⚛️}

_{Comment @coderabbitai help to get the list of available commands.}

[Copilot]

Pull request overview

Adds a new, comprehensive Markdown document describing the orchestrator_mapreduce.py module (Plan → Map → Reduce flow, public API/classes, customization points, protocol calling behavior, persistence, and examples) to help developers integrate and extend the orchestrator.

Changes:

• Introduces full module documentation for orchestrator_mapreduce.py in docs/orchestrator_mapreduce.md.
• Adds architecture/lifecycle diagrams, API reference, customization contracts, and usage examples.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/orchestrator_mapreduce.md`:
- Line 21: The fenced code block at line 21 in the orchestrator_mapreduce.md
file is missing a language identifier, which violates the MD040 markdown linting
rule. Add the language identifier `text` immediately after the opening triple
backticks on the line containing the fenced code block start marker to specify
that this is a text/ASCII art block, ensuring compliance with markdown
standards.
- Line 520: The fenced code block at line 520 in the orchestrator_mapreduce.md
file is missing a language identifier, which violates the MD040 Markdown linting
rule. Add the language identifier `text` to the opening of the fenced code block
(the line with the opening triple backticks) since this contains an ASCII
diagram. Change the opening ``` to ```text to specify that the content is plain
text.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 1cc1a810-26b5-4476-a2fa-6f3e527537c5

📥 Commits

Reviewing files that changed from the base of the PR and between 1637fd3 and 2a5099a.

📒 Files selected for processing (1)

• docs/orchestrator_mapreduce.md

[groupthinking]

Gtg

[Copilot]

🟡 Changes recommended

Resolve the two remaining documentation inconsistencies.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (1)

docs/orchestrator_mapreduce.md:185

• This new clarification conflicts with the lifecycle diagram, which still shows Apply timeout followed unconditionally by Track outcome. A reader can therefore still infer that timed-out subtasks are tracked, contrary to the implementation and the text here; update the diagram to show tracking only for successful returns and non-timeout exceptions.

- Tracks successful returns and non-timeout exceptions via `track_outcome()` (non-blocking, via `run_in_executor`); timeout failures are not sent to this tracker

• Files reviewed: 2/2 changed files
• Comments generated: 1
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

Update the remaining examples that pass inputs to the zero-argument health checker.

Review details

Suppressed comments (1)

docs/orchestrator_mapreduce.md:289

• This clarification still leaves the runnable endpoint examples above (lines 148–149) and in the fan-out section (lines 392–394) using api_health_checker with per-endpoint inputs. That protocol is zero-argument, so _run_protocol_isolated() ignores those values and the examples do not validate the listed endpoints. Update both examples to use the separate parameterized protocol named by this section, or remove the per-endpoint inputs and describe the existing checker’s behavior.

The repository's existing `api_health_checker.task()` takes no arguments and checks its own endpoint list. To use the parameterized examples, implement a separate protocol like the following and use its name in `task_list`:

• Files reviewed: 2/2 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

🟡 Changes recommended

The documentation inaccurately presents max_job_history as a persistent storage limit even though it only bounds in-memory job history, leaving an active clarification issue.

1 open issue

• Clarify max_job_history as an in-memory limit · New

1 resolved since last review

• The documentation now states that api_health_checker.task() takes no arguments, but the runnable…

_{Review effort: Lite}

[Copilot]

🟢 Approval recommended

No active review issues are reported, and the documentation, runnable examples, and focused behavior tests address the current Map/Reduce behavior and documented limitations.

No open issues 🎉

1 resolved since last review

• Clarify max_job_history as an in-memory limit

_{Review effort: Lite}

[Copilot]

🟢 Approval recommended

No active review issues are reported, and the changes are focused on aligning documentation and examples with current behavior while adding targeted coverage for the documented execution semantics.

No open issues 🎉

_{Review effort: Lite}