[AISOS-1883] Forge Workflow Statistics and Automated Reporting

.env.example

@@ -245,3 +245,21 @@ CI_FIX_MAX_RETRIES=5
 CI_IGNORED_CHECKS=tide
 # Webhook acknowledgment timeout in seconds
 WEBHOOK_ACK_TIMEOUT=0.5
+
+# =============================================================================
+# Stats Cost Alert Configuration
+# =============================================================================
+# Enable cost alerting in workflow stats summaries. When enabled and aggregate
+# token usage (input + output across all stages) exceeds the threshold, the
+# stats summary will include a cost alert.
+STATS_ALERT_ENABLED=true
+# Total token count threshold that triggers a cost alert (default: 1,000,000).
+# Applies to aggregate token usage across all workflow stages.
+STATS_ALERT_THRESHOLD_TOKENS=1000000
+# Dollar cost threshold for cost alerts. When set, compares total dollar cost against
+# this value instead of using the token-based threshold above.
+# STATS_ALERT_THRESHOLD_COST=10.00
+# LLM pricing table as a JSON-encoded string mapping model name substrings to
+# per-million-token rates (input and output in $/MTok). Longest key match wins.
+# Default rates are pre-populated; override only if prices change.
+# LLM_PRICING={"claude-opus-4":{"input":15.00,"output":75.00},"claude-sonnet-4":{"input":3.00,"output":15.00}}

.gitignore

@@ -42,6 +42,7 @@ ENV/
 
 # Testing
 .pytest_cache/
+.mypy_cache/
 .coverage
 htmlcov/
 *.cover

CLAUDE.md

@@ -122,6 +122,8 @@ podman rm $(podman ps -a --filter name=forge- -q)
 | `!` | Revision request — triggers regeneration with feedback |
 | `?` or `@forge ask` | Question — triggers Q&A answer |
 | `>option N` | RCA option selection (RCA Option Gate only) |
+| `/forge stats` | Post current workflow statistics as a Jira comment (read-only) |
+| `/forge stats retry` | Re-post stats comment, forcing a fresh calculation |
 | _(no prefix)_ | Informational — workflow ignores it |
 
 ## GitHub PR Comment Commands

containers/entrypoint.py

@@ -457,6 +457,54 @@ async def run_agent_task(
         else:
             result = await agent.ainvoke(initial_message, config=config)
 
+        # Extract and aggregate tokens from usage_metadata
+        try:
+            total_input_tokens = 0
+            total_output_tokens = 0
+            messages = result.get("messages", []) if isinstance(result, dict) else []
+            for message in messages:
+                msg_type = type(message).__name__
+                if msg_type in ("AIMessage", "AIMessageChunk"):
+                    usage = getattr(message, "usage_metadata", None)
+                    if not usage:
+                        resp_metadata = getattr(message, "response_metadata", {})
+                        if isinstance(resp_metadata, dict):
+                            usage = resp_metadata.get("token_usage") or resp_metadata.get("usage")
+
+                    if isinstance(usage, dict):
+                        total_input_tokens += (
+                            usage.get("input_tokens", 0) or usage.get("prompt_tokens", 0) or 0
+                        )
+                        total_output_tokens += (
+                            usage.get("output_tokens", 0) or usage.get("completion_tokens", 0) or 0
+                        )
+                    elif usage is not None:
+                        total_input_tokens += (
+                            getattr(usage, "input_tokens", 0)
+                            or getattr(usage, "prompt_tokens", 0)
+                            or 0
+                        )
+                        total_output_tokens += (
+                            getattr(usage, "output_tokens", 0)
+                            or getattr(usage, "completion_tokens", 0)
+                            or 0
+                        )
+
+            metrics_dir = workspace / ".forge"
+            metrics_dir.mkdir(parents=True, exist_ok=True)
+            metrics_file = metrics_dir / "metrics.json"
+            metrics_file.write_text(
+                json.dumps(
+                    {"input_tokens": total_input_tokens, "output_tokens": total_output_tokens},
+                    indent=2,
+                )
+            )
+            logger.info(
+                f"Saved container metrics to {metrics_file}: input_tokens={total_input_tokens}, output_tokens={total_output_tokens}"
+            )
+        except Exception as e:
+            logger.warning(f"Failed to record token usage inside sandbox: {e}")
+
         # Flush Langfuse traces before exit
         if langfuse_enabled:
             try:

docs/guide/bug-workflow.md

@@ -119,6 +119,7 @@ At any approval gate, Forge classifies your comment by its prefix:
 - **`!` prefix** — revision request: Forge regenerates the current artifact with your feedback
 - **`?` prefix or `@forge ask`** — question: Forge answers and stays paused
 - **`>option N`** — RCA option selection (RCA Option Gate only)
+- **`/forge stats`** — posts current workflow statistics as a Jira comment (read-only)
 - **No prefix** — informational: ignored by the workflow
 
 ---

docs/guide/feature-workflow.md

@@ -185,7 +185,7 @@ Start a comment with `!` followed by your feedback. Forge regenerates the curren
 ```
 
 !!! note
-    Comments without a recognized prefix (`!`, `?`, `@forge ask`) are treated as informational and ignored by the workflow. Only `!`-prefixed comments trigger regeneration.
+    Comments without a recognized prefix (`!`, `?`, `@forge ask`, `/forge stats`) are treated as informational and ignored by the workflow. Only `!`-prefixed comments trigger regeneration.
 
 ## Handling Failures
 
@@ -199,6 +199,52 @@ To retry, add the `forge:retry` label. Forge resumes from the exact node that fa
 !!! tip "CI retries"
     If CI fix attempts are exhausted, `forge:retry` resets the attempt counter for a fresh budget of retries.
 
+## Workflow Statistics
+
+At the end of a workflow execution (when the ticket reaches a terminal state, including **Completed**, **Blocked**, or **Failed**), Forge aggregates execution data and automatically posts a comprehensive summary on the Jira ticket. This ensures that even when a workflow is blocked or fails, stakeholders can inspect the resource usage and performance metrics up to that point. This helps teams track efficiency, analyze execution bottlenecks, and monitor LLM token costs.
+
+### Summary Format
+
+The summary is generated as a Markdown table with the following columns:
+
+| Column | Description |
+|---|---|
+| **Stage** | The name of the pipeline stage (e.g., PRD, Spec, Epics, Tasks, Implementation, CI, Review). |
+| **Iterations** | The number of attempts or iterations executed during that stage. |
+| **Machine Time** | Monotonic duration of active processing by Forge during that stage (formatted as `1h 2m 3s`). |
+| **Input Tokens** | Estimated number of LLM input tokens consumed during that stage. |
+| **Output Tokens** | Estimated number of LLM output tokens consumed during that stage. |
+| **Cost** | Calculated cost based on the stage's token consumption and LLM pricing mappings. |
+
+At the bottom of the table, a **Total** rollup row displays sum totals across all executed stages.
+
+### Cost Alerting
+
+If the cumulative resource consumption exceeds specified safety thresholds, a prominent warning alert is appended to the statistics summary comment.
+
+Alert thresholds are defined globally (or can be customized in the configuration):
+- **Token Threshold:** Triggers if cumulative input + output tokens exceed a specified value (default: `1,000,000` tokens).
+- **Dollar Threshold:** Triggers if cumulative calculated cost exceeds a specified monetary value (default: disabled/`None`).
+
+When triggered, a cost warning similar to the following is displayed directly below the summary table:
+
+```text
+⚠️ WARNING: This workflow run exceeded the configured cost/token limits!
+Please review the resource usage details above for potential optimizations.
+```
+
+---
+
+## On-Demand Stats Commands
+
+In addition to automatic summary posting at the end of a successful workflow run, team members can request or force-refresh stats at any time using Jira comment commands.
+
+| Command | Action | Description |
+|---|---|---|
+| `/forge stats` | Request Stats | Generates the current statistics table and posts it as a comment on the Jira ticket, reflecting metrics up to the current stage of execution. |
+| `/forge stats retry` | Refresh Stats | Forces a fresh recalculation of statistics and re-posts the summary table. This ensures the stats comment remains updated as the final comment on the Jira issue. |
+
+
 ## Labels Summary
 
 See [Jira Labels](labels.md) for the complete reference.

docs/guide/labels.md

@@ -42,7 +42,7 @@ These labels advance the pipeline. Forge watches for label changes via Jira webh
 
 **Asking questions:** Start a comment with `?` or `@forge ask`. Forge answers without advancing or regenerating.
 
-**Informational comments:** Comments without a recognized prefix (`!`, `?`, `@forge ask`, `>option`) are ignored by the workflow — use them for team discussion without triggering Forge.
+**Informational comments:** Comments without a recognized prefix (`!`, `?`, `@forge ask`, `>option`, `/forge stats`) are ignored by the workflow — use them for team discussion without triggering Forge.
 
 **Handling failures:** When `forge:blocked` appears, read the Forge comment for the error. Fix the underlying issue if needed, then add `forge:retry`.
 

docs/guide/weekly-reporting.md

@@ -0,0 +1,63 @@
+# Weekly Reporting System
+
+Forge includes an automated, weekly aggregation and reporting system that compiles and publishes metrics across all managed tickets for a specific Jira project. This documentation explains how the reporting system operates behind the scenes.
+
+## Quick Start
+
+Generate a weekly report for your project (e.g., `PROJ`) with the following command:
+
+```bash
+forge weekly-report --project PROJ
+```
+
+> **Note:** The `forge weekly-report` command requires active Redis access and must be run from the Forge project directory containing `.env` to load configurations.
+
+## Aggregation Logic
+
+When you run `forge weekly-report` (or trigger it via automated schedules), the reporting system performs the following steps:
+
+1. **Query Active/Historical Checkpoints:** Forge scans the Redis event and state checkpoints for the specified project (`PROJECT_KEY`). It uses a key scanning pattern `checkpoint:{PROJECT_KEY}-*` to find all state checkpoints.
+2. **Filter by Sliding Window:** Metrics are collected and filtered based on a sliding window of `N` days (by default, `7` days). A checkpoint falls within the reporting window if its `updated_at` timestamp or any stage `started_at`/`ended_at` timestamp is greater than or equal to the cutoff (`now - N days`).
+3. **Aggregate Stats per Stage:** Data is aggregated across all feature and bug workflows, tracking:
+   - **Ticket Rollups:** Total numbers of active, completed, or blocked workflows.
+   - **Machine Time:** Cumulative active machine processing time (monotonic durations) across all stages.
+   - **LLM Token Costs:** Sum of all input and output tokens consumed, translating them into actual dollar costs based on LLM pricing mappings.
+   - **Feature Rollups:** Metrics aggregated per epic-linked ticket and feature. Ancestry traversal resolves the parent/grandparent Feature for each ticket in Jira up to two hops (e.g., ticket -> Epic -> Feature). Tickets without a resolved Feature are grouped under the "Unassigned" bucket.
+   - **Bottleneck Analysis:** Identifies the slowest stage by average duration, ranks stages by iteration count, and calculates the CI fix rate.
+
+## Idempotency & Ticket Publishing
+
+To avoid cluttering Jira with duplicate reports every week, the reporting system is designed to be completely **idempotent** when publishing to Jira via the `--create-ticket` flag.
+
+- **Ticket Naming Convention:** The ticket summary is formatted dynamically based on the project key and current date:
+  ```text
+  Forge Weekly Report - {PROJECT} - Week of {date}
+  ```
+  Where `{PROJECT}` is the project key, and `{date}` is the first day of the reporting week (i.e. `today - N + 1 days`).
+- **Label Identification:** The system uses the special `forge:weekly-report` and `forge:generated` labels to identify and tag report tickets.
+- **Idempotency Guard:**
+  - When `--create-ticket` is run, Forge first searches Jira using the following JQL:
+    ```jql
+    project = "{PROJECT}" AND labels = "forge:weekly-report" AND summary ~ "Week of {date}"
+    ```
+  - If a matching ticket is found, Forge updates that existing ticket's description with the newly compiled statistics instead of creating a new one.
+  - If no matching ticket exists, Forge creates a new Jira Task issue, assigns the `forge:weekly-report` and `forge:generated` labels, and sets the description.
+
+## Stakeholder Notifications
+
+When using the `--notify` option alongside `--create-ticket`, Forge automatically mentions and notifies designated stakeholders.
+
+### Notification List Compilation
+
+The notification list is compiled hierarchically to allow easy overriding (highest priority first):
+
+1. **Jira Project Property (Highest Priority):** Forge attempts to read the `forge.weekly-report.notify` project property from Jira. This property must contain a JSON array of Jira Account IDs (e.g., `["account-id-1", "account-id-2"]`) or a comma-separated string of account IDs.
+2. **Environment Variable (Global Fallback):** If no project-specific property is set, Forge falls back to the `FORGE_WEEKLY_REPORT_NOTIFY` environment variable in `.env`. This variable should contain a comma-separated list of Jira Account IDs or the keyword `"project-leads"`. The special value `"project-leads"` instructs Forge to query the per-project Jira property.
+3. **No Recipients:** If neither is configured, no notifications are triggered.
+
+### How Notifications are Delivered
+
+Once the recipient account IDs are resolved:
+- Forge posts a comment directly on the generated weekly report Jira ticket.
+- The comment mentions each stakeholder using Jira's native `[~accountid:{id}]` mention syntax.
+- This triggers email and/or Slack notifications based on the users' individual Atlassian notification preferences, ensuring visibility to project leads and management.

docs/index.md

@@ -22,6 +22,8 @@ graph TD
 
 - [Getting Started](getting-started.md) — Set up Forge in 10 minutes
 - [Feature Workflow](guide/feature-workflow.md) — How features flow through Forge
+- [Weekly Reporting Guide](guide/weekly-reporting.md) — Automated project-wide metrics and notifications
+- [CLI Reference](reference/cli.md) — Command-line interface documentation
 - [Developer Guide](developer-guide.md) — Full local development reference
 - [Skills System](skills/index.md) — Customize Forge for your stack
 - [Contributing](dev/contributing.md) — How to contribute

docs/reference/api.md

@@ -44,7 +44,7 @@ Receives Jira webhook events. Validates the signature and enqueues the event for
 
 - `jira:issue_created` — triggers new workflow if `forge:managed` label is present
 - `jira:issue_updated` — handles label changes (approvals, retry)
-- `jira:issue_commented` — handles Q&A and revision requests
+- `jira:issue_commented` — handles Q&A, revision requests, and `/forge stats` commands
 
 Returns HTTP 200 immediately. Processing is asynchronous.