From edecb41581405388a13c28428fe7af83c31c83f8 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 09:08:04 +0000 Subject: [PATCH 1/3] docs(ops): AI cost per document, monthly limits and action at the limit (#109) operations.md gets an "AI cost" section: cost per document from the two sample recordings (input + output incl. thinking) at dated list prices, marked unconfirmed for gemini-3.5-flash; what the upload caps bound and what they do not (no output or input cap per call -> #123); proposed monthly limits for showcase and pilot; the budget alert as an open account action; the steps at the limit. A note under the vendor table links it. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0117LRiQKt8ycJEHW7vxLWG3 --- docs/technical/architecture.md | 3 ++ docs/technical/operations.md | 52 ++++++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+) diff --git a/docs/technical/architecture.md b/docs/technical/architecture.md index c03e5e0..cca2b78 100644 --- a/docs/technical/architecture.md +++ b/docs/technical/architecture.md @@ -79,6 +79,9 @@ failure at any step ─► retry with backoff ─► dead letter ─► requests | Vertex AI (`eu` endpoint, gemini-3.5-flash) | extraction | showcase + customer; local dev may use the Gemini free tier with synthetic data | | ERP | export target | pilot: `erp-mock`; contract `contracts/erp-export.openapi.yaml` | +AI cost per document, monthly limits (showcase 10 EUR, pilot 50 EUR, proposed) and what happens at the limit: +[operations.md → AI cost](operations.md#ai-cost) (#109). + No secrets in this document; every variable is documented in `.env.example`. ## Database roles diff --git a/docs/technical/operations.md b/docs/technical/operations.md index 8b03204..e555ec3 100644 --- a/docs/technical/operations.md +++ b/docs/technical/operations.md @@ -160,6 +160,58 @@ endpoints are disabled except `set-active` (404) – every change goes through t confirm the notification; reset the variable afterwards. - Out of scope: error-rate alerts from log aggregation, monitoring of a customer environment. +## AI cost + +Issue #109. Every processed document is **one** model call (`process-request.ts`: one call per +document of a request; `.msg` attachments travel inside their document). Tokens are recorded per run +(`extraction_runs.total_tokens`, per document in `extraction_runs.documents`); the query for cost per +request is signal S4 of the pilot metrics page (`docs/product/metrics.md`, #113). + +**Cost per document (estimate, 2026-10-16).** Formula: `input_tokens × input price + (total_tokens − +input_tokens) × output price` – the difference is output **plus thinking** tokens, which Google bills at +the output price. Token source: the two sample recordings (`src/features/samples/data/*.recording.json`, +`gemini-3.5-flash`, prompt `extract_v2`). Prices per 1M tokens, Gemini API standard (paid) tier: + +| Price basis | Input / output (USD) | „Pumpe P204“ (1,280 in / 3,242 out) | „Werk Ost“ (1,281 in / 2,277 out) | +|---|---|---|---| +| `gemini-3.5-flash` – **unconfirmed**: Google's pricing page (updated 2026-10-15) no longer lists it; figure from third-party price lists | 1.50 / 9.00 | 0.031 | 0.022 | +| `gemini-3.6-flash` (Google's recommended successor), until 2026-12-31 | 0.75 / 3.75 | 0.013 | 0.010 | +| `gemini-3.6-flash` from 2027-01-01 | 1.50 / 7.50 | 0.026 | 0.019 | + +Assumptions: the same token counts for 3.6 (not measured – a model change runs the eval gate and a new +recording first); Vertex AI list prices are not confirmed here. On the free tier (showcase until the D11 +exception ends, at the latest 2026-10-31) the cost is 0 – only the quota counts. + +**What bounds the cost – and what does not.** The upload caps bound the number of calls, not their size: +`UPLOAD_MAX_PER_HOUR` (showcase: 20 per person) × `UPLOAD_MAX_FILES` (showcase: 3) = at most 60 calls per +person and hour – about 1.90 USD per hour at the worst figure above, so 10 USD after about 5 hours of +uploading without pause. The model call itself has **no output limit and no input-size limit** (no +`max_output_tokens`; the document text goes in whole, up to the 20 MB file cap), so one large document can +cost far more than the samples – follow-up #123 (cap output tokens and document text per call). + +**Monthly limits** (proposed – the orchestrator confirms them when setting up the budget): + +| Environment | Limit / month | ≈ document calls at the worst figure | Basis | +|---|---|---|---| +| Showcase (invite-only, one person) | 10 EUR | ~320 | brief → Risks ("10 €/month GCP budget alert") | +| Pilot (3–5 test users) | 50 EUR | ~1,600 | assumption: 5 users × ~15 documents per working day × 21 days ≈ 1,575; reviewed after the first month | + +**Budget alert (account action, orchestrator).** A Cloud Billing budget on the project that pays for the +model (paid Gemini API key or Vertex), amount = the limit above, alerts at 50 %, 90 % and 100 % of actual +spend, recipients: billing admins (the orchestrator) – no address in the repo. Status: **not yet +configured** (the showcase still runs on the free tier). A budget alert only notifies; it never stops +spending. + +**At the limit** (any 100 % alert, or 90 % before mid-month): + +1. Stop new model calls: pause the Vercel project `requestflow-ai` (uploads then end in a visible + processing error once their retries are used up), or – to stop all processing – remove `CRON_SECRET` + and set `JOB_DRAIN_INLINE=false` (`deployment-vercel.md` §9). +2. Find the cause with the S4 query (tokens per request) and the `jobs.drain_run` logs: one large + document, a retry loop, or simply more use. +3. Resume only after the orchestrator raises the limit or the cause is fixed; reprocess requests that went + to `ERROR` in between. + ## Frequent failures | Symptom | Cause | Action | From 0a24a2b665145fef5a9636311295eb99af0f687e Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 09:08:44 +0000 Subject: [PATCH 2/3] docs(ops): name the hourly upload cap as the runbook example (#109) Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0117LRiQKt8ycJEHW7vxLWG3 --- docs/technical/operations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/technical/operations.md b/docs/technical/operations.md index e555ec3..5d57484 100644 --- a/docs/technical/operations.md +++ b/docs/technical/operations.md @@ -183,7 +183,7 @@ recording first); Vertex AI list prices are not confirmed here. On the free tier exception ends, at the latest 2026-10-31) the cost is 0 – only the quota counts. **What bounds the cost – and what does not.** The upload caps bound the number of calls, not their size: -`UPLOAD_MAX_PER_HOUR` (showcase: 20 per person) × `UPLOAD_MAX_FILES` (showcase: 3) = at most 60 calls per +`UPLOAD_MAX_PER_HOUR` (runbook example for the showcase: 20 per person) × `UPLOAD_MAX_FILES` (showcase: 3) = at most 60 calls per person and hour – about 1.90 USD per hour at the worst figure above, so 10 USD after about 5 hours of uploading without pause. The model call itself has **no output limit and no input-size limit** (no `max_output_tokens`; the document text goes in whole, up to the 20 MB file cap), so one large document can From 7973361f0bf816f523e45c44cb3fbea636fda12b Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 09:10:45 +0000 Subject: [PATCH 3/3] docs(ops): count retried model calls, fix rounding and units (#124 review) A retried processing job calls the model again for every document (up to 6 attempts); failed attempts are not in the recorded tokens. Werk Ost on 3.6 rounds to 0.009; EUR limits are counted as USD (conservative). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0117LRiQKt8ycJEHW7vxLWG3 --- docs/technical/operations.md | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/docs/technical/operations.md b/docs/technical/operations.md index 5d57484..010607d 100644 --- a/docs/technical/operations.md +++ b/docs/technical/operations.md @@ -162,20 +162,25 @@ endpoints are disabled except `set-active` (404) – every change goes through t ## AI cost -Issue #109. Every processed document is **one** model call (`process-request.ts`: one call per -document of a request; `.msg` attachments travel inside their document). Tokens are recorded per run +Issue #109. Every processed document is **one** model call per attempt (`process-request.ts`: one call +per document of a request; `.msg` attachments travel inside their document). A failed attempt repeats +the calls: an invalid or failed model answer (`model_output_invalid`, `model_error`) is retryable, and +the processing job then runs again – up to **6 attempts** (`retryLimit: 5`, `queues.ts`) – calling the +model again for **every** document of the request, including those that already succeeded. A paid but +unusable answer can thus be billed up to 6 times per document, plus each manual reprocess ("Erneut verarbeiten"). Tokens are recorded per run (`extraction_runs.total_tokens`, per document in `extraction_runs.documents`); the query for cost per -request is signal S4 of the pilot metrics page (`docs/product/metrics.md`, #113). +request is signal S4 of the pilot metrics page (`docs/product/metrics.md`, pending in #121). Tokens are +stored only for a successful run, so calls of failed attempts are **not** in these figures. **Cost per document (estimate, 2026-10-16).** Formula: `input_tokens × input price + (total_tokens − input_tokens) × output price` – the difference is output **plus thinking** tokens, which Google bills at the output price. Token source: the two sample recordings (`src/features/samples/data/*.recording.json`, `gemini-3.5-flash`, prompt `extract_v2`). Prices per 1M tokens, Gemini API standard (paid) tier: -| Price basis | Input / output (USD) | „Pumpe P204“ (1,280 in / 3,242 out) | „Werk Ost“ (1,281 in / 2,277 out) | +| Price basis | Input / output (USD) | "Pumpe P204" (1,280 in / 3,242 out) | "Werk Ost" (1,281 in / 2,277 out) | |---|---|---|---| | `gemini-3.5-flash` – **unconfirmed**: Google's pricing page (updated 2026-10-15) no longer lists it; figure from third-party price lists | 1.50 / 9.00 | 0.031 | 0.022 | -| `gemini-3.6-flash` (Google's recommended successor), until 2026-12-31 | 0.75 / 3.75 | 0.013 | 0.010 | +| `gemini-3.6-flash` (Google's recommended successor), until 2026-12-31 | 0.75 / 3.75 | 0.013 | 0.009 | | `gemini-3.6-flash` from 2027-01-01 | 1.50 / 7.50 | 0.026 | 0.019 | Assumptions: the same token counts for 3.6 (not measured – a model change runs the eval gate and a new @@ -183,15 +188,15 @@ recording first); Vertex AI list prices are not confirmed here. On the free tier exception ends, at the latest 2026-10-31) the cost is 0 – only the quota counts. **What bounds the cost – and what does not.** The upload caps bound the number of calls, not their size: -`UPLOAD_MAX_PER_HOUR` (runbook example for the showcase: 20 per person) × `UPLOAD_MAX_FILES` (showcase: 3) = at most 60 calls per -person and hour – about 1.90 USD per hour at the worst figure above, so 10 USD after about 5 hours of -uploading without pause. The model call itself has **no output limit and no input-size limit** (no +`UPLOAD_MAX_PER_HOUR` (runbook example for the showcase: 20 per person) × `UPLOAD_MAX_FILES` (showcase: 3) = at most 60 documents per +person and hour – about 1.90 USD per hour at the worst figure above (10 EUR after roughly 5 hours of +uploading without pause), and up to 6 times that while answers keep failing (360 calls per hour). The model call itself has **no output limit and no input-size limit** (no `max_output_tokens`; the document text goes in whole, up to the 20 MB file cap), so one large document can cost far more than the samples – follow-up #123 (cap output tokens and document text per call). **Monthly limits** (proposed – the orchestrator confirms them when setting up the budget): -| Environment | Limit / month | ≈ document calls at the worst figure | Basis | +| Environment | Limit / month | ≈ document calls at the worst figure (1 EUR counted as 1 USD – conservative) | Basis | |---|---|---|---| | Showcase (invite-only, one person) | 10 EUR | ~320 | brief → Risks ("10 €/month GCP budget alert") | | Pilot (3–5 test users) | 50 EUR | ~1,600 | assumption: 5 users × ~15 documents per working day × 21 days ≈ 1,575; reviewed after the first month |