[High] Structured Log Pipeline with Correlation IDs and Log-Based Alerting

[james2177]

Scope note (prior work): Log shipping (LOG_SHIPPING_ENABLED, Winston HTTP transport in src/common/logger.ts) and request-ID propagation already exist. (see #116 and #88).

Do not rebuild the shipping transport or request IDs. Scope is: a documented structured log schema enforced by the logger API, AsyncLocalStorage context (trace/intent/solver fields), migration of core-module log call sites, OTel collector config, and log-based security alerts.

Description:
Standardise structured JSON logging with a stable schema (request id, trace id, intent id, solver, chain), ship to Loki/OpenSearch via an OTel collector, and define log-based alerts for security events.

Problem Statement & Context:
Log messages are free-form strings (e.g., [event-ingestion] poll failed: …), making queries brittle. Security-relevant events (signature failures, admin actions, policy violations) need reliable detection.

Scope & Acceptance Criteria:

• Log schema documented; logger API enforces structured fields (logger.info(msg, { intentId })).
• Existing log statements migrated in core modules.
• OTel collector config for log shipping in the observability compose profile.
• Log-based alerts: signature-failure spikes, signer policy violations, admin action anomalies.
• Out of scope: vendor-specific SIEM.

Implementation Guidelines:

1. Key Files/Modules: src/common/logger.ts, src/common/logging.interceptor.ts, ops/otel-collector/, all src/** logging call sites in core modules.
2. Design/Architecture: AsyncLocalStorage-based context for request-scoped fields.
3. Edge Cases/Constraints: Logging overhead < 5% CPU at 1k rps; redaction applied.
4. Testing: Schema conformance tests on emitted logs.

Definition of "Done": Common DoD.

Resources:

• Design reference: N/A (backend-only)
• https://opentelemetry.io/docs/specs/otel/logs/data-model/

---

Common Definition of "Done" (applies in addition to the criteria above):

• Code written, tested, and documented (TSDoc on public APIs, README/runbook/ADR updates where behaviour changes).
• All acceptance criteria met; npm run lint, npm run typecheck, npm test, npm run test:e2e pass in CI.
• PR follows .github/PULL_REQUEST_TEMPLATE, uses a Conventional Commit title (enforced by commitlint), includes test output / metrics screenshots, and references the issue.
• New env vars are added to .env.example variants and src/config/env.validation.ts (the check:env-drift script must pass).
• Reviewed and approved by at least one CODEOWNER.