diff --git a/CHANGELOG.md b/CHANGELOG.md index afd7ae3..b81efdf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), - **Alert rules API and UI** — CRUD endpoints under `/api/jobs/{job_id}/alert-rules` plus management controls on the job detail page. - **Alembic migration `c3d4e5f6a7b8`** — creates the `alert_rules` table with job and workspace foreign keys. - **Workspace-scoped alert dispatch** — alert notifications and escalation policies now dispatch only through channels/policies in the alert's workspace. +- **Log aggregation** — Loki and Elasticsearch log shipping via `LOG_SINK`, `LOKI_URL` and `ELASTICSEARCH_URL` environment variables; Promtail agent-based shipping for Docker deployments added to `docker-compose.observability.yml`; label schema documented in `docs/log-aggregation.md`. +- **Notification provider extensions** — PagerDuty dedup key (`dedup_key`), OpsGenie alert priority (`priority`) and custom payload template (`payload_template`) for richer provider-specific delivery configuration. ### Changed diff --git a/backend/tests/conftest.py b/backend/tests/conftest.py index 31105ac..39112ae 100644 --- a/backend/tests/conftest.py +++ b/backend/tests/conftest.py @@ -7,6 +7,9 @@ _TEST_DB_PATH.unlink(missing_ok=True) os.environ["DATABASE_URL"] = f"sqlite+aiosqlite:///./{_TEST_DB_PATH}" os.environ["REDIS_URL"] = "redis://localhost:0/0" +os.environ["JWT_SECRET_KEY"] = ( + "test-secret-key-autoflowops-ci-placeholder-do-not-use-in-production" +) import pytest from fastapi.testclient import TestClient diff --git a/docs/release-notes-v1.2.0.md b/docs/release-notes-v1.2.0.md index 553d231..1fe8677 100644 --- a/docs/release-notes-v1.2.0.md +++ b/docs/release-notes-v1.2.0.md @@ -61,6 +61,25 @@ Application logs now use `structlog` with context-variable injection. - Authenticated requests bind `user_id`; workspace-scoped requests bind `workspace_id`. - Fully compatible with existing `logging.getLogger()` usage throughout the codebase. +### Log aggregation + +Structured logs can now be shipped directly to Loki or Elasticsearch in addition to printing to stdout. + +- `LOG_SINK` selects the active shipping mode: `stdout` (default), `loki`, `elasticsearch`, or a comma-separated combination for dual shipping. +- `LOKI_URL` and `ELASTICSEARCH_URL` configure target endpoints for direct-push mode. +- `docker-compose.observability.yml` updated with a Loki + Promtail stack for agent-based log collection from Docker container stdout — the recommended mode for standard self-hosted deployments. +- All log streams carry a consistent label schema (`app`, `service`, `env`, `level`, `logger`) plus structured metadata fields (`job_id`, `execution_id`, `workspace_id`, `request_id`). +- Direct-push and agent-based modes can be active simultaneously. +- Full configuration reference and local setup instructions in `docs/log-aggregation.md`. + +### Notification provider extensions + +PagerDuty and OpsGenie delivery now supports additional provider-specific fields. + +- `dedup_key` on PagerDuty channels for alert deduplication across the Events API v2 lifecycle. +- `priority` on OpsGenie channels (`P1`–`P5`) for routing to on-call schedules by severity. +- `payload_template` on both channel types for fully custom JSON payloads when the built-in format does not match provider expectations. + ### Prometheus + Grafana stack A ready-to-run observability compose stack is provided in `docker-compose.observability.yml`. diff --git a/docs/roadmap.md b/docs/roadmap.md index 95ef923..78dc21e 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -1,6 +1,6 @@ # Roadmap -This document tracks the feature status of AutoFlowOps across completed, planned and future phases. +This document tracks the feature status of AutoFlowOps across completed and future phases. --- @@ -9,6 +9,8 @@ This document tracks the feature status of AutoFlowOps across completed, planned | Feature | Details | | --- | --- | | **Conditional alert rules** | Per-job rules trigger alerts from HTTP status thresholds, execution duration thresholds, response body matching and consecutive failure counts; CRUD API, job detail UI and backend/frontend tests | +| **Log aggregation** | Loki and Elasticsearch log shipping via `LOG_SINK`, `LOKI_URL` and `ELASTICSEARCH_URL`; Promtail agent-based shipping for Docker deployments; label schema documented in `docs/log-aggregation.md` | +| **Notification provider extensions** | PagerDuty dedup key, OpsGenie alert priority and custom payload templates for provider-specific delivery configuration | | **Workspace membership enforcement** | Non-admin users receive `403 Forbidden` when supplying an `X-Workspace-ID` they are not a member of; admin role bypasses the check; 5 new tests | | **Encryption key documentation** | `NOTIFICATION_ENCRYPTION_KEY` added to `docs/deployment.md` env reference and production checklist; backup and rotation guide added to `docs/security.md` | | **Prometheus metrics** | `/metrics` endpoint in Prometheus text format via `prometheus-fastapi-instrumentator`; HTTP histogram auto-instrumented; `autoflowops_job_executions_total` and `autoflowops_alerts_created_total` business counters | @@ -194,14 +196,9 @@ These features are implemented, tested and validated in the current MVP. --- -## Future (planned, not yet scheduled) +## Future -These features are planned but not yet in active development. - -| Feature | Description | -| --- | --- | -| **Notification provider extensions** | Richer provider-specific delivery options (PagerDuty dedup keys, OpsGenie alias, custom payload templates) | -| **Log aggregation integration** | First-class Loki or Elastic shipping configuration with documented label schema | +No features are currently scheduled. Proposals and contributions are welcome via the issue tracker. ---