Conversation
The chart ships a `PendingMigrations` alert at severity critical, plus two recording rules, all keyed on `django_migrations_unapplied_total`. The default authentik deployment never exports that metric, so on a stock install these are silent by construction -- see goauthentik/authentik#9941, open since 2024-06. Three things have to be true for the metric to appear, and none of them is: 1. django-prometheus only exports it when PROMETHEUS_EXPORT_MIGRATIONS is set (DjangoPrometheusConfig.ready()), and authentik does not set it. 2. Even set, it would not surface. preload_app = True loads Django's app registry -- and so ExportMigrations() -- in the gunicorn master, while post_fork only installs MultiProcessValue after the fork. The gauge lands in the master's plain in-process storage, which the multiprocess collector never reads. 3. It would measure nothing anyway: run_migrations() runs at module level in lifecycle/gunicorn.conf.py, so migrations are already applied before anything observes them. The gauge would read 0 on every successful boot. Point 3 is why this gates rather than fixes. PendingMigrations is an alert for deployments where migrations are a separate step and the app can serve against an un-migrated database. That configuration is real, and those users are exactly who the alert was written for -- so the rules stay, behind an opt-in. Both blocks are gated, not just the alert: the "authentik Aggregate migrations" recording group reads the same dead metric. Defaulting to false cannot regress anyone, because the rules it disables cannot currently fire. Signed-off-by: Pirat83 <Pirat83@users.noreply.github.com>
Member
|
closing in favour of goauthentik/authentik#26352 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The chart ships a
PendingMigrationsalert atseverity: critical, plus two recording rules in the authentik Aggregate migrations group, all keyed ondjango_migrations_unapplied_total. The default authentik deployment never exports that metric, so on a stock install those rules are silent by construction.This is goauthentik/authentik#9941, open since 2024-06. I added a confirmation there reproducing it on 2026.8.1 via this chart.
Why gate rather than make the metric work
Three things must be true for the metric to appear, and none of them is. Traced through the source at
version/2026.8.1:django-prometheusonly exports it whenPROMETHEUS_EXPORT_MIGRATIONSis set (DjangoPrometheusConfig.ready()), and authentik does not set it.preload_app = Trueloads Django's app registry — and soExportMigrations()— in the gunicorn master, whilepost_forkonly installsMultiProcessValueafter the fork. The gauge lands in the master's plain in-process storage, which the multiprocess collector never reads. This is the part the issue reporter flagged as unexplained in 2024.run_migrations()runs at module level inlifecycle/gunicorn.conf.py, so migrations are already applied before anything observes them — the gauge would read 0 on every successful boot.Point 3 is the reason for gating instead of fixing.
PendingMigrationsis an alert for deployments where migrations are a separate step and the app can serve against an un-migrated database. That setup is real and those users are exactly who the alert was written for — so the rules stay, behind an opt-in.The change
One new value,
prometheus.rules.migrations.enabled, defaulting tofalse, gating both the alert and the recording-rule group — the latter reads the same dead metric, so gating only the alert would leave it behind.Defaulting to
falsecannot regress anyone, because the rules it disables cannot currently fire.Verified
Rendered three ways with
helm template:PrometheusRule(unchanged —prometheus.rules.enabledis alreadyfalse)prometheus.rules.enabled=truePendingMigrations; the other 3 alerts and 4 recording groups intactenabled=truemain— 10 objects, all 6 groups, all 4 alerts. The only textual delta is 3 blank lines absorbed by{{-chompingThat third case is the one that matters: the gate is purely additive for anyone who opts in.
Also run:
helm lintclean with the gate both off and on;yamllint -c .github/configs/lintconf.yamlclean onvalues.yamlandChart.yaml;./scripts/helm-docs.shregeneratedREADME.md(one row added) and is idempotent on a repeat run. No chart version bump —check-version-incrementisfalseand the chart version tracksappVersionon release bumps.Happy to change the approach
If you would rather simply delete these rules than carry a flag for them, say so and I will amend — I chose the flag only to preserve the split-migration-step use case, and it is your call which is the better trade.