From 368e27087cfb0de43df1d25a338b8329ae52a4ef Mon Sep 17 00:00:00 2001 From: Darren Janeczek Date: Tue, 26 May 2026 08:29:26 -0400 Subject: [PATCH 1/2] fix(wait-for-grafana): lower startupTimeout default from 300s to 60s MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The two-phase polling introduced in v1.0.3 was based on the hypothesis that Grafana startups on contested CI runners are slow-but-alive. Recent evidence (see grafana/grafana#122993, mitigated in grafana/grafana#123034 and grafana/logs-drilldown#1886) shows that an important class of Playwright matrix failures is actually a *crash* during provisioning — specifically, a SQLITE_BUSY self-deadlock between the Grafana Advisor checktyperegisterer and legacy provisioning, which prevents the HTTP listener from ever binding. In that scenario the v1.0.3 default of 300 seconds makes the failure 5x slower to diagnose than v1.0.2 was: the action waits 5 minutes observing only "Current status: 000" before timing out, during which the operator has no signal at all. wait-for-grafana cannot distinguish "still booting" from "dead before booting" — both produce the same ECONNREFUSED. Lowering the default to 60 seconds restores the v1.0.2 fail-fast behaviour for crash scenarios while keeping the two-phase split (Phase 1 TCP-bind vs Phase 2 health endpoint) available for repos that have a genuine slow-start need and want to opt into a higher value explicitly. A follow-up will propose a separate `grafana-startup-logs` sibling action that dumps `docker compose ps` and a filtered tail of Grafana's own logs (with secret re-masking and configurable redaction) when wait-for-grafana fails — that is the right place to surface the real signal, rather than waiting longer at this layer. Co-Authored-By: Claude Opus 4.7 (1M context) --- wait-for-grafana/README.md | 4 ++-- wait-for-grafana/action.yml | 4 ++-- wait-for-grafana/wait-for-grafana.sh | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/wait-for-grafana/README.md b/wait-for-grafana/README.md index 22f81bf7..77325378 100644 --- a/wait-for-grafana/README.md +++ b/wait-for-grafana/README.md @@ -22,11 +22,11 @@ The time to wait between each check, in seconds. Default is `0.5`. ### `startupTimeout` (optional) -The maximum time to wait for the server's TCP port to bind, in seconds. Default is `300`. +The maximum time to wait for the server's TCP port to bind, in seconds. Default is `60`. This covers the window between the container starting and Grafana's HTTP listener becoming active. During this phase the action polls every 5 seconds. Once the port responds (with any status other than `000`), normal health polling begins using the `timeout` and `interval` values above. -On contested CI runners, Grafana's HTTP listener can take longer to bind than the default health-check window allows, regardless of Grafana version. Increasing this value gives the process more time to start without affecting the health-check phase. +Raise this value only if Grafana genuinely takes longer than the default to start. A long default delays diagnosis of startup *crashes* (for example, a provisioning deadlock that prevents the HTTP listener from ever binding), since the action can only observe `000` and cannot distinguish "still booting" from "dead before booting." Pair this action with a separate diagnostic step that dumps container logs on failure to recover the real signal. ## How to use? diff --git a/wait-for-grafana/action.yml b/wait-for-grafana/action.yml index b742ccba..4e9cbc8d 100644 --- a/wait-for-grafana/action.yml +++ b/wait-for-grafana/action.yml @@ -18,9 +18,9 @@ inputs: required: true default: "0.5" startupTimeout: - description: "Seconds to wait for the TCP port to bind before health polling begins. On contested CI runners, Grafana's HTTP listener can take longer to start than the default health-check window allows." + description: "Seconds to wait for the TCP port to bind before health polling begins. Raise this only if Grafana genuinely takes longer than the default to start — a long default delays diagnosis of startup crashes (e.g. provisioning deadlocks), which never recover no matter how long we wait." required: true - default: "300" + default: "60" runs: using: "composite" steps: diff --git a/wait-for-grafana/wait-for-grafana.sh b/wait-for-grafana/wait-for-grafana.sh index 5da6bac9..7ae478e5 100755 --- a/wait-for-grafana/wait-for-grafana.sh +++ b/wait-for-grafana/wait-for-grafana.sh @@ -4,7 +4,7 @@ url="$1" expected_response_code="$2" timeout="$3" interval="$4" -startup_timeout="${5:-300}" +startup_timeout="${5:-60}" echo "Checking URL: $url" echo "Expected response code: $expected_response_code" From ceee5c4afa21f07275c4d1dedfe86ff21bd66fec Mon Sep 17 00:00:00 2001 From: Darren Janeczek Date: Wed, 27 May 2026 14:39:04 -0400 Subject: [PATCH 2/2] feat(grafana-startup-logs): add diagnostic action for wait-for-grafana failures MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Introduces a sibling action that pairs with wait-for-grafana on the failure path. wait-for-grafana can only observe "Current status: 000" (connection refused) and cannot distinguish a slow-but-alive Grafana from one that crashed before binding :3000. This action recovers the missing signal. On failure, the action: 1. Discovers the Grafana container via a heuristic chain (explicit container/service inputs → first compose service matching /grafana/ → docker ps publish-port lookup → literal "grafana" container). 2. Pulls full container logs to $RUNNER_TEMP and applies a configurable regex redaction pipeline (default catches Bearer tokens, glsa_/glc_ service-account tokens, JWT prefixes, ?token= URL parameters, and inline password/secret/token assignments). 3. Prints three workflow log groups: - "Grafana errors and warnings (filtered)" — level-aware, multi-line so stack-trace continuations after an error line stay attached - "Container state" — docker compose ps -a / docker ps -a - "Recent Grafana logs (last N lines, all levels, redacted)" 4. Optionally uploads the full redacted logs as an actions artifact with a 7-day default retention and surfaces the artifact URL in the job summary. PII trust model is documented in the README. Defaults treat container logs as semi-sensitive (shorter retention than GitHub's 90-day default, opt-out artifact upload, regex redaction layered on top of GHA's secret masker). The masker only protects log streams; redact-patterns covers the artifact path. Co-Authored-By: Claude Opus 4.7 (1M context) --- .release-please-manifest.json | 1 + grafana-startup-logs/LICENSE | 201 +++++++++++++++++++ grafana-startup-logs/README.md | 110 ++++++++++ grafana-startup-logs/action.yml | 118 +++++++++++ grafana-startup-logs/grafana-startup-logs.sh | 159 +++++++++++++++ release-please-config.json | 4 + 6 files changed, 593 insertions(+) create mode 100644 grafana-startup-logs/LICENSE create mode 100644 grafana-startup-logs/README.md create mode 100644 grafana-startup-logs/action.yml create mode 100755 grafana-startup-logs/grafana-startup-logs.sh diff --git a/.release-please-manifest.json b/.release-please-manifest.json index 8caf11e0..77ff7757 100644 --- a/.release-please-manifest.json +++ b/.release-please-manifest.json @@ -4,6 +4,7 @@ "bundle-types": "1.0.3", "create-plugin-update": "2.0.2", "e2e-version": "2.0.0", + "grafana-startup-logs": "1.0.0", "is-compatible": "1.0.3", "package-plugin": "1.2.0", "playwright-gh-pages/deploy-report-pages": "1.1.0", diff --git a/grafana-startup-logs/LICENSE b/grafana-startup-logs/LICENSE new file mode 100644 index 00000000..7a2f8dac --- /dev/null +++ b/grafana-startup-logs/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright 2024 Grafana Labs + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/grafana-startup-logs/README.md b/grafana-startup-logs/README.md new file mode 100644 index 00000000..c7a8ee1f --- /dev/null +++ b/grafana-startup-logs/README.md @@ -0,0 +1,110 @@ +# Grafana startup logs + +Diagnostic companion to [`wait-for-grafana`](../wait-for-grafana). When wait-for-grafana fails, this action discovers the Grafana container, prints a level-filtered summary plus a bounded tail to the workflow log, and uploads the full redacted logs as a short-retention artifact. + +## When to use it + +`wait-for-grafana` can only observe `Current status: 000` — connection refused. It cannot distinguish "Grafana is still booting" from "Grafana crashed before binding `:3000`." Pair this action with it on the failure path to recover the missing signal. + +## Example + + +```yaml +- name: Wait for Grafana to start + id: wait + uses: grafana/plugin-actions/wait-for-grafana@wait-for-grafana/v1.0.5 + +- name: Dump Grafana startup logs on failure + if: ${{ failure() && steps.wait.outcome == 'failure' }} + uses: grafana/plugin-actions/grafana-startup-logs@grafana-startup-logs/v1.0.0 + with: + additional-secrets: | + ${{ env.HL_TOKEN }} + ${{ env.GRAFANACLOUD_USAGE_TOKEN }} +``` + + +The two-step pattern keeps the polling action narrow and lets the diagnostic action live on the failure branch only. + +## Inputs + +All inputs are optional. Most have heuristic discovery that handles the default `docker-compose.yaml` + `:3000` setup with no configuration. + +### `url` + +URL passed to wait-for-grafana. The port is parsed and used as a container-discovery fallback. Default `http://localhost:3000/`. + +### `service` + +Compose service name for Grafana. Empty triggers heuristic discovery (first compose service whose name contains `grafana`, then port-based `docker ps` lookup). Set to `none` to skip docker compose and use plain `docker logs`. Default empty. + +### `container` + +Explicit container name. Wins over `service` when set. Used with plain `docker logs`. Default empty. + +### `compose-file` + +Path to docker-compose file. Default uses discovery (`docker-compose.yaml` in `working-directory`). + +### `working-directory` + +Working directory for docker commands. Default `.`. + +### `level` + +Comma-separated log levels included in the filtered summary group. Multi-line continuations after a matching line (stack traces) are carried until the next timestamp-led line. Default `error,warn,crit,eror` — `eror` is Grafana's truncated rendering of `error`. + +### `tail` + +Tail line count for the unfiltered group printed inline. Set to `0` to skip this group (the artifact still contains the full logs). Default `500`. + +### `upload-artifact` + +Whether to upload the full redacted logs as an artifact. Default `true`. + +### `artifact-name` + +Artifact name. Default `grafana-startup-logs`. For matrix workflows, prefix with `${{ matrix.GRAFANA_IMAGE.NAME }}-${{ matrix.GRAFANA_IMAGE.VERSION }}-` to keep matrix cells from colliding. + +### `artifact-retention-days` + +Artifact retention. Default `7` — shorter than GitHub's 90-day default to limit the window during which container logs are downloadable from CI. + +### `redact-patterns` + +Newline-separated regex patterns redacted from all output and from the artifact. Each non-empty, non-comment line is an extended-regex pattern; the entire match is replaced with `[REDACTED]`. Default catches `Bearer …`, `glsa_…`, `glc_…`, `eyJ…` (JWT prefix), `?token=…` / `?api_key=…` URL parameters, and inline `password=…` / `token: …` style assignments. + +This applies on top of GitHub Actions' built-in secret masker, which only covers values registered via the `secrets` context or `::add-mask::`. Use `additional-secrets` to register plugin-specific values that the workflow has in scope but did not pass through `secrets:`. + +### `additional-secrets` + +Newline-separated literal values registered with `::add-mask::` before any docker commands run. The masker affects log streams only — artifact contents are protected by `redact-patterns`. + +## Outputs + +### `artifact-url` + +URL of the uploaded artifact, when `upload-artifact` is true and the upload succeeded. Also written to the job summary as a markdown link. + +## PII and trust model + +Container logs are not designed to be a public surface. Default behavior treats them as semi-sensitive: + +- Artifact retention defaults to **7 days** (vs. GitHub's 90) +- `redact-patterns` strips common token / JWT / URL-parameter shapes before content leaves the runner +- `additional-secrets` re-masks plugin-specific values for the log streams that GHA would otherwise miss +- The artifact is **opt-out**: set `upload-artifact: false` for repos that consider container logs too sensitive even after redaction + +The redactor is regex-based and best-effort. Treat the artifact as semi-sensitive in proportion to what your plugins log. + +## How it discovers the container + +In order: + +1. `container` input (explicit, wins over everything) +2. `service` input (explicit compose service) +3. First compose service whose name contains `grafana` (case-insensitive) +4. `docker ps --filter publish=` where `` is parsed from `url` +5. A container literally named `grafana` + +If none match, the action prints a warning and exits 0 — it never blocks the workflow on its own discovery miss. diff --git a/grafana-startup-logs/action.yml b/grafana-startup-logs/action.yml new file mode 100644 index 00000000..659bc8e9 --- /dev/null +++ b/grafana-startup-logs/action.yml @@ -0,0 +1,118 @@ +name: "Grafana startup logs" +description: "On a wait-for-grafana failure, dump docker compose status, a level-filtered summary of Grafana's own logs, and upload the full redacted logs as a short-retention artifact." +inputs: + url: + description: "URL passed to wait-for-grafana. The port is used as a container-discovery fallback when service/container inputs are not set." + required: false + default: "http://localhost:3000/" + service: + description: "Compose service name for Grafana. Empty triggers heuristic discovery (first compose service whose name contains 'grafana', then port-based docker ps lookup). Set to 'none' to skip docker compose entirely and use plain docker logs." + required: false + default: "" + container: + description: "Explicit container name. Wins over service when set. Used with plain docker logs." + required: false + default: "" + compose-file: + description: "Path to docker-compose file. Empty uses the default discovery (docker-compose.yaml in working-directory)." + required: false + default: "" + working-directory: + description: "Working directory for docker commands." + required: false + default: "." + level: + description: "Comma-separated log levels to include in the filtered summary group. Multi-line continuations after a matching line are included until the next timestamp-led line." + required: false + default: "error,warn,crit,eror" + tail: + description: "Tail line count for the unfiltered log group printed inline. Set to 0 to skip the unfiltered group (the artifact still contains the full logs)." + required: false + default: "500" + upload-artifact: + description: "Whether to upload the full redacted logs as an actions artifact. Set to false for repos that consider container logs sensitive enough to keep inline-only." + required: false + default: "true" + artifact-name: + description: "Artifact name. Defaults to a name that includes the matrix job context so concurrent matrix cells do not collide." + required: false + default: "grafana-startup-logs" + artifact-retention-days: + description: "Artifact retention. Default is shorter than the GitHub-default 90 days to limit the window during which container logs are downloadable from CI." + required: false + default: "7" + redact-patterns: + description: "Newline-separated regex patterns redacted from all output and from the artifact. Applied on top of GitHub Actions' built-in secret masking, which only covers values registered via secrets context or ::add-mask::." + required: false + default: | + Bearer\s+[A-Za-z0-9._\-]+ + glsa_[A-Za-z0-9_]+ + glc_[A-Za-z0-9_]+ + eyJ[A-Za-z0-9._\-]+ + [?&](token|api_key|apikey|auth|password|secret)=[^&[:space:]]+ + (password|secret|api[_-]?key|token)[\"']?\s*[:=]\s*[\"']?[^\s\"',}]+ + additional-secrets: + description: "Newline-separated literal values to register with GitHub Actions' secret masker (::add-mask::) before any docker commands run. Use this for plugin-specific env vars (HL_TOKEN, etc.) that the workflow exposed but did not pass via the secrets context." + required: false + default: "" + +outputs: + artifact-url: + description: "URL of the uploaded artifact, when upload-artifact is true and the upload succeeded." + value: ${{ steps.upload.outputs.artifact-url }} + +runs: + using: "composite" + steps: + - name: Register additional secrets with the masker + shell: bash + env: + GSL_ADDITIONAL_SECRETS: ${{ inputs.additional-secrets }} + run: | + # Each non-empty line is masked from subsequent log output. + # ::add-mask:: directives only affect log streams, not artifact contents — + # the redact-patterns regex pipeline covers the artifact side. + while IFS= read -r line; do + [ -n "$line" ] || continue + echo "::add-mask::$line" + done <<< "$GSL_ADDITIONAL_SECRETS" + + - name: Collect and redact Grafana startup logs + id: collect + shell: bash + working-directory: ${{ inputs.working-directory }} + env: + GSL_URL: ${{ inputs.url }} + GSL_SERVICE: ${{ inputs.service }} + GSL_CONTAINER: ${{ inputs.container }} + GSL_COMPOSE_FILE: ${{ inputs.compose-file }} + GSL_LEVEL: ${{ inputs.level }} + GSL_TAIL: ${{ inputs.tail }} + GSL_REDACT_PATTERNS: ${{ inputs.redact-patterns }} + run: ${{ github.action_path }}/grafana-startup-logs.sh + + - name: Upload full redacted logs as artifact + id: upload + if: ${{ inputs.upload-artifact == 'true' && steps.collect.outputs.log-file != '' }} + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 + with: + name: ${{ inputs.artifact-name }} + path: ${{ steps.collect.outputs.log-dir }} + retention-days: ${{ inputs.artifact-retention-days }} + if-no-files-found: warn + + - name: Write artifact link to job summary + if: ${{ inputs.upload-artifact == 'true' && steps.upload.outputs.artifact-url != '' }} + shell: bash + env: + GSL_ARTIFACT_URL: ${{ steps.upload.outputs.artifact-url }} + GSL_ARTIFACT_NAME: ${{ inputs.artifact-name }} + GSL_RETENTION: ${{ inputs.artifact-retention-days }} + run: | + { + echo "### Grafana startup logs" + echo + echo "- Artifact: [\`$GSL_ARTIFACT_NAME\`]($GSL_ARTIFACT_URL)" + echo "- Retention: $GSL_RETENTION days" + echo "- Contents are redacted per the action's regex pipeline. Treat the artifact as semi-sensitive: it contains application logs, not secrets, but unredacted patterns may slip through." + } >> "$GITHUB_STEP_SUMMARY" diff --git a/grafana-startup-logs/grafana-startup-logs.sh b/grafana-startup-logs/grafana-startup-logs.sh new file mode 100755 index 00000000..42fd0b2c --- /dev/null +++ b/grafana-startup-logs/grafana-startup-logs.sh @@ -0,0 +1,159 @@ +#!/bin/bash +# Dump Grafana container logs on failure with level-aware filtering, regex +# redaction, and artifact upload. Designed to run as a follow-up step after +# wait-for-grafana fails — never invoked on success. +# +# Inputs come from environment variables (see action.yml). The script +# discovers the container via a heuristic chain (explicit container, +# compose service, compose-service-name regex, docker-ps port lookup), +# writes the full redacted logs to $RUNNER_TEMP/grafana-startup-logs/, and +# prints three log groups: filtered errors+warns, container state, and a +# bounded tail of the full log. + +set -uo pipefail + +mkdir -p "$RUNNER_TEMP/grafana-startup-logs" +log_dir="$RUNNER_TEMP/grafana-startup-logs" +full_log="$log_dir/grafana-full.log" +redacted_log="$log_dir/grafana.log" +ps_log="$log_dir/docker-compose-ps.txt" + +# Surface paths for the composite step's outputs. +echo "log-dir=$log_dir" >> "$GITHUB_OUTPUT" + +# ---------- compose helpers ---------- + +compose_args=() +if [ -n "${GSL_COMPOSE_FILE:-}" ]; then + compose_args+=(-f "$GSL_COMPOSE_FILE") +fi + +run_compose() { + docker compose "${compose_args[@]}" "$@" +} + +# ---------- container discovery ---------- + +container="" +compose_service="" + +if [ -n "${GSL_CONTAINER:-}" ]; then + container="$GSL_CONTAINER" +elif [ "${GSL_SERVICE:-}" = "none" ]; then + # Skip compose entirely; fall through to port-based docker ps lookup. + : +elif [ -n "${GSL_SERVICE:-}" ]; then + compose_service="$GSL_SERVICE" +else + # Heuristic 1: first compose service whose name contains "grafana". + if compose_services=$(run_compose ps --services 2>/dev/null); then + compose_service=$(printf '%s\n' "$compose_services" | grep -i grafana | head -n 1 || true) + fi +fi + +# Heuristic 2 (fallback): port-based docker ps lookup. +if [ -z "$container" ] && [ -z "$compose_service" ]; then + port=$(printf '%s' "$GSL_URL" | sed -E 's|^[a-z]+://[^:/]+:([0-9]+).*|\1|; t; s|.*|3000|') + if [ -n "$port" ]; then + container=$(docker ps --filter "publish=$port" --format '{{.Names}}' | head -n 1 || true) + fi +fi + +# Heuristic 3 (last resort): literal container named "grafana". +if [ -z "$container" ] && [ -z "$compose_service" ]; then + if docker inspect grafana >/dev/null 2>&1; then + container="grafana" + fi +fi + +if [ -z "$container" ] && [ -z "$compose_service" ]; then + echo "::warning::grafana-startup-logs: could not locate a Grafana container. Set the 'container' or 'service' input explicitly." + exit 0 +fi + +# ---------- collect ---------- + +if [ -n "$compose_service" ]; then + echo "Collecting logs from compose service: $compose_service" + run_compose ps -a > "$ps_log" 2>&1 || true + run_compose logs --no-color --no-log-prefix "$compose_service" > "$full_log" 2>&1 || true +else + echo "Collecting logs from container: $container" + docker ps -a --filter "name=$container" > "$ps_log" 2>&1 || true + docker logs "$container" > "$full_log" 2>&1 || true +fi + +if [ ! -s "$full_log" ]; then + echo "::warning::grafana-startup-logs: log file is empty. Container may not have produced any output." +fi + +# ---------- redaction ---------- + +# Build sed args from the multi-line redact-patterns input. Empty lines and +# comments (lines starting with #) are ignored. Each pattern is wrapped in +# an extended-regex substitution that replaces the entire match with +# [REDACTED]. Pattern delimiter is ~ because | clashes with regex alternation +# inside patterns (e.g. "(password|secret)=..."), / is common in URLs, and +# : / = appear in many log lines and patterns. +sed_args=() +while IFS= read -r pat; do + # Trim whitespace. + pat="${pat#"${pat%%[![:space:]]*}"}" + pat="${pat%"${pat##*[![:space:]]}"}" + [ -z "$pat" ] && continue + case "$pat" in \#*) continue ;; esac + sed_args+=(-E -e "s~$pat~[REDACTED]~g") +done <<< "${GSL_REDACT_PATTERNS:-}" + +if [ "${#sed_args[@]}" -gt 0 ]; then + if ! sed "${sed_args[@]}" "$full_log" > "$redacted_log"; then + echo "::warning::grafana-startup-logs: redaction sed pipeline failed; falling back to copying full log verbatim." + cp "$full_log" "$redacted_log" + fi +else + cp "$full_log" "$redacted_log" +fi + +# Drop the unredacted intermediate so the artifact never contains it. +rm -f "$full_log" +echo "log-file=$redacted_log" >> "$GITHUB_OUTPUT" + +# ---------- filtered summary ---------- + +# Build alternation regex from comma-separated levels. +levels_alt=$(printf '%s' "${GSL_LEVEL:-error,warn,crit}" | tr ',' '|' | tr -d '[:space:]') + +# awk-based multi-line filter: +# - lines that match lvl= or "level":"" → emit and mark in-match +# - lines that look like a new log line (t=, JSON {, or ISO timestamp) and +# do NOT match the level filter → stop emitting +# - other lines while in-match (stack frames, continuation) → emit +# +# Regex notes: \b is a GNU extension, not POSIX ERE — awk would warn and treat +# it as literal. The level keyword in Grafana text logs is always followed by +# a space, so dropping the boundary is safe in practice. Likewise [{] avoids +# awk's interval-expression interpretation of a bare {. +filter_re="(lvl=(${levels_alt})|\"level\":\"(${levels_alt})\")" +new_line_re="^(t=|[{]|[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9])" + +echo "::group::Grafana errors and warnings (filtered to: ${GSL_LEVEL})" +awk -v fr="$filter_re" -v nr="$new_line_re" ' + $0 ~ fr { in_match=1; print; next } + $0 ~ nr { in_match=0; next } + in_match == 1 { print } +' "$redacted_log" || true +echo "::endgroup::" + +# ---------- container state ---------- + +echo "::group::Container state" +cat "$ps_log" || true +echo "::endgroup::" + +# ---------- bounded tail ---------- + +if [ "${GSL_TAIL:-500}" -gt 0 ] 2>/dev/null; then + echo "::group::Recent Grafana logs (last ${GSL_TAIL} lines, all levels, redacted)" + tail -n "$GSL_TAIL" "$redacted_log" || true + echo "::endgroup::" +fi diff --git a/release-please-config.json b/release-please-config.json index 8fdfa3b8..4f498625 100644 --- a/release-please-config.json +++ b/release-please-config.json @@ -69,6 +69,10 @@ "package-name": "e2e-version", "extra-files": ["README.md"] }, + "grafana-startup-logs": { + "package-name": "grafana-startup-logs", + "extra-files": ["README.md"] + }, "is-compatible": { "package-name": "is-compatible", "extra-files": ["README.md"]