Skip to content

chore: audit cleanup pass (2026-05-22)#234

Merged
JacobPEvans merged 7 commits into
mainfrom
chore/audit-cleanup
May 25, 2026
Merged

chore: audit cleanup pass (2026-05-22)#234
JacobPEvans merged 7 commits into
mainfrom
chore/audit-cleanup

Conversation

@JacobPEvans
Copy link
Copy Markdown
Collaborator

@JacobPEvans JacobPEvans commented May 22, 2026

Summary

  • Audit pass on gitignore, Python toolchain config, pre-commit hooks, and documentation
  • Added test fixtures to enforce placeholder IP pattern; replaced leaked Splunk IPs with shared constants
  • Expanded security and contribution guides with contact info, version matrix, and pre-commit setup steps

Changes

Gitignore and caching:

  • Excluded .envrc and tightened Python cache gitignores (.pyc, pycache, .mypy_cache, .pytest_cache)

Python toolchain:

  • Added mypy strict mode config to pyproject.toml
  • Added coverage gate (fail_under=80) to pyproject.toml

Pre-commit and secrets:

  • Added gitleaks pre-commit hook for secrets scanning

Documentation:

  • Expanded SECURITY.md with Contact and Supported Versions sections
  • Added Pre-Commit Setup section to CONTRIBUTING.md

Tests:

  • Created tests/fixtures.py with SAMPLE_HEC_URL and SAMPLE_SPLUNK_MGMT_URL constants (192.168.0.200 range)
  • Refactored test_unit.py::TestUrlPresentInOutputsYaml to use shared fixtures instead of hardcoded IPs

Test Plan

  • make test-unit — verify new fixtures and refactored URL test pass
  • make test-all — run full E2E suite to ensure no regressions
  • Pre-commit hooks run cleanly (gitleaks, yamllint, mypy, coverage)
  • CI validation passes (validate.yml, e2e-tests.yml)

Related

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request enhances the project's security and development tooling by adding gitleaks for secret scanning, providing detailed pre-commit setup instructions, and configuring mypy and code coverage. Feedback suggests updating the pre-commit installation command to include the 'pre-commit' hook type so that all checks run as intended, and refining the coverage configuration to omit test files from the report to ensure accurate metrics.

Comment thread CONTRIBUTING.md Outdated
Comment thread pyproject.toml
JacobPEvans added a commit that referenced this pull request May 24, 2026
The previous install command only registered commit-msg and pre-push
hooks, so default-stage hooks (ruff, shellcheck, yamllint, gitleaks)
never ran on git commit. Add --hook-type pre-commit to cover them.

Addresses gemini-code-assist review feedback on PR #234.
JacobPEvans added a commit that referenced this pull request May 24, 2026
…nstant

PR #234 commit 5cbb851 introduced "10.0.1.200" into two test cases — the
live Splunk address observed in Cribl Stream's outputs.yml, pasted
verbatim instead of using the placeholder pattern (192.168.0.x) already
used in adjacent tests. The existing no-real-ips hook missed it: it only
scanned *.yaml/*.sh under k8s/, scripts/, docker/.

Three layers:

1. tests/fixtures.py — single source of truth for sample URLs. Tests
   import SAMPLE_HEC_URL / SAMPLE_SPLUNK_MGMT_URL instead of inlining
   literals, removing the "what URL goes here" decision that leads to
   copy-paste from live tool output.

2. scripts/check-no-real-ips.py — replaces the inline bash hook. Runs
   on every committed file type (yaml/yml/sh/py/md/json/toml/j2/conf/
   cfg/txt/env/example), receives staged paths from pre-commit, and
   matches IPs against a strict allowlist: 192.168.0.0/24, loopback,
   0.0.0.0, broadcast, link-local metadata. Skips version pins.

3. CLAUDE.md / CONTRIBUTING.md — documents the convention so the next
   agent reaching for an example URL has an explicit rule to follow
   instead of inferring from observed system state.

Hook verified against the original leak and full tree.

Assisted-by: Claude <noreply@anthropic.com>
JacobPEvans added a commit that referenced this pull request May 24, 2026
PR #234 commit 5cbb851 introduced "10.0.1.200" into two new test cases —
the live Splunk address observed in Cribl Stream's outputs.yml, pasted
verbatim instead of using the placeholder pattern (192.168.0.x) already
used in adjacent tests.

Refactor TestUrlPresentInOutputsYaml to import SAMPLE_HEC_URL /
SAMPLE_SPLUNK_MGMT_URL from tests/fixtures.py, removing the "what URL
goes here" decision that lets agents copy values from live tool output.

Enforcement of the no-real-ips convention is moving to the content-guards
plugin (PreToolUse Write|Edit hook), which blocks bad values at write
time instead of commit time.

Assisted-by: Claude <noreply@anthropic.com>
@JacobPEvans JacobPEvans force-pushed the chore/audit-cleanup branch from c2de315 to 32abe34 Compare May 24, 2026 02:04
JacobPEvans added a commit that referenced this pull request May 24, 2026
The previous install command only registered commit-msg and pre-push
hooks, so default-stage hooks (ruff, shellcheck, yamllint, gitleaks)
never ran on git commit. Add --hook-type pre-commit to cover them.

Addresses gemini-code-assist review feedback on PR #234.
JacobPEvans added a commit that referenced this pull request May 24, 2026
PR #234 commit 5cbb851 introduced "10.0.1.200" into two new test cases —
the live Splunk address observed in Cribl Stream's outputs.yml, pasted
verbatim instead of using the placeholder pattern (192.168.0.x) already
used in adjacent tests.

Refactor TestUrlPresentInOutputsYaml to import SAMPLE_HEC_URL /
SAMPLE_SPLUNK_MGMT_URL from tests/fixtures.py, removing the "what URL
goes here" decision that lets agents copy values from live tool output.

Enforcement of the no-real-ips convention is moving to the content-guards
plugin (PreToolUse Write|Edit hook), which blocks bad values at write
time instead of commit time.

Assisted-by: Claude <noreply@anthropic.com>
@JacobPEvans JacobPEvans force-pushed the chore/audit-cleanup branch from 32abe34 to 1148943 Compare May 24, 2026 02:39
JacobPEvans added a commit to dryvist/docs that referenced this pull request May 24, 2026
The CI/CD overview documented the four runner tiers but didn't say
what a self-hosted runner has to actually be. The recurring
token-refresh failure in orbstack-kubernetes
(dryvist/orbstack-kubernetes#234, #237) shows the cost of leaving
this implicit.

Adds a single subsection between "Runner tiers" and "The shape of every
IaC pipeline" listing the five non-negotiables for any self-hosted
runner: GitHub App auth (not PAT), digest-pinned image, process
healthcheck, dead-man's-switch heartbeat, pre-flight secret check.
Links to the orbstack-kubernetes runner as the reference implementation.

Companion PRs codify the same rules at the AI-agent layer:
- dryvist/ai-assistant-instructions#654 (org-wide ci-cd-policy rule)
- dryvist/claude-code-plugins#321 (self-hosted-runners skill)

Supersedes the earlier standalone runner-topology-page draft in this
PR's history — the four-tier CI/CD section landed in #23 in the
meantime, making a separate topology page redundant.

Assisted-by: Claude <noreply@anthropic.com>
JacobPEvans added a commit that referenced this pull request May 24, 2026
Three additive hardening changes to the self-hosted runner; addresses the
recurring token-refresh failure loop that blocked #234 at merge.

- Pin myoung34/github-runner to the current ubuntu-jammy multi-arch
  manifest digest (sha256:0d48...) so pulls are deterministic and
  Renovate's docker-compose manager tracks future builds.
- Drop DISABLE_AUTO_UPDATE=1 so the actions/runner binary self-updates
  on registration; combined with the Renovate-managed image digest this
  keeps both layers current without manual docker pull cycles.
- Add a Docker healthcheck (curl https://api.github.com/zen) and surface
  it through runner-doctor-container so reachability regressions show up
  in `docker compose ps` and the doctor target instead of failing silently.
- Add runner-preflight as a dependency of runner-foreground and
  runner-start; doppler-injected GH_PAT_RUNNER_TOKEN is asserted non-empty
  before docker compose up so the LaunchAgent stops the silent 30s retry
  loop and surfaces the actionable error in stderr.log.

Follow-up: migrate from fine-grained PAT to GitHub App auth
(myoung34/github-runner natively supports APP_ID/APP_PRIVATE_KEY and
mints installation tokens internally — no expiry, no manual rotation).
Captured as a separate issue because it requires creating a GitHub App
in the org and updating Doppler.

Assisted-by: Claude <noreply@anthropic.com>
JacobPEvans added a commit to dryvist/claude-code-plugins that referenced this pull request May 24, 2026
…ements

The skill covered RunsOn vs github-hosted but said nothing about on-prem
self-hosted runners — yet two repos in the org use them and the
recurring token-refresh failure in orbstack-kubernetes (dryvist/orbstack-kubernetes#234, #237)
shows what happens when the on-prem path has no documented requirements.

- Reframe the lead: RunsOn is default, on-prem is the documented
  exception for hardware-bound jobs. Update the H1, intro paragraphs,
  and skill description frontmatter to match.
- Add an on-prem row to the decision table with the two actual in-use
  labels.
- New "On-prem runner requirements" section listing the five
  non-negotiables: GitHub App auth (not PAT), digest-pinned image,
  process healthcheck, dead-man's-switch heartbeat, pre-flight secret
  check. Link to the orbstack-kubernetes runner as the reference impl.
- Side fix: reword "App Runner + CloudWatch" so the wrapped "+ " at
  line start doesn't get parsed as a markdown list bullet (was causing
  cascading MD032/MD004 errors after my main edit).

Companion PR: dryvist/ai-assistant-instructions#654 codifies the
same rule in the org-wide ci-cd-policy rule.

Assisted-by: Claude <noreply@anthropic.com>
JacobPEvans added a commit that referenced this pull request May 24, 2026
The ignorePatterns entry pointed at `kubernetes-monitoring` — the
repo's previous name before c21d9ca renamed it to `orbstack-kubernetes`.
The pattern hasn't matched any actual self-references since that rename,
which means every github.com self-ref in CHANGELOG.md (release-please
compare links), docs, and READMEs has been getting hit by markdown-link-check
and randomly failing pre-commit with transient GitHub 502s. PRs #234,
#237, #240, #241 have all hit this flake in the last 24 hours.

Fix: update the pattern to the current repo name. Self-references are
safe to skip because the CHANGELOG compare links and issue links are
machine-generated by release-please and can't have typos; cross-repo
links are still validated.

Assisted-by: Claude <noreply@anthropic.com>
JacobPEvans added a commit that referenced this pull request May 24, 2026
…242)

The ignorePatterns entry pointed at `kubernetes-monitoring` — the
repo's previous name before c21d9ca renamed it to `orbstack-kubernetes`.
The pattern hasn't matched any actual self-references since that rename,
which means every github.com self-ref in CHANGELOG.md (release-please
compare links), docs, and READMEs has been getting hit by markdown-link-check
and randomly failing pre-commit with transient GitHub 502s. PRs #234,
#237, #240, #241 have all hit this flake in the last 24 hours.

Fix: update the pattern to the current repo name. Self-references are
safe to skip because the CHANGELOG compare links and issue links are
machine-generated by release-please and can't have typos; cross-repo
links are still validated.

Assisted-by: Claude <noreply@anthropic.com>
JacobPEvans added a commit that referenced this pull request May 24, 2026
The configmap template for Cribl Stream wraps the HEC URL in double
quotes (k8s/monitoring/cribl-stream-standalone/configmap-cribl-config.yaml
line 12), so after sed substitution the deployed outputs.yml renders as:

    url: "https://10.0.1.200:8088/services/collector"

The helper's regex required strictly unquoted values, giving a false
negative on every run and failing test_splunk_hec_url_matches_secret.
This has blocked PR #234 and #237 on E2E for two days.

Fix: allow an optional matching pair of single or double quotes around
the URL value. Tested against six representative cases (double-quoted,
single-quoted, unquoted, mismatched-quote, unsubstituted placeholder,
different URL) — all behave correctly.

Assisted-by: Claude <noreply@anthropic.com>
JacobPEvans added a commit that referenced this pull request May 24, 2026
The configmap template for Cribl Stream wraps the HEC URL in double
quotes (k8s/monitoring/cribl-stream-standalone/configmap-cribl-config.yaml
line 12), so after sed substitution the deployed outputs.yml renders as:

    url: "https://10.0.1.200:8088/services/collector"

The helper's regex required strictly unquoted values, giving a false
negative on every run and failing test_splunk_hec_url_matches_secret.
This has blocked PR #234 and #237 on E2E for two days.

Fix: allow an optional matching pair of single or double quotes around
the URL value. Tested against six representative cases (double-quoted,
single-quoted, unquoted, mismatched-quote, unsubstituted placeholder,
different URL) — all behave correctly.

Assisted-by: Claude <noreply@anthropic.com>
JacobPEvans added a commit that referenced this pull request May 24, 2026
…241)

The configmap template for Cribl Stream wraps the HEC URL in double
quotes (k8s/monitoring/cribl-stream-standalone/configmap-cribl-config.yaml
line 12), so after sed substitution the deployed outputs.yml renders as:

    url: "https://10.0.1.200:8088/services/collector"

The helper's regex required strictly unquoted values, giving a false
negative on every run and failing test_splunk_hec_url_matches_secret.
This has blocked PR #234 and #237 on E2E for two days.

Fix: allow an optional matching pair of single or double quotes around
the URL value. Tested against six representative cases (double-quoted,
single-quoted, unquoted, mismatched-quote, unsubstituted placeholder,
different URL) — all behave correctly.

Assisted-by: Claude <noreply@anthropic.com>
The previous install command only registered commit-msg and pre-push
hooks, so default-stage hooks (ruff, shellcheck, yamllint, gitleaks)
never ran on git commit. Add --hook-type pre-commit to cover them.

Addresses gemini-code-assist review feedback on PR #234.
PR #234 commit 5cbb851 introduced "10.0.1.200" into two new test cases —
the live Splunk address observed in Cribl Stream's outputs.yml, pasted
verbatim instead of using the placeholder pattern (192.168.0.x) already
used in adjacent tests.

Refactor TestUrlPresentInOutputsYaml to import SAMPLE_HEC_URL /
SAMPLE_SPLUNK_MGMT_URL from tests/fixtures.py, removing the "what URL
goes here" decision that lets agents copy values from live tool output.

Enforcement of the no-real-ips convention is moving to the content-guards
plugin (PreToolUse Write|Edit hook), which blocks bad values at write
time instead of commit time.

Assisted-by: Claude <noreply@anthropic.com>
@JacobPEvans JacobPEvans force-pushed the chore/audit-cleanup branch from 1148943 to c5d7380 Compare May 25, 2026 00:19
JacobPEvans added a commit that referenced this pull request May 25, 2026
Three additive hardening changes to the self-hosted runner; addresses the
recurring token-refresh failure loop that blocked #234 at merge.

- Pin myoung34/github-runner to the current ubuntu-jammy multi-arch
  manifest digest (sha256:0d48...) so pulls are deterministic and
  Renovate's docker-compose manager tracks future builds.
- Drop DISABLE_AUTO_UPDATE=1 so the actions/runner binary self-updates
  on registration; combined with the Renovate-managed image digest this
  keeps both layers current without manual docker pull cycles.
- Add a Docker healthcheck (curl https://api.github.com/zen) and surface
  it through runner-doctor-container so reachability regressions show up
  in `docker compose ps` and the doctor target instead of failing silently.
- Add runner-preflight as a dependency of runner-foreground and
  runner-start; doppler-injected GH_PAT_RUNNER_TOKEN is asserted non-empty
  before docker compose up so the LaunchAgent stops the silent 30s retry
  loop and surfaces the actionable error in stderr.log.

Follow-up: migrate from fine-grained PAT to GitHub App auth
(myoung34/github-runner natively supports APP_ID/APP_PRIVATE_KEY and
mints installation tokens internally — no expiry, no manual rotation).
Captured as a separate issue because it requires creating a GitHub App
in the org and updating Doppler.

Assisted-by: Claude <noreply@anthropic.com>
@JacobPEvans JacobPEvans merged commit f1ef5c0 into main May 25, 2026
17 checks passed
@JacobPEvans JacobPEvans deleted the chore/audit-cleanup branch May 25, 2026 16:48
JacobPEvans added a commit that referenced this pull request May 25, 2026
…ck (#237)

Three additive hardening changes to the self-hosted runner; addresses the
recurring token-refresh failure loop that blocked #234 at merge.

- Pin myoung34/github-runner to the current ubuntu-jammy multi-arch
  manifest digest (sha256:0d48...) so pulls are deterministic and
  Renovate's docker-compose manager tracks future builds.
- Drop DISABLE_AUTO_UPDATE=1 so the actions/runner binary self-updates
  on registration; combined with the Renovate-managed image digest this
  keeps both layers current without manual docker pull cycles.
- Add a Docker healthcheck (curl https://api.github.com/zen) and surface
  it through runner-doctor-container so reachability regressions show up
  in `docker compose ps` and the doctor target instead of failing silently.
- Add runner-preflight as a dependency of runner-foreground and
  runner-start; doppler-injected GH_PAT_RUNNER_TOKEN is asserted non-empty
  before docker compose up so the LaunchAgent stops the silent 30s retry
  loop and surfaces the actionable error in stderr.log.

Follow-up: migrate from fine-grained PAT to GitHub App auth
(myoung34/github-runner natively supports APP_ID/APP_PRIVATE_KEY and
mints installation tokens internally — no expiry, no manual rotation).
Captured as a separate issue because it requires creating a GitHub App
in the org and updating Doppler.

Assisted-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant