Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .agents/skills
14 changes: 14 additions & 0 deletions .claude.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Global Regex and ReDoS Policy

Whenever you are writing, modifying, or reviewing Regular Expressions (Regex) in this codebase, you **MUST** adhere to the project's structural-extraction accuracy and ReDoS immunity standards.

1. **ReDoS Immunity & Boundary Correctness**: You must read and follow the 12 engine rules defined in `gitgalaxy/standards/how_to_add_a_language.md`.
2. **Hardening Process**: You must follow the 5-stage pipeline checklist and avoid the recurring bug classes outlined in `tests/extraction/how_to_harden_extraction.md`.
3. **Use the Tooling**: Always verify regex empirically against the real compiled regex using `tests/extraction/tools/verify_candidates.py` and run the scaling checks. Do not guess whether a payload matches or scales.
4. **Skills**: If you are deepening or fixing a language's structural-extraction accuracy (e.g. `func_start`, `args`), you must activate and follow the `harden-language-extraction` skill.

# Core Engine Modification CI Checklist

When you are preparing to push a fix or open a PR that touches GitGalaxy's core parsing logic (`language_standards.py`, `detector.py`, `prism.py`), you **MUST** ensure the CI validation gauntlet is handled.

1. **Invoke the ci-push-checklist skill**: You must activate and follow the `.claude/skills/ci-push-checklist/SKILL.md` before pushing to ensure all Golden Masters, Tri-Comparison, and Tree-Sitter baselines are accurately regenerated and validated.
11 changes: 11 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
### Description
<!-- Describe your changes in detail -->

### Core Engine Modification Checklist
If your PR changes the core parsing engine (`language_standards.py`, `detector.py`, `prism.py`), you **MUST** complete the following validation gauntlet before merging:

- [ ] **Golden Master Verification**: Ran `crucible_check.py` to verify diffs, and ran `crucible_check.py --update --yes` if the baseline changed due to intentional parsing improvements.
- [ ] **Tri-Comparison Audit**: Handled by CI automatically, but you should run `python tests/tools/tri_comparison_chart.py --all --ci` locally first. If you intentionally improved precision, regenerate the baseline: `python tests/tools/tri_comparison_chart.py --regenerate --languages <lang>`.
- [ ] **Tree-Sitter Accuracy**: Ran `python tests/tools/tree_sitter_accuracy_audit.py --ci --all`. If ground truth drifted for valid reasons, regenerated the baseline with `python tests/tools/tree_sitter_accuracy_audit.py --regenerate --lang <lang>`.

> **Note for AI Agents**: If you are an AI agent generating this PR, you MUST invoke the `ci-push-checklist` skill to ensure these steps are completed properly.
14 changes: 14 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Global Regex and ReDoS Policy

Whenever you are writing, modifying, or reviewing Regular Expressions (Regex) in this codebase, you **MUST** adhere to the project's structural-extraction accuracy and ReDoS immunity standards.

1. **ReDoS Immunity & Boundary Correctness**: You must read and follow the 12 engine rules defined in `gitgalaxy/standards/how_to_add_a_language.md`.
2. **Hardening Process**: You must follow the 5-stage pipeline checklist and avoid the recurring bug classes outlined in `tests/extraction/how_to_harden_extraction.md`.
3. **Use the Tooling**: Always verify regex empirically against the real compiled regex using `tests/extraction/tools/verify_candidates.py` and run the scaling checks. Do not guess whether a payload matches or scales.
4. **Skills**: If you are deepening or fixing a language's structural-extraction accuracy (e.g. `func_start`, `args`), you must activate and follow the `harden-language-extraction` skill.

# Core Engine Modification CI Checklist

When you are preparing to push a fix or open a PR that touches GitGalaxy's core parsing logic (`language_standards.py`, `detector.py`, `prism.py`), you **MUST** ensure the CI validation gauntlet is handled.

1. **Invoke the ci-push-checklist skill**: You must activate and follow the `.agents/skills/ci-push-checklist/SKILL.md` before pushing to ensure all Golden Masters, Tri-Comparison, and Tree-Sitter baselines are accurately regenerated and validated.
23 changes: 16 additions & 7 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,17 +59,26 @@ To ensure your contribution integrates smoothly into the Zero-Trust ecosystem:

---

## 🔒 Updating the Golden Crucible Baseline
## 🔒 CI Pipeline & Baselines

`tests/golden_master_audit.json` and `tests/golden_master_zero_dep_audit.json` are the accepted-good snapshots that every PR's output is diffed against (see the `crucible-audit` check). A failing diff means GitGalaxy's output changed -- that's either a bug you should fix, or an intentional improvement whose new baseline needs to be deliberately re-blessed.
When you modify GitGalaxy's core engine, several CI workflows will rigorously test your changes against established baselines. If your changes intentionally improve parsing precision, you must regenerate the corresponding baselines.

**Never** hand-copy fresh output over these files. Always use:
1. **Golden Crucible Baselines (`tests/golden_master_audit.json`)**
This validates exact extraction output.
- Run `python tests/tools/crucible_check.py` to view structural drift.
- To re-bless the new state: `python tests/tools/crucible_check.py --update --yes`

```
python tests/tools/update_golden_master.py
```
2. **Tri-Comparison Ledger (`docs/self_scan/tri_comparison_ledger.json`)**
This compares GitGalaxy precision against ctags and tree-sitter.
- Run `python tests/tools/tri_comparison_chart.py --all --ci` to test.
- If you improved precision, lock it in: `python tests/tools/tri_comparison_chart.py --regenerate --languages <lang>`

3. **Tree-Sitter Accuracy (`docs/self_scan/tree_sitter_accuracy_history.csv`)**
This audits AST accuracy against tree-sitter ground truth.
- Run `python tests/tools/tree_sitter_accuracy_audit.py --ci --all`.
- If your regex fix legitimately drops false positives causing ground truth drift, regenerate the baseline: `python tests/tools/tree_sitter_accuracy_audit.py --regenerate --lang <lang>`

It shows you exactly what's about to change before writing anything, and requires explicit confirmation. If your PR touches either fixture, **explain why in the PR description** (e.g. "improved the Rust parser, now correctly detects async trait bounds") -- a CI check flags any PR that modifies these files so it's never invisible in a large diff.
If your PR touches any baseline fixtures, **explain why in the PR description** (e.g. "improved the Rust parser, now correctly detects async trait bounds"). A CI check flags any PR that modifies these files so it's never invisible in a large diff.

---

Expand Down
10 changes: 5 additions & 5 deletions docs/self_scan/tree_sitter_accuracy_chart.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
45 changes: 45 additions & 0 deletions docs/self_scan/tree_sitter_accuracy_history.csv
Original file line number Diff line number Diff line change
Expand Up @@ -4818,3 +4818,48 @@ timestamp_utc,commit_sha,language,files_scanned,real_functions,found_functions,e
2026-08-22T23:56:51+00:00,5b8e2c3f2ec195fa4d1cc541cd0b99fdee3cbfc8,sqlite,0,0,0,0,0,0,0,0,0,,,,,0,0,0,0,0,,,,,,1,0
2026-08-22T23:56:51+00:00,5b8e2c3f2ec195fa4d1cc541cd0b99fdee3cbfc8,yacc,0,0,0,0,0,0,0,0,0,,,,,0,0,0,0,0,,,,,,1,1
2026-08-22T23:56:51+00:00,5b8e2c3f2ec195fa4d1cc541cd0b99fdee3cbfc8,yaml,7,0,0,0,0,0,0,0,0,,,,,0,0,0,0,0,,,,,,1,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,apex,4,38,38,0,4,4,0,38,38,100.0,100.0,100.0,100.0,38,0,4,0,38,100.0,100.0,100.0,100.0,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,c,31,1719,1702,9,61,61,0,1702,1702,99.0,99.5,100.0,100.0,1719,71,61,27,1702,100.0,96.0,100.0,69.3,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,cpp,29,1491,1296,60,65,65,0,1296,1233,86.9,95.6,100.0,100.0,1491,0,65,106,1296,100.0,100.0,100.0,38.0,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,csharp,6,964,964,0,33,33,0,964,962,100.0,100.0,100.0,100.0,647,0,24,0,647,67.1,100.0,72.7,100.0,67.1,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,css,4,3,3,0,90,90,0,3,3,100.0,100.0,100.0,100.0,3,0,90,0,3,100.0,100.0,100.0,100.0,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,dart,7,1748,1738,13,200,200,0,1738,1519,99.4,99.3,100.0,100.0,1748,0,200,0,1738,100.0,100.0,100.0,100.0,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,fortran,6,141,139,0,11,11,0,139,138,98.6,100.0,100.0,100.0,123,0,11,0,121,87.2,100.0,100.0,100.0,87.1,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,go,14,897,897,0,69,69,0,897,822,100.0,100.0,100.0,100.0,897,0,69,0,897,100.0,100.0,100.0,100.0,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,groovy,0,0,0,0,0,0,0,0,0,,,,,0,0,0,0,0,,,,,,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,haskell,7,146,146,1,16,16,0,146,136,100.0,99.3,100.0,100.0,146,0,16,0,146,100.0,100.0,100.0,100.0,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,html,13,0,0,0,0,0,0,0,0,,,,,0,0,0,0,0,,,,,,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,java,7,318,318,0,35,35,0,318,318,100.0,100.0,100.0,100.0,318,0,35,0,318,100.0,100.0,100.0,100.0,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,javascript,18,599,599,10,29,29,0,599,554,100.0,98.4,100.0,100.0,599,2,29,0,599,100.0,99.7,100.0,100.0,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,kotlin,5,28,28,0,7,7,0,28,28,100.0,100.0,100.0,100.0,28,0,7,0,28,100.0,100.0,100.0,100.0,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,lua,0,0,0,0,0,0,0,0,0,,,,,0,0,0,0,0,,,,,,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,makefile,1,1,1,0,0,0,0,1,1,100.0,100.0,,,1,0,0,0,1,100.0,100.0,,,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,matlab,6,23,23,0,0,0,0,23,22,100.0,100.0,,,23,0,0,0,23,100.0,100.0,,,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,objective-c,5,153,151,1,5,5,0,151,151,98.7,99.3,100.0,100.0,153,0,5,0,151,100.0,100.0,100.0,100.0,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,perl,22,941,941,1,20,20,0,941,929,100.0,99.9,100.0,100.0,941,122,20,0,170,100.0,88.5,100.0,100.0,18.1,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,php,35,1702,1702,1,29,29,0,1702,1702,100.0,99.9,100.0,100.0,1702,0,29,0,1702,100.0,100.0,100.0,100.0,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,powershell,6,110,110,0,7,7,0,110,105,100.0,100.0,100.0,100.0,110,0,7,0,110,100.0,100.0,100.0,100.0,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,python,258,3625,3625,16,459,459,0,3625,3625,100.0,99.6,100.0,100.0,3625,0,459,0,3625,100.0,100.0,100.0,100.0,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,ruby,7,124,124,0,18,18,0,124,115,100.0,100.0,100.0,100.0,124,0,18,0,124,100.0,100.0,100.0,100.0,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,rust,43,1927,1927,0,270,270,0,1927,1927,100.0,100.0,100.0,100.0,1775,0,245,0,1775,92.1,100.0,90.7,100.0,92.1,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,scala,7,541,541,0,17,17,0,541,525,100.0,100.0,100.0,100.0,541,0,17,0,541,100.0,100.0,100.0,100.0,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,shell,1,3,3,0,0,0,0,3,3,100.0,100.0,,,3,0,0,0,0,100.0,100.0,,,0.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,solidity,7,99,99,6,7,7,0,99,99,100.0,94.3,100.0,100.0,99,0,7,0,99,100.0,100.0,100.0,100.0,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,swift,6,132,131,1,38,38,0,131,130,99.2,99.2,100.0,100.0,132,0,38,0,131,100.0,100.0,100.0,100.0,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,tcl,2,144,142,1,0,0,0,142,142,98.6,99.3,,,144,0,0,0,142,100.0,100.0,,,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,typescript,23,2789,2721,12,842,842,0,2721,2721,97.6,99.6,100.0,100.0,2789,125,842,0,2721,100.0,95.7,100.0,100.0,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,zig,32,2751,2751,0,656,648,1,2751,2751,100.0,100.0,98.8,99.8,2751,0,656,0,2751,100.0,100.0,100.0,100.0,100.0,0,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,abap,7,0,124,0,0,12,0,0,0,,100.0,,100.0,0,0,0,0,0,,,,,,1,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,ada,0,0,0,0,0,0,0,0,0,,,,,0,0,0,0,0,,,,,,1,0
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,agc_assembly,10,0,812,0,0,0,0,0,0,,100.0,,,0,0,0,0,0,,,,,,1,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,assembly,8,0,102,0,0,0,0,0,0,,100.0,,,0,0,0,0,0,,,,,,1,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,cobol,34,0,167,0,0,19,0,0,0,,100.0,,100.0,0,0,0,0,0,,,,,,1,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,dockerfile,4,0,71,0,0,77,0,0,0,,100.0,,100.0,0,0,0,0,0,,,,,,1,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,embedded_python,0,0,0,0,0,0,0,0,0,,,,,0,0,0,0,0,,,,,,1,0
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,jcl,3,0,3,0,0,0,0,0,0,,100.0,,,0,0,0,0,0,,,,,,1,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,livecode,2,0,0,0,0,0,0,0,0,,,,,0,0,0,0,0,,,,,,1,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,m4,7,0,39,0,0,4,0,0,0,,100.0,,100.0,0,0,0,0,0,,,,,,1,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,scheme,4,0,58,0,0,0,0,0,0,,100.0,,,0,0,0,0,0,,,,,,1,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,sqlite,0,0,0,0,0,0,0,0,0,,,,,0,0,0,0,0,,,,,,1,0
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,yacc,0,0,0,0,0,0,0,0,0,,,,,0,0,0,0,0,,,,,,1,1
2026-08-23T18:10:41+00:00,f6b44668ea453601465c4a9fcd4679402b28cda1,yaml,7,0,0,0,0,0,0,0,0,,,,,0,0,0,0,0,,,,,,1,1
Loading
Loading