Skip to content

Split android-reviewer review-rules.md into conditional rule files#11266

Open
jonathanpeppers wants to merge 6 commits intomainfrom
jonathanpeppers/split-reviewer-rules
Open

Split android-reviewer review-rules.md into conditional rule files#11266
jonathanpeppers wants to merge 6 commits intomainfrom
jonathanpeppers/split-reviewer-rules

Conversation

@jonathanpeppers
Copy link
Copy Markdown
Member

@jonathanpeppers jonathanpeppers commented May 1, 2026

Split the monolithic review-rules.md (297 lines, 150 rules) into 8 focused files that the skill loads conditionally based on which file types changed in the PR.

Always loaded:

  • repo-conventions.md — formatting, style, localization, repo-specific patterns
  • ai-pitfalls.md — common AI-generated code mistakes

Conditionally loaded:

  • csharp-rules.md (46 rules) — nullable, async, error handling, performance, code organization — when .cs files changed
  • msbuild-rules.md (20 rules) — MSBuild task conventions, process management, and target/XML rules — when .targets/.props/.csproj changed
  • native-rules.md (27 rules) — memory management, C++ best practices, symbol visibility, platform-specific code — when .c/.cpp/.h/.hpp changed
  • interop-rules.md (5 rules) — managed-native boundary (P/Invoke, JNI, marshalling) — when diff contains interop markers (DllImport, [Register], JNIEnv, [MarshalAs], [StructLayout])
  • testing-rules.md (7 rules) — test conventions and best practices — when tests/ or **/Tests/ files changed
  • security-rules.md (3 rules) — zip slip, command injection, path traversal — when any code files changed

Deep Audit

All 150 original rules verified preserved via word-for-word content comparison:

Category Count
Identical 112
Encoding-only (UTF-8) 27
Intentionally modified 8
Missing 0
New rules added 6

6 new rules (from cross-referencing with dotnet/android-tools#355)

Rule File
Confidently wrong domain facts ai-pitfalls.md
Over-mocking ai-pitfalls.md
Debug.WriteLine for logging ai-pitfalls.md
Use appropriate log levels msbuild-rules.md
Don't redirect stdout/stderr without draining msbuild-rules.md
Include stdout in error diagnostics msbuild-rules.md

8 intentionally modified rules (aligned wording with android-tools)

Rule Change
Zip Slip Added ZipFile.ExtractToDirectory() warning
Command injection Added "never interpolate user/external input"
Path traversal Added example bypass + directory boundary check
Bug fixes need tests #1234#N (generalized for sharing)
Test assertions Added NUnit constraint examples (Does.Contain, Is.EqualTo)
Reinventing the wheel Made generic (repo-specific utilities still in repo-conventions.md)
Swallowed errors Added exit code checking note
Unused parameters Added string interpolation injection risk

Evaluation

Tested the split against 3 real merged PRs with different file-type profiles:

PR Profile Rules loaded Rules skipped Savings Coverage gaps?
#11246 MSBuild + C# + tests 124 / 156 32 (native, interop) 21% None
#11237 C# bug fix + test 104 / 156 52 (msbuild, native, interop) 33% None
#11035 Large C# + tests (22 files) 104 / 156 52 (msbuild, native, interop) 33% None

Key findings:

  • Zero coverage gaps across all 3 PRs — every finding came from rules that were loaded
  • 21–33% context reduction per review, freeing tokens for actual diff analysis
  • Zero false positives from the 6 new rules — they correctly stayed silent on PRs where they don't apply
  • The file-type heuristic correctly determined which rule categories to load in all cases
  • Interop-rules loading triggers on diff content (interop markers like DllImport, [Register]) rather than just file presence

jonathanpeppers and others added 4 commits May 1, 2026 16:01
Split the monolithic review-rules.md (297 lines) into 8 focused files
that the skill loads conditionally based on which file types changed
in the PR:

Always loaded:
- repo-conventions.md: formatting, style, localization, repo patterns
- ai-pitfalls.md: common AI-generated code mistakes

Conditionally loaded:
- csharp-rules.md: nullable, async, error handling, performance, code org
- msbuild-rules.md: MSBuild task conventions and target/XML rules
- native-rules.md: C/C++ memory, best practices, symbols, platform code
- interop-rules.md: managed-native boundary (P/Invoke, JNI, marshalling)
- testing-rules.md: test conventions and best practices
- security-rules.md: zip slip, command injection, path traversal

Updated SKILL.md step 5 to describe the conditional loading logic.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ile presence

The previous heuristic loaded interop rules only when both C# and native
files changed. This missed cases where a C#-only diff touches JNI/P/Invoke
boundary code (e.g., DllImport, [Register], JNIEnv, [MarshalAs],
[StructLayout]). Now triggers based on actual interop markers in the diff.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Cross-referenced with dotnet/android-tools#355 to align shared rules
and adopt new relevant ones.

New rules added (6):
- ai-pitfalls: Confidently wrong domain facts
- ai-pitfalls: Over-mocking
- ai-pitfalls: Debug.WriteLine for logging
- msbuild-rules: Use appropriate log levels (MessageImportance)
- msbuild-rules: Don't redirect stdout/stderr without draining
- msbuild-rules: Include stdout in error diagnostics

Wording aligned with android-tools for sharing:
- security: Zip Slip adds ZipFile.ExtractToDirectory() warning
- security: Path traversal adds directory boundary example
- security: Command injection adds "never interpolate" note
- security: Reorganized into Archive & Process subsections
- ai-pitfalls: Reinventing the wheel adds "hundreds of lines" emphasis
- ai-pitfalls: Swallowed errors adds exit code checking
- ai-pitfalls: Null-forgiving expanded with IsNullOrEmpty suggestion
- ai-pitfalls: Unused parameters adds injection risk note
- testing: Bug fixes uses generic "#N" placeholder
- testing: Assertions adds NUnit constraint examples

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jonathanpeppers jonathanpeppers marked this pull request as ready for review May 1, 2026 21:31
Copilot AI review requested due to automatic review settings May 1, 2026 21:31
@jonathanpeppers
Copy link
Copy Markdown
Member Author

/review

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 1, 2026

Android PR Reviewer completed successfully!

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the android-reviewer skill’s rule reference material by replacing the single monolithic review-rules.md with multiple focused rule files intended to be loaded conditionally based on what a PR changes.

Changes:

  • Delete the legacy monolithic references/review-rules.md.
  • Add 8 focused rule reference files (repo conventions, AI pitfalls, and per-area rule sets like C#, MSBuild, native, interop, testing, security).
  • Update SKILL.md to describe the new always/conditional rule-loading behavior.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
.github/skills/android-reviewer/SKILL.md Updates step 5 to load the new split rule files (always + conditional sets).
.github/skills/android-reviewer/references/review-rules.md Removes the legacy monolithic rule file.
.github/skills/android-reviewer/references/repo-conventions.md Adds always-loaded repo-specific conventions (formatting/localization/patterns).
.github/skills/android-reviewer/references/ai-pitfalls.md Adds always-loaded AI pitfall guidance.
.github/skills/android-reviewer/references/csharp-rules.md Adds conditional C# rules (nullable/async/error handling/perf/organization).
.github/skills/android-reviewer/references/msbuild-rules.md Adds conditional MSBuild rules (tasks/targets/process management).
.github/skills/android-reviewer/references/native-rules.md Adds conditional native rules (memory/C++/visibility/platform).
.github/skills/android-reviewer/references/interop-rules.md Adds conditional interop boundary rules (P/Invoke/JNI/shared structs).
.github/skills/android-reviewer/references/testing-rules.md Adds conditional test guidance.
.github/skills/android-reviewer/references/security-rules.md Adds conditional security checks (zip slip/path traversal/command injection).

Comment thread .github/skills/android-reviewer/SKILL.md
Comment thread .github/skills/android-reviewer/references/interop-rules.md Outdated
- android-reviewer.md: Remove reference to deleted review-rules.md,
  point to SKILL.md which handles rule file loading (step 5)
- interop-rules.md: Align header load-condition wording with SKILL.md
  to mention interop markers (DllImport, [Register], etc.)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM

Well-structured refactoring that splits the monolithic 297-line review-rules.md into 8 focused files with conditional loading. Verified:

  • 0 rules lost — All 150 original rules present across the 8 new files
  • 6 new rules added — Domain facts, over-mocking, logging patterns, process management (matches PR description)
  • 8 enhanced rules — Security rules (Zip Slip, path traversal, command injection) meaningfully improved with concrete examples and additional guidance
  • SKILL.md updated correctly — Conditional loading instructions match the actual file set
  • Postmortem references — All consistently backtick-wrapped (#N) to prevent GitHub auto-linking
  • CI is green — All 6 checks passing

Positive callouts:

  • The conditional loading strategy (21-33% savings per the evaluation) is a pragmatic way to reduce context for the AI reviewer without losing coverage
  • Grouping security rules into "Archive & Path Safety" and "Process & Command Safety" subsections improves scannability
  • Generalizing #1234#N in testing-rules.md makes the rules portable across repos

1 suggestion filed inline about reducing duplication between always-loaded and conditionally-loaded null-forgiving operator rules.

Generated by Android PR Reviewer for issue #11266 · ● 11M

Comment thread .github/skills/android-reviewer/references/ai-pitfalls.md Outdated
Keep ai-pitfalls.md concise with a cross-reference to csharp-rules.md
for the full NRT details. Saves ~500 tokens of duplicate context since
ai-pitfalls is always loaded and csharp-rules is loaded for most PRs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jonathanpeppers jonathanpeppers added the ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable). label May 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants