feat: add quarterly audit CLI, grace period automation, workflows, and agent skill

.agents/skills/quarterly-ladder-analysis/SKILL.md

@@ -0,0 +1,107 @@
+---
+name: quarterly-ladder-analysis
+description: Instructions and operational workflow for conducting the Flutter Contributor Ladder quarterly audits (Community Audit and Partner Team Verification).
+---
+
+# Quarterly Contributor Ladder Analysis Skill
+
+This skill guides you through executing Flutter's quarterly contributor reviews across the 30+ day review cycle.
+
+## Step 0: Determine Current Phase in the 30+ Day Process
+
+Before running any mutating commands, **first ask the user (if not already clear from their request) where in the 30+ day quarterly review process we are** (you can also check open `quarterly-review` / `quarterly-review-checklist` issues and open PRs via `gh issue list` and `gh pr list`):
+
+1. **Phase 1 — Kickoff & Opening the PR / Review Issues (Day 1)**: Starting the quarterly review after the automated checklist issue is filed.
+2. **Phase 2 — Mid-Review Check-In (Days 1–30)**: Auditing responses on the open review issue, checking an individual contributor (`tool/check_user.dart`), or tracking partner confirmations during the 30-day window.
+3. **Phase 3 — Finalization & Merging (Day 30+)**: Incorporating 30-day responses into the open quarterly review PR, landing the PR, syncing GitHub/Discord permissions, and closing the issues.
+
+Proceed to the matching phase below.
+
+---
+
+## Phase 1: Kickoff, Ban-History Vetting, Single PR & Review Issues (Day 1)
+
+### 1.1 Preview Nominated Candidates & Ask User for Ban-History Check
+1. Ensure you are on a fresh branch off the latest `main` (which is updated daily by `.github/workflows/sync_activity.yml`):
+   ```bash
+   git checkout -b YYYY-QX-quarterly-review
+   ```
+2. Run `--prepare` in dry-run, non-interactive mode first to see the nominated candidates without writing any files (`tool/quarterly_audit.dart` automatically queries live `flutter` GitHub organization and team membership so every current org member is evaluated):
+   ```bash
+   dart run tool/quarterly_audit.dart --prepare --dry-run --no-prompt
+   ```
+3. **Present the nominated usernames** (promotions to Committer, returning Dasher Emeriti, and prospective external new joiners) to the user and **ask them to check those usernames against the internal Code of Conduct / ban history list**.
+   - Contributors who were previously banned for Code of Conduct violations remain welcome to contribute as external Contributors after a temporary ban expires, but are not eligible for ladder membership.
+   - Wait for the user to confirm whether any usernames should be excluded before generating files on disk so no one is singled out in reports or commits.
+
+### 1.2 Generate Reports & Update Roster for the Single PR
+Once the user confirms any exclusions (or confirms none are needed), run `--prepare`:
+```bash
+dart run tool/quarterly_audit.dart --prepare --no-prompt --exclude user1,user2
+```
+*(Omit `--exclude` if no candidates are excluded.)*
+This single command:
+- Writes `reports/YYYY-QX-community-ladder-audit.md` and `reports/YYYY-QX-partner-verification-audit.md` (with excluded users completely omitted).
+- Updates `data/contributors.yaml`, `ROSTER.md`, and `EMERITUS.md`.
+
+### 1.3 Open the Single Quarterly Review PR
+Commit the generated reports and proposed roster updates into **1 PR**:
+```bash
+git add reports/ data/contributors.yaml ROSTER.md EMERITUS.md
+git commit -m "chore: QX YYYY quarterly contributor ladder review"
+```
+This PR stays open throughout the 30-day community review period and does not land until Phase 3 (Day 30+).
+
+### 1.4 Draft & Open the Public Review Issues
+Use the tool to preview or open the two public tracking issues from the vetted reports to start the 30-day review period:
+```bash
+# Preview drafted issues and gh CLI commands:
+dart run tool/quarterly_audit.dart --draft-issues --pr <PR_URL>
+
+# Or open both issues directly on GitHub via gh CLI:
+dart run tool/quarterly_audit.dart --open-issues --pr <PR_URL>
+```
+
+---
+
+## Phase 2: Manage the 30-Day Grace Period (Days 1–30)
+
+- Inspect responses on the Community Audit issue (`gh issue view <ISSUE_NUMBER> --comments`):
+  - Collect Discord usernames from newly invited contributors (`user=discord_handle`).
+  - Summarize reported qualitative contributions that cannot be quantified automatically (issue triage comments, reproduction verification, [RFC](https://github.com/flutter/rfc) reviews, mentoring) for the user to review.
+  - To inspect a specific contributor's 12-month metrics and current roster status:
+    ```bash
+    dart run tool/check_user.dart <username>
+    ```
+- Remind the user to reach out to partner teams to confirm active status for `@flutter/partners` members.
+- *(During the 30-day window, `.github/workflows/check_grace_period.yml` runs once a week to post a weekly countdown update on the open review issues, and pings the assignee once 30 days have elapsed.)*
+
+---
+
+## Phase 3: Update the Open PR, Land It & Provision Access (Day 30+)
+
+1. Gather 30-day adjustments:
+   - Read all comments on the open Community Audit issue (`gh issue view <ISSUE_NUMBER> --comments`) and confirm with the user:
+     - `--retain`: Contributors in the 30-day grace period who shared verified qualitative contributions.
+     - `--exclude`: Any previously excluded usernames plus any invited contributors who declined.
+     - `--discord`: `github_user=discord_handle` mappings shared by accepted new joiners, returning Emeriti, and promoted members.
+2. Update the open PR branch:
+   On the open `YYYY-QX-quarterly-review` PR branch, run `--finalize`:
+   ```bash
+   dart run tool/quarterly_audit.dart --finalize \
+     --retain user1,user2 \
+     --exclude excluded_or_declined_user \
+     --discord user=discord_handle
+   ```
+3. Validate schema and roster sync:
+   ```bash
+   dart run tool/validate_roster.dart
+   dart run tool/generate_roster.dart --check
+   ```
+4. Commit, push & land the PR:
+   Commit the updated `data/contributors.yaml`, `ROSTER.md`, and `EMERITUS.md` to the PR branch and merge the PR.
+5. Post-merge provisioning checklist:
+   - Update GitHub organization teams (`@flutter/reviewers`, `@flutter/committers`, and `@flutter/maintainers` as child teams of `@flutter/flutter-hackers`, along with `@flutter/partners` child teams) to match `ROSTER.md` (`@flutter/collaborators` is automatically managed by internal tooling and should not be modified manually).
+   - Remove members who transitioned to Dasher Emeritus from the `flutter` GitHub organization.
+   - Grant the Discord team member role to newly added ladder members using their collected Discord handles.
+   - Close the public review issues and the quarterly checklist issue.

.agents/skills/quarterly-ladder-analysis/references/bots.txt

@@ -0,0 +1,44 @@
+dependabot
+dependabot[bot]
+fluttergithubbot
+google-oss-robot
+google-oss-robot[bot]
+github-actions
+github-actions[bot]
+engine-flutter-autoroll
+engine-flutter-autoroll[bot]
+skia-flutter-autoroll
+skia-flutter-autoroll[bot]
+flutter-autoroll
+flutter-autoroll[bot]
+auto-submit
+auto-submit[bot]
+google-cla
+google-cla[bot]
+googlebot
+copybara-service
+copybara-service[bot]
+renovate
+renovate[bot]
+flutteractionsbot
+flutter-pub-roller-bot
+flutter-skills-sync-bot
+flutter-intellij-kokoro
+flutter-website-bot
+fluttercerberus
+flutterbuildbot
+flutterdeploybot
+fluttergalleryreleasebot
+fluttermirroringbot
+DartDevtoolWorkflowBot
+buildbot
+agentfu
+kenzie-agent
+reidbaker-agent
+google-admin
+google-admin-automation-1
+google-admin-automation-2
+google-admin-backup
+google-ospo-team
+gemini-code-assist
+gemini-code-assist[bot]

.agents/skills/quarterly-ladder-analysis/references/criteria.md

@@ -0,0 +1,19 @@
+# Contributor Ladder Metric Criteria Reference
+
+## 12-Month Rolling Criteria Rubric
+
+| Role | GitHub Role | Criteria (Rolling 12 Months) |
+| :--- | :--- | :--- |
+| **Contributor** | `Read` (External) | Any material contribution (code, docs, issue triage, QA). |
+| **Reviewer** | `Triage` (Org Member) | $\ge 10$ qualifying contributions (merged PRs, valid issues, reviews on merged PRs, reproducible issue triage). |
+| **Committer** | `Write` (Org Member) | $\ge 30$ qualifying contributions and $\ge 10$ merged PRs, demonstrating depth, code quality, and technical judgment. |
+| **Maintainer** | `Write` + Org Mod (Org Member) | All members of `@flutter/googlers` and `@flutter/partners` (there are no Maintainers outside of Googlers and Partners). |
+| **Dasher Emeritus** | `Read` (External Alumni) | Inactive community role-holder ($< 10$ contributions in past 12 months after 30-day grace period). Removed from the GitHub org and listed in `EMERITUS.md`. |
+
+## Special Policies
+
+Contributors who have previously been banned for Code of Conduct violations remain welcome to contribute as external Contributors after a temporary ban expires, but are not eligible for ladder membership (`Reviewer`, `Committer`, `Maintainer`). When running `tool/quarterly_audit.dart --prepare` locally, the person running the audit checks the nominated usernames against the internal Code of Conduct / ban history list and excludes any ineligible candidates before any report or roster files are generated, ensuring no one is singled out.
+
+Community members who joined `flutter-hackers` within the 12 months prior to ladder launch receive a one-time 12-month Reviewer transition grace period (`grace_until` in `data/contributors.yaml`). Once that initial year expires, no further recent-joiner carve-outs apply.
+
+Googlers and Partners are automatically classified as Maintainers (while their 12-month contribution history is still recorded in `data/activity_summary.json`), and contributions that cannot be quantified automatically—such as issue triage, reproduction verifications, [RFC](https://github.com/flutter/rfc) feedback, design docs, and mentoring—count toward Reviewer and Committer qualifications when shared during the 30-day review period.

.agents/skills/quarterly-ladder-analysis/references/known_maintainers.txt

@@ -0,0 +1,8 @@
+# These are known maintainers, listed here by exception due to a sync issue with internal tooling and GitHub.
+CarlosMendonca
+LouiseHsu
+abdallahshaban557
+andywolff
+gaaclarke
+gmackall
+jesswrd

.github/workflows/check_grace_period.yml

@@ -0,0 +1,42 @@
+name: Weekly Grace Period Check
+
+on:
+  schedule:
+    # 03:23 UTC every Monday (Weekly check for open quarterly review issues, offset from top of hour)
+    - cron: '23 3 * * 1'
+  workflow_dispatch:
+    inputs:
+      dry_run:
+        description: 'Dry run (do not post comments or add labels)'
+        type: boolean
+        default: false
+
+permissions:
+  contents: read
+  issues: write
+
+concurrency:
+  group: check-grace-period
+  cancel-in-progress: false
+
+jobs:
+  grace_period_check:
+    name: Check Open Quarterly Review Issues
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+
+      - name: Set up Dart SDK
+        uses: dart-lang/setup-dart@e51d8e571e22473a2ddebf0ef8a2123f0ab2c02c # v1.7.1
+
+      - name: Install dependencies
+        run: dart pub get
+
+      - name: Check Open Quarterly Review Issues
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          DRY_RUN: ${{ inputs.dry_run || false }}
+        run: dart run tool/check_grace_period.dart

.github/workflows/quarterly_review.yml

@@ -0,0 +1,115 @@
+name: Quarterly Ladder Review
+
+on:
+  schedule:
+    # 03:17 UTC on the 1st of January, April, July, and October (Quarterly Kickoff, offset from top of hour)
+    - cron: '17 3 1 1,4,7,10 *'
+  workflow_dispatch:
+    inputs:
+      dry_run:
+        description: 'Dry run (do not file issues)'
+        type: boolean
+        default: false
+
+permissions:
+  contents: read
+  issues: write
+
+concurrency:
+  group: quarterly-review
+  cancel-in-progress: false
+
+jobs:
+  file_checklist:
+    name: File Quarterly Review Checklist Issue
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+
+      - name: File Quarterly Review Checklist Issue for @Piinks
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          DRY_RUN: ${{ inputs.dry_run || false }}
+        run: |
+          QUARTER="Q$(( ($(date +%-m) - 1) / 3 + 1 )) $(date +%Y)"
+          QUARTER_SLUG="$(date +%Y)-Q$(( ($(date +%-m) - 1) / 3 + 1 ))"
+
+          cat <<EOF > checklist_body.md
+          # Quarterly Contributor Ladder Review Checklist (${QUARTER})
+
+          This issue tracks the facilitator tasks for the **${QUARTER}** Flutter Contributor Ladder review.
+          See [docs/QUARTERLY_REVIEW.md](docs/QUARTERLY_REVIEW.md) for full details.
+
+          ## Phase 1: Kickoff & Opening the Review PR (Day 1)
+          - [ ] **1. Run the audit locally & vet against the internal ban history list**
+            Pull the latest \`main\` branch (with daily synced activity data), create a branch for the quarterly review PR, and run:
+            \`\`\`bash
+            git checkout -b ${QUARTER_SLUG}-quarterly-review
+            dart run tool/quarterly_audit.dart --prepare
+            \`\`\`
+            - The tool lists all nominated promotions, returning Emeriti, and prospective external new joiners so you can manually check those usernames against the internal Code of Conduct / ban history list.
+            - Enter any ineligible usernames when prompted (or pass \`--exclude user1,user2\`) so they are excluded before any files are written and no one is singled out.
+            - This generates \`reports/${QUARTER_SLUG}-community-ladder-audit.md\` and \`reports/${QUARTER_SLUG}-partner-verification-audit.md\` and updates \`data/contributors.yaml\`, \`ROSTER.md\`, and \`EMERITUS.md\`.
+          - [ ] **2. Open the single Quarterly Review PR (do not merge until after 30-day grace period)**
+            Commit the generated reports and roster updates into a single PR:
+            \`\`\`bash
+            git add reports/ data/contributors.yaml ROSTER.md EMERITUS.md
+            git commit -m "chore: ${QUARTER} quarterly contributor ladder review"
+            \`\`\`
+            *(Hold this PR open throughout the 30-day review period.)*
+          - [ ] **3. Open the public Community Review & Partner Verification issues**
+            Use the tool to draft or open the two public tracking issues from your vetted reports to start the 30-day review period:
+            \`\`\`bash
+            # Preview drafted issues and gh commands:
+            dart run tool/quarterly_audit.dart --draft-issues --pr <PR_URL>
+
+            # Or open both issues directly via gh CLI:
+            dart run tool/quarterly_audit.dart --open-issues --pr <PR_URL>
+            \`\`\`
+
+          ## Phase 2: 30-Day Review Period (Days 1–30)
+          - [ ] **4. Monitor community responses & confirm partner members**
+            - Collect Discord usernames from newly invited contributors on the Community Audit issue.
+            - Review qualitative contributions that cannot be quantified automatically (triage, reproductions, [RFCs](https://github.com/flutter/rfc), mentoring).
+            - Reach out to partner teams to confirm active status for \`@flutter/partners\` members.
+            - *(The weekly \`.github/workflows/check_grace_period.yml\` workflow automatically posts a weekly countdown update on the open review issues during the 30-day window and pings the assignee once 30 days have elapsed.)*
+
+          ## Phase 3: Finalization & Provisioning (Day 30+)
+          - [ ] **5. Update the open PR with 30-day adjustments & land it**
+            On your \`${QUARTER_SLUG}-quarterly-review\` PR branch, run the tool with any adjustments from the 30-day window:
+            \`\`\`bash
+            dart run tool/quarterly_audit.dart --finalize \\
+              --retain user1,user2 \\
+              --exclude declined_or_excluded_user \\
+              --discord new_user=discord_handle
+            \`\`\`
+            Commit, push to the PR branch, and merge the PR.
+          - [ ] **6. Synchronize GitHub teams, Discord access & close issues**
+            - Update \`@flutter/reviewers\`, \`@flutter/committers\`, and \`@flutter/maintainers\` (child teams of \`@flutter/flutter-hackers\`, where \`@flutter/maintainers\` contains all members of \`@flutter/googlers\` and \`@flutter/partners\`) and \`@flutter/partners\` child teams to match \`ROSTER.md\` (\`@flutter/collaborators\` is automatically managed by internal tooling and is not updated manually).
+            - Remove members who transitioned to Dasher Emeritus from the \`flutter\` GitHub organization.
+            - Grant the Discord team member role to newly added ladder members using their collected Discord handles.
+            - Close the public review issues and this checklist issue.
+          EOF
+
+          if [ "$DRY_RUN" = "true" ]; then
+            echo "Dry run enabled. Would file checklist issue:"
+            cat checklist_body.md
+            exit 0
+          fi
+
+          gh label create "quarterly-review-checklist" --description "Quarterly Review Facilitator Checklist" --color "0052CC" --force || true
+
+          EXISTING_CHECKLIST=$(gh issue list --state all --label "quarterly-review-checklist" --search "${QUARTER}" --json number --jq '.[0].number')
+          if [ -z "$EXISTING_CHECKLIST" ]; then
+            CHECKLIST_URL=$(gh issue create \
+              --title "[Quarterly Review Checklist]: ${QUARTER} Contributor Ladder Audit" \
+              --body-file checklist_body.md \
+              --label "quarterly-review-checklist" \
+              --assignee "Piinks")
+            echo "Created Quarterly Review Checklist Issue: $CHECKLIST_URL"
+          else
+            echo "Quarterly Review Checklist Issue already exists (#$EXISTING_CHECKLIST)."
+          fi

.github/workflows/validate.yml

@@ -5,6 +5,7 @@ on:
     branches: [main]
   pull_request:
     branches: [main]
+  workflow_dispatch:
 
 permissions:
   contents: read
@@ -32,7 +33,13 @@ jobs:
         run: dart format --output=none --set-exit-if-changed .
 
       - name: Analyze Dart code
-        run: dart analyze
+        run: dart analyze --fatal-infos
+
+      - name: Validate contributors.yaml
+        run: dart run tool/validate_roster.dart
+
+      - name: Verify ROSTER.md and EMERITUS.md are in sync
+        run: dart run tool/generate_roster.dart --check
 
       - name: Run unit test suite
         run: dart test