ci: Readability-score delta Github Action

.github/workflows/README.md

@@ -144,3 +144,47 @@ git add .
 git commit -m "Sync code examples from docs-code-eval"
 git push
 ```
+
+## Readability delta
+
+**Workflow**: `readability-delta.yml`
+
+Posts an informational, **non-blocking** PR comment describing how the PR affects the readability of the English docs it changes (DOCS-2626). It reports the *delta* (before/after) for well-established formulas (Flesch-Kincaid grade, Flesch reading ease, Gunning fog, SMOG), word-weighted across the changed pages, plus an optional AI-agent-comprehension rating from a W&B Inference LLM judge.
+
+### Triggers
+
+- **Pull request**: `opened`, `synchronize`, `reopened` on PRs that touch `**/*.mdx`
+- **Manual**: `workflow_dispatch` (writes the report to the job summary instead of a comment)
+
+### What it does
+
+1. Diffs the PR base and head, scoring each changed English `.mdx` file (localized content under `ja/`, `ko/`, and `fr/` is skipped).
+2. Extracts narrative prose and scores it with `textstat` via the analyzer in the `coreweave/docs-skills` submodule (`.claude/scripts/_readability.py`).
+3. Optionally runs the AI agent comprehension judge (W&B Inference) when `WANDB_API_KEY` is set.
+4. Upserts a single PR comment identified by the `<!-- readability-delta-report -->` marker.
+
+The check **never fails** a PR. If scoring is unavailable it posts a brief notice and exits successfully.
+
+### Configuration
+
+- **Python**: 3.11
+- **Permissions**: `contents: read`, `pull-requests: write`
+- **Report glue**: `scripts/readability/pr_report.py`
+- **Scoring logic**: `.claude/scripts/_readability.py` and `_docs_eval_lib.py` (submodule)
+
+### Authentication
+
+- The main checkout uses the default `GITHUB_TOKEN`.
+- The private, cross-org `coreweave/docs-skills` submodule is initialized in a separate step with the `DOCENGINE_TOKEN` secret (the same `x-access-token` credential used for the `gitsubmodule` ecosystem in `.github/dependabot.yml`; it rotates ~every 30 days and needs no `wandb/docs` scope).
+- The AI agent comprehension judge calls W&B Inference with the `WANDB_DOCS_INFERENCE_API_KEY` secret (a W&B API key whose entity has Inference credits), passed to the scorer as `WANDB_API_KEY`. When that secret is absent, the deterministic `textstat` delta still runs.
+
+### Forks
+
+Fork PRs have no access to repo secrets, so the first step detects a fork, posts an Actions notice, and makes the whole job a no-op (still reporting success). Forks are uncommon in `wandb/docs` and coreweave repos cannot use forks at all.
+
+### Related Files
+
+- **Report glue**: `scripts/readability/pr_report.py`
+- **Dependencies**: `scripts/readability/requirements.txt`
+- **Tests**: `scripts/readability/tests/`
+- **Documentation**: `scripts/readability/README.md`

.github/workflows/readability-delta.yml

@@ -0,0 +1,217 @@
+# ------------------------------------------------------------------
+# Readability delta check (DOCS-2626)
+# ------------------------------------------------------------------
+#
+# What this workflow does
+# -----------------------
+# Posts an informational, non-blocking PR comment describing how the PR affects
+# the readability of the English docs it touches. The scoring logic lives in the
+# coreweave/docs-skills submodule (.claude/scripts/_readability.py and
+# _docs_eval_lib.py); scripts/readability/pr_report.py is the wandb/docs glue
+# that diffs base..head, scores each changed .mdx file, and builds the Markdown.
+#
+# This check never fails a PR. Every step exits 0; the worst case is a comment
+# noting that scoring was unavailable.
+#
+# Auth
+# ----
+# - Main checkout uses the default GITHUB_TOKEN (the workflow runs in the
+#   wandb/docs PR context).
+# - The coreweave/docs-skills submodule is private and cross-org, so a separate
+#   step initializes it with DOCENGINE_TOKEN (the same x-access-token credential
+#   used for the gitsubmodule ecosystem in .github/dependabot.yml). That token
+#   needs no wandb/docs scope.
+# - The AI agent comprehension judge calls W&B Inference with the
+#   WANDB_DOCS_INFERENCE_API_KEY secret (a W&B key with Inference credits),
+#   passed to the scorer as WANDB_API_KEY. When that secret is absent the
+#   deterministic textstat delta still runs.
+#
+# Forks
+# -----
+# Fork PRs have no access to repo secrets, so neither the submodule fetch nor the
+# judge can run. The first step detects a fork and makes the whole job a no-op
+# (posts an Actions notice, exits 0). Forks are uncommon in wandb/docs and
+# coreweave repos cannot use forks at all, so the lost coverage is negligible.
+# ------------------------------------------------------------------
+
+name: Readability delta
+
+on:
+  pull_request:
+    types: [opened, synchronize, reopened]
+    paths:
+      - "**/*.mdx"
+  workflow_dispatch:
+
+concurrency:
+  group: readability-delta-${{ github.ref }}
+  cancel-in-progress: true
+
+permissions:
+  contents: read
+  pull-requests: write
+
+jobs:
+  readability:
+    name: Report readability delta
+    runs-on: ubuntu-latest
+    steps:
+      # Fork no-op guard: fork PRs cannot reach the private submodule or W&B
+      # Inference, so skip the whole check (still reports success).
+      - name: Skip on fork PRs
+        id: fork
+        run: |
+          if [ "${{ github.event_name }}" = "pull_request" ] && \
+             [ "${{ github.event.pull_request.head.repo.fork }}" = "true" ]; then
+            echo "is_fork=true" >> "$GITHUB_OUTPUT"
+            echo "::notice::Readability check skipped: fork PRs have no access to the docs-skills submodule or W&B Inference."
+          else
+            echo "is_fork=false" >> "$GITHUB_OUTPUT"
+          fi
+
+      - name: Check out repository
+        if: steps.fork.outputs.is_fork != 'true'
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
+        with:
+          # Need full history to diff base..head.
+          fetch-depth: 0
+          submodules: false
+
+      # The docs-skills submodule is a private repo in another org. Initialize it
+      # with DOCENGINE_TOKEN so the main checkout token needs no extra scope.
+      - name: Check out docs-skills submodule
+        # Non-fatal: a transient auth/network failure here must not fail the PR.
+        # The report step below detects the missing analyzer and posts the
+        # informational fallback comment instead.
+        continue-on-error: true
+        if: steps.fork.outputs.is_fork != 'true'
+        env:
+          DOCENGINE_TOKEN: ${{ secrets.DOCENGINE_TOKEN }}
+        run: |
+          if [ -z "${DOCENGINE_TOKEN:-}" ]; then
+            echo "::notice::Readability check: DOCENGINE_TOKEN not set; skipping docs-skills submodule checkout."
+            exit 0
+          fi
+          # One-shot url rewrite so the token is never written to the runner's
+          # global git config, where later steps (which don't get the secret in
+          # their env) could recover it from ~/.gitconfig.
+          git -c url."https://x-access-token:${DOCENGINE_TOKEN}@github.com/".insteadOf="https://github.com/" \
+            submodule update --init --recursive .claude
+
+      - name: Set up Python 3.11
+        if: steps.fork.outputs.is_fork != 'true'
+        uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
+        with:
+          python-version: "3.11"
+
+      - name: Cache pip packages
+        if: steps.fork.outputs.is_fork != 'true'
+        uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6
+        with:
+          path: ~/.cache/pip
+          key: ${{ runner.os }}-pip-readability-${{ hashFiles('scripts/readability/requirements.txt') }}
+          restore-keys: |
+            ${{ runner.os }}-pip-readability-
+            ${{ runner.os }}-pip-
+
+      - name: Install dependencies
+        # Non-fatal: if deps fail to install, the report step still runs and
+        # falls back to the informational notice rather than failing the PR.
+        continue-on-error: true
+        if: steps.fork.outputs.is_fork != 'true'
+        run: |
+          pip install textstat==0.7.13
+          pip install -r scripts/readability/requirements.txt || true
+
+      # Resolve base and head SHAs. For pull_request, GitHub provides them. For
+      # workflow_dispatch, diff the merge-base of the current ref against the
+      # default branch.
+      - name: Resolve base and head
+        id: refs
+        if: steps.fork.outputs.is_fork != 'true'
+        env:
+          EVENT_NAME: ${{ github.event_name }}
+          PR_BASE: ${{ github.event.pull_request.base.sha }}
+          PR_HEAD: ${{ github.event.pull_request.head.sha }}
+          DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
+        run: |
+          set -euo pipefail
+          if [ "$EVENT_NAME" = "pull_request" ]; then
+            echo "base=$PR_BASE" >> "$GITHUB_OUTPUT"
+            echo "head=$PR_HEAD" >> "$GITHUB_OUTPUT"
+          else
+            git fetch --no-tags --depth=200 origin "$DEFAULT_BRANCH" || true
+            BASE=$(git merge-base "origin/${DEFAULT_BRANCH}" HEAD || git rev-parse HEAD~1)
+            echo "base=$BASE" >> "$GITHUB_OUTPUT"
+            echo "head=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
+          fi
+
+      - name: Build readability report
+        id: report
+        if: steps.fork.outputs.is_fork != 'true'
+        env:
+          # Optional: enables the AI agent comprehension judge via W&B Inference.
+          # The scorer reads WANDB_API_KEY; the repo secret is named
+          # WANDB_DOCS_INFERENCE_API_KEY (a W&B key with Inference credits).
+          WANDB_API_KEY: ${{ secrets.WANDB_DOCS_INFERENCE_API_KEY }}
+          RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
+        run: |
+          set -euo pipefail
+          JUDGE_FLAG=""
+          if [ -n "${WANDB_API_KEY:-}" ]; then
+            JUDGE_FLAG="--judge"
+          fi
+          # Never fail the PR: capture failures and fall back to a notice.
+          if python scripts/readability/pr_report.py \
+              --base "${{ steps.refs.outputs.base }}" \
+              --head "${{ steps.refs.outputs.head }}" \
+              --include-marker \
+              --run-url "${RUN_URL}" \
+              --run-id "${GITHUB_RUN_ID}" \
+              ${JUDGE_FLAG} \
+              > comment-body.md 2> report-errors.log; then
+            echo "ok=true" >> "$GITHUB_OUTPUT"
+          else
+            echo "ok=false" >> "$GITHUB_OUTPUT"
+            echo "::warning::Readability report failed to build; see logs."
+            cat report-errors.log || true
+            {
+              echo "<!-- readability-delta-report -->"
+              echo ""
+              echo "## Readability impact"
+              echo ""
+              echo "The readability check could not run for this push. This is informational and does not block the PR."
+            } > comment-body.md
+          fi
+
+      - name: Upsert readability report on PR
+        # Non-fatal: a transient gh api/comment failure (rate limit, network,
+        # permissions) must not fail the PR. The check stays informational.
+        continue-on-error: true
+        if: steps.fork.outputs.is_fork != 'true' && github.event_name == 'pull_request'
+        env:
+          GH_TOKEN: ${{ github.token }}
+          PR_NUMBER: ${{ github.event.pull_request.number }}
+        run: |
+          MARKER="<!-- readability-delta-report -->"
+          COMMENT_ID=$(gh api \
+            "repos/${{ github.repository }}/issues/${PR_NUMBER}/comments" \
+            --paginate -q ".[] | select(.body | contains(\"${MARKER}\")) | .id" \
+          | head -n1)
+
+          if [ -n "${COMMENT_ID}" ]; then
+            gh api \
+              "repos/${{ github.repository }}/issues/comments/${COMMENT_ID}" \
+              -X PATCH -F "body=@comment-body.md"
+            echo "Updated existing readability report comment ${COMMENT_ID}"
+          else
+            gh pr comment "${PR_NUMBER}" --body-file comment-body.md
+            echo "Created new readability report comment"
+          fi
+
+      - name: Write readability report to job summary
+        # Non-fatal: if comment-body.md is missing due to an earlier unexpected
+        # failure, writing the summary must not fail the otherwise-green job.
+        continue-on-error: true
+        if: steps.fork.outputs.is_fork != 'true' && github.event_name == 'workflow_dispatch'
+        run: cat comment-body.md >> "${GITHUB_STEP_SUMMARY}"

models/artifacts/data-privacy-and-compliance.mdx

@@ -4,7 +4,7 @@ description: Learn where W&B files are stored by default. Explore how to save, s
 title: Artifact data privacy and compliance
 ---
 
-Files are uploaded to a Google Cloud bucket managed by W&B when you log artifacts. The contents of the bucket are encrypted both at rest and in transit. Artifact files are only visible to users who have access to the corresponding project.
+When you log artifacts, W&B uploads the files to a Google Cloud bucket. The contents of the bucket are encrypted at rest and in transit. Only users with access to the project can see the files.
 
 <Frame>
     <img src="/images/artifacts/data_and_privacy_compliance_1.png" alt="GCS W&B Client Server diagram"  />

scripts/readability/README.md

@@ -0,0 +1,53 @@
+# Readability delta check
+
+An informational, non-blocking CI check that reports how a pull request affects
+the readability of the docs it touches (DOCS-2626). It posts a single PR comment
+with a word-weighted Flesch-Kincaid grade delta across the changed pages, plus an
+optional AI-agent-comprehension rating.
+
+We report the *delta*, not an absolute grade. Technical docs have an inherently
+high reading grade level, so "this page went from 12.4 to 11.1" is meaningful
+where "12.4" on its own is not. A lower Flesch-Kincaid grade and a higher Flesch
+reading ease both mean easier to read.
+
+## How it works
+
+The scoring logic lives in the `coreweave/docs-skills` submodule (`.claude/`),
+shared with `coreweave/documentation`:
+
+- `.claude/scripts/_readability.py` extracts narrative prose from MDX (stripping
+  frontmatter, code, JSX/Mintlify components, tables, and links) and scores it
+  with `textstat` (Flesch reading ease, Flesch-Kincaid grade, Gunning fog, SMOG).
+- `.claude/scripts/_docs_eval_lib.py` provides the AI-agent-comprehension LLM
+  judge (W&B Inference, authenticated with `WANDB_API_KEY`).
+- `.claude/scripts/readability_baselines.json` holds per-doc-type baseline grade
+  levels (regenerate with `.claude/scripts/readability_baselines.py`).
+
+This directory holds the wandb/docs plumbing:
+
+- `pr_report.py` diffs the PR base and head, scores each changed English `.mdx`
+  file, aggregates a word-weighted delta, and builds the Markdown report.
+- `tests/` covers the parsing and Markdown-building logic (no network needed).
+
+The workflow is `.github/workflows/readability-delta.yml`.
+
+## Behavior
+
+- **Non-blocking**: the check never fails a PR. It only posts a comment.
+- **Forks**: skipped (no access to the submodule or W&B Inference); the workflow
+  posts an Actions notice and exits successfully.
+- **Deterministic backbone**: the `textstat` delta always runs. The LLM judge is
+  supplementary and degrades gracefully when the W&B Inference key is absent. In
+  CI the key comes from the `WANDB_DOCS_INFERENCE_API_KEY` repo secret, passed to
+  the scorer as `WANDB_API_KEY`.
+- **Localized content** under `ja/`, `ko/`, and `fr/` is skipped.
+
+## Run locally
+
+```bash
+pip install -r scripts/readability/requirements.txt
+git submodule update --init .claude
+python3 scripts/readability/pr_report.py --base main --head HEAD
+```
+
+Add `--judge` to include the comprehension judge (requires `WANDB_API_KEY`).