Skip to content

Generate LLMs text during site build - #29

Merged
koriym merged 2 commits into
masterfrom
codex/ci-generate-llms-full
Jun 12, 2026
Merged

koriym merged 2 commits into
masterfrom
codex/ci-generate-llms-full

Conversation

@koriym

@koriym koriym commented May 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • add a Pages workflow that generates llms-full.txt before the Jekyll build
  • add an llms-full.txt generator based on English manual markdown
  • ignore and stop tracking generated llms-full.txt

Verification

  • zsh -ic "sphp85; php -l bin/generate_llms_full.php"
  • zsh -ic "sphp85; php bin/generate_llms_full.php"
  • git diff --cached --check

Note: local Jekyll build was not run successfully because required bundle gems are not installed in this local environment.

Summary by CodeRabbit

  • Chores

    • Added an automated CI/CD workflow to build and deploy the site to GitHub Pages.
    • Enhanced the local development serve process to run preprocessing steps before serving.
    • Added a script to generate a consolidated documentation file and updated ignore rules for generated output.
  • Documentation

    • Removed the previous full documentation content from the consolidated docs file.

Review Change Stack

@coderabbitai

coderabbitai Bot commented May 21, 2026 •

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

This pull request adds a PHP script that aggregates llms.txt with manual markdown files into llms-full.txt, runs that generator in the local serve script and a new GitHub Actions workflow that builds and deploys the Jekyll site, and updates .gitignore to exclude the generated llms-full.txt.

Changes

Content generation and build automation

Layer / File(s) Summary
Content generation script
bin/generate_llms_full.php
PHP script validates input files exist, collects and sorts markdown files from the manual directory (excluding a skip-list), strips frontmatter from each file, aggregates the content with <!-- Source: ... --> markers, and writes llms-full.txt with a write-completeness check.
Build and serve integration
bin/serve_local.sh, .github/workflows/jekyll.yml
bin/serve_local.sh runs Ruby and PHP preprocessing before jekyll serve; the GitHub Actions workflow runs the same generation steps, builds with bundle exec jekyll build using the Pages base path, uploads the Pages artifact, and deploys via actions/deploy-pages to the github-pages environment with pages: write and id-token: write permissions.
Generated file configuration
.gitignore
Adds llms-full.txt to .gitignore and the repository no longer contains the checked-in llms-full.txt contents (now generated at build time).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Poem

I'm a rabbit with a tiny quill,
I hop through docs to fit the bill,
I stitch each markdown, strip the front,
Bundle knowledge — then I sprint and stunt. 🐇📚

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: implementing automated generation of llms-full.txt during the site build process, which is the primary objective across all modified files.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/ci-generate-llms-full

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@claude

claude Bot commented May 21, 2026

Copy link
Copy Markdown

Code Review — PR #29: Generate LLMs text during site build

Overview

This PR moves llms-full.txt from a tracked, manually-maintained file to a CI-generated artifact. The approach is sound: generate content from the source-of-truth Markdown files rather than duplicating it in version control. The PHP script is clean and the workflow integration is correct.


Positives

  • Deleting 569 lines of duplicated content from git is a net win — the manual files are the source of truth.
  • declare(strict_types=1) and explicit early-exit error checks for missing paths are good PHP hygiene.
  • The frontmatter regex anchors to \A so it only strips from the very start of the file (once), avoiding false positives on --- horizontal rules in body content.
  • sort($files, SORT_STRING) correctly orders NN-slug.md files since the full path prefix is the same for all files.
  • HTML source comments (<!-- Source: manuals/1.0/en/... -->) are helpful for LLM debugging without affecting the readable output.
  • bundler-cache: true keeps CI fast.
  • cancel-in-progress: false on the deploy concurrency group is the right choice — never interrupt a live deploy.

Issues

1. Unchecked file_put_contents return value (bin/generate_llms_full.php, last write)

file_put_contents($outputFile, $content);

If the write fails (disk full, permission error, etc.) the script exits 0 and reports success. The Jekyll build then proceeds without the generated file.

if (file_put_contents($outputFile, $content) === false) {
    fwrite(STDERR, "Failed to write {$outputFile}\n");
    exit(1);
}

2. "Included files" count is pre-filter (bin/generate_llms_full.php)

count($files) is reported after the sort() but before the loop that skips empty files. The count may overstate the number of files actually written into the output.

$included = 0;
foreach ($files as $file) {
    // ...
    if ($markdown === '') { continue; }
    $content .= ...;
    $included++;
}
echo 'Included files: ' . $included . "\n";

3. serve_local.sh has no fail-fast guard

If merge_md_files.rb or generate_llms_full.php exit non-zero, the script continues to start the Jekyll server, which will serve stale or missing generated content without any warning.

set -e   # add at top, after the shebang

Minor / Worth Noting

Supply-chain pinning in the workflow — ruby/setup-ruby@v1, shivammathur/setup-php@v2, actions/upload-pages-artifact@v3, and actions/deploy-pages@v4 are pinned to major version tags. Pinning to commit SHAs (uses: ruby/setup-ruby@<sha>) is more resistant to tag mutation attacks. This is a low-risk project, so this is optional, but worth knowing.

Local build not verified — noted in the PR description. The critical path (CI workflow generating files before jekyll build) is straightforward enough that this is low risk, but running bundle exec jekyll build locally after the scripts is the definitive check.


Summary

The overall direction is correct and the implementation is clean. The two functional issues (#1 and #2) are worth fixing before merge: a silent write failure could silently ship an incomplete llms-full.txt to production. The set -e fix (#3) is also straightforward and prevents confusing local dev sessions.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
bin/serve_local.sh (1)

1-8: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Fail fast if preprocessing commands fail.

Without strict shell mode, failed generators can still fall through to jekyll serve.

Proposed fix
 #!/bin/bash
+set -euo pipefail
+
 # This script is used to serve the Jekyll site locally with automatic rebuilding.
 # 'bundle exec' ensures we're using the correct versions of each gem according to our Gemfile.lock.
 # 'jekyll serve' starts a Jekyll development server.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@bin/serve_local.sh` around lines 1 - 8, Enable strict-failure behavior in the
serve_local.sh script so preprocessing failures stop execution: ensure the
script sets strict shell options (e.g., exit on error, undefined var, and
failing pipes) before running the preprocessing commands, and/or explicitly
check the exit status of the calls to "ruby bin/merge_md_files.rb" and "php
bin/generate_llms_full.php" and exit with a non-zero status if either fails so
that "bundle exec jekyll serve --watch --trace" does not run when preprocessing
fails.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/jekyll.yml:
- Around line 8-12: The workflow currently grants pages: write and id-token:
write at the top-level permissions which apply to all jobs; move those two
permissions into only the deploy job's permissions block so only the deploy job
gets pages: write and id-token: write while keeping contents: read at top-level.
Locate the top-level permissions block and the deploy job definition (the deploy
job name) and remove pages and id-token from the global permissions, then add
them under the deploy job's permissions section (ensuring pages: write and
id-token: write appear only there).
- Around line 21-23: The checkout step using actions/checkout@v4 currently
leaves default token credentials available; update the "Checkout" step
(actions/checkout@v4) to add persist-credentials: false so the GITHUB_TOKEN is
not persisted to subsequent steps—modify the checkout step configuration to
include the persist-credentials: false property under the uses entry.

In `@bin/generate_llms_full.php`:
- Around line 49-53: The script silently ignores failures from file_get_contents
and file_put_contents which can lead to partial output and false success
messages; update the logic around the $content assembly and the final write so
that each file_get_contents($llmsFile) and file_get_contents($file) call in the
foreach is checked for a false return and handled (log/processLogger->error or
fwrite to STDERR and exit with non-zero) with the filename included, and
likewise check the return value of file_put_contents when writing the final
output (lines referenced around where $content is written) and abort/report on
failure instead of proceeding to print “Generated … successfully.” Ensure the
checks reference the existing variables ($llmsFile, $file, $content) and exit
with a non-zero status on error.

---

Outside diff comments:
In `@bin/serve_local.sh`:
- Around line 1-8: Enable strict-failure behavior in the serve_local.sh script
so preprocessing failures stop execution: ensure the script sets strict shell
options (e.g., exit on error, undefined var, and failing pipes) before running
the preprocessing commands, and/or explicitly check the exit status of the calls
to "ruby bin/merge_md_files.rb" and "php bin/generate_llms_full.php" and exit
with a non-zero status if either fails so that "bundle exec jekyll serve --watch
--trace" does not run when preprocessing fails.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e5efde77-4dc7-4197-8bbc-8810d07a0a5f

📥 Commits

Reviewing files that changed from the base of the PR and between 150db3d and 050daa4.

📒 Files selected for processing (5)
  • .github/workflows/jekyll.yml
  • .gitignore
  • bin/generate_llms_full.php
  • bin/serve_local.sh
  • llms-full.txt
💤 Files with no reviewable changes (1)
  • llms-full.txt

Comment thread .github/workflows/jekyll.yml
Comment thread .github/workflows/jekyll.yml
Comment thread bin/generate_llms_full.php Outdated
@koriym

koriym commented May 21, 2026

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented May 21, 2026

Copy link
Copy Markdown
Contributor
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
.github/workflows/jekyll.yml (1)

21-60: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Pin GitHub Actions to full-length commit SHAs.

This workflow references third-party actions via mutable tags (@v1/@v2/@v3/@v4/@v5) instead of 40-char commit SHAs (e.g., actions/checkout@v4, ruby/setup-ruby@v1, shivammathur/setup-php@v2, actions/configure-pages@v5, actions/upload-pages-artifact@v3, actions/deploy-pages@v4). Replace each uses: value with the corresponding commit SHA to reduce supply-chain risk.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/jekyll.yml around lines 21 - 60, Replace mutable action
tags with fixed 40-character commit SHAs for every third-party action used in
the workflow: actions/checkout@v4, ruby/setup-ruby@v1,
shivammathur/setup-php@v2, actions/configure-pages@v5,
actions/upload-pages-artifact@v3, and actions/deploy-pages@v4. Locate the steps
named "Checkout", "Setup Ruby", "Setup PHP", "Setup Pages", "Upload artifact",
and "Deploy to GitHub Pages" and change each uses: entry to the corresponding
action@<full-commit-sha> (obtain the exact SHA from the action's official GitHub
repo tags/releases) so the workflow references immutable commit SHAs instead of
version tags.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In @.github/workflows/jekyll.yml:
- Around line 21-60: Replace mutable action tags with fixed 40-character commit
SHAs for every third-party action used in the workflow: actions/checkout@v4,
ruby/setup-ruby@v1, shivammathur/setup-php@v2, actions/configure-pages@v5,
actions/upload-pages-artifact@v3, and actions/deploy-pages@v4. Locate the steps
named "Checkout", "Setup Ruby", "Setup PHP", "Setup Pages", "Upload artifact",
and "Deploy to GitHub Pages" and change each uses: entry to the corresponding
action@<full-commit-sha> (obtain the exact SHA from the action's official GitHub
repo tags/releases) so the workflow references immutable commit SHAs instead of
version tags.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ed6d3772-f60a-47fb-9945-42f2dad1f6c1

📥 Commits

Reviewing files that changed from the base of the PR and between 050daa4 and 3fd5bb0.

📒 Files selected for processing (3)
  • .github/workflows/jekyll.yml
  • bin/generate_llms_full.php
  • bin/serve_local.sh

@koriym
koriym merged commit a2bd228 into master Jun 12, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant