infra: simplify Motoko sync after upstream PR #6132#268
Open
marc0olo wants to merge 7 commits into
Open
Conversation
Upstream caffeinelabs/motoko PR #6132 transformed doc/md/ into a
Starlight-ready state: clean filenames, sidebar.order frontmatter,
aside syntax normalized, mo:base→mo:core rewrites, numeric prefixes
removed, and index.md stubs added for subdirectory groups. This PR
adopts those changes in the developer-docs sync pipeline.
Changes:
- scripts/sync-motoko.sh: rewrite as plain rsync (~35 lines vs ~180).
All the complex bash transforms (frontmatter injection, .mdx→.md
conversion, numeric prefix stripping, slug collision guards) are
no-ops now that upstream handles them.
- scripts/postprocess-motoko.mjs: remove seven steps that upstream
now handles. Critical fix: the old script explicitly deleted the
new index.md stubs PR #6132 added, breaking autogenerate. Also
adds a general docs.internetcomputer.org/ prefix rule to handle
links that upstream PR #6132 §5 rewrote from the retired portal
format to the current docs.internetcomputer.org/ domain.
- sidebar-motoko.mjs: replace 130-line explicit page list with a
section-level autogenerate config. Uses explicit section labels
("ICP features") to avoid the "Icp Features" label that plain
autogenerate would produce for the icp-features/ directory.
- ec.config.mjs + syntaxes/bnf.tmLanguage.json: register BNF grammar
for Shiki so the 30 BNF blocks in language-manual.md no longer
emit language-not-found warnings.
- .docs-plan/motoko-repo-sync-proposal.md: delete (resolved by PR
#6132; tracked in issue #265).
- .sources/motoko: bump to a9cc50f (master after PR #6132 merge;
no release tag cut yet — this is a pre-release test pin).
- docs/languages/motoko/**: full resync from upstream (77 content
files updated + 6 new index.md stubs for autogenerate groups).
Build: 208 pages, no errors.
|
🤖 Here's your preview: https://lvbky-nyaaa-aaaam-ai7tq-cai.icp0.io |
The new autogenerate approach in sidebar-motoko.mjs broke the Motoko
sidebar in two ways:
1. Group labels showed raw directory names ("actors", "basic-syntax")
instead of "Actors", "Basic syntax". In Starlight v0.38, groupFromDir
uses the raw directory key as the label — it does not read index.md
frontmatter to derive the label.
2. The index.md stubs (added by upstream PR #6132) appeared as duplicate
page entries within the autogenerate groups ("Actors" page inside the
"actors" group), and the root "Motoko" page showed as "Motoko" instead
of "Overview".
Fix:
- sidebar-motoko.mjs: restore explicit items for fundamentals/ (matches
the live site structure) and reference/ (needed for "Motoko changelog"
label). icp-features/ keeps autogenerate — it is a flat list of pages
with correct sidebar.order values and no subdirectories.
- postprocess-motoko.mjs: add sidebar.hidden: true to all subdirectory
index.md stubs. This prevents them from appearing as stray page entries
in autogenerate results. The root index.md (which has content) is
excluded from this rule.
When .sources/motoko/doc/md/sidebar.mjs exists (added in upstream caffeinelabs/motoko docs/sidebar-extraction), postprocess-motoko.mjs now generates sidebar-motoko.mjs automatically by: - importing the upstream sidebar array - prefixing every slug with "languages/motoko/" - prefixing every autogenerate.directory with "languages/motoko/" - wrapping in the motokoSidebar export with label "Motoko" When sidebar.mjs is absent (submodule on an older release), the step is skipped and the committed sidebar-motoko.mjs is used unchanged. Once the upstream PR merges and the submodule is bumped to a release that includes sidebar.mjs, sidebar-motoko.mjs becomes a generated artifact — no hand-editing required.
Member
Author
Waiting on: caffeinelabs/motoko
|
sidebar.mjs will exist in doc/md/ once caffeinelabs/motoko#6139 merges. postprocess-motoko.mjs reads it directly from the submodule path; it should not be copied into docs/languages/motoko/ as a stray non-content file.
…sion sidebar.mjs moved to doc/site/ in caffeinelabs/motoko (site config, not content). Update UPSTREAM_SIDEBAR path accordingly. The rsync exclusion added in the previous commit is no longer needed since doc/site/ is never part of the rsync source.
.sources/motoko bumped to b75b9aa91 (master after caffeinelabs/motoko PR #6139: sidebar extraction to doc/site/sidebar.mjs). sidebar-motoko.mjs is now auto-generated by postprocess-motoko.mjs from the upstream sidebar definition. The hidden:true injection for index.md stubs and the Motoko changelog label override are both no-ops — both handled upstream in #6139.
The upstream changelog.md stub uses a build-time file include (```md file=<motokoRoot>/Changelog.md```) that works locally but produces an empty page in the icp deploy CI recipe (the submodule is not in scope when the asset canister recipe runs npm run build). Fix: expand the include in postprocess-motoko.mjs before walkAndProcess so the file is fully self-contained in docs/ and em-dash replacement and link rewrites apply to the inlined content too.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Upstream
caffeinelabs/motokoPRs #6132 and #6139 transformeddoc/md/into a Starlight-ready state and extracted the sidebar to a shareabledoc/site/sidebar.mjs. This PR adopts those changes in the developer-docs sync pipeline, closing issue #265.Changes
scripts/sync-motoko.sh— rewritten as plain rsync (~35 lines, down from ~180). All the complex bash transforms (frontmatter injection,.mdx→.mdconversion, numeric prefix stripping, slug-collision guards, H1 removal) are no-ops now that upstream handles them.scripts/postprocess-motoko.mjs— seven steps removed (all no-ops after PR #6132). Remaining responsibilities:internetcomputer.org/docs/anddocs.internetcomputer.org/links to internal paths.core//base/relative links to mops.one (safety net).sidebar.hidden: truefor subdirectoryindex.mdstubs (transitional fallback — now a no-op since upstream PR #6139 addedhidden: truedirectly in the stubs).sidebar-motoko.mjsfromdoc/site/sidebar.mjsin the submodule. Slugs andautogenerate.directoryvalues get thelanguages/motoko/prefix. Ifsidebar.mjsis absent (older submodule pin), generation is skipped and the committed file is used as-is.sidebar-motoko.mjs— now auto-generated from the upstreamdoc/site/sidebar.mjs(added in caffeinelabs/motoko PR #6139). The upstream file is the single source of truth for Motoko navigation. To change the sidebar, editdoc/site/sidebar.mjsincaffeinelabs/motokoand runnpm run sync:motoko.ec.config.mjs+syntaxes/bnf.tmLanguage.json— register BNF TextMate grammar for Shiki so the 30 BNF blocks inlanguage-manual.mdno longer emit language-not-found warnings at build time..docs-plan/motoko-repo-sync-proposal.md— deleted. The proposal it tracked is now complete..sources/motoko— bumped tob75b9aa91(master after PR #6139 merge). This is a pre-release pin; once the Motoko team cuts a release tag including #6139, the automatedsync-motoko.ymlworkflow will take over for future updates.docs/languages/motoko/**— full resync from updated submodule.Build
208 pages, no errors, no Shiki language-not-found warnings.
Sync recommendation
sync from caffeinelabs/motoko doc/md/— after this PR, Motoko docs are maintained purely by runningnpm run sync:motokoagainst the pinned submodule. Navigation structure is driven bydoc/site/sidebar.mjsupstream.Testing
Closes #265