Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 21 additions & 4 deletions docs/features/publisher.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ src/core/publisher/
├── escapeProps.ts — escape string props at the render boundary, dispatched per-prop on the schema control `type`
├── classInjection.ts — inject author classIds into rendered HTML
├── classCss.ts — compile user StyleRule entries → CSS, including supported raw @keyframes
├── responsiveBackground.ts — rewrite media-library background-image url(...) values to optimized image-set(...) ladders
├── cssCollector.ts — CssCollector + collectClassCSS + sanitizeModuleCSS
├── reset.ts — PUBLISHER_RESET_CSS (cross-browser baseline)
├── frameworkCss.ts — site framework CSS (spacing scale, typography)
Expand All @@ -52,7 +53,7 @@ server/publish/
├── publishedHtmlPipeline.ts — post-process (sanitize + plugin filters + injections)
├── siteCssBundle.ts — server-side hashing + file emission
├── frontendInjections.ts — splice plugin <script>/<link>/<meta> into HTML
├── mediaPresentation.ts — <picture>/<srcset> materialization at publish time
├── mediaPresentation.ts — media URL materialization for originals + responsive variants
├── renderTreeWalk.ts — walkRenderTree: visits every node that contributes to a rendered page (page nodes + VC definition trees, cycle-guarded); single source of truth for loop-prefetch and media-prefetch
├── mediaPrefetch.ts, loopPrefetch.ts — pre-warm caches needed by the renderer
├── republish.ts — bulk re-publish on site-level changes
Expand Down Expand Up @@ -93,7 +94,7 @@ For each node, bottom-up:
2. resolvedProps = resolveProps(node, breakpoint) ← merge breakpoint overrides
3. dynamicProps = resolveDynamicProps(...) ← apply data bindings
4. safeProps = escapeProps(dynamicProps, schema) ← escape per schema TYPE
5. attachResolvedMediaByKey(safeProps, def, ...) ← attach <picture>/<srcset>
5. attachResolvedMediaByKey(safeProps, def, ...) ← attach srcset-capable media payloads
6. attachAutoSizes(safeProps, def, ...) ← auto <img sizes>
7. { html, css } = def.render(safeProps, children) ← MODULE BOUNDARY
8. acc.cssMap.set(moduleId, sanitizeModuleCSS(css)) ← neutralise </style (Constraint #228), dedup by moduleId
Expand Down Expand Up @@ -205,6 +206,21 @@ style-<hash>.css = collectClassCSS(site) ← user-defi
userStyles-<hash>.css = collectUserStylesheetCss(site, page) ← author stylesheets, scoped to this page
```

Media-library background images are optimized in the same publish pass as
`<img srcset>`. `mediaPrefetch.ts` collects `/uploads/...` URLs from
image/media module props, node `inlineStyles.backgroundImage`, and StyleRule
`backgroundImage` values (including breakpoint/context overrides), then
batch-fetches their media rows. During CSS emission,
`responsiveBackground.ts` rewrites each matched `url('/uploads/original.png')`
to two `background-image` declarations: an optimized variant URL fallback and
an `image-set(...)` ladder built only from `media_assets.variants_json`. The
uploaded original is excluded whenever variants exist, so a 20 MB PNG used as
a background does not become the selectable source for modern browsers. CSS
`image-set()` supports density descriptors (`1x`, `2x`, etc.), not HTML
`srcset` width descriptors (`640w` + `sizes`), so the background ladder is a
DPR-oriented mirror of the same variant policy rather than a literal copy of
the `<img sizes>` algorithm.

`reset` / `framework` / `style` are page-invariant — every page on the site
shares the same hash. `userStyles` is **page-scoped**: each author stylesheet
(`site.files[type === 'style']`) carries a `SiteStyleRuntimeConfig` (in
Expand Down Expand Up @@ -349,8 +365,9 @@ Because `serializeCsp` sorts, the same plugins + adapters always emit a **byte-i
| `server/publish/republish.ts` | Bulk re-publish on settings change (touches every page). |
| `server/publish/publishScheduler.ts` | Scheduled publish jobs (cron-style). |
| `server/publish/frontendInjections.ts` | Compute plugin `<script>`/`<link>`/`<meta>` tags + CSP entries. |
| `server/publish/mediaPresentation.ts` | At publish time, build `<picture>` / `<img srcset>` markup from `media_assets.variants_json`. |
| `server/publish/mediaPrefetch.ts` | Collect every image/media-typed prop from the full render tree — including VC definition trees — via `walkRenderTree`, then batch-fetch matching `media_assets` rows into a `Map<publicPath, MediaAsset>` before render. Uses `MEDIA_ASSET_COLUMNS` and `mapMediaAssetRow` from `server/repositories/mediaAssetMapping.ts` (shared with the admin repository) so the published page and the admin panel always see one identical asset shape. |
| `server/publish/mediaPresentation.ts` | Materialize media paths (originals + responsive variants) for publisher consumers. |
| `src/core/publisher/responsiveBackground.ts` | Convert media-library `background-image: url(...)` values into optimized variant fallback + `image-set(...)` declarations. |
| `server/publish/mediaPrefetch.ts` | Collect every image/media-typed prop and every media-library background-image URL from the full render tree — including VC definition trees — via `walkRenderTree`, then batch-fetch matching `media_assets` rows into a `Map<publicPath, MediaAsset>` before render. Uses `MEDIA_ASSET_COLUMNS` and `mapMediaAssetRow` from `server/repositories/mediaAssetMapping.ts` (shared with the admin repository) so the published page and the admin panel always see one identical asset shape. |
| `server/publish/loopPrefetch.ts` | Collect every `base.loop` node from the full render tree — including VC definition trees — via `walkRenderTree`, fetch each source's items, and return a `Map<nodeId, ResolvedLoopData>` before render so the walker is purely synchronous. Also exports `canonicalRenderQuery(searchParams)` — strips all non-loop-pagination params from a URL's query, returning only `loop_<nodeId>_page` keys in sorted order (or `''` when none remain). Used by `publicRouter.ts` to normalise the Layer B cache key and Layer A fast-path eligibility. |
| `server/publish/renderTreeWalk.ts` | `walkRenderTree(nodes, rootNodeId, site, onNode)` — visits every node that contributes to a rendered page: all page-tree nodes reachable from `rootNodeId`, plus all nodes inside each referenced VC's definition tree (recursively, cycle-guarded by a `Set<vcId>`). Used by both `mediaPrefetch.ts` and `loopPrefetch.ts` so their traversal logic can't drift apart. |
| `server/publish/runtime/packageServer.ts` | Serve per-site `bun install` workspace under `/_instatic/runtime/cache/`. |
Expand Down
2 changes: 1 addition & 1 deletion server/handlers/cms/data/preview.ts
Original file line number Diff line number Diff line change
Expand Up @@ -100,11 +100,11 @@ export async function handleRowPreview(
// resolution, etc.) operate against this seed.
const draftPublishedRow: PublishedDataRow = synthesisePublishedRow(row, table, draftCells)

const cssBundle = buildSiteCssBundle(snapshot.site, registry, merged)
const [loopData, mediaAssets] = await Promise.all([
prefetchLoopData(merged, snapshot.site, db),
prefetchMediaAssets(merged, snapshot.site, registry, db),
])
const cssBundle = buildSiteCssBundle(snapshot.site, registry, merged, { mediaAssets })

const publicPath = buildEntryPublicPath(table.routeBase, draftPublishedRow.slug)
const syntheticUrl = new URL(`http://localhost${publicPath}`)
Expand Down
5 changes: 5 additions & 0 deletions server/publish/mediaPrefetch.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@

import type { Page, SiteDocument } from '@core/page-tree'
import type { IModuleRegistry } from '@core/module-engine'
import { collectNodeBackgroundImagePaths, collectSiteStyleBackgroundImagePaths } from '@core/publisher'
import { walkRenderTree } from './renderTreeWalk'
import type { DbClient } from '../db/client'
import type { MediaAsset } from '../repositories/media'
Expand All @@ -43,6 +44,7 @@ function collectMediaPaths(page: Page, site: SiteDocument, registry: IModuleRegi
walkRenderTree(page.nodes, page.rootNodeId, site, (node) => {
const def = registry.get(node.moduleId)
if (!def) return
collectNodeBackgroundImagePaths(node, paths)
for (const [propKey, control] of Object.entries(def.schema)) {
// Only `image` / `media` controls participate in the responsive
// pipeline. Plain `text` URL fields (rare) aren't auto-upgraded.
Expand All @@ -52,6 +54,9 @@ function collectMediaPaths(page: Page, site: SiteDocument, registry: IModuleRegi
paths.add(value)
}
})
for (const path of collectSiteStyleBackgroundImagePaths(site)) {
paths.add(path)
}
return paths
}

Expand Down
2 changes: 1 addition & 1 deletion server/publish/publicRenderer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -92,12 +92,12 @@ async function renderMergedTemplate(
ctx: RenderPublishedSnapshotContext,
): Promise<{ html: string; jsModuleIds: string[]; publishVersion: number; cssBundle: SiteCssBundle }> {
const publishVersion = ctx.publishVersion ?? getPublishVersion()
const cssBundle = buildPublishedSiteCssBundle(snapshot.site, registry, merged, publishVersion)
const moduleJsMap = buildPublishedSiteModuleJsMap(snapshot.site, registry)
const [loopData, mediaAssets] = await Promise.all([
prefetchLoopData(merged, snapshot.site, ctx.db, ctx.url),
prefetchMediaAssets(merged, snapshot.site, registry, ctx.db),
])
const cssBundle = buildPublishedSiteCssBundle(snapshot.site, registry, merged, publishVersion, { mediaAssets })
const published = publishPage(merged, snapshot.site, registry, {
templateContext,
runtimeAssets: snapshot.runtimeAssets,
Expand Down
4 changes: 3 additions & 1 deletion server/publish/publishSite.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ import {
serializeImportmapForCsp,
} from './runtime/packageImportmap'
import { renderPublishedNotFound, renderPublishedSnapshot } from './publicRenderer'
import { prefetchMediaAssets } from './mediaPrefetch'
import { applyPublishedHtmlPipeline } from './publishedHtmlPipeline'
import {
NOT_FOUND_ARTEFACT_URL_PATH,
Expand Down Expand Up @@ -222,7 +223,8 @@ async function publishDraftSiteLocked(
for (const snapshot of snapshots) {
const page = snapshot.site.pages.find((p) => p.id === snapshot.pageRowId)
if (!page || isTemplatePage(page)) continue // template pages only ever wrap; never baked at their own slug
collectCssFiles(buildPublishedSiteCssBundle(snapshot.site, registry, page, nextPublishVersion))
const mediaAssets = await prefetchMediaAssets(page, snapshot.site, registry, db)
collectCssFiles(buildPublishedSiteCssBundle(snapshot.site, registry, page, nextPublishVersion, { mediaAssets }))
}
for (const asset of runtimeAssetFiles) {
if (!assetsByPath.has(asset.publicPath)) assetsByPath.set(asset.publicPath, asset.bytes)
Expand Down
35 changes: 28 additions & 7 deletions server/publish/siteCssBundle.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,13 @@ import type { IModuleRegistry } from '@core/module-engine'
import {
PUBLISHER_RESET_CSS,
collectClassCSS,
collectSiteStyleBackgroundImagePaths,
buildSiteFrameworkCss,
collectUserStylesheetCss,
} from '@core/publisher'
import type {
CssBundleFile,
ResponsiveCssOptions,
SiteCssBundle,
SiteCssBundleId,
} from '@core/publisher'
Expand Down Expand Up @@ -70,9 +72,10 @@ export function buildSiteCssBundle(
site: SiteDocument,
registry: IModuleRegistry,
page?: Page,
options: ResponsiveCssOptions = {},
): SiteCssBundle {
return {
...computePageInvariantBundles(site, registry),
...computePageInvariantBundles(site, registry, options),
userStyles: makeBundleFile('userStyles', collectUserStylesheetCss(site, page)),
}
}
Expand Down Expand Up @@ -100,9 +103,10 @@ export function buildPublishedSiteCssBundle(
registry: IModuleRegistry,
page?: Page,
publishVersion: number = getPublishVersion(),
options: ResponsiveCssOptions = {},
): SiteCssBundle {
return {
...memoizedPageInvariantBundles(site, registry, publishVersion),
...memoizedPageInvariantBundles(site, registry, publishVersion, options),
userStyles: makeBundleFile('userStyles', collectUserStylesheetCss(site, page)),
}
}
Expand All @@ -111,11 +115,12 @@ export function buildPublishedSiteCssBundle(
function computePageInvariantBundles(
site: SiteDocument,
registry: IModuleRegistry,
options: ResponsiveCssOptions,
): PageInvariantBundles {
return {
reset: makeBundleFile('reset', PUBLISHER_RESET_CSS),
framework: makeBundleFile('framework', buildFrameworkCss(site, registry)),
style: makeBundleFile('style', collectClassCSS(site)),
style: makeBundleFile('style', collectClassCSS(site, options)),
}
}

Expand All @@ -126,7 +131,7 @@ function computePageInvariantBundles(
// Deliberately NOT keyed on the site object: every consumer loads the snapshot
// fresh (DB JSON parse per query), so an identity key would never hit — that
// was exactly the bug that made every Layer B miss re-walk the whole site.
let pageInvariantCache: { version: number; bundles: PageInvariantBundles } | null = null
let pageInvariantCache: { version: number; mediaSignature: string; bundles: PageInvariantBundles } | null = null
registerVersionedCacheReset(() => {
pageInvariantCache = null
})
Expand All @@ -139,15 +144,31 @@ function memoizedPageInvariantBundles(
site: SiteDocument,
registry: IModuleRegistry,
version: number,
options: ResponsiveCssOptions,
): PageInvariantBundles {
if (pageInvariantCache && pageInvariantCache.version === version) {
const mediaSignature = styleMediaSignature(site, options)
if (pageInvariantCache && pageInvariantCache.version === version && pageInvariantCache.mediaSignature === mediaSignature) {
return pageInvariantCache.bundles
}
const bundles = computePageInvariantBundles(site, registry)
pageInvariantCache = { version, bundles }
const bundles = computePageInvariantBundles(site, registry, options)
pageInvariantCache = { version, mediaSignature, bundles }
return bundles
}

function styleMediaSignature(site: SiteDocument, options: ResponsiveCssOptions): string {
if (!options.mediaAssets || options.mediaAssets.size === 0) return ''
const paths = collectSiteStyleBackgroundImagePaths(site)
if (paths.size === 0) return ''

const parts: string[] = []
for (const path of [...paths].sort()) {
const media = options.mediaAssets.get(path)
if (!media || media.variants.length === 0) continue
parts.push(`${path}=${media.variants.map((v) => `${v.width}:${v.path}`).join('|')}`)
}
return parts.join(';')
}

/**
* Build the `framework.css` body: site-wide platform CSS plus any plugin
* module CSS used anywhere on the site.
Expand Down
15 changes: 13 additions & 2 deletions server/router.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { renderNotFoundResponse, renderPublicResolution } from './publish/public
import { readStaticAsset } from './publish/staticArtefact'
import { getLatestSnapshotForVersion } from './publish/publishedSnapshotCache'
import { getPublishVersion, registerVersionedCacheReset } from './publish/publishState'
import { prefetchMediaAssets } from './publish/mediaPrefetch'
import { getSetupStatusCached } from './repositories/setup'
import { getPublishedRuntimeAsset } from './repositories/runtimeAsset'
import { handleLoopRequest, isLoopRuntimeAssetPath, serveLoopRuntimeAsset } from './handlers/cms/loop'
Expand Down Expand Up @@ -605,12 +606,22 @@ async function rebuildSiteCssFromSnapshot(

const pages = bundleId === 'userStyles' ? snapshot.site.pages : snapshot.site.pages.slice(0, 1)
for (const page of pages) {
const file: CssBundleFile = buildPublishedSiteCssBundle(snapshot.site, registry, page, version)[bundleId]
const mediaAssets = await prefetchMediaAssets(page, snapshot.site, registry, db)
const file: CssBundleFile = buildPublishedSiteCssBundle(snapshot.site, registry, page, version, { mediaAssets })[bundleId]
if (file.hash === requestedHash) return file.content
}
// Page-agnostic view (every enabled stylesheet) — covers a hash that
// predates a scope change but is still referenced somewhere.
const fallback: CssBundleFile = buildPublishedSiteCssBundle(snapshot.site, registry, undefined, version)[bundleId]
const fallbackMediaAssets = snapshot.site.pages[0]
? await prefetchMediaAssets(snapshot.site.pages[0], snapshot.site, registry, db)
: undefined
const fallback: CssBundleFile = buildPublishedSiteCssBundle(
snapshot.site,
registry,
undefined,
version,
{ mediaAssets: fallbackMediaAssets },
)[bundleId]
if (fallback.hash === requestedHash) return fallback.content

return null
Expand Down
63 changes: 63 additions & 0 deletions src/__tests__/canvas/classStyleInjector.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import {
} from '@site/canvas/canvasClassCss'
import { generateFrameworkColorUtilityClasses } from '@core/framework'
import { classKindSelector, type StyleRule } from '@core/page-tree'
import type { RenderResolvedMedia } from '@core/publisher'

function makeClass(
id: string,
Expand All @@ -25,6 +26,23 @@ function makeClass(
}
}

function resolvedMedia(path = '/uploads/hero.png'): RenderResolvedMedia {
return {
publicPath: path,
mimeType: 'image/png',
width: 2400,
height: 1200,
altText: '',
blurHash: null,
variants: [
{ width: 320, height: 160, format: 'webp', path: '/uploads/hero-w320.webp', sizeBytes: 12_000 },
{ width: 1024, height: 512, format: 'webp', path: '/uploads/hero-w1024.webp', sizeBytes: 82_000 },
{ width: 2048, height: 1024, format: 'webp', path: '/uploads/hero-w2048.webp', sizeBytes: 190_000 },
],
posterPath: null,
}
}

describe('generateCanvasClassCSS', () => {
it('prepends the unscoped publisher reset so the iframe cascade matches the published page', () => {
const css = generateCanvasClassCSS({}, [])
Expand Down Expand Up @@ -70,6 +88,30 @@ describe('generateCanvasClassCSS', () => {
expect(css).not.toContain('[data-breakpoint-id="mobile"] .title')
})

it('rewrites class background images to optimized image-set candidates in the canvas CSS', () => {
const css = generateCanvasClassCSS(
{
hero: makeClass('hero', { backgroundImage: "url('/uploads/hero.png')" }),
},
[],
[],
null,
null,
null,
null,
null,
{
mediaAssets: new Map([['/uploads/hero.png', resolvedMedia()]]),
mediaSignature: 'hero',
},
)

expect(css).toContain('background-image: url("/uploads/hero-w2048.webp");')
expect(css).toContain('background-image: image-set(')
expect(css).toContain('url("/uploads/hero-w1024.webp") 1x')
expect(css).not.toContain('/uploads/hero.png')
})

it('emits sanitized raw @keyframes rules, matching the published output', () => {
// Regression: the canvas used to skip `rawCss` rules entirely, so
// imported keyframe animations published fine but never played in the
Expand Down Expand Up @@ -190,6 +232,27 @@ describe('createCanvasClassCssMemo', () => {
expect(calls()).toBe(3)
})

it('regenerates when the responsive background media signature changes', () => {
const { memo, calls } = countingMemo()
const mediaAssets = new Map([['/uploads/hero.png', resolvedMedia()]])

memo(classes, breakpoints, conditions, null, null, null, null, null, {
mediaAssets,
mediaSignature: 'a',
})
memo(classes, breakpoints, conditions, null, null, null, null, null, {
mediaAssets,
mediaSignature: 'a',
})
expect(calls()).toBe(1)

memo(classes, breakpoints, conditions, null, null, null, null, null, {
mediaAssets,
mediaSignature: 'b',
})
expect(calls()).toBe(2)
})

it('the exported generator produces identical CSS for cached and fresh inputs', () => {
const cached = generateCanvasClassCSS(classes, breakpoints)
expect(generateCanvasClassCSS(classes, breakpoints)).toBe(cached)
Expand Down
Loading