Add safe deterministic world processing

.changeset/safe-world-processing.md

@@ -0,0 +1,5 @@
+---
+"@blurengine/cli": minor
+---
+
+Add deterministic world processors with immutable Bedrock observations, canonical JSON or TypeScript runtime pointers, verified staged-world mutations, content-addressed artifacts, processed development and packaging, and authored-versus-processed publication safety. Update archive, compiler, and LevelDB dependencies, and preserve caller-owned chunk coordinates across the LevelDB compatibility boundary.

.gitignore

@@ -6,3 +6,4 @@ dist/
 npm-debug.log*
 .env.local
 .blr/
+docs/superpowers/

packages/blr/README.md

@@ -28,6 +28,7 @@ npm run dev
 - [Command Reference](./docs/reference/commands.md)
 - [Config Reference](./docs/reference/config.md)
 - [Generated Project Reference](./docs/reference/generated-project.md)
+- [World Processing](./docs/reference/world-processing.md)
 - [Changelog](./CHANGELOG.md)
 
 ## Development

packages/blr/docs/README.md

@@ -7,6 +7,7 @@ This is the canonical reference set for the `blr` tool and the projects it gener
 - [Command Reference](./reference/commands.md)
 - [Config Reference](./reference/config.md)
 - [Generated Project Reference](./reference/generated-project.md)
+- [World Processing](./reference/world-processing.md)
 
 ## Standards
 
@@ -26,6 +27,7 @@ These docs cover:
 - machine-local overrides via environment variables and CLI flags
 - generated project files and optional state files
 - staged build output and packaged artifacts under `dist/`
+- immutable world observations, derived artifacts, and verified processed worlds
 - shared Bedrock engineering standards used by generated projects
 
 ## Maintenance

packages/blr/docs/reference/commands.md

@@ -6,6 +6,7 @@
 - Generated projects run the installed binary through package-manager scripts, for example `npm run dev`, `pnpm run dev`, `yarn run dev`, or `bun run dev`
 - Generated projects also expose a `system` package script for support diagnostics
 - Generated projects also expose a `world` package script for world backend operations
+- Generated projects expose `assets:check` for a read-only world-processor freshness check
 
 Windows PowerShell note:
 
@@ -231,6 +232,7 @@ Build output:
 - generated project code should not import `@blurengine/bebe/tooling/*`; those Node-only build surfaces are resolved by `blr` when it bakes Bebe assets
 - dynamic Link usage such as assigning or destructuring `Link.event` or `Link.snapshot` fails the offline build with a clear error because `blr` cannot safely erase it
 - `local-deploy` consumes the offline staged behavior pack; `local-server` consumes the BDS staged behavior pack
+- configured artifact processors run before content staging; transform processors are validated but only materialise a world when a consuming workflow needs one
 
 Bebe integration:
 
@@ -541,6 +543,7 @@ Purpose:
 Remote object layout for the S3 backend:
 
 - `<keyPrefix>/<worldName>.zip`
+- `<keyPrefix>/processed/<worldName>.zip`
 - `<keyPrefix>/<worldName>.lock.json`
 
 Notes:
@@ -567,6 +570,36 @@ Notes:
 - internal runtime and materialization bookkeeping lives under `.blr/state/world-state.json`
 - generated projects ignore raw world contents by default but still allow `worlds/worlds.json` to be committed
 - if the remote fingerprint drifts, `blr` ignores the stale pin until the next successful remote world action refreshes it
+- authored and processed world publication use separate lineage state; processed output never replaces the authored project world
+
+### `blr world build`
+
+Runs the configured processor graph against an immutable snapshot and produces or verifies a content-addressed processed world.
+
+Syntax:
+
+```text
+blr world build [worldName]
+```
+
+Behavior:
+
+- never opens or modifies the authored world database
+- publishes configured immutable artifact sets and runtime pointers
+- applies declarative transform mutations only to a disposable staged copy
+- reopens and verifies the processed world before publishing its lineage
+- retains the previous coherent build after any failure or superseded run
+
+Flags:
+
+- `--check`: verify committed artifacts and the processed-world lineage without project or cache writes
+- `--dry-run`: alias for `--check`
+- `--processor <id...>`: narrow the run to named processors plus dependencies
+- `--audit`: write configured audit reports after a successful build
+- `--output <path>`: copy the verified world to a new explicit directory
+- `--json`: emit the result as JSON
+
+See [World Processing](./world-processing.md) for the processor and mutation contracts.
 
 ### `blr world list`
 
@@ -974,5 +1007,8 @@ Generated projects expose these scripts:
 - `<packageManager> run minecraft -- <subcommand>` -> `blr minecraft <subcommand>`
 - `<packageManager> run system -- <subcommand>` -> `blr system <subcommand>`
 - `<packageManager> run world -- <subcommand>` -> `blr world <subcommand>`
+- `<packageManager> run assets:check` -> `blr world build --check`
 - `<packageManager> run clean` -> `blr clean`
 - `<packageManager> run upgrade` -> `blr upgrade`
+
+New projects also default `check` to `npm run assets:check && blr build`. `blr upgrade` reconciles `assets:check` but preserves a project's existing custom `check` workflow.

packages/blr/docs/reference/config.md

@@ -157,6 +157,7 @@ Optional world source backend configuration.
 Fields:
 
 - `backend`: `local | s3`
+- `pushPolicy`: `authored | processed`
 - `s3.bucket`: bucket name for remote world storage
 - `s3.region`: region for the S3 client
 - `s3.endpoint`: optional custom S3-compatible endpoint
@@ -168,6 +169,7 @@ Fields:
 Defaults if omitted:
 
 - `backend`: `local`
+- `pushPolicy`: `authored`
 - `s3.bucket`: empty
 - `s3.region`: empty in config, then resolved from `AWS_REGION` / `AWS_DEFAULT_REGION`, then `us-east-1`
 - `s3.endpoint`: empty
@@ -226,6 +228,59 @@ Notes:
   - `blr world push`
   - `blr dev` project-world remote sync
 - `blr dev` stays lenient about local world edits and uses `worlds/worlds.json` as the project truth instead of trying to infer remote freshness from local files alone
+- `pushPolicy: processed` makes `blr world push` require and publish the exact current verified processed-world build on a separate remote channel
+
+### `worldProcessors`
+
+Optional ordered trusted processors that derive artifacts or staged mutations from an immutable world snapshot.
+
+Example:
+
+```json
+{
+  "worldProcessors": [
+    {
+      "id": "project-locations",
+      "module": "./tooling/world/locations.ts",
+      "export": "createLocationsProcessor",
+      "sourceWorld": "Bedrock level",
+      "capabilities": ["artifact", "transform"],
+      "dependsOn": [],
+      "inputPaths": ["world-data/Bedrock level/locations.source.json"],
+      "outputRoot": "world-data/Bedrock level/generated/locations",
+      "payloadFileNames": {
+        "locations": "locations.json"
+      },
+      "runtimePointerPath": "world-data/Bedrock level/generated/locations/current.generated.json",
+      "applyOn": {
+        "dev": true,
+        "build": true,
+        "package": true,
+        "check": true,
+        "worldBuild": true,
+        "worldPush": true
+      }
+    }
+  ]
+}
+```
+
+Fields:
+
+- `id`: portable unique processor id
+- `module`: explicit project-relative module or bare package export
+- `export`: named factory export
+- `sourceWorld`: authoritative selected authored world
+- `capabilities`: any of `observer`, `artifact`, `transform`
+- `dependsOn`: preceding processor ids whose results are provided as immutable dependencies
+- `inputPaths`: declared project-relative logical input files
+- `outputRoot`: required immutable-set root for artifact processors
+- `payloadFileNames`: artifact id to relative payload filename map
+- `runtimePointerPath`: optional generated `.json` or `.ts` pointer to the current immutable set. Prefer JSON in world-owned data directories; use TypeScript only when the consumer explicitly owns generated source code.
+- `auditOutputPath`: optional audit output outside the runtime artifact set
+- `applyOn`: intent switches for `dev`, `build`, `package`, `check`, `worldBuild`, and `worldPush`
+
+Processor ids, input paths, and outputs are validated for portability and project containment. Output ownership cannot overlap across processors. See [World Processing](./world-processing.md) for source-snapshot, observation, cache, publication, and staged-mutation guarantees.
 
 ### `package`
 

packages/blr/docs/reference/generated-project.md

@@ -52,6 +52,7 @@ Generated scripts:
 - `minecraft`
 - `system`
 - `world`
+- `assets:check`
 - `clean`
 - `upgrade`
 
@@ -287,6 +288,7 @@ Usage:
 - `blr package mctemplate` and `blr package mcworld` use this as the source world payload
 - `blr world pull` can materialize this folder from a remote S3-compatible backend
 - `blr world push` can publish this folder back to the configured remote backend
+- `blr world build` derives a verified processed copy without opening or mutating this folder
 - `blr create` does not generate this folder by default because an empty folder looks like a valid world when it is not
 - world-aware commands require a real Bedrock world with a `db/` directory
 
@@ -400,6 +402,7 @@ Typical contents may include:
 - BDS zip cache
 - provisioned BDS server files
 - cached remote world archives and extracted world sources under `.blr/cache/worlds/`
+- content-addressed processor caches and verified processed-world lineages
 - CLI-owned transient state under `.blr/state/`, such as prompt-silence state in `.blr/state/cli.json`
 - other CLI-owned runtime state
 
@@ -412,6 +415,8 @@ During `local-server` runs, `blr` may also manage runtime world hook files such
 
 Those files are only written when the corresponding pack automation is enabled for the current project and current run.
 
+Configured `worldProcessors` may also publish immutable JSON sets and a stable generated TypeScript pointer at project-owned paths declared in `blr.config.json`. These are generated inputs, not hand-authored runtime state. Transform results live under the CLI workspace or an explicit output path; they never overwrite `worlds/<worldName>/`.
+
 ## Build Output
 
 ### `dist/stage/`

packages/blr/docs/reference/world-processing.md

@@ -0,0 +1,139 @@
+# World Processing
+
+## Purpose
+
+World processors turn a selected authored Bedrock world into deterministic build inputs without opening or modifying that authored world. A project can use several processors for independent jobs such as extracting sign-authored locations, compiling zones or paths, auditing a build, and preparing a processed world for development or publication.
+
+`blr` owns the unsafe and repetitive infrastructure:
+
+- resolving the selected and, when configured, remotely pinned world
+- taking a verified immutable snapshot before opening LevelDB
+- decoding blocks, block states, all subchunk layers, block entities, and both sign faces
+- bounding and ordering observation queries
+- loading trusted TypeScript processor modules
+- tracking logical inputs and invalidating cached work
+- publishing content-addressed artifact sets and typed runtime pointers
+- checking declarative world mutations for conflicts and preconditions
+- applying mutations only to a disposable staged copy
+- reopening and verifying the processed copy before it becomes usable
+- keeping authored and processed remote publication lineages separate
+
+The project or reusable package owns meaning. `blr` does not decide what `@name`, `@object`, a location, a zone, furniture, or a train means.
+
+## Processor contract
+
+Processors import the supported SDK from `@blurengine/cli/world-processing` and are created with `defineWorldProcessor`.
+
+```ts
+import { defineWorldProcessor } from "@blurengine/cli/world-processing";
+
+export const createLocationsProcessor = defineWorldProcessor(() => ({
+  implementationRevision: "locations-v1",
+  logicalInputs: [
+    { id: "definition", kind: "file", path: "locations.source.json" },
+  ],
+  async run(input) {
+    return {
+      logicalInputs: input.logicalInputs,
+      artifacts: [],
+      diagnostics: [],
+      mutations: [],
+    };
+  },
+}));
+```
+
+Processor modules are trusted local build code, not a sandbox. They receive no LevelDB handle or output path from `blr`, and cancellation is cooperative. Publication tokens prevent a superseded run from publishing stale results.
+
+Pipeline intent and check/bake mode select when BLR invokes a processor and how
+it publishes the result; they are not processor inputs. A processor therefore
+returns the same semantic artifacts, diagnostics, and mutation plan for the
+same immutable inputs across `dev`, `build`, `package`, and world workflows.
+This keeps one declared output path content-addressed and stable across commands.
+
+A processor may return optional canonical JSON in `audit`. When its config declares `auditOutputPath`, `blr world build --audit` writes that value beside the processor identity, revision, logical-input hash, and diagnostics. Audit data is excluded from runtime artifact sets and generated pointers unless the processor also chooses to publish it as an artifact.
+
+Capabilities are explicit:
+
+- `observer`: diagnostics only
+- `artifact`: immutable JSON payloads and a generated runtime pointer
+- `transform`: a declarative mutation plan for a staged world
+
+All selected processors observe the same immutable base snapshot. They do not read one another's mutations. Dependencies can consume preceding processor results, while conflicting mutation claims fail instead of using last-writer-wins ordering.
+
+## Sign observations
+
+Sign observations are data, not a directive standard. For each supported sign, the SDK preserves:
+
+- its integer block location and complete palette state
+- front and back faces independently
+- exact raw text
+- newline-normalised text
+- every row, including empty rows and rows beyond four
+- a discriminated orientation
+
+Standing signs expose all 16 `ground_sign_direction` states. Wall signs expose the four horizontal `facing_direction` states. Hanging signs retain their states for consumers that support them. Processors decide which face syntax is valid and which observations become artifacts.
+
+Use `input.observations.signs()` for location-style processors. It walks the world's block-entity records once, resolves each coherent sign's exact layer-zero block and orientation, and yields canonical dimension/X/Y/Z order. This cost scales with block entities instead of the volume between distant signs. An optional dimension and inclusive bounds filter the results; independent per-query and per-run sign limits protect builds without charging the ordinary block-position budget. Use `blocks(...)` for spatial block evidence such as routes, terrain, and bounded assemblies.
+
+For example, a project may treat line 1 as an explicit id or `~`, line 2 as any `@directive`, and all later lines as opaque consumer data. That grammar and any automatic id allocation belong to the processor or feature package, not to `blr`.
+
+## Artifacts
+
+An artifact processor declares its output root, payload filenames, and optional runtime pointer path in `blr.config.json`. `blr` canonicalises JSON, hashes payload bytes, writes an immutable set under `<outputRoot>/sets/<artifactSetId>/`, and replaces the stable pointer only after the complete set is valid. A detached manifest lists payload members and never hashes itself.
+
+Runtime pointers may be canonical `.json` data with payload values embedded, or generated `.ts` modules that statically import the immutable payloads. World-derived data should normally use a project-level world sidecar such as `world-data/<world>/generated/`; placing it under an authored `src/` tree is a consumer choice, not a BLR requirement.
+
+Processors that bind one payload to another can call
+`canonicalizeWorldDerivedJson(value)` or `hashWorldDerivedJson(value)` from
+`@blurengine/cli/world-processing`. These are the exact canonical byte and
+lowercase SHA-256 contracts used by BLR publication, including lexical object
+keys, finite-number validation, `-0` normalisation, two-space indentation, and
+one terminal newline. Pair the embedded hash with an artifact `hashReferences`
+entry so BLR independently verifies the binding before publication.
+
+`blr world build --check` recomputes and compares the contract without writing project files or caches. It is suitable for CI and the generated `assets:check` package script.
+
+When no processor applies to the selected world, the check is a successful read-only no-op and does not require a world directory. This keeps the managed script safe for ordinary projects before they add processors.
+
+## World mutations
+
+Transform processors return typed, declarative operations rather than editing LevelDB:
+
+- set one block with an exact observed/literal palette entry
+- replace matching blocks in a bounded volume
+- remove an expected block entity
+- consume a sign as one block-and-block-entity operation
+
+Every operation has a stable id, exact dimension/location/layer, and explicit preconditions. Palette entries include type id, complete states, and Bedrock palette version. The writer preserves untouched layers and block entities, updates affected height data, applies the raw key changes atomically to the staged database, then reopens and verifies the result. Initial support intentionally edits only existing chunks and subchunks.
+
+Authored marker signs are retained by default. A consumer must explicitly emit a mutation if its processed world should remove or replace one.
+
+## Commands and workflow
+
+Use these commands from a configured project:
+
+```text
+blr world build
+blr world build --check
+blr world build --dry-run --json
+blr world build --processor locations --audit
+blr world build --output dist/world-preview
+```
+
+`build` and `package` run the processors enabled for those intents. Packages that include a world consume the verified processed world; pack-only targets do not materialise unnecessary world transforms.
+
+In `dev`, artifact processors may trigger a script/content reload. A world transform causes a stopped-server rebuild and reseed. Processed-world development cannot be combined with `watch-world`: capturing a derived runtime world back over the marker-bearing authored source is rejected. Use two explicit modes:
+
+- author mode: raw world with capture enabled
+- play/verification mode: processed world with capture disabled
+
+## Remote publication
+
+Set `world.pushPolicy` to `processed` to make ordinary `blr world push` require a current verified `blr world build`. The processed world is uploaded to a separate remote channel from the authored source. A push acquires the world lock before checking and uploading the exact current build.
+
+`--channel authored|processed` overrides the configured policy for an explicit invocation. Pulling continues to operate on the authored channel; processed output is a publication artifact and is never materialised over the project source.
+
+## Source safety
+
+No processor run opens the authored LevelDB. `blr` inventories source contents, copies them to a unique temporary snapshot, inventories source and snapshot again, and proceeds only when they agree. Failed, aborted, conflicting, or stale runs leave the source and previous published artifacts untouched.

packages/blr/package.json

@@ -21,6 +21,12 @@
     "url": "https://github.com/BlurEngine/cli/issues"
   },
   "type": "module",
+  "exports": {
+    "./world-processing": {
+      "types": "./dist/world-processing.d.ts",
+      "import": "./dist/world-processing.js"
+    }
+  },
   "bin": {
     "blr": "./dist/blr.js"
   },
@@ -34,6 +40,7 @@
     "CHANGELOG.md",
     "NOTICE"
   ],
+  "types": "./dist/world-processing.d.ts",
   "scripts": {
     "sync:legal": "node ./scripts/sync-legal-files.mjs",
     "generate:schema": "node ./scripts/generate-config-schema.mjs",
@@ -58,12 +65,12 @@
   "dependencies": {
     "@8crafter/leveldb-zlib": "^1.6.0",
     "@aws-sdk/client-s3": "3.1068.0",
-    "adm-zip": "^0.5.17",
+    "adm-zip": "^0.6.0",
     "chokidar": "^5.0.0",
     "commander": "^14.0.3",
-    "esbuild": "^0.28.0",
+    "esbuild": "^0.28.2",
     "kleur": "^4.1.5",
-    "mcbe-leveldb": "^1.15.0",
+    "mcbe-leveldb": "^1.21.0",
     "node-pty": "^1.1.0",
     "picomatch": "^4.0.4",
     "pngjs": "^7.0.0",
@@ -73,7 +80,6 @@
     "typescript": "^5.9.3"
   },
   "devDependencies": {
-    "@types/adm-zip": "^0.5.8",
     "@types/pngjs": "^6.0.5",
     "ts-json-schema-generator": "^2.4.0"
   }