From ea45ef3db3319c896e7a79aa2fcfa45b8c4a7114 Mon Sep 17 00:00:00 2001 From: Joshua Natarajan Date: Tue, 16 Jun 2026 16:54:53 -0400 Subject: [PATCH] docs: update README + project state for intent-based agency, telemetry, and Python client - README: update intro, architecture diagram, project structure, roadmap to reflect shipped intent-based client agency, telemetry bus, browser modularization, and Python debug client - LILA_PROJECT_STATE: update architecture diagram, repo structure, current status, planning docs. Add Step 4.1/4.2 for intent-based agency and telemetry. Mark #77 (telemetry) as shipped. --- LILA_PROJECT_STATE.md | 199 ++++++++++++++++++++++++++++++------------ README.md | 91 ++++++++++++------- 2 files changed, 202 insertions(+), 88 deletions(-) diff --git a/LILA_PROJECT_STATE.md b/LILA_PROJECT_STATE.md index deb4b2c..d9df8a3 100644 --- a/LILA_PROJECT_STATE.md +++ b/LILA_PROJECT_STATE.md @@ -11,13 +11,13 @@ **Tagged release: v0.0.1-alpha** — published, repo public on GitHub. -līlā is a BYOM (Bring Your Own Model) ecosystem simulation engine. Users define a world in JSON — species, biome, soil, water — and the engine grows an autonomous ecosystem from simple rules. The server runs the hybrid automaton (ecology, physics, ML inference); clients render the result via WebSocket at 10 Hz. +līlā is a BYOM (Bring Your Own Model) ecosystem simulation engine. Users define a world in JSON — species, biome, soil, water — and the engine grows an autonomous ecosystem from simple rules. The server runs the hybrid automaton at ~0.5 Hz, emitting intent packets (state, drives, motion latents, reference positions). Clients execute local agency at 60 Hz and send heartbeats upstream. Divergence is treated as emergence, not error. The project thesis — explored in ["The Unseen Hand"](https://www.hellolifeforms.com/p/the-unseen-hand) — is that the most impactful AI is small, specialized, and invisible. Tiny ML models guide lifelike motion and behavior; the user never sees inference happening, they just see a world that feels alive. The name comes from the Sanskrit concept of [līlā](https://www.embodiedphilosophy.com/what-is-lila/) — the spontaneous, purposeless creative unfolding of reality. There's no win condition. The world plays as itself. -**Current direction:** The engine is transitioning from hand-crafted per-species rules to a **trait-based architecture** using allometric scaling laws (Metabolic Theory of Ecology). Species become points in trait space; the engine derives all behavior parameters from body mass and functional traits. This also makes līlā a compelling **substrate for automated ALife search** (ASAL framework) — an ecologically-grounded simulation where FM-guided search discovers interesting ecosystem configurations. +**Current direction:** The trait-based architecture is shipped. The intent-based client agency layer is shipped (PR #83). The telemetry bus is shipped for ML training data generation. Next: calibration & regression testing with all 8 species, spatial hash for O(1) neighbor queries, and trait-based ASAL search. **Copyright:** BioSynthArt Studios LLC. **License:** Apache 2.0. **Source control:** GitHub at `github.com/hellolifeforms/lila` (org: hellolifeforms). @@ -29,49 +29,56 @@ The name comes from the Sanskrit concept of [līlā](https://www.embodiedphiloso ## Architecture Overview ``` -┌─────────────────────────┐ -│ Browser Visualizer │ ← v0.0.1-alpha (shipped, single HTML file) -│ Godot 4.x Client │ ← deferred to Milestone 4 -│ Headless Renderer │ ← Shipped (PIL, 256×256, for ASAL search) -└──────────┬──────────────┘ - │ WebSocket (delta-encoded tick packets) -┌──────────▼──────────────┐ -│ Worker │ ← Shipped. HTTP + WS on single port -│ (one per session) │ Serves viz HTML, streams ticks -└──────────┬──────────────┘ - │ -┌──────────▼─────────────────────────────────────────────────────────┐ -│ ecosim (Python package, stdlib only) │ -│ ┌─────────────────┐ ┌───────────────────────────────────────┐ │ -│ │ Hybrid Automaton│ │ Trait System (Milestone 2) │ │ -│ │ Flow + Guards │ │ TraitVector + Compiler │ │ -│ ├─────────────────┤ │ Allometric Derivations │ │ -│ │ Voxel Manager │ │ Interaction Templates │ │ -│ │ 5 layers (M2) │ ├───────────────────────────────────────┤ │ -│ │ Water System │ │ Actor Effects Architecture │ │ -│ │ Dynamic levels │ │ EffectBus + Flow/Guard/IX Actors │ │ -│ ├─────────────────┤ │ Dual-path: trait-based / legacy │ │ -│ │ Two-Pool Soil │ ├───────────────────────────────────────┤ │ -│ │ Fast/Slow (M2) │ │ BYOM Adapters │ │ -│ └─────────────────┘ │ mlp/static/random │ │ -│ ├───────────────────────────────────────┤ │ -│ │ World Randomizer │ │ -│ │ D4 transforms │ │ -│ └───────────────────────────────────────┘ │ -└────────────────────────────────────────────────────────────────────┘ - │ -┌──────────▼────────────────────────────────────────────┐ -│ search/ (Shipped — Track A, rate-tuning search) │ -│ ASAL Substrate Protocol (Init/Step/Render) │ -│ Headless PIL Renderer (256×256) │ -│ CLIP ViT-B/32 Evaluator │ -│ Illumination Search (diversity GA) │ -│ Simulation Atlas (UMAP + grid sampling) │ -│ ───────────────────────────────────────────────── │ -│ Target Search (CMA-ES + text prompts) pending │ -│ Open-Ended Search (temporal novelty) pending │ -│ Trait-Based θ Expansion (Milestone 2 dep) pending │ -└───────────────────────────────────────────────────────┘ +┌─────────────────────────────┐ ┌─────────────────────────────┐ +│ Browser Visualizer │ │ Python Debug Client │ +│ Modular JS (shipped) │ │ ImGui + telemetry (shipped)│ +│ 60 Hz agency + reconciliation│ │ Replay from JSONL logs │ +│ Heartbeats @ 1 Hz │ │ │ +└─────────┬───────────────────┘ └──────────┬──────────────────┘ + │ WebSocket │ WebSocket + │ intent packets (0.5 Hz) │ /ws + /telemetry + │ heartbeats (1 Hz) │ + │ │ +┌─────────▼─────────────────────────────────▼──────────────────┐ +│ Worker │ +│ HTTP + WS on single port (one per session) │ +│ Serves viz, streams intents, absorbs heartbeats │ +└─────────┬────────────────────────────────────────────────────┘ + │ +┌─────────▼────────────────────────────────────────────────────┐ +│ ecosim (Python package, stdlib only) │ +│ ┌─────────────────┐ ┌───────────────────────────────────┐ │ +│ │ Hybrid Automaton│ │ Trait System (Milestone 2) │ │ +│ │ Intent emit + │ │ TraitVector + Compiler │ │ +│ │ HB absorption │ │ Allometric Derivations │ │ +│ ├─────────────────┤ │ Interaction Templates │ │ +│ │ Voxel Manager │ ├───────────────────────────────────┤ │ +│ │ 5 layers (M2) │ │ Actor Effects Architecture │ │ +│ │ Water System │ │ EffectBus + Flow/Guard/IX Actors │ │ +│ │ Dynamic levels │ ├───────────────────────────────────┤ │ +│ │ Two-Pool Soil │ │ BYOM Adapters │ │ +│ │ Fast/Slow (M2) │ │ mlp/static/random │ │ +│ └─────────────────┘ ├───────────────────────────────────┤ │ +│ │ Telemetry Bus (shipped) │ │ +│ │ JSONL: config + timeseries + evt │ │ +│ ├───────────────────────────────────┤ │ +│ │ World Randomizer │ │ +│ │ D4 transforms │ │ +│ └───────────────────────────────────┘ │ +└──────────────────────────────────────────────────────────────┘ + │ +┌─────────▼───────────────────────────────────────────┐ +│ search/ (Shipped — Track A, rate-tuning search) │ +│ ASAL Substrate Protocol (Init/Step/Render) │ +│ Headless PIL Renderer (256×256) │ +│ CLIP ViT-B/32 Evaluator │ +│ Illumination Search (diversity GA) │ +│ Simulation Atlas (UMAP + grid sampling) │ +│ ─────────────────────────────────────────────── │ +│ Target Search (CMA-ES + text prompts) pending │ +│ Open-Ended Search (temporal novelty) pending │ +│ Trait-Based θ Expansion (M2 dep) pending │ +└─────────────────────────────────────────────────────┘ ``` --- @@ -86,7 +93,7 @@ lila/ │ ├── uv.lock # deterministic dependency lockfile │ ├── ecosim/ # core simulation library │ │ ├── __init__.py -│ │ ├── engine.py # hybrid automaton (trait-based, actor-driven, ~737 lines) +│ │ ├── engine.py # hybrid automaton + intent emit/absorption (~782 lines after decomposition) │ │ ├── entities.py # entity schemas, init_entity() │ │ ├── biome.py # biome presets → BiomeConfig │ │ ├── config.py # [M2] SIM_CONFIG loader — tunable params from JSON @@ -94,7 +101,8 @@ lila/ │ │ ├── world_processes.py # World-process handlers (evaporation, water replenish, soil drain/deposit, nutrient pool dynamics) │ │ ├── environment_manager.py # [M2] Environment state — biome, climate, voxels, water sources, rain │ │ ├── model_adapter.py # MotorAdapter protocol, ContextSpec -│ │ ├── worker.py # async WS tick loop + HTTP viz server +│ │ ├── worker.py # WS server: intent emit (0.5 Hz), heartbeat absorption, telemetry WS +│ │ ├── telemetry.py # Telemetry bus — JSONL event stream (config, timeseries, events) │ │ ├── traits.py # [M2] TraitVector, allometric derivations │ │ ├── interactions.py # [M2] InteractionTemplate grammar │ │ ├── trait_compiler.py # [M2] TraitCompiler: traits → engine params @@ -128,13 +136,35 @@ lila/ │ │ ├── test_movement_actor.py # [M3] movement actor behavior tests (36) │ │ ├── test_voxel_grid.py # VoxelGrid protocol + query_overlap/walk_layer (28 tests) │ │ ├── test_nutrients.py # [M2] two-pool nutrient flow tests (~20) -│ │ └── test_reproduction_actor.py # reproduction actor behavior tests (~15) +│ │ ├── test_reproduction_actor.py # reproduction actor behavior tests (~15) +│ │ └── client_harness.py # Headless client test harness (intent protocol + reconciliation) │ └── weights/ │ └── (motion_v0.json) # placeholder for trained weights │ ├── client/ -│ ├── browser/ -│ │ └── index.html # canvas-based 2D ecosystem visualizer +│ ├── browser/ # Modular JS visualizer (client-side agency + reconciliation) +│ │ ├── index.html # Thin shell (46 lines) +│ │ ├── css/style.css # All visual styles +│ │ ├── js/agency.js # Client-side behavior engine (60 Hz) +│ │ ├── js/heartbeat.js # Upstream position/event reporting (1 Hz) +│ │ ├── js/reconciliation.js # Gravity-well position reconciliation +│ │ ├── js/world-model.js # Local entity registry + spatial queries +│ │ ├── js/renderer.js # Canvas rendering (entities, water, heatmap, particles) +│ │ ├── js/particles.js # Particle system for event visualizations +│ │ ├── js/constants.js # Colors, grid config, tick rates +│ │ └── js/main.js # Entry point — WS setup, render loop, UI controls +│ ├── python/ # ImGui debug client (telemetry + replay) +│ │ ├── pyproject.toml +│ │ └── lila_client/ +│ │ ├── main.py # Entry point +│ │ ├── agency.py # Client-side behavior engine (mirrors browser) +│ │ ├── websocket.py # Async WS client (simulation + telemetry streams) +│ │ ├── world_model.py # Local scene graph +│ │ ├── imgui_view.py # ImGui render: world view, telemetry timeline, inspector +│ │ ├── reconciliation.py # Position reconciliation +│ │ ├── replay.py # Post-mortem replay from JSONL with tick scrubber +│ │ ├── constants.py # Shared constants +│ │ └── pygame_renderer.py # Fallback renderer │ └── godot/ # [M4] Godot 4.x client │ ├── search/ # ASAL substrate + search (shipped Track A) @@ -170,6 +200,8 @@ lila/ ├── docs/ │ ├── model_adapter_spec.md # BYOM guide — how to build adapters │ ├── data_contract.md # v0.2 protocol spec +│ ├── intent_based_architecture.md # Intent protocol, reconciliation, client agency spec +│ ├── ECOSIM_PARAMETER_TELEMETRY_SPACE.md # Telemetry streams, parameter space, hybrid BYOM plan │ ├── architecture.md │ ├── species_spec.md # 0.1-alpha species + skeleton rigs │ ├── lessons_learned.md # debugging war stories @@ -183,7 +215,8 @@ lila/ ├── LICENSE # Apache 2.0 ├── README.md # project overview, quick start, roadmap ├── TRAIT_TRANSITION_PLAN.md # detailed Phase 1-3 implementation plan -└── TWO_POOL_NUTRIENT_SPEC.md # two-pool soil nutrient spec +├── TWO_POOL_NUTRIENT_SPEC.md # two-pool soil nutrient spec +└── LILA_PROJECT_STATE.md # This file — project state, milestones, architecture ``` Items marked `[M2]`, `[M3]`, `[M4]` indicate which milestone introduces them. @@ -397,6 +430,54 @@ Five species, two skeletons, five interaction chains: **Reference documents:** `TRAIT_TRANSITION_PLAN.md` (Phase 1), `TWO_POOL_NUTRIENT_SPEC.md` +### Step 4.1 — Intent-Based Client Agency Architecture ✅ + +The engine was refactored from an authoritative-server model (streaming exact positions at 10 Hz) to an **intent-based architecture** where the server guides via desire and disposition, and the client executes local agency between ticks. Divergence is treated as emergence, not error. + +**Server-side changes (`engine.py`, `worker.py`):** +- `_build_tick_packet()` — emits intent fields (`state`, `drive`, `motion_latent`, `ref_position`, `_can_*` eligibility flags) instead of authoritative positions +- `absorb_client_positions(positions)` — reconciles client-reported positions using soft-nudge within bounds, snap + `_ack` on large divergence +- `absorb_client_events(events)` — absorbs client-reported interactions (consumption, predation, pollination, repro) with validation and rate caps +- `get_species_definitions()` — exports lightweight species reference (`type`, `movement_speed`, `diet_order`, `flee_targets`, `is_pollinator`, `pollination_targets`) for client-side agency +- `DEFAULT_TICK_RATE` changed from 0.1s (10 Hz) → 2.0s (0.5 Hz) +- `session_started` ack includes `species` definitions +- `"heartbeat"` added to `CONTROL_HANDLERS` +- `process_request()` extended to serve arbitrary static files from viz directory (css/, js/) with proper MIME types + +**Browser client modularization:** Monolithic `index.html` (~1200 lines) split into modular files: +- `js/agency.js` (370 lines) — client-side behavior engine: drive evaluation → target selection → movement → interaction triggers with cooldowns +- `js/heartbeat.js` (65 lines) — upstream position/event reporting at 1 Hz +- `js/reconciliation.js` (50 lines) — gravity-well position reconciliation (trust within bounds, nudge/snap, `_ack` handling) +- `js/world-model.js` (280 lines) — local entity registry, species defs, spatial queries (`findNearest`, `findNearestWater`, `findNearestMate`) +- `js/renderer.js` (510 lines) — all canvas drawing functions +- `js/main.js` (340 lines) — entry point: WS setup, render loop, UI controls +- `js/constants.js` (61 lines), `js/particles.js` (38 lines), `css/style.css` (176 lines) + +**Python debug client (`client/python/`):** +- ImGui-based real-time visualization with telemetry timeline, entity inspector +- WebSocket client bridging asyncio ↔ ImGui via queues +- Dual-stream connection: `/ws` (simulation) + `/telemetry` (event stream) +- Post-mortem replay from JSONL logs with tick scrubber +- Client-side agency engine mirroring browser (flee → drink → mate → forage → hunt → pollinate → wander) + +**Telemetry bus (`ecosim/telemetry.py`):** +- Three streams: config snapshot (once), time-series aggregates (every K ticks), event log (batched) +- Writes to `~/.lila/logs/.jsonl` +- Streams over `/telemetry` WebSocket endpoint +- Tracks: intent emit, event log, absorption trace + +**Test harness (`tests/client_harness.py`):** +- Headless Python client simulating full browser session +- Validates intent packet format, heartbeat reconciliation, event absorption, divergence snap + `_ack` + +**Reference doc:** [`docs/intent_based_architecture.md`](docs/intent_based_architecture.md) (full protocol spec, reconciliation strategy, action mapping, migration notes) + +#### Step 4.2 — Telemetry Infrastructure ✅ +- Config snapshots, time-series aggregates, event batching +- Parameter space summary: ~299 constants → ~48 actionable dimensions +- JSONL logging pipeline for ML training data generation +- `docs/ECOSIM_PARAMETER_TELEMETRY_SPACE.md` — architecture reference + ### Completed Steps ✅ #### Step 2.1 — Audit Current Hard-Coded Parameters ✅ @@ -743,13 +824,13 @@ Each phase has a **physics default** and an optional **model override**. The eng ### Phase Plan + Issues -| # | Issue | Role | -|---|-------|------| -| [#77](https://github.com/hellolifeforms/lila/issues/77) | Telemetry emitter: config snapshot + time-series aggregates + event batching | Foundation — three-stream data collection | -| [#78](https://github.com/hellolifeforms/lila/issues/78) | Surrogate model + sensitivity analysis over sim_config parameter space | Understanding — which of ~48 tunable params actually matter | -| [#79](https://github.com/hellolifeforms/lila/issues/79) | Learned diffusion: replace or gate voxel nutrient diffusion with a model | First replacement — O(N×4) → O(1) forward pass | -| [#20](https://github.com/hellolifeforms/lila/issues/20) | Behavior-level adapter | Guard augmentation — learned threshold bias per entity | -| [#21](https://github.com/hellolifeforms/lila/issues/21) | Narrative-level adapter | Macro intelligence — ecosystem-scale event injection | +| # | Issue | Role | Status | +|---|-------|------|--------| +| [#77](https://github.com/hellolifeforms/lila/issues/77) | Telemetry emitter: config snapshot + time-series aggregates + event batching | Foundation — three-stream data collection | **shipped** | +| [#78](https://github.com/hellolifeforms/lila/issues/78) | Surrogate model + sensitivity analysis over sim_config parameter space | Understanding — which of ~48 tunable params actually matter | pending | +| [#79](https://github.com/hellolifeforms/lila/issues/79) | Learned diffusion: replace or gate voxel nutrient diffusion with a model | First replacement — O(N×4) → O(1) forward pass | pending | +| [#20](https://github.com/hellolifeforms/lila/issues/20) | Behavior-level adapter | Guard augmentation — learned threshold bias per entity | pending | +| [#21](https://github.com/hellolifeforms/lila/issues/21) | Narrative-level adapter | Macro intelligence — ecosystem-scale event injection | pending | ### Telemetry Streams (from #77) @@ -968,6 +1049,8 @@ Expand the shipped search pipeline from 17-dim rate tuning to trait-space search - **docs/ECOSIM_PARAMETER_TELEMETRY_SPACE.md** — Hybrid model + simulation architecture. Covers telemetry streams (config snapshot, time-series aggregates, event batching), parameter space stratification (~299 constants → ~48 actionable), concrete use cases (surrogate model, learned diffusion, sensitivity analysis, early-warning predictor), and five-phase implementation plan with issue tracking. +- **docs/intent_based_architecture.md** — Intent-based client agency architecture. Covers intent protocol (state, drives, motion_latent, ref_position, eligibility flags), reconciliation strategy (soft-nudge, snap + _ack), heartbeat format, client-side behavior engine, action mapping (who decides what), browser client modularization, test harness, and migration notes. + --- ## Key References diff --git a/README.md b/README.md index cbadaa1..cd3c920 100755 --- a/README.md +++ b/README.md @@ -18,7 +18,7 @@ `līlā` is an open-source engine that grows autonomous ecosystems from simple rules. You define species, biomes, and resources — the engine handles hunger cycles, predator-prey loops, soil nutrient flows, water depletion, dormancy, and recovery. Organisms don't follow scripts; their behavior emerges from continuous state variables, hybrid automata guards, and environmental feedback. -> **What you see right now is a 2D debug visualizer** — a window into the engine's state, not the final form. The engine is the product: a headless simulation server that streams tick packets over WebSocket to any client. A 3D Godot client with skeletal animation is planned for v0.1.0. The thesis isn't pretty graphics — it's that tiny ML models, invisible to the user, make a world feel alive. See ["The Unseen Hand"](https://www.hellolifeforms.com/p/the-unseen-hand) for the full argument. +> **What you see is an intent-based client** — the server emits state, drives, and reference positions; the client decides movement, triggers interactions, and sends heartbeats back upstream. Divergence is treated as emergence, not error. Two clients ship today: a browser visualizer (modular JS, 60 Hz local agency) and a Python debug client (ImGui, telemetry timeline, replay). A 3D Godot client with skeletal animation is planned for v0.1.0. The thesis isn't pretty graphics — it's that tiny ML models, invisible to the user, make a world feel alive. See ["The Unseen Hand"](https://www.hellolifeforms.com/p/the-unseen-hand) for the full argument. ### Built for @@ -35,7 +35,7 @@ cd lila/deploy/compose docker compose up --build ``` -Open **http://localhost:8001** — a temperate meadow is already running at 10 Hz. Click **☔ Rain** to trigger recovery cycles, or **⏺ Record** to capture dynamics as WebM (convert to GIF with `ffmpeg -i lila-recording.webm -vf "fps=15,scale=480:-1" -loop 0 docs/assets/demo.gif`). Stop with `docker compose down`. +Open **http://localhost:8001** — a temperate meadow is running. The server computes ecology at 0.5 Hz and streams intent packets to the client; the browser renders at 60 Hz with local agency. Click **☔ Rain** to trigger recovery cycles, or **⏺ Record** to capture dynamics as WebM (convert to GIF with `ffmpeg -i lila-recording.webm -vf "fps=15,scale=480:-1" -loop 0 docs/assets/demo.gif`). Stop with `docker compose down`. ### Why līlā? @@ -51,6 +51,7 @@ Open **http://localhost:8001** — a temperate meadow is already running at 10 H - **Define a world** — [Define a world in JSON](#define-a-world-in-json) just below - **Bring Your Own Model** — [`docs/model_adapter_spec.md`](docs/model_adapter_spec.md) for the adapter protocol, context specs, and export pipeline - **Trait architecture** — [`LILA_PROJECT_STATE.md`](LILA_PROJECT_STATE.md#milestone-2-trait-based-architecture--partial--two-pool-nutrients-pending) for allometric scaling and interaction templates +- **Intent-based agency** — [`docs/intent_based_architecture.md`](docs/intent_based_architecture.md) for the intent protocol, reconciliation, and client-side behavior engine - **ASAL search** — [`search/README.md`](search/README.md) for illumination search, CLIP evaluation, and atlas generation - **Develop & contribute** — [`DEVELOPING.md`](DEVELOPING.md) for the `uv` workflow, tests, and architecture deep dives @@ -147,20 +148,24 @@ See [`server/examples/`](server/examples/) for the full demo and additional pres ## How it works ``` - Browser / Godot client - │ - │ WebSocket (tick packets at 10 Hz) - │ - ┌──────▼───────┐ - │ Worker │ One per active ecosystem - │ HTTP + WS │ Serves viz + streams ticks - └──────┬───────┘ + Browser client (60 Hz agency) │ Python debug client (ImGui + telemetry) + │ │ + │ WebSocket │ WebSocket + │ intent packets (0.5 Hz) │ /ws + /telemetry + │ heartbeats (1 Hz) │ + │ │ + ┌──────▼────────────────────────────────▼────────┐ + │ Worker │ + │ HTTP + WS on single port (one per session) │ + │ Serves viz, streams intents, absorbs HB │ + └──────┬─────────────────────────────────────────┘ │ ┌──────▼───────────────────────────────────┐ │ ecosim (Python, stdlib only) │ │ │ │ engine ─── hybrid automaton │ - │ flow equations + guards │ + │ intent emit + absorption │ + │ flow/guard/interaction │ │ hysteresis on transitions │ │ │ │ traits ── species as trait vectors │ @@ -175,10 +180,14 @@ See [`server/examples/`](server/examples/) for the full demo and additional pres │ nutrients_fast, slow ✅ │ │ moisture, temperature, │ │ organic matter │ + │ │ + │ telemetry ── JSONL event stream │ + │ config snapshot + timeseries │ + │ for training / post-mortem │ └──────────────────────────────────────────┘ ``` -Each tick, the engine runs seven phases: continuous flow updates, entity interactions, guard condition checks (with hysteresis to prevent oscillation), voxel layer effects, motor model inference, removals, and spawns. The result is a delta-encoded tick packet streamed to the client over WebSocket. +Each tick, the engine runs seven phases: flow actors, interactions, guard actors, voxel effects, world processes, motor inference, and lifecycle (removals/spawns). The result is an **intent packet** — state, drives, motion latents, reference positions, and eligibility flags — streamed to clients at ~0.5 Hz. Clients execute local agency at 60 Hz and send **heartbeats** (positions + interaction events) upstream. The server absorbs heartbeats through a reconciliation strategy: soft-nudge within bounds, snap + `_ack` on large divergence. The engine has **zero external dependencies** — stdlib Python only. All numeric constants live in a single `constants.py` module. The actor system (flow, guard, interaction) uses immutable effects applied atomically via EffectBus. The worker adds `websockets`. That's the entire server. @@ -231,6 +240,8 @@ Three model levels are defined: | Behavior | every tick | Influences state transition decisions | designed, not yet wired | | Narrative | every N ticks | Shapes macro-scale ecosystem dynamics | designed, not yet wired | +**Intent-based client agency** (shipped) bridges motor inference with client-side behavior: the server streams `state`, `drive` variables, `motion_latent`, `ref_position`, and `_can_*` eligibility flags at ~0.5 Hz. The client evaluates a priority chain (flee → drink → mate → forage → hunt → pollinate → wander) and executes movement locally at 60 Hz. Heartbeats report positions and interactions upstream for server validation. See [`docs/intent_based_architecture.md`](docs/intent_based_architecture.md). + See [`docs/model_adapter_spec.md`](docs/model_adapter_spec.md) for the full guide to building your own adapter. ## Use cases @@ -276,31 +287,34 @@ Three interaction chains emerge without scripting: lila/ ├── server/ │ ├── ecosim/ # Core simulation (stdlib only) -│ │ ├── engine.py # Hybrid automaton +│ │ ├── engine.py # Hybrid automaton + intent emit/absorption │ │ ├── entities.py # Entity schemas │ │ ├── traits.py # Trait definitions │ │ ├── trait_compiler.py# Trait → derived params compiler │ │ ├── interactions.py # Interaction templates │ │ ├── biome.py # Biome presets +│ │ ├── config.py # SIM_CONFIG loader (tunable params from JSON) │ │ ├── voxel_manager.py # VoxelGrid protocol + UniformVoxelGrid (multi-resolution ready) │ │ ├── world_processes.py # World-process handlers (evaporation, water replenish, soil drain/deposit) │ │ ├── constants.py # Universal simulation constants (single source of truth) │ │ ├── model_adapter.py # BYOM protocol │ │ ├── effects.py # Effect dataclasses + EffectBus (immutable effect pipeline) +│ │ ├── telemetry.py # Telemetry bus — JSONL event stream │ │ ├── actors/ # Actor system (flow, guard, interaction, movement) │ │ │ ├── __init__.py # InteractionContext, FlowActor/GuardActor bases, registries -│ │ │ ├── flow_actors.py # ConsumerFlowActor, ProducerFlowActor, DecomposerFlowActor (+ MovementActor integration) +│ │ │ ├── flow_actors.py # ConsumerFlowActor, ProducerFlowActor, DecomposerFlowActor │ │ │ ├── guard_actors.py # ConsumerGuardActor, ProducerGuardActor, DecomposerGuardActor │ │ │ ├── interaction_actors.py # FleeActor, PredationActor, HerbivoryActor, PollinationActor │ │ │ └── movement_actors.py # MovementActor — target selection as effect-emitting actor │ │ ├── layout.py # LayoutManager — world loading, D4 transforms, randomization │ │ ├── spatial_index.py # SpatialIndex protocol + BruteForceSpatialIndex (neighbor queries) │ │ ├── movement_system.py # MovementSystem — gate policy + kinematics for mobile entities -│ │ ├── worker.py # WebSocket server +│ │ ├── worker.py # WebSocket server (intent emit, heartbeat absorption) │ │ └── adapters/ # Built-in motor models │ ├── examples/ # Demo world definitions │ └── tests/ │ ├── test_voxel_grid.py # VoxelGrid protocol + query_overlap/walk_layer (28 tests) +│ └── client_harness.py # Headless client test harness (intent protocol + reconciliation) ├── search/ │ ├── lila_search/ # ASAL-compatible search (see below) │ │ ├── substrate.py # Init/Step/Render protocol @@ -312,12 +326,22 @@ lila/ │ ├── scripts/ # CLI entry points │ └── tests/ ├── client/ -│ ├── browser/ # Canvas-based 2D visualizer +│ ├── browser/ # Modular JS visualizer (client-side agency + reconciliation) +│ │ ├── index.html # Thin shell +│ │ ├── js/agency.js # Client-side behavior engine (60 Hz) +│ │ ├── js/heartbeat.js # Upstream position/event reporting (1 Hz) +│ │ ├── js/reconciliation.js # Gravity-well position reconciliation +│ │ ├── js/world-model.js # Local entity registry + spatial queries +│ │ ├── js/renderer.js # Canvas rendering +│ │ └── js/main.js # Entry point — WS setup, render loop, UI +│ ├── python/ # ImGui debug client (telemetry + replay) +│ │ └── lila_client/ # agency, imgui_view, websocket, world_model, replay │ └── godot/ # 3D client (in development) ├── training/ # Example ML training pipeline ├── deploy/ │ └── compose/ # Docker Compose (start here) └── docs/ + └── intent_based_architecture.md # Intent protocol + reconciliation spec ``` ### Actor-based architecture @@ -402,18 +426,19 @@ means writing a JSON trait vector, not new Python code. The interaction templates (herbivory, predation, pollination, decomposition) handle the combinatorics. -The second architectural shift is **intent-based motion**. Today the -engine streams discrete state and positions to the client, which renders -them flat. The target architecture has the server emit small latent -vectors — *intent* — and the client integrate a learned body law from -them. The body is modeled as a bank of coupled Hopf oscillators; style -becomes the parameter set of that law, and gaits become limit cycles -of it. Wire cost drops by orders of magnitude because the client holds -the decoder. In multi-observer worlds, this generalizes into a split -between a **consensus base** (causal facts that must agree across all -clients) and a **cosmetic fiber** (the local performance of those facts), -with reconciliation issued as intent rather than as coordinate snaps. -See ["The Mathematics of Līlā"](https://www.hellolifeforms.com/p/the-mathematics-of-lila) +The second architectural shift is **intent-based motion**. The server +emits state, drives, motion latents, and reference positions at ~0.5 Hz. +Clients execute local agency at 60 Hz and send heartbeats upstream. +Divergence is treated as emergence, not error — the server soft-nudges +within bounds and snaps + acknowledges on large divergence. This first +layer is shipped. The deeper layer models the body as a bank of coupled +Hopf oscillators; style becomes the parameter set of that law, and gaits +become limit cycles of it. Wire cost drops by orders of magnitude because +the client holds the decoder. In multi-observer worlds, this generalizes +into a split between a **consensus base** (causal facts that must agree +across all clients) and a **cosmetic fiber** (the local performance of +those facts), with reconciliation issued as intent rather than as +coordinate snaps. See ["The Mathematics of Līlā"](https://www.hellolifeforms.com/p/the-mathematics-of-lila) for the full argument. **Shipped:** @@ -434,22 +459,28 @@ for the full argument. - Headless renderer for FM-guided evaluation (PIL, 256×256) - Illumination search — diversity-driven GA with CLIP ViT-B/32 - Simulation atlas — UMAP projection of discovered ecosystems +- Intent-based client agency — server emits state/drives/latents/ref_position at 0.5 Hz; client executes 60 Hz local behavior + 1 Hz heartbeats +- Browser client modularization — split from monolithic HTML into modular JS (agency, heartbeat, reconciliation, world-model, renderer) +- Python debug client — ImGui-based with telemetry timeline, entity inspector, post-mortem replay from JSONL logs +- Telemetry bus — config snapshots, time-series aggregates, event batching (JSONL to `~/.lila/logs/`) +- Client reconciliation — soft-nudge within bounds, snap + `_ack` on divergence, per-target interaction cooldowns **Near-term:** -- Telemetry infrastructure — config snapshots, time-series aggregates, event batching for ML training pipelines -- Hybrid BYOM — extend pluggable adapters from motor inference to full simulation phases (learned diffusion, behavior bias, narrative control) - Spatial hash for O(1) neighbor queries (SpatialIndex strategy swap) - Calibration & regression testing (2000-tick baseline with two-pool nutrients) - Emergent dynamics validation with 8 species (trophic cascades, Lotka-Volterra oscillations) - Trait-based search — θ encodes organism traits, not just rate multipliers - Target search — CMA-ES optimization toward text prompts via CLIP +- Surrogate model + sensitivity analysis over ~48-dim sim_config parameter space **Medium-term:** +- Hybrid BYOM — extend pluggable adapters from motor inference to full simulation phases (learned diffusion, behavior bias, narrative control) - Base/fiber architecture for multi-observer worlds — consensus on causal facts, private performance, reconciliation as homing intent - Searchable physics — allometric exponents as θ dimensions - Godot 3D client with latent-driven skeletal animation - Open-ended search — temporal novelty across simulation rollouts +- Multi-client support — spectator mode via read-only WebSocket ## Contributing