render: a Deploy to Render button for a metrics pipeline

.github/workflows/render-template.yml

@@ -0,0 +1,35 @@
+name: render-template
+
+# The Deploy to Render template under render/. Runs when it, the Blueprint or
+# this file changes.
+on:
+  push:
+    branches: [main]
+    paths: ["render/**", "render.yaml", ".github/workflows/render-template.yml"]
+  pull_request:
+    paths: ["render/**", "render.yaml", ".github/workflows/render-template.yml"]
+
+jobs:
+  validate:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+      - name: Configs, the generated migration, and render.yaml against Render's schema
+        run: make -C render validate
+      - name: A render.yaml that Render would refuse is refused
+        run: |
+          sed 's/plan: 0.5c-512mb/plan: not-a-plan/' render.yaml > "$RUNNER_TEMP/bad.yaml"
+          status=0
+          render/bin/validate-render.py "$RUNNER_TEMP/bad.yaml" || status=$?
+          if [ "$status" != 1 ]; then
+            echo "expected status 1 for an invalid plan, got $status" >&2
+            exit 1
+          fi
+
+  e2e:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Post metrics and read them back at every grain
+        run: make -C render test

.gitignore

@@ -5,6 +5,8 @@ __pycache__/
 .venv/
 dev/volume/
 bin/
+# The Deploy to Render template's scripts are source, not build output.
+!/render/bin/
 dist/
 .claude/
 

CHANGELOG.md

@@ -42,6 +42,9 @@
 
 ### Added
 
+- A Deploy to Render button. `render.yaml` and `render/` deploy a Postgres, a
+  webhook pipeline that aggregates a generic metric by minute, and a
+  `sqlflow serve` API over a six-grain rollup ladder. See `render/README.md`.
 - `GET /healthz` on the webhook source's address. It answers 200 while the
   source admits deliveries and 503 once it is closing, needs no signature,
   and is not counted in `webhook_requests_total`. A platform that routes one

README.md

@@ -11,6 +11,12 @@ SQLFlow is a stream processing engine that lets you define pipelines with just S
 - [Coverage and invariant matrix](docs/coverage/matrix.md): what is tested, and
   separately, what is *proven* — regenerated from the suites on every push.
 
+[![Deploy to Render](https://render.com/images/deploy-to-render-button.svg)](https://render.com/deploy?repo=https://github.com/turbolytics/sql-flow)
+
+One click runs sqlflow on Render: a webhook that accepts metrics, a one-minute
+aggregation into Postgres, and an HTTP API that reads them back. It asks for
+one secret and is a paid deploy. See [`render/`](render/README.md).
+
 # Quick Start (Getting Started in 5 Minutes)
 
 1. Build a binary (see [Installation](#installation) for prebuilt binaries and Docker):

docs/coverage/features.yml

@@ -202,6 +202,12 @@ features:
     description: Generates rollup tables, the triggers that keep them current, and the serve datasets that read them from one declaration, and checks the generated files have not drifted.
     requires: [unit, integration]
 
+  # The template is YAML, SQL and shell, so what it promises under concurrency
+  # is tested from Go against its migrations, in internal/rendertemplate.
+  - id: template.render
+    description: The Deploy to Render template's schema keeps the minute table and every rollup exact when several pipeline processes write to one database.
+    requires: [integration]
+
   # Release only. The defect this catches -- build args that never reached the
   # -ldflags symbol path, so the binary reports "dev" -- exists only in a built
   # image, and no unit test can see it.

docs/coverage/invariants.yml

@@ -538,6 +538,36 @@ invariants:
     requires: []
     enforced: true
 
+  # --- Resilience: more than one process writing one destination -------------
+  # The engine gives each pipeline process its own window, and a windowed
+  # pipeline publishes a closed minute by replacing the row for its key. That
+  # is right for one process and wrong for two: instances behind a load
+  # balancer each hold part of a minute, and keyed on the series alone the
+  # second to publish replaces the first. Two real instances, sent 7 and 5 of
+  # one minute, stored 7 (#331). Two instances also run during every deploy,
+  # the old beside the new.
+  #
+  # Nothing in the engine prevents it, so the claim is about what a destination
+  # ends up holding, and it is proven where a destination's schema makes it
+  # true: the Deploy to Render template keys its write on the process and has
+  # Postgres merge the processes. A pipeline configuration with no such schema
+  # has no evidence here, and that empty cell is accurate.
+  - id: pipeline.writers.merge_exactly
+    family: resilience
+    class: safety
+    applies_to: pipeline
+    claim: >
+      When several pipeline processes publish the same window of the same key
+      to one destination, the destination holds the merge of what every
+      process last published: no process's rows replace another's, and a
+      process that republishes replaces only its own. It holds at every grain
+      derived from that window, under concurrent publishes, without deadlock.
+      Proven in internal/rendertemplate against the Deploy to Render
+      template's migrations and the Postgres sink, not by the conformance
+      harness: the property belongs to the destination's schema.
+    verified_by: harness
+    requires: []
+
   # --- Cache: serve's response cache ----------------------------------------
   # Serve is never told that its backend changed, so the cache invalidates
   # nothing and bounds everything. These say what the bounds are. Each is

docs/coverage/matrix.md

@@ -60,19 +60,20 @@ added, and this page changes only when a status does.
 | `cli.dev_invoke` | Runs a pipeline against a fixture file, without a source. | ✅ | — | ✅ |
 | `cli.serve` | Serves a config's named SQL datasets over HTTP, with client ids, typed params, grains and limits. | ✅ | ✅ | ✅ |
 | `cli.rollup` | Generates rollup tables, the triggers that keep them current, and the serve datasets that read them from one declaration, and checks the generated files have not drifted. | ✅ | ✅ | — |
+| `template.render` | The Deploy to Render template's schema keeps the minute table and every rollup exact when several pipeline processes write to one database. | ⚠️ skipped | ✅ | — |
 | `cli.version` | The shipped binary reports the version it was built from. | — | — | ✅ |
 | `tooling.conformance` | The harness proves the declared invariants for any integration. | ✅ | — | — |
 | `tooling.coverage` | Tests attribute to features and invariants, and the registries match the code. | ✅ | — | — |
 
-**41 features declared. 41 have at least one passing test attributed at every level they require, so 0 gap(s).**
+**42 features declared. 41 have at least one passing test attributed at every level they require, so 0 gap(s).**
 
 That sentence counts attribution, not proof. A feature is green here when
 a test named for it ran and passed; it says nothing about whether the
 integration behind it keeps a batch it could not deliver, or commits
 offsets only after a flush. Those are invariants, they are counted
 separately below, and the two numbers are not interchangeable.
 
-**50 invariants declared: 42 safety and 8 liveness. Of 178 (invariant, integration) cells: 94 proven, 50 missing, 0 skipped, 0 failing, 34 exempt. 0 gap(s).**
+**51 invariants declared: 43 safety and 8 liveness. Of 180 (invariant, integration) cells: 95 proven, 51 missing, 0 skipped, 0 failing, 34 exempt. 0 gap(s).**
 
 Safety says nothing bad happens. Liveness says something good
 eventually does, and the two are not interchangeable: a sink that
@@ -116,6 +117,17 @@ until an invariant's `requires` is filled in, and none is yet.
 | `sink.probe.fails_start` | A Prober whose destination is unreachable fails the start once, without retrying. | ✅ i | — exempt | — exempt | ✅ i | — exempt | ✅ i | — exempt |
 | `sink.flush.idempotent_on_key` | Delivering the same batch twice leaves the destination holding it once. The engine promises at-least-once; a sink that identifies rows by key is what turns that into exactly-once for the reader. A sink that declares no key is exempt, with a proof that it declares none. | — exempt | — exempt | — exempt | — exempt | — exempt | ✅ i | — exempt |
 
+These resilience invariants are properties of the consume loop
+rather than of anything a config file names. The columns are
+its configurations, and `internal/conformance` runs each one
+through every path that reaches a batch: the batch filling, the
+flush interval elapsing, the source closing, and a cancel that
+drains. An invariant holds only if it holds on all four.
+
+| Invariant | Claim | `pipeline.stateful` | `pipeline.stateless` |
+| --- | --- | --- | --- |
+| `pipeline.writers.merge_exactly` | When several pipeline processes publish the same window of the same key to one destination, the destination holds the merge of what every process last published: no process's rows replace another's, and a process that republishes replaces only its own. It holds at every grain derived from that window, under concurrent publishes, without deadlock. Proven in internal/rendertemplate against the Deploy to Render template's migrations and the Postgres sink, not by the conformance harness: the property belongs to the destination's schema. | ❌ missing | ✅ i |
+
 ## Safety invariants: checkpoint
 
 | Invariant | Claim | `source.kafka` | `source.webhook` | `source.websocket` | `manager.watermark` |

docs/coverage/status/features.yml

@@ -37,6 +37,7 @@ source.websocket: {unit: covered, integration: not_required, release: covered}
 state.corruption: {unit: covered, integration: not_required, release: covered}
 state.durability: {unit: covered, integration: not_required, release: covered}
 state.offsets: {unit: covered, integration: not_required, release: covered}
+template.render: {unit: skipped, integration: covered, release: not_required}
 tooling.conformance: {unit: covered, integration: not_required, release: not_required}
 tooling.coverage: {unit: covered, integration: not_required, release: not_required}
 validate.schema: {unit: covered, integration: not_required, release: not_required}

docs/coverage/status/pipeline.stateful.yml

@@ -12,3 +12,4 @@ pipeline.flush.eventually: {unit: covered, integration: missing, release: missin
 pipeline.progress.no_silent_stall: {unit: missing, integration: missing, release: missing}
 pipeline.shutdown.commits_only_delivered: {unit: covered, integration: missing, release: missing}
 pipeline.state.with_offsets: {unit: covered, integration: missing, release: missing}
+pipeline.writers.merge_exactly: {unit: missing, integration: missing, release: missing}

docs/coverage/status/pipeline.stateless.yml

@@ -12,3 +12,4 @@ pipeline.flush.eventually: {unit: covered, integration: missing, release: missin
 pipeline.progress.no_silent_stall: {unit: missing, integration: missing, release: missing}
 pipeline.shutdown.commits_only_delivered: {unit: covered, integration: missing, release: missing}
 pipeline.state.with_offsets: {unit: exempt, integration: exempt, release: exempt}
+pipeline.writers.merge_exactly: {unit: skipped, integration: covered, release: missing}