Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions documentation/changelog.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,28 @@ description: Recent updates and improvements to the QuestDB documentation.

This page tracks significant updates to the QuestDB documentation.

## October 2026

### New

- [ALTER LIVE VIEW RESUME WAL](/docs/query/sql/alter-live-view-resume-wal/) - New reference page for resuming a live view whose own WAL was suspended after an error, and how to find suspended live views through `wal_tables()`

### Reference

- Added the dedicated live view refresh worker pool settings [`live.view.refresh.worker.count`](/docs/configuration/live-views/#liveviewrefreshworkercount), `.affinity`, `.haltOnError`, `.nap.threshold`, `.sleep.threshold`, `.sleep.timeout`, and `.yield.threshold`, plus [`cairo.live.view.checkpoint.purge.interval`](/docs/configuration/live-views/#cairoliveviewcheckpointpurgeinterval). Live views run on their own refresh pool, not the materialized view pool as previously documented
- Added the `checkpoint_seal_failures` column to [`live_views()`](/docs/query/functions/meta/#live_views) and corrected the `lag_seqtxn`, `last_processed_seqtxn`, `lag_micros`, and `writer_stall_micros` descriptions
- Corrected the defaults of [`mat.view.refresh.worker.count`](/docs/configuration/materialized-views/#matviewrefreshworkercount) and [`wal.apply.worker.count`](/docs/configuration/wal/#walapplyworkercount), and documented that `mat.view.refresh.worker.count=0` disables materialized view refresh

### Updated

- [Live views](/docs/concepts/live-views/) - Corrected the [supported window functions](/docs/concepts/live-views/#supported-window-functions) to match QuestDB 10.0.1: the statistical functions, EMA, and VWEMA run only in anchored windows, `ksum` does not support `RANGE` frames, `lag` needs a bounded frame or an anchor, and `VARCHAR`, `UUID`, `LONG256`, `DECIMAL`, `BINARY`, and array partition keys are rejected. Added a [rolling standard deviation](/docs/concepts/live-views/#rolling-standard-deviation) recipe for Bollinger bands, rewrote [Freshness](/docs/concepts/live-views/#freshness) to reflect that interval-filtered reads, aggregates, and `ASOF JOIN` see unflushed rows while `LATEST ON` does not, and added the `UPDATE` and `REBASE WAL` invalidation triggers, base WAL retention, and replica freshness. Documented that an out-of-order base row rewrites view rows already emitted, that `view_status` stays `active` while the view's own WAL is suspended, and what to change in `SHOW CREATE LIVE VIEW` output before replaying it
- [CREATE LIVE VIEW](/docs/query/sql/create-live-view/) - Fixed the `OWNED BY` example, which requires a parenthesized query, documented the designated-timestamp projection requirement and the remaining anchored-window rules, and expanded the error reference with verified messages. Added a grammar for anchored windows and [examples](/docs/query/sql/create-live-view/#examples) of a `RANGE` frame with a `WHERE` filter and an anchored session window
- [Live view configuration](/docs/configuration/live-views/) - Corrected the descriptions of the in-memory buffer growth budget, partition compaction threshold, flush retry budgets, and refresh memory limit
- [Shared workers](/docs/configuration/shared-workers/) - The write pool no longer lists WAL apply and materialized view refresh, which run on [dedicated pools](/docs/configuration/shared-workers/#dedicated-pools) together with live view refresh
- [EMA and VWEMA](/docs/query/functions/window-functions/reference/#avg) - Corrected the frame behavior: both reject a frame clause instead of ignoring it
- [ALTER TABLE RESUME WAL](/docs/query/sql/alter-table-resume-wal/), [Monitoring and alerting](/docs/operations/monitoring-alerting/), and [Write-ahead log](/docs/concepts/write-ahead-log/) - Suspended materialized views and live views now point to their own `RESUME WAL` commands, since `ALTER TABLE` fails on them; [`wal_tables()`](/docs/query/functions/meta/#wal_tables) now lists live views
- [ALTER MATERIALIZED VIEW RESUME WAL](/docs/query/sql/alter-mat-view-resume-wal/) - Corrected the Enterprise permission to `RESUME WAL`

## September 2026

### QuestDB Enterprise Releases
Expand Down
377 changes: 289 additions & 88 deletions documentation/concepts/live-views.md

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion documentation/concepts/views.md
Original file line number Diff line number Diff line change
Expand Up @@ -352,7 +352,8 @@ Use a [live view](/docs/concepts/live-views/) instead when you need to
incrementally maintain a row-per-input window computation, such as a moving
average, running total, or ranking.

The output of a live view is one row per base row.
The output of a live view is one row per base row that passes the view's
`WHERE` filter and `START FROM` boundary.

## Security with views

Expand Down
5 changes: 5 additions & 0 deletions documentation/concepts/write-ahead-log.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,11 @@ If WAL transactions are suspended (rare), resume them:
ALTER TABLE prices RESUME WAL;
```

For a suspended materialized view or live view, use
[`ALTER MATERIALIZED VIEW RESUME WAL`](/docs/query/sql/alter-mat-view-resume-wal/)
or [`ALTER LIVE VIEW RESUME WAL`](/docs/query/sql/alter-live-view-resume-wal/)
instead.

## How WAL works

When data is written to a WAL table:
Expand Down
7 changes: 4 additions & 3 deletions documentation/configuration/cairo-engine.md
Original file line number Diff line number Diff line change
Expand Up @@ -931,8 +931,7 @@ edit `server.conf` and call
materialized view refreshes, and WAL apply batches use the updated limits; work
already running keeps its original limit. A live view acquires its limit when it
is first compiled and keeps it across refreshes, so a reloaded value reaches an
existing view only after a full teardown: invalidation, recreation, or a server
restart.
existing view only after the view is recreated or the server restarts.

When a workload exceeds its limit, QuestDB raises an out-of-memory error at the
allocation that crossed the line and aborts that workload, while unrelated
Expand Down Expand Up @@ -1051,7 +1050,9 @@ you can reproduce as a plain query and measure the same way:
the view retains between refreshes: the `IN MEMORY` tier, whose capacity
[`live_views()`](/docs/query/functions/meta/#live_views) reports in its
`in_mem_bytes` column, and the window state. Add these to the transient
figure, and keep the limit above the
figure. Window state grows with the number of distinct partition keys the
view has seen and does not shrink when keys go quiet, so leave headroom for
key growth, and keep the limit above the
[allocation floor](/docs/configuration/live-views/#cairoliveviewrefreshmemorylimitbytes)
described with the key.

Expand Down
174 changes: 136 additions & 38 deletions documentation/configuration/live-views.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
---
title: Live views
description: Configuration settings for live views in QuestDB.
description:
"Live view server settings: refresh worker pool size, per-view memory limit,
in-memory tier buffers, checkpoints, and out-of-order repair budgets."
---

These settings control live view SQL support and the background refresh job that
maintains live views incrementally. For a conceptual overview, see
[Live views](/docs/concepts/live-views/).
These settings control live view SQL support, the refresh worker pool, and the
background refresh job that maintains live views incrementally. For a conceptual
overview, see [Live views](/docs/concepts/live-views/).

Live view refresh shares the materialized view refresh worker pool, so the
worker-pool settings under
[Materialized views](/docs/configuration/materialized-views/)
(`mat.view.refresh.worker.count`, `.affinity`, `.haltOnError`) also govern live
view refresh. There are no dedicated live-view worker-pool properties.
Live view refresh runs on its own worker pool, separate from the
[materialized view](/docs/configuration/materialized-views/) refresh pool. Size
it with [`live.view.refresh.worker.count`](#liveviewrefreshworkercount).

## cairo.live.view.checkpoint.compaction.interval

Expand All @@ -20,7 +20,10 @@ view refresh. There are no dedicated live-view worker-pool properties.

Number of checkpoint seals between physical compaction attempts. Compaction
repacks live state pages from sparse data segments so obsolete segments can be
reclaimed. A value of `0` disables compaction.
reclaimed. A value of `0` disables compaction. Without compaction, a superseded
page is reclaimed only once its whole segment becomes unreferenced, so dead
bytes left behind by out-of-order repairs can accumulate. Enable it for views
that receive frequent out-of-order corrections.

## cairo.live.view.checkpoint.max.duration.micros

Expand All @@ -31,6 +34,17 @@ Maximum wall-clock interval, in microseconds, between head-checkpoint writes.
This caps restart and out-of-order replay work for low-rate views that do not
reach the row-count trigger.

## cairo.live.view.checkpoint.purge.interval

- **Default**: `1`
- **Reloadable**: no

Number of checkpoint seals between sweeps that delete checkpoint segments no
longer referenced by any checkpoint generation. A value of `0` disables the
periodic sweep. Unreferenced segments are then reclaimed only by the sweep that
runs once per server start, so their disk space is not released until a
restart.

## cairo.live.view.checkpoint.repair.replay.max.rows

- **Default**: `1000000`
Expand Down Expand Up @@ -65,53 +79,70 @@ limit is crossed, QuestDB uses the conservative unlocalized bound. A value of
- **Default**: `1000000`
- **Reloadable**: no

Number of newly flushed rows after which the refresh worker writes a head
checkpoint. Smaller values shorten restart replay at the cost of more checkpoint
writes.
Number of live view output rows applied since the previous head checkpoint after
which the refresh worker writes a new one. Smaller values shorten restart and
out-of-order replay at the cost of more checkpoint writes. The value is also the
per-turn row budget of the initial seed sweep, so lowering it also shortens each
seed turn.

## cairo.live.view.enabled

- **Default**: `true`
- **Reloadable**: no

Enables or disables SQL support and the refresh job for live views. When
disabled, `CREATE LIVE VIEW` fails with `live views are disabled`.
disabled, `CREATE LIVE VIEW` fails with `live views are disabled`, and existing
live views are neither refreshed nor listed in
[`live_views()`](/docs/query/functions/meta/#live_views). They stay queryable as
plain tables without their in-memory tier, and the base table's WAL is no longer
retained on their behalf. Re-enabling live views rebuilds each existing view
from the applied base table without invalidating it.

## cairo.live.view.flush.retry.max

- **Default**: `5`
- **Reloadable**: no

Maximum number of consecutive flush attempts before a view is marked invalid. A
flush persists the in-memory rows to the view's disk tier.
Maximum number of consecutive failed refresh cycles, such as a failed flush to
the view's disk tier, before the view is marked invalid. The view is invalidated
as soon as either this count or
[`cairo.live.view.flush.retry.max.duration.micros`](#cairoliveviewflushretrymaxdurationmicros)
is exhausted. Failures caused by a transiently missing base table count only
against the duration budget. A breach of
[`cairo.live.view.refresh.memory.limit.bytes`](#cairoliveviewrefreshmemorylimitbytes)
invalidates the view without any retry.

## cairo.live.view.flush.retry.max.duration.micros

- **Default**: `60000000` (60 seconds)
- **Reloadable**: no

Maximum total time, in microseconds, spent retrying a stalled flush before the
view is marked invalid.
Maximum time, in microseconds, that a view may keep failing refresh cycles
before it is marked invalid. See
[`cairo.live.view.flush.retry.max`](#cairoliveviewflushretrymax) for how the two
budgets interact.

## cairo.live.view.in.memory.buffer.growth.bytes

- **Default**: `16777216` (16 MiB)
- **Reloadable**: no

Fast-path growth budget for the in-memory tier. Once the published buffer's
footprint reaches this size, refresh falls back to a swap that evicts expired
rows and may shrink the buffer instead of continuing to append in place. Raise
it for `IN MEMORY` windows expected to exceed the default. A value of `0` or
less forces this compaction path on every publish. Accepts a size suffix such as
`16M`.
Bytes of aged rows that the in-memory tier may hold before a refresh compacts
it. Aged rows are rows already flushed to disk and older than the view's
`IN MEMORY` window. Until this many bytes of aged rows accumulate, refresh keeps
appending in place; once they do, it swaps to a compacted buffer that evicts
them. The value therefore bounds how much RAM the tier holds beyond the
`IN MEMORY` window. A value of `0` or less compacts on every publish. Accepts a
size suffix such as `16M`.

## cairo.live.view.in.memory.buffer.initial.bytes

- **Default**: `65536` (64 KiB)
- **Reloadable**: no

Initial size of a live view's in-memory tier buffer. Accepts a size suffix such
as `64K`.
Initial size of each column buffer in the in-memory tier. The tier keeps two
buffer slots, and each slot holds one buffer per output column. Accepts a size
suffix such as `64K`.

## cairo.live.view.in.memory.max

Expand All @@ -126,30 +157,40 @@ Upper bound on the `IN MEMORY` retention window. A `CREATE LIVE VIEW` whose
- **Default**: `100000`
- **Reloadable**: no

Anchor-map tombstone threshold that triggers compaction. Compaction also runs
when tombstones exceed half of the anchor map, regardless of this absolute
threshold.
Threshold for the sweep that drops anchored-window partition state left behind
the previous anchor boundary. The sweep runs only when all of the following
hold: the anchor map has more partitions than this value, at least this many of
them are reclaimable, and the reclaimable partitions make up at least half of
the map. Only anchors that return `TIMESTAMP` and advance with the designated
timestamp compact. An anchor value that is not monotonic or is `NULL`, and any
`ANCHOR EXPRESSION` that returns `LONG` or `INT`, never compacts.

## cairo.live.view.refresh.memory.limit.bytes

- **Default**: `0` (unlimited)
- **Reloadable**: yes

Per-live-view limit on the peak memory used during a refresh cycle. It includes
persistent window state, the `IN MEMORY` recent-row tier, staging memory, and
transient buffers such as Parquet row-group decode buffers. Exceeding the limit
invalidates the view immediately; recovery requires dropping and recreating it.
Per-live-view limit on tracked native memory. It covers persistent window state,
the `IN MEMORY` recent-row tier, staging memory, and transient buffers such as
Parquet row-group decode buffers. Exceeding the limit invalidates the view
immediately; recovery requires dropping and recreating it. Process-wide
out-of-memory errors are not limit breaches and are retried instead.

This is one of the four workload
[memory limits](/docs/configuration/cairo-engine/#memory-limits) and counts
tracked native allocations only. A view acquires the limit when it is first
compiled and keeps it across refreshes, so a reloaded value reaches an existing
view only after invalidation, recreation, or a server restart.
compiled and keeps it until it is dropped or invalidated, so the limit applies
to the view's running footprint across refreshes rather than to a single cycle.
Persistent state grows with the number of distinct partition keys the view has
seen and does not shrink when keys go quiet, and the in-memory tier does not
shrink after a burst. A limit sized from an early measurement can therefore be
breached later without any change in workload. A reloaded value reaches an
existing view only after it is recreated or the server restarts.

Size the limit above the refresh workload's allocation floor. In particular, a
bounded `ROWS` frame allocates at least one `cairo.sql.window.store.page.size`
page (1 MiB by default), and a Parquet-backed refresh may decode a complete row
group.
`RANGE` frame, or a `ROWS` frame or `lag()` with `PARTITION BY`, allocates at
least one `cairo.sql.window.store.page.size` page (1 MiB by default) on its
first partition, and a Parquet-backed refresh may decode a complete row group.

## cairo.live.view.refresh.turn.max.commits

Expand All @@ -166,3 +207,60 @@ before yielding to other views.

Maximum wall-clock time, in microseconds, a refresh worker spends on one view
per turn before yielding.

## live.view.refresh.worker.affinity

- **Default**: none
- **Reloadable**: no

Comma-separated list of CPU core indexes, one per refresh worker.

## live.view.refresh.worker.count

- **Default**: `2` on hosts with up to 8 CPU cores, `3` on up to 32 cores, `4`
on larger hosts
- **Reloadable**: no

Number of worker threads that refresh live views. A single view is refreshed by
one worker at a time, so more workers help when several views are busy at once,
not when one view is slow.

A value of `0` turns live view refresh off. `CREATE LIVE VIEW` then fails, and
existing views behave as described for
[`cairo.live.view.enabled`](#cairoliveviewenabled) set to `false`.

## live.view.refresh.worker.haltOnError

- **Default**: `false`
- **Reloadable**: no

Flag that indicates if a refresh worker thread must stop when an unexpected
error occurs.

## live.view.refresh.worker.nap.threshold

- **Default**: `7000`
- **Reloadable**: no

Number of idle worker-loop iterations before a refresh worker naps.

## live.view.refresh.worker.sleep.threshold

- **Default**: `10000`
- **Reloadable**: no

Number of idle worker-loop iterations before a refresh worker sleeps.

## live.view.refresh.worker.sleep.timeout

- **Default**: `10ms`
- **Reloadable**: no

Sleep duration when a refresh worker has no views to refresh.

## live.view.refresh.worker.yield.threshold

- **Default**: `1000`
- **Reloadable**: no

Number of idle worker-loop iterations before a refresh worker yields.
21 changes: 14 additions & 7 deletions documentation/configuration/materialized-views.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@ description:
---

These settings control materialized view SQL support and the background refresh
job. Materialized views can use dedicated worker threads or share the server's
common pool.
job. Materialized views refresh on a dedicated worker pool sized by
[`mat.view.refresh.worker.count`](#matviewrefreshworkercount); `0` disables
refresh.

To cap the native memory a single refresh may allocate, see
[`cairo.mat.view.refresh.memory.limit.bytes`](/docs/configuration/cairo-engine/#memory-limits).
Expand Down Expand Up @@ -72,18 +73,24 @@ has no effect; deferred out-of-memory retries use this backoff.

## mat.view.refresh.worker.affinity

- **Default**: equal to the CPU core count
- **Default**: none
- **Reloadable**: no

Comma-separated list of numerical CPU core indexes.
Comma-separated list of numerical CPU core indexes, one per refresh worker.

## mat.view.refresh.worker.count

- **Default**: `0`
- **Default**: `2` on hosts with up to 8 CPU cores, `3` on up to 32 cores, `4`
on larger hosts. This is the same CPU-based default as
[`wal.apply.worker.count`](/docs/configuration/wal/#walapplyworkercount);
setting that key does not change this one.
- **Reloadable**: no

Number of dedicated worker threads assigned to refresh materialized views.
When `0`, uses the shared worker pool.
Number of dedicated worker threads assigned to refresh materialized views. A
value of `0` disables materialized view refresh: `CREATE MATERIALIZED VIEW`
still succeeds, but nothing refreshes the view. Live views use their own pool,
sized by
[`live.view.refresh.worker.count`](/docs/configuration/live-views/#liveviewrefreshworkercount).

## mat.view.refresh.worker.haltOnError

Expand Down
Loading
Loading