Skip to content

docs(live-views): align live view docs with QuestDB 10.0.1 - #580

Merged
javier merged 4 commits into
mainfrom
puzpuzpuz_fix_lv_docs
Oct 15, 2026
Merged

javier merged 4 commits into
mainfrom
puzpuzpuz_fix_lv_docs

Conversation

@puzpuzpuz

@puzpuzpuz puzpuzpuz commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The live view docs listed variance, stddev, covariance and correlation as supported, but 10.0.1 rejects them with bounded frames (incremental snapshot is not supported for this function yet). This PR fixes that and the other mismatches found by auditing every live view page, plus the pages that link to them, against the 10.0.1 code and a running 10.0.1 server.

Window functions (concepts, CREATE LIVE VIEW)

  • The statistical functions, EMA and VWEMA work only in anchored windows. This replaces the flat list with a function × frame table.
  • ksum rejects RANGE frames.
  • lag needs either a bounded frame or an anchor.
  • VARCHAR, UUID, LONG256, DECIMAL, BINARY and array partition keys are rejected. Casting a VARCHAR key to SYMBOL avoids this.
  • count does not accept BINARY or array arguments.
  • Adds a rolling standard deviation recipe for Bollinger bands, built from bounded avg(). It matches stddev_pop on test data, and the sample variant is given as SQL because n / (n - 1) is integer division in QuestDB.
  • EMA and VWEMA reject a frame clause everywhere, not only in live views. The window function reference said they ignore it.

Freshness and lifecycle

  • The "disk tier only" list was mostly wrong. Interval-filtered reads, projections, aggregates and ASOF/LT JOIN right-hand sides all see unflushed rows.
  • The reads that do lag are LATEST ON, views over dedup base tables, and views on Enterprise replicas, which have no in-memory lead.
  • The view is not append-only: an out-of-order base row rewrites view rows already emitted, so polling for rows newer than the last one seen misses corrections.
  • New invalidation triggers documented: UPDATE on the base table (even of unreferenced columns), REBASE WAL, retry budget exhaustion, and columns used only in WHERE.
  • DROP PARTITION/TRUNCATE/TTL on the base table neither reset window state nor remove view rows, so dropping and recreating the view is the only way to reclaim its disk space.
  • Base WAL is retained for lagging views.
  • The recovery recipe now covers replaying SHOW CREATE LIVE VIEW output: START FROM NOW re-resolves, a trailing ; breaks the replay, and the Enterprise OWNED BY names the caller.

Monitoring

  • lag_seqtxn is meaningful only for active views; it reads 0 while a view is still seeding.
  • A view whose own WAL is suspended still reports view_status = 'active'. The concepts page and live_views() now say so and show how to find such views.
  • ALTER TABLE ... RESUME WAL fails on materialized and live views. The ALTER TABLE RESUME WAL page, the monitoring runbook and the WAL concept page now point to the view-specific commands, and wal_tables() lists live views.

Configuration and live_views()

  • Live views have their own refresh pool, live.view.refresh.worker.*. The docs said they share the mat view pool. This PR documents the pool and cairo.live.view.checkpoint.purge.interval.
  • Fixes the descriptions of:
    • the in-memory growth budget
    • the partition compaction threshold, which applies only to TIMESTAMP anchors
    • the retry budgets (either budget invalidates the view)
    • the memory-limit reload behaviour and allocation floor
  • Adds the missing checkpoint_seal_failures column.
  • Corrects lag_seqtxn (counts applied base txns, not committed), last_processed_seqtxn, lag_micros and writer_stall_micros.
  • Fixes the mat.view.refresh.worker.count and wal.apply.worker.count defaults. Both use the same CPU-based default, but neither drives the other. 0 disables mat view refresh rather than using the shared pool.
  • Shared workers: the write pool no longer claims WAL apply and mat view refresh, which run on dedicated pools alongside live view refresh.

SQL reference

  • CREATE LIVE VIEW:
    • The OWNED BY example was broken; the query must be in parentheses. Documents who can be named as owner by non-admin creators.
    • Uses { } grammar and adds a grammar for anchored WINDOW clauses.
    • Documents that the query must project the designated timestamp.
    • Adds the remaining anchor rules.
    • Adds examples for a RANGE frame with a WHERE filter, and an anchored session window with ranking, EMA/VWEMA, lag and a bounded window alongside it.
    • Replaces the error table with messages verified on the server. Some were wrong; for example, a regular view base doesn't hit the WAL error.
  • New ALTER LIVE VIEW RESUME WAL page. The concepts page already referenced the statement. A suspension clears on the view's next flush after a base commit, so only views over an idle base table need it.
  • ALTER MATERIALIZED VIEW RESUME WAL requires the RESUME WAL permission, not ALTER MATERIALIZED VIEW.
  • DROP LIVE VIEW, SHOW CREATE LIVE VIEW and SHOW CREATE DATABASE (LIVE_VIEWS category): corrected error text and Enterprise wording.

Possible engine issues found during the audit (not docs fixes):

  • ALTER LIVE VIEW ... SUSPEND WAL is cleared by the view's next refresh, so it never pauses the view.
  • The "incremental snapshot is not supported" error is also what you get for an unsupported PARTITION BY key type or a missing PARTITION BY. That makes it hard to act on.
  • For row_number(), that error prints the Java class name instead of the function name.
  • SHOW CREATE LIVE VIEW (and mat views) keep the creator's trailing ; inside AS ( ... ), so the output fails to replay with ')' expected.
  • A WHERE-clause subquery over another table (WHERE symbol IN (SELECT ... FROM other)) is accepted. It appears to be re-evaluated on each refresh with no dependency tracking on the other table; it should probably be rejected.
  • o3_rejected_count double-counts: one late row below the START FROM boundary moved it from 0 to 2.
  • Unconfirmed: invalidation_reason may be overwritten by later base DDL, and window inheritance (w2 AS (w ROWS 3 PRECEDING)) works in a plain SELECT but fails in a live view.

🤖 Generated with Claude Code

puzpuzpuz and others added 2 commits October 12, 2026 10:39
Audit the live view pages against the 10.0.1 release and fix the
mismatches:

- Statistical window functions, EMA and VWEMA are only accepted in
  anchored windows; bounded ROWS/RANGE frames fail with "incremental
  snapshot is not supported". ksum rejects RANGE frames, lag needs a
  bounded frame or an anchor, and VARCHAR, UUID, LONG256 and DECIMAL
  partition keys are rejected. Add a rolling standard deviation
  (Bollinger bands) recipe built from bounded avg().
- Freshness: interval-filtered reads, aggregates and ASOF JOIN see
  unflushed rows; LATEST ON, dedup-base views and replica views do not.
- Live views have their own refresh worker pool
  (live.view.refresh.worker.*), not the mat view pool. Document the pool
  and cairo.live.view.checkpoint.purge.interval, and fix the growth
  budget, compaction threshold, retry budget and memory limit
  descriptions.
- live_views(): add checkpoint_seal_failures, fix lag_seqtxn,
  last_processed_seqtxn, lag_micros and writer_stall_micros.
- CREATE LIVE VIEW: fix the OWNED BY example (query must be
  parenthesized), document the designated timestamp projection rule and
  the remaining anchor rules, and rewrite the error tables with the
  verified messages.
- Add an ALTER LIVE VIEW RESUME WAL page, add the UPDATE and REBASE WAL
  invalidation triggers, and correct the mat view and WAL apply worker
  count defaults.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The statement is intentionally undocumented.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

🚀 Build success!

Latest successful preview: https://preview-580--questdb-documentation.netlify.app/docs/

Commit SHA: 93559fb

📦 Build generates a preview & updates the link on each commit.

Concepts:
- Say that out-of-order base rows rewrite view rows already emitted
- Monitoring: lag is meaningful only for active views; a suspended view
  WAL still shows active, with a query to find such views
- Give the sample stddev correction as SQL (integer division trap)
- Retention: dropping and recreating is the only way to reclaim space
- Scope the subquery/CTE rule to FROM; recovery recipe caveats for
  START FROM NOW, trailing semicolons, and Enterprise OWNED BY

SQL reference:
- CREATE LIVE VIEW: { } grammar, anchored WINDOW grammar, RANGE/WHERE
  and anchored session examples, corrected error causes, new rows,
  OWNED BY rules for non-admin creators
- RESUME WAL: precise description and parameters, self-clearing
  suspension on the next flush after a base commit
- Point ALTER TABLE RESUME WAL runbooks to the view-specific commands
- EMA/VWEMA reject a frame clause rather than ignoring it
- Mat view RESUME WAL needs the RESUME WAL permission

Configuration:
- Mat view and shared worker pool descriptions match 10.0.1 dedicated
  pools and CPU-based defaults
- Anchor compaction applies only to TIMESTAMP anchors

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@javier
javier merged commit 130fd1b into main Oct 15, 2026
3 checks passed
@javier
javier deleted the puzpuzpuz_fix_lv_docs branch October 15, 2026 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants