docs: correct outdated Nightly contributor guide content

.gitignore

@@ -35,3 +35,6 @@ static/skills/
 
 # Used by Codex to store temporary artifacts
 tmp/
+
+# Local MCP server configuration. Contains connection credentials.
+/.mcp.json

docs/contributor-guide/datanode/data-persistence-indexing.md

@@ -5,19 +5,23 @@ description: Explanation of data persistence and indexing in GreptimeDB, includi
 
 # Data Persistence and Indexing
 
-Similar to all LSMT-like storage engines, data in MemTables is persisted to durable storage, for example, the local disk file system or object storage service. GreptimeDB adopts [Apache Parquet][1] as its persistent file format.
+Like other LSM-tree storage engines, GreptimeDB persists data from memtables to durable storage such as a local filesystem or object storage. It uses [Apache Parquet][1] as the persistent file format.
 
 ## SST File Format
 
 Parquet is an open source columnar format that provides fast data querying and has already been adopted by many projects, such as Delta Lake.
 
-Parquet has a hierarchical structure like "row groups-columns-data pages". Data in a Parquet file is horizontally partitioned into row groups, in which all values of the same column are stored together to form a data page. Data page is the minimal storage unit. This structure greatly improves performance.
+Parquet organizes data as row groups, column chunks, and pages. A row group contains one column chunk for each column, and each column chunk contains one or more pages. Pages are the units of encoding and compression; column chunks are the I/O units for reading selected columns.
 
 First, clustering data by column makes file scanning more efficient, especially when only a few columns are queried, which is very common in analytical systems.
 
-Second, data of the same column tends to be homogeneous which helps with compression when apply techniques like dictionary and Run-Length Encoding (RLE).
+Second, values within a column tend to be similar, which improves compression with techniques such as dictionary encoding and run-length encoding (RLE).
 
-<img src="/parquet-file-format.png" alt="Parquet file format" width="500"/>
+The following diagram from the Apache Parquet specification also shows the physical file layout: column chunks are stored by row group, while file metadata and its length are written in the footer.
+
+<img src="/parquet-file-layout.gif" alt="Apache Parquet file layout" width="601"/>
+
+*Source: Apache Parquet [FileLayout.gif](https://github.com/apache/parquet-format/blob/master/doc/images/FileLayout.gif). Copyright 2014 The Apache Software Foundation, licensed under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0).*
 
 ## Data Persistence
 
@@ -27,17 +31,17 @@ When the size of data buffered in MemTables reaches that threshold, GreptimeDB w
 
 ## Indexing Data in SST Files
 
-Apache Parquet file format provides inherent statistics in headers of column chunks and data pages, which are used for pruning and skipping.
+Parquet stores row-group column statistics such as minimum, maximum, and null count in each column chunk's metadata. Page metadata and optional column indexes can provide finer-grained statistics.
 
-<img src="/column-chunk-header.png" alt="Column chunk header" width="350"/>
+![A name predicate uses Parquet column statistics to skip one row group while retaining another as a read candidate.](/parquet-row-group-statistics.svg)
 
-For example, in the above Parquet file, if you want to filter rows where `name` = `Emily`, you can easily skip row group 0 because the max value for `name` field is `Charlie`. This statistical information reduces IO operations.
+For example, a query filtering for `name` = `Emily` can skip row group 0 because the maximum `name` value is `Charlie`. This avoids reading that row group.
 
 ## Index Files
 
-For each SST file, GreptimeDB not only maintains an internal index but also generates a separate file to store the index structures specific to that SST file.
+When an SST has one or more configured index outputs, GreptimeDB writes them to a Puffin file associated with that SST. An SST with no applicable index does not need a Puffin file.
 
-The index files utilize the [Puffin][3] format, which offers significant flexibility, allowing for the storage of additional metadata and supporting a broader range of index structures.
+Puffin provides a container for index blobs and their metadata, allowing different index structures to share one file.
 
 ![Puffin](/puffin.png)
 
@@ -57,13 +61,13 @@ The inverted index enables GreptimeDB to skip data segments that do not meet que
 
 ![Inverted index searching](/inverted-index-searching.png)
 
-For instance, the query above uses the inverted index to identify data segments where `job` equals `apiserver`, `handler` matches the regex `.*users`, and `status` matches the regex `4...`. It then scans these data segments to produce the final results that meet all conditions, significantly reducing the number of IO operations.
+The query above uses the inverted index to identify data segments where `job` equals `apiserver`, `handler` matches `.*users`, and `status` matches `4..`. It scans only those segments before applying the remaining filters.
 
 ### Inverted Index Format
 
-![Inverted index format](/inverted-index-format.png)
+![An inverted-index blob contains one index per column followed by footer metadata; each column index contains a null bitmap, posting bitmaps, and an FST.](/inverted-index-blob-layout.svg)
 
-GreptimeDB builds inverted indexes by column, with each inverted index consisting of an FST and multiple Bitmaps.
+GreptimeDB builds inverted indexes by column. Each column index contains a null bitmap, multiple posting bitmaps, and an FST. The blob footer records the offsets, sizes, and metadata needed to locate and decode the column indexes.
 
 The FST (Finite State Transducer) enables GreptimeDB to store mappings from column values to Bitmap positions in a compact format and provides excellent search performance and supports complex search capabilities (such as regular expression matching). The Bitmaps maintain a list of data segment IDs, with each bit representing a data segment.
 
@@ -77,7 +81,7 @@ The number of rows in a data segment is controlled by the engine option `index.i
 
 ## Unified Data Access Layer: OpenDAL
 
-GreptimeDB uses [OpenDAL][2] to provide a unified data access layer, thus, the storage engine does not need to interact with different storage APIs, and data can be migrated to cloud-based storage like AWS S3 seamlessly.
+GreptimeDB uses [OpenDAL][2] to provide a common access layer for local filesystems and object stores. Changing the configured storage backend does not migrate existing data.
 
 [1]: https://parquet.apache.org
 [2]: https://github.com/datafuselabs/opendal

docs/contributor-guide/datanode/memtable.md

@@ -0,0 +1,100 @@
+---
+keywords: [memtable, Mito engine, write buffer, flush, time partition, BulkMemtable]
+description: How Mito organizes mutable Region data in memtables and moves it into SST files.
+---
+
+# Memtable design
+
+A memtable is Mito's in-memory write buffer for a Region. It makes writes available to reads before a flush creates SST files. A Region version identifies the memtables and SST files that a scan may read. Together with a committed-sequence fence, it keeps the scan consistent while writes and flushes advance the current version.
+
+## Write and flush lifecycle
+
+For a normal WAL-backed write, Mito uses this order:
+
+```text
+write request
+     |
+     v
+WAL append -> mutable memtable -> publish committed sequence
+                   |
+                 freeze
+                   v
+            immutable memtable -> SST write -> manifest edit
+```
+
+The Region worker assigns sequence numbers and a WAL entry ID before appending the mutation to the [write-ahead log](wal.md). If the WAL append fails, Mito does not update the memtable. After the memtable update succeeds, Mito publishes the committed sequence and the rows become visible to new reads. A Region configured with `skip_wal` omits the WAL append, but keeps the same memtable and visibility ordering.
+
+A flush freezes the mutable memtables and installs a new mutable set before starting the background SST write. New writes therefore continue without changing the frozen data. The flush writes the immutable memtables to SST files, then persists a manifest edit containing the files and the flushed WAL and sequence checkpoints. Only after that edit is durable does Mito remove the flushed memtables from the current Region version. If the flush fails, the immutable memtables remain available for a later attempt.
+
+## Region versions and time partitions
+
+Each Region has one mutable `TimePartitions` container, which can hold more than one memtable:
+
+```text
+Region version
+├─ mutable TimePartitions
+│  ├─ [t0, t1) -> memtable
+│  └─ [t1, t2) -> memtable
+├─ immutable memtables
+└─ SST files
+```
+
+Mito routes each row to a partition by its time-index value. Partition ranges are half-open and aligned to a fixed duration. The duration follows the Region's compaction time window; Mito uses one day until a compaction window is available. An out-of-order write can create an earlier partition alongside the latest one.
+
+Freezing a Region freezes all mutable time partitions together. Mito moves their memtables to the immutable list and creates a new `TimePartitions` container. A failed flush can leave more than one generation of immutable memtables, so reads and later flushes must not assume that the list contains a single item.
+
+## Memtable implementations
+
+Mito selects a memtable implementation from the Region's SST format, primary-key encoding, and memtable options:
+
+```text
+flat SST format (the default) or sparse primary-key encoding -> BulkMemtable
+memtable.type=bulk                                           -> BulkMemtable, and forces flat SST
+primary_key SST with dense encoding (legacy)                 -> a legacy implementation
+```
+
+With the default engine configuration, a Region without an explicit SST format uses `flat`, so `BulkMemtable` is the normal path and the rest of this page describes it. The rules exist to prevent incompatible combinations: flat format or sparse primary-key encoding requires `BulkMemtable`, and explicitly selecting the bulk implementation forces flat format.
+
+### BulkMemtable
+
+`BulkMemtable` stores writes as parts in the flat Arrow layout instead of inserting rows into per-series buffers:
+
+```text
+BulkMemtable
+├─ unordered_part
+│  └─ small BulkPart batches
+└─ parts
+   ├─ BulkPart        (Arrow RecordBatch)
+   ├─ MultiBulkPart   (raw RecordBatches)
+   └─ EncodedBulkPart (in-memory Parquet)
+```
+
+Small parts accumulate in `unordered_part`; larger parts enter `parts` directly. Background memtable compaction merge-sorts eligible parts into a `MultiBulkPart` or encodes them as an `EncodedBulkPart`. Scans use part statistics to prune ranges, and flush can write encoded ranges to SST without decoding and encoding the rows again. For the design rationale and performance results, see [Scaling Time Series to Millions of Cardinalities: GreptimeDB's Flat Format](https://greptime.com/blogs/2025-12-22-flat-format).
+
+### Legacy implementations
+
+Regions on the legacy `primary_key` SST format with dense primary-key encoding still use `TimeSeriesMemtable`, which groups rows by encoded primary key rather than storing flat parts. A Region with no primary-key columns gets `SimpleBulkMemtable` from the same builder. Both are compatibility code for existing tables and may be removed once the `primary_key` format is retired; new work targets the bulk and flat path.
+
+The removed `partition_tree` memtable is not a third implementation. The option parser accepts `memtable.type=partition_tree` for compatibility, but it does not recreate that implementation. The Region uses the bulk and flat path.
+
+## Read snapshots
+
+A scan obtains the Region version and committed sequence together from `VersionControl`. It selects the version before applying the sequence fence. Reading the sequence separately before the version could pair that sequence with a later version after flush or compaction removes an older input, producing an incomplete snapshot.
+
+The selected version supplies mutable memtables, immutable memtables, and SST files. Mito first prunes sources by time range, then asks each memtable for ranges using the scan's projection, predicate, and sequence bounds. The scan merges the resulting ranges with SST ranges and applies the same ordering, deletion, and merge semantics across all sources. References held by the scan keep an older memtable alive even after a newer Region version removes it.
+
+## Memory pressure
+
+Each memtable tracks its estimated heap allocation through the engine's write-buffer manager. Freezing a memtable removes its allocation from the mutable-memory count, but total usage includes the allocation until all references to that memtable are released. The mutable-memory count therefore tracks data that can still accept writes, while total usage continues to include memory retained by active scans.
+
+The global write-buffer limit causes workers to select Regions for flush. If memory remains above the configured limits, Mito stalls writes and can reject them at a higher threshold. An optional per-Region limit applies the same pressure to one hot Region without stalling unrelated Regions. Periodic, manual, and Region lifecycle operations can also request a flush.
+
+## Constraints for changes
+
+Changes to memtable code must preserve these properties:
+
+- For a WAL-backed Region, append to the WAL before installing rows in a memtable. Publish the committed sequence only after installation succeeds.
+- Keep frozen memtables readable and retryable until the SST files and manifest edit are durable.
+- Obtain the Region version and committed sequence from the same `VersionControl` snapshot; never read the sequence separately before the version.
+- Preserve the ordering and metadata that scans and flushes need to apply the same deletion, deduplication, and merge rules across memtable and SST ranges.
+- Charge allocations to the write-buffer manager and release them only when the underlying memory can no longer be referenced.

docs/contributor-guide/datanode/metric-engine.md

@@ -7,9 +7,9 @@ description: Overview of the Metric engine in GreptimeDB, its concepts, architec
 
 ## Overview
 
-The `Metric` engine is a component of GreptimeDB, and it's an implementation of the storage engine. It mainly targets scenarios with a large number of small tables for observable metrics.
+The `Metric` engine stores workloads with many small metric tables.
 
-Its main feature is to use synthetic physical wide tables to store a large amount of small table data, achieving effects such as reuse of the same column and metadata. This reduces storage overhead for small tables and improves columnar compression efficiency. The concept of a table becomes even more lightweight under the `Metric` engine.
+It maps those logical tables onto shared physical wide tables so they can reuse columns and metadata. This reduces per-table storage overhead and improves columnar compression.
 
 ## Concepts
 
@@ -19,7 +19,7 @@ The `Metric` engine introduces two new concepts: "logical table" and "physical t
 
 A logical table refers to user-defined tables. Just like any other ordinary table, its definition includes the name of the table, column definitions, index definitions etc. All operations such as queries or write-ins by users are based on these logical tables. Users don't need to worry about differences between logical and ordinary tables during usage.
 
-From an implementation standpoint, a logical table is virtual; it doesn't directly read or write physical data but maps read/write requests into corresponding requests for physical tables in order to implement data storage and querying.
+A logical table is virtual. The engine maps its read and write requests to the corresponding physical table instead of storing data for it directly.
 
 ### Physical Table
 
@@ -29,16 +29,14 @@ A physical table is a table that actually stores data, possessing several physic
 
 The main design architecture of the `Metric` engine is as follows:
 
-![Arch](/metric-engine-arch.png)
+![Multiple logical tables map through the Metric engine to shared data and metadata Regions managed by Mito.](/metric-engine-architecture.svg)
 
-In the current version implementation, the `Metric` engine reuses the `Mito` engine to achieve storage and query capabilities for physical data. It also provides access to both physical tables and logical tables simultaneously.
+The `Metric` engine delegates physical storage and queries to the `Mito` engine. Each physical Region group contains a data Region, which stores rows from its mapped logical tables, and a metadata Region, which stores the logical-table and logical-column mappings.
 
-Regarding partitioning, logical tables have identical partition rules and Region distribution as physical tables. This makes sense because the data of logical tables are directly stored in physical tables, so their partition rules are consistent.
+Logical tables associated with the same physical table share its partition layout. During writes, the engine records the logical table identity with each row. During reads, it adds a logical-table filter before scanning the physical Region.
 
-Concerning routing metadata, the routing address of a logical table is a logical address - what its corresponding physical table is - then through this physical table for secondary routing to obtain the real physical address. This indirect routing method can significantly reduce the number of metadata modifications required when Region migration scheduling occurs in Metric engines.
+A logical table's route stores only the ID of its physical table; the physical table route resolves that to the Datanodes holding the Regions. Because logical routes do not name peers, migrating a physical Region rewrites one physical route instead of every logical route that maps to it.
 
-Operationally speaking, The `Metric` engine supports standard DML operations (INSERT, DELETE, SELECT) on logical tables. However, it only supports limited operations on physical tables to prevent misoperations - for example, writing directly to a physical table is prohibited as it could affect user's logical table data. Generally speaking, users can consider that they have read-only access to these physical tables.
+Logical tables support normal INSERT, DELETE, and SELECT operations. Direct writes to a physical Region are rejected because they would bypass the logical-table mapping; querying a physical table remains supported.
 
-To improve performance during simultaneous DDL (Data Definition Language) operations on many tables, the 'Metric' engine has introduced some batch DDL operations. These batch DDL operations can merge lots of DDL actions into one request thereby reducing queries and modifications times for metadata thus enhancing performance. This feature is particularly beneficial in scenarios such as the automatic creation requests brought about by large amounts of metrics during Prometheus Remote Write cold start-up, as well as the modification requests for numerous route-tables mentioned earlier during migration of many physical regions.
-
-Apart from physical data regions belonging to physical tables, the 'Metric' engine creates an additional metadata region physically for each individual physical data region used in storing some metadata needed by itself while maintaining mapping and other states. This metadata includes the mapping relationship between logical tables and physical tables, the mapping relationship between logical columns and physical columns etc.
+Batch DDL operations reduce metadata work when many logical tables are created or updated together, such as during Prometheus Remote Write auto-creation or physical Region migration.