From 3bb1a6ca1197edf80490462764c5678bee763db1 Mon Sep 17 00:00:00 2001 From: Justin Mclean Date: Thu, 3 Sep 2026 00:26:12 +0000 Subject: [PATCH] docs: add a description to every docs page Only 6 of 76 pages carried a description in frontmatter. Every page without one gives search results, og:description, the docs search snippet and any generated index nothing to show but a title. Each description is one line drawn from the page's own opening, under 130 characters, and says what the page covers rather than restating its title. --- content/docs/ai/introduction.mdx | 1 + content/docs/ai/mcp.mdx | 1 + content/docs/binary-protocol/cluster.mdx | 1 + content/docs/binary-protocol/commands.mdx | 1 + content/docs/binary-protocol/connection-lifecycle.mdx | 1 + content/docs/binary-protocol/encodings.mdx | 1 + content/docs/binary-protocol/framing.mdx | 1 + content/docs/binary-protocol/index.mdx | 1 + content/docs/binary-protocol/messages.mdx | 1 + content/docs/cli/start.mdx | 1 + content/docs/connectors/introduction.mdx | 1 + content/docs/connectors/observability.mdx | 1 + content/docs/connectors/runtime.mdx | 1 + content/docs/connectors/sdk.mdx | 1 + content/docs/connectors/sinks/clickhouse.mdx | 1 + content/docs/connectors/sinks/delta.mdx | 1 + content/docs/connectors/sinks/doris.mdx | 1 + content/docs/connectors/sinks/elasticsearch.mdx | 1 + content/docs/connectors/sinks/http.mdx | 1 + content/docs/connectors/sinks/iceberg.mdx | 1 + content/docs/connectors/sinks/influxdb.mdx | 1 + content/docs/connectors/sinks/meilisearch.mdx | 1 + content/docs/connectors/sinks/mongodb.mdx | 1 + content/docs/connectors/sinks/postgres.mdx | 1 + content/docs/connectors/sinks/quickwit.mdx | 1 + content/docs/connectors/sinks/s3.mdx | 1 + content/docs/connectors/sinks/sink.mdx | 1 + content/docs/connectors/sinks/stdout.mdx | 1 + content/docs/connectors/sinks/surrealdb.mdx | 1 + content/docs/connectors/sources/elasticsearch.mdx | 1 + content/docs/connectors/sources/influxdb.mdx | 1 + content/docs/connectors/sources/postgres.mdx | 1 + content/docs/connectors/sources/random.mdx | 1 + content/docs/connectors/sources/source.mdx | 1 + content/docs/connectors/transforms.mdx | 1 + content/docs/faq/faq.mdx | 1 + content/docs/index.mdx | 1 + content/docs/introduction/about.mdx | 1 + content/docs/introduction/architecture.mdx | 1 + content/docs/introduction/concepts.mdx | 1 + content/docs/introduction/getting-started.mdx | 1 + content/docs/sdk/connection-strings.mdx | 1 + content/docs/sdk/cpp/intro.mdx | 1 + content/docs/sdk/csharp/examples.mdx | 1 + content/docs/sdk/csharp/guide.mdx | 1 + content/docs/sdk/csharp/high-level-sdk.mdx | 1 + content/docs/sdk/csharp/intro.mdx | 1 + content/docs/sdk/go/examples.mdx | 1 + content/docs/sdk/go/intro.mdx | 1 + content/docs/sdk/introduction.mdx | 1 + content/docs/sdk/java/examples.mdx | 1 + content/docs/sdk/java/intro.mdx | 1 + content/docs/sdk/node/examples.mdx | 1 + content/docs/sdk/node/intro.mdx | 1 + content/docs/sdk/php/intro.mdx | 1 + content/docs/sdk/python/examples.mdx | 1 + content/docs/sdk/python/intro.mdx | 1 + content/docs/sdk/rust/examples.mdx | 1 + content/docs/sdk/rust/high-level-sdk.mdx | 1 + content/docs/sdk/rust/intro.mdx | 1 + content/docs/sdk/rust/stream-builder.mdx | 1 + content/docs/server/benchmarking.mdx | 1 + content/docs/server/configuration.mdx | 1 + content/docs/server/docker.mdx | 1 + content/docs/server/introduction.mdx | 1 + content/docs/server/networking.mdx | 1 + content/docs/server/security.mdx | 1 + content/docs/server/storage-engine.mdx | 1 + content/docs/server/topic-options.mdx | 1 + content/docs/web_ui/start.mdx | 1 + 70 files changed, 70 insertions(+) diff --git a/content/docs/ai/introduction.mdx b/content/docs/ai/introduction.mdx index ab113471c3..c6b3243acd 100644 --- a/content/docs/ai/introduction.mdx +++ b/content/docs/ai/introduction.mdx @@ -1,5 +1,6 @@ --- title: Introduction +description: "The AI-related components of Apache Iggy, and where the MCP server is documented." --- This module contains the AI-related components of the Apache Iggy message streaming infrastructure. diff --git a/content/docs/ai/mcp.mdx b/content/docs/ai/mcp.mdx index c53922bd68..8e8e716b0b 100644 --- a/content/docs/ai/mcp.mdx +++ b/content/docs/ai/mcp.mdx @@ -1,5 +1,6 @@ --- title: MCP Server +description: "Run and configure the Apache Iggy MCP server, including transport, Iggy credentials and TLS." --- The [Model Context Protocol](https://modelcontextprotocol.io) (MCP) is an open protocol that standardizes how applications provide context to LLMs. The Apache Iggy MCP Server is an implementation of the MCP protocol for the message streaming infrastructure. It is built using the `rmcp` crate and supports both **stdio** and **HTTP** transports. diff --git a/content/docs/binary-protocol/cluster.mdx b/content/docs/binary-protocol/cluster.mdx index 0d0e2aff00..954af26ad1 100644 --- a/content/docs/binary-protocol/cluster.mdx +++ b/content/docs/binary-protocol/cluster.mdx @@ -1,5 +1,6 @@ --- title: Server-to-server +description: "The replica-to-replica plane: its dedicated TCP port, command discriminants, and traffic that never reaches a client." --- Replicas talk to each other with the same 256-byte [framing](/docs/binary-protocol/framing) the client protocol uses, on a **dedicated TCP port**, with their own set of `command` discriminants. None of these frames ever appears on a client connection, and a client cannot reach this plane: it is a separate listener, gated by a handshake. diff --git a/content/docs/binary-protocol/commands.mdx b/content/docs/binary-protocol/commands.mdx index 5a9687c61c..525805036d 100644 --- a/content/docs/binary-protocol/commands.mdx +++ b/content/docs/binary-protocol/commands.mdx @@ -1,5 +1,6 @@ --- title: Commands +description: "The catalog of command codes, their operation bytes, and which commands are replicated." --- ## Catalog diff --git a/content/docs/binary-protocol/connection-lifecycle.mdx b/content/docs/binary-protocol/connection-lifecycle.mdx index 3c6e49d7fc..8add77d8fe 100644 --- a/content/docs/binary-protocol/connection-lifecycle.mdx +++ b/content/docs/binary-protocol/connection-lifecycle.mdx @@ -1,5 +1,6 @@ --- title: Connection lifecycle +description: "How the protocol version is packed and checked, and the login-register handshake that authenticates a connection." --- ## Protocol version diff --git a/content/docs/binary-protocol/encodings.mdx b/content/docs/binary-protocol/encodings.mdx index aa55d3dece..7f482db489 100644 --- a/content/docs/binary-protocol/encodings.mdx +++ b/content/docs/binary-protocol/encodings.mdx @@ -1,5 +1,6 @@ --- title: Shared encodings +description: "The primitives command payloads are built from: names, identifiers, consumers and the other shared encodings." --- Primitives the [command payloads](/docs/binary-protocol/commands) are built from. diff --git a/content/docs/binary-protocol/framing.mdx b/content/docs/binary-protocol/framing.mdx index ec0fc0801e..e3bf5148a3 100644 --- a/content/docs/binary-protocol/framing.mdx +++ b/content/docs/binary-protocol/framing.mdx @@ -1,5 +1,6 @@ --- title: Framing +description: "The fixed 256-byte header every binary-transport message rides in, and the request, reply and eviction shapes it carries." --- Every message on a binary transport is a fixed **256-byte header** followed by an optional body. The header is a `#[repr(C)]` struct decoded by pointer cast (zero-copy), so field offsets are fixed and enforced at compile time in the server. Three header shapes cross the client boundary: diff --git a/content/docs/binary-protocol/index.mdx b/content/docs/binary-protocol/index.mdx index d37057f822..a97d18a3d9 100644 --- a/content/docs/binary-protocol/index.mdx +++ b/content/docs/binary-protocol/index.mdx @@ -1,5 +1,6 @@ --- title: Binary Protocol +description: "Binary protocol 0.11.0: one wire format across TCP, QUIC and WebSocket, covering both the client and replica planes." --- Iggy speaks one binary protocol over its three binary transports: TCP, QUIC, and WebSocket. Commands, responses, data models, and status codes are the same on all of them. The HTTP transport is separate: it exposes the same operations as JSON REST endpoints, listed in [server.http](https://github.com/apache/iggy/blob/master/core/server/server.http). diff --git a/content/docs/binary-protocol/messages.mdx b/content/docs/binary-protocol/messages.mdx index 12634a88e7..9a7a62ccd5 100644 --- a/content/docs/binary-protocol/messages.mdx +++ b/content/docs/binary-protocol/messages.mdx @@ -1,5 +1,6 @@ --- title: Message batches +description: "The batch layout messages travel and rest in, from the 256-byte batch header down to individual message frames." --- Messages travel and rest in exactly one layout, the **batch**: the `SendMessages` body, the replicated record, the persisted segment record, and the poll reply all share it. There is no other message encoding. diff --git a/content/docs/cli/start.mdx b/content/docs/cli/start.mdx index 327ca1d772..330a431f6c 100644 --- a/content/docs/cli/start.mdx +++ b/content/docs/cli/start.mdx @@ -1,5 +1,6 @@ --- title: Iggy CLI +description: "Install the Iggy CLI, connect to a server, authenticate, and manage connection contexts." --- The Iggy CLI is a command-line interface for managing an Iggy server: streams, topics, partitions, messages, users, tokens and more. It lives in the core repository under the `core/cli` directory. The crate is named `iggy-cli`, the installed binary `iggy`. diff --git a/content/docs/connectors/introduction.mdx b/content/docs/connectors/introduction.mdx index 7a65dc903b..52acead375 100644 --- a/content/docs/connectors/introduction.mdx +++ b/content/docs/connectors/introduction.mdx @@ -1,5 +1,6 @@ --- title: Introduction +description: "The connector runtime: dynamically loaded source and sink plugins, how data flows through them, and what ships today." --- The highly performant and modular runtime for statically typed, yet dynamically loaded connectors. Ingest the data from the external sources and push it further to the Iggy streams, or fetch the data from the Iggy streams and push it further to the external sources. Create your own Rust plugins by simply implementing either the `Source` or `Sink` trait and build custom pipelines for the data processing. diff --git a/content/docs/connectors/observability.mdx b/content/docs/connectors/observability.mdx index 0ca8b65b68..279451156e 100644 --- a/content/docs/connectors/observability.mdx +++ b/content/docs/connectors/observability.mdx @@ -1,5 +1,6 @@ --- title: Observability +description: "Logs, Prometheus metrics, the stats endpoint, OpenTelemetry export and per-source state files for the connector runtime." --- The connector runtime exposes its health through logs, Prometheus metrics, a stats endpoint, OpenTelemetry export, and per-source state files. diff --git a/content/docs/connectors/runtime.mdx b/content/docs/connectors/runtime.mdx index 16a31f6b1a..80cad6e3f0 100644 --- a/content/docs/connectors/runtime.mdx +++ b/content/docs/connectors/runtime.mdx @@ -1,5 +1,6 @@ --- title: Runtime +description: "How the connector runtime loads plugins, resolves its configuration, and manages connector lifecycles." --- Runtime is responsible for managing the lifecycle of the connectors and providing the necessary infrastructure for the connectors to run. diff --git a/content/docs/connectors/sdk.mdx b/content/docs/connectors/sdk.mdx index e7a64ddd61..34091fa4aa 100644 --- a/content/docs/connectors/sdk.mdx +++ b/content/docs/connectors/sdk.mdx @@ -1,5 +1,6 @@ --- title: SDK +description: "The Sink and Source traits, the connector macros, and the decoders and encoders used to read and write Iggy streams." --- SDK provides the commonly used structs and traits such as `Sink` and `Source`, along with the `sink_connector` and `source_connector` macros to be used when developing connectors. diff --git a/content/docs/connectors/sinks/clickhouse.mdx b/content/docs/connectors/sinks/clickhouse.mdx index 618a08c064..0d7a141a7a 100644 --- a/content/docs/connectors/sinks/clickhouse.mdx +++ b/content/docs/connectors/sinks/clickhouse.mdx @@ -1,5 +1,6 @@ --- title: ClickHouse Sink +description: "Insert messages from Iggy streams into ClickHouse tables over the HTTP interface." --- The ClickHouse sink connector consumes messages from Iggy streams and inserts them into ClickHouse tables over the HTTP interface. It supports three insert formats: `json_each_row` (default), `row_binary`, and `string` passthrough for pre-formatted CSV/TSV/JSON payloads. diff --git a/content/docs/connectors/sinks/delta.mdx b/content/docs/connectors/sinks/delta.mdx index a0c569c857..988c6d264f 100644 --- a/content/docs/connectors/sinks/delta.mdx +++ b/content/docs/connectors/sinks/delta.mdx @@ -1,5 +1,6 @@ --- title: Delta Lake Sink +description: "Write messages from Iggy streams into Delta Lake tables on local disk, S3, Azure Blob Storage or Google Cloud Storage." --- The Delta Lake sink connector consumes messages from Iggy streams and writes them to Delta Lake tables on the local filesystem, AWS S3, Azure Blob Storage, or Google Cloud Storage. Each batch is flushed and committed as one atomic Delta transaction. diff --git a/content/docs/connectors/sinks/doris.mdx b/content/docs/connectors/sinks/doris.mdx index 9ed1282562..7dc573914a 100644 --- a/content/docs/connectors/sinks/doris.mdx +++ b/content/docs/connectors/sinks/doris.mdx @@ -1,5 +1,6 @@ --- title: Apache Doris Sink +description: "Load JSON messages from Iggy streams into a pre-created Apache Doris table through the Stream Load HTTP API." --- The Apache Doris sink connector consumes JSON messages from Iggy streams and writes them to a pre-created Doris table via Doris's Stream Load HTTP API. Batches are loaded under deterministic labels so that in-request retries deduplicate instead of doubling rows. diff --git a/content/docs/connectors/sinks/elasticsearch.mdx b/content/docs/connectors/sinks/elasticsearch.mdx index cc357cdb8b..0d99db5054 100644 --- a/content/docs/connectors/sinks/elasticsearch.mdx +++ b/content/docs/connectors/sinks/elasticsearch.mdx @@ -1,5 +1,6 @@ --- title: Elasticsearch Sink +description: "Index messages from Iggy streams into an Elasticsearch index using the bulk API." --- The Elasticsearch sink connector consumes messages from Iggy streams and indexes them into an Elasticsearch index using the bulk API. diff --git a/content/docs/connectors/sinks/http.mdx b/content/docs/connectors/sinks/http.mdx index 846fffb161..4220d6d5b8 100644 --- a/content/docs/connectors/sinks/http.mdx +++ b/content/docs/connectors/sinks/http.mdx @@ -1,5 +1,6 @@ --- title: HTTP Sink +description: "Deliver messages from Iggy streams to any HTTP endpoint, from webhooks and REST APIs to serverless functions." --- The HTTP Sink connector consumes messages from Iggy streams and delivers them to any HTTP endpoint: webhooks, REST APIs, serverless functions, or SaaS integrations. This is the transport-level, generic HTTP connector. It's distinct from sinks such as Elasticsearch or Quickwit, which use HTTP internally to talk to one specific system. You bring the URL, headers, and batching strategy, and the sink handles transport, retries, and metadata wrapping. diff --git a/content/docs/connectors/sinks/iceberg.mdx b/content/docs/connectors/sinks/iceberg.mdx index f960458f9a..d0b74a7bfa 100644 --- a/content/docs/connectors/sinks/iceberg.mdx +++ b/content/docs/connectors/sinks/iceberg.mdx @@ -1,5 +1,6 @@ --- title: Iceberg Sink +description: "Consume messages from Iggy topics and store them in Iceberg tables, with REST catalogs and S3-compatible storage." --- The Iceberg Sink Connector allows you to consume messages from Iggy topics and store them in Iceberg tables. diff --git a/content/docs/connectors/sinks/influxdb.mdx b/content/docs/connectors/sinks/influxdb.mdx index b90e0acd40..c5848e1003 100644 --- a/content/docs/connectors/sinks/influxdb.mdx +++ b/content/docs/connectors/sinks/influxdb.mdx @@ -1,5 +1,6 @@ --- title: InfluxDB Sink +description: "Write messages from Iggy streams to InfluxDB as line-protocol points, on both InfluxDB V2 and V3." --- The InfluxDB sink connector consumes messages from Iggy streams and writes them to InfluxDB as line-protocol points. It supports both InfluxDB V2 (OSS 2.x / Cloud) and InfluxDB V3 (Core / Enterprise), selected with the `version` option. diff --git a/content/docs/connectors/sinks/meilisearch.mdx b/content/docs/connectors/sinks/meilisearch.mdx index a9f3aaaaa2..26b996b275 100644 --- a/content/docs/connectors/sinks/meilisearch.mdx +++ b/content/docs/connectors/sinks/meilisearch.mdx @@ -1,5 +1,6 @@ --- title: Meilisearch Sink +description: "Index messages from Iggy streams as documents in a Meilisearch index through the official Rust SDK." --- The Meilisearch sink connector consumes messages from Iggy streams and indexes them as documents in a Meilisearch index through the official Rust SDK. It can create the index on startup and waits for Meilisearch indexing tasks to complete by default, so task failures surface as connector errors. diff --git a/content/docs/connectors/sinks/mongodb.mdx b/content/docs/connectors/sinks/mongodb.mdx index 9fde5eace6..c7e75f60fc 100644 --- a/content/docs/connectors/sinks/mongodb.mdx +++ b/content/docs/connectors/sinks/mongodb.mdx @@ -1,5 +1,6 @@ --- title: MongoDB Sink +description: "Insert messages from Iggy streams into a MongoDB collection, one document per message." --- The MongoDB sink connector writes messages from Iggy streams to a MongoDB database. Each message is inserted as a document into the configured collection. diff --git a/content/docs/connectors/sinks/postgres.mdx b/content/docs/connectors/sinks/postgres.mdx index 2039238ea9..723c98ef2e 100644 --- a/content/docs/connectors/sinks/postgres.mdx +++ b/content/docs/connectors/sinks/postgres.mdx @@ -1,5 +1,6 @@ --- title: Postgres Sink +description: "Store messages from Iggy topics in a PostgreSQL table, with optional table creation and configurable payload storage." --- The PostgreSQL sink connector consumes messages from Iggy topics and stores them in a PostgreSQL table, with optional automatic table creation and configurable payload storage. diff --git a/content/docs/connectors/sinks/quickwit.mdx b/content/docs/connectors/sinks/quickwit.mdx index 4486eb0f61..3d177e26fb 100644 --- a/content/docs/connectors/sinks/quickwit.mdx +++ b/content/docs/connectors/sinks/quickwit.mdx @@ -1,5 +1,6 @@ --- title: Quickwit Sink +description: "Send messages from Iggy streams to a Quickwit index over HTTP, creating the index when it does not exist." --- The Quickwit connector allows you to send data to the Quickwit API using HTTP. This sink will ensure that the index exists (create it if it doesn't) and will append the data to the index using the same batch size as specified in the Iggy configuration. diff --git a/content/docs/connectors/sinks/s3.mdx b/content/docs/connectors/sinks/s3.mdx index 1a77fdb071..c4fb11205e 100644 --- a/content/docs/connectors/sinks/s3.mdx +++ b/content/docs/connectors/sinks/s3.mdx @@ -1,5 +1,6 @@ --- title: S3 Sink +description: "Write messages from Iggy streams to Amazon S3 and S3-compatible object stores such as MinIO, R2, Spaces and B2." --- The S3 sink connector writes messages from Iggy streams to Amazon S3 and S3-compatible object stores (MinIO, Cloudflare R2, DigitalOcean Spaces, Backblaze B2). Messages are buffered in memory and uploaded as files, rotated by size or message count, under configurable path templates with offset ranges in the object names. diff --git a/content/docs/connectors/sinks/sink.mdx b/content/docs/connectors/sinks/sink.mdx index e67d7bb0d8..2359c39761 100644 --- a/content/docs/connectors/sinks/sink.mdx +++ b/content/docs/connectors/sinks/sink.mdx @@ -1,5 +1,6 @@ --- title: Sink +description: "What sink connectors do, and the Sink trait every one of them implements." --- ## Overview diff --git a/content/docs/connectors/sinks/stdout.mdx b/content/docs/connectors/sinks/stdout.mdx index 7d10b7f1fa..0fea9c2317 100644 --- a/content/docs/connectors/sinks/stdout.mdx +++ b/content/docs/connectors/sinks/stdout.mdx @@ -1,5 +1,6 @@ --- title: Stdout Sink +description: "Print messages from Iggy streams to standard output, for debugging and verifying a connector pipeline." --- The stdout sink connector prints messages from Iggy streams to the standard output. This is useful for debugging, development, and verifying that your connector pipeline is working correctly. diff --git a/content/docs/connectors/sinks/surrealdb.mdx b/content/docs/connectors/sinks/surrealdb.mdx index f5f64cae57..0ec515d7e0 100644 --- a/content/docs/connectors/sinks/surrealdb.mdx +++ b/content/docs/connectors/sinks/surrealdb.mdx @@ -1,5 +1,6 @@ --- title: SurrealDB Sink +description: "Write messages from Iggy streams into SurrealDB over the HTTP API, one bulk insert per batch." --- The SurrealDB sink connector writes messages from Iggy streams into SurrealDB over the HTTP API. Each batch becomes one SurrealQL bulk `INSERT IGNORE`. Every record gets a deterministic record id derived from stream, topic, partition, offset, and message id, so replayed batches are idempotent and existing records are left untouched. diff --git a/content/docs/connectors/sources/elasticsearch.mdx b/content/docs/connectors/sources/elasticsearch.mdx index c7f21278bf..63f53882c4 100644 --- a/content/docs/connectors/sources/elasticsearch.mdx +++ b/content/docs/connectors/sources/elasticsearch.mdx @@ -1,5 +1,6 @@ --- title: Elasticsearch Source +description: "Poll documents from an Elasticsearch index into Iggy streams, incrementally when a timestamp field is configured." --- The Elasticsearch source connector polls documents from an Elasticsearch index and produces them to Iggy streams. With a timestamp field configured, each poll only fetches documents newer than the last one seen, giving incremental, restart-safe ingestion. diff --git a/content/docs/connectors/sources/influxdb.mdx b/content/docs/connectors/sources/influxdb.mdx index b2623d6138..ac0a2879cc 100644 --- a/content/docs/connectors/sources/influxdb.mdx +++ b/content/docs/connectors/sources/influxdb.mdx @@ -1,5 +1,6 @@ --- title: InfluxDB Source +description: "Poll InfluxDB on an interval and produce the resulting rows into an Iggy stream, on both InfluxDB V2 and V3." --- The InfluxDB source connector polls InfluxDB on an interval and produces the resulting rows as messages into an Iggy stream. It supports both InfluxDB V2 (OSS 2.x / Cloud, Flux queries) and InfluxDB V3 (Core / Enterprise, SQL queries), selected with the `version` option. A timestamp cursor stored in persistent connector state tracks the position, so restarts resume where they left off. diff --git a/content/docs/connectors/sources/postgres.mdx b/content/docs/connectors/sources/postgres.mdx index e41676be05..ebef4c8e76 100644 --- a/content/docs/connectors/sources/postgres.mdx +++ b/content/docs/connectors/sources/postgres.mdx @@ -1,5 +1,6 @@ --- title: Postgres Source +description: "Stream rows from PostgreSQL into Iggy topics by incremental table polling or change data capture over logical replication." --- The PostgreSQL source connector fetches data from PostgreSQL databases and streams it to Iggy topics. It supports incremental table polling and Change Data Capture (CDC) via logical replication. diff --git a/content/docs/connectors/sources/random.mdx b/content/docs/connectors/sources/random.mdx index e3d6c388e5..a4e052a279 100644 --- a/content/docs/connectors/sources/random.mdx +++ b/content/docs/connectors/sources/random.mdx @@ -1,5 +1,6 @@ --- title: Random Source +description: "Generate random messages into Iggy streams for testing, benchmarking and verifying a connector pipeline." --- The random source connector generates random messages and sends them to Iggy streams. This is useful for testing, benchmarking, and verifying that your connector pipeline is working correctly without needing an external data source. diff --git a/content/docs/connectors/sources/source.mdx b/content/docs/connectors/sources/source.mdx index 0f8ec778bd..0fd29b5de0 100644 --- a/content/docs/connectors/sources/source.mdx +++ b/content/docs/connectors/sources/source.mdx @@ -1,5 +1,6 @@ --- title: Source +description: "What source connectors do, and the Source trait every one of them implements." --- ## Overview diff --git a/content/docs/connectors/transforms.mdx b/content/docs/connectors/transforms.mdx index a33548c2f7..ed3a25714a 100644 --- a/content/docs/connectors/transforms.mdx +++ b/content/docs/connectors/transforms.mdx @@ -1,5 +1,6 @@ --- title: Transforms +description: "Mutate, filter and convert messages as they pass through the connector runtime, and how each transform is configured." --- Transforms mutate, filter, or convert messages as they flow through the connector runtime. They run inside the runtime process, not inside the plugins. For source connectors they're applied after the plugin's messages are decoded and before producing to Iggy. For sink connectors, after consuming from Iggy and before handing the messages to the plugin. diff --git a/content/docs/faq/faq.mdx b/content/docs/faq/faq.mdx index 26764ac23a..72f6b2bb2f 100644 --- a/content/docs/faq/faq.mdx +++ b/content/docs/faq/faq.mdx @@ -1,5 +1,6 @@ --- title: FAQ +description: "Common questions about Apache Iggy, including how it compares to Kafka and other message brokers." --- ## Q: What is the difference between Iggy and traditional message brokers like Kafka? diff --git a/content/docs/index.mdx b/content/docs/index.mdx index 3ad40fcc2f..c75dcc98cb 100644 --- a/content/docs/index.mdx +++ b/content/docs/index.mdx @@ -1,5 +1,6 @@ --- title: Welcome +description: "Apache Iggy documentation: how Iggy organizes data into streams, topics, partitions and segments, and where to start." --- diff --git a/content/docs/introduction/about.mdx b/content/docs/introduction/about.mdx index 46a109df1f..87a58a7064 100644 --- a/content/docs/introduction/about.mdx +++ b/content/docs/introduction/about.mdx @@ -1,5 +1,6 @@ --- title: About +description: "What Apache Iggy is: a persistent message streaming platform written in Rust, with QUIC, TCP, WebSocket and HTTP transports." --- **Iggy** is a persistent message streaming platform written in Rust, supporting QUIC, TCP, WebSocket (custom binary specification) and HTTP (regular REST API) transport protocols, **capable of processing millions of messages per second at ultra-low latency**. diff --git a/content/docs/introduction/architecture.mdx b/content/docs/introduction/architecture.mdx index e453ca535f..3dbe281015 100644 --- a/content/docs/introduction/architecture.mdx +++ b/content/docs/introduction/architecture.mdx @@ -1,5 +1,6 @@ --- title: Architecture +description: "Inside the Iggy server: how work is scheduled across cores, how requests are routed, and how a message reaches disk." --- This page covers the internals of the Iggy server: how work is scheduled across CPU cores, how requests are routed, and how data ends up on disk. If you're looking for the domain model (streams, topics, partitions, consumer groups, message format), see [concepts](/docs/introduction/concepts) first. diff --git a/content/docs/introduction/concepts.mdx b/content/docs/introduction/concepts.mdx index 47ff815f73..70d4af2f09 100644 --- a/content/docs/introduction/concepts.mdx +++ b/content/docs/introduction/concepts.mdx @@ -1,5 +1,6 @@ --- title: Concepts +description: "The domain model behind Iggy, and how an append-only streaming log differs from a message broker." --- Iggy is a persistent message streaming platform: messages are stored in a form of an **append-only log**. You can create multiple streams, consisting of topics, which might have one or more partitions assigned, e.g. to achieve the horizontal scalability between many independent consumers or higher system resiliency. You can think of Iggy as an alternative to Kafka or RabbitMQ streams. diff --git a/content/docs/introduction/getting-started.mdx b/content/docs/introduction/getting-started.mdx index 7d012e120b..2547616b45 100644 --- a/content/docs/introduction/getting-started.mdx +++ b/content/docs/introduction/getting-started.mdx @@ -1,5 +1,6 @@ --- title: Getting started +description: "A first program with the low-level Rust SDK: create a stream and topic, send messages, then poll them back." --- ## Before we start diff --git a/content/docs/sdk/connection-strings.mdx b/content/docs/sdk/connection-strings.mdx index 4f72e32a0c..fbc92bb080 100644 --- a/content/docs/sdk/connection-strings.mdx +++ b/content/docs/sdk/connection-strings.mdx @@ -1,5 +1,6 @@ --- title: Connection Strings +description: "The canonical reference for Iggy connection strings, covering every scheme, credential form and option." --- A connection string configures an Iggy client in a single line: transport, credentials, server address, and options. This page is the canonical reference for every scheme, credential form, and option key. diff --git a/content/docs/sdk/cpp/intro.mdx b/content/docs/sdk/cpp/intro.mdx index 541575e9b4..cc8833344f 100644 --- a/content/docs/sdk/cpp/intro.mdx +++ b/content/docs/sdk/cpp/intro.mdx @@ -1,5 +1,6 @@ --- title: C++ SDK (WIP) +description: "The C++ SDK, a CXX bridge over the Rust SDK, and how to build it from source with Bazel." --- The Iggy C++ SDK wraps the Rust SDK through a [CXX](https://cxx.rs) FFI bridge, so it can use all transport protocols (TCP, QUIC, HTTP, WebSocket) via connection strings. The source code is available at [github.com/apache/iggy/tree/master/foreign/cpp](https://github.com/apache/iggy/tree/master/foreign/cpp). diff --git a/content/docs/sdk/csharp/examples.mdx b/content/docs/sdk/csharp/examples.mdx index 427922e3bf..43302389dd 100644 --- a/content/docs/sdk/csharp/examples.mdx +++ b/content/docs/sdk/csharp/examples.mdx @@ -1,5 +1,6 @@ --- title: Examples +description: "Producer and consumer samples for the C# SDK, built on the high-level publisher and consumer." --- These samples use the [High-level SDK](/docs/sdk/csharp/high-level-sdk) - the recommended way to build producers and consumers. For the low-level, per-call equivalents, see the [Guide](/docs/sdk/csharp/guide). diff --git a/content/docs/sdk/csharp/guide.mdx b/content/docs/sdk/csharp/guide.mdx index d6b5d900d7..ac5e5d71cb 100644 --- a/content/docs/sdk/csharp/guide.mdx +++ b/content/docs/sdk/csharp/guide.mdx @@ -1,5 +1,6 @@ --- title: Guide +description: "Client configuration and the full low-level IIggyClient API surface for the C# SDK." --- This guide covers client configuration and the full `IIggyClient` API surface - the low-level, per-call operations. For the ergonomic producer/consumer abstractions built on top of these, see the [High-level SDK](/docs/sdk/csharp/high-level-sdk). diff --git a/content/docs/sdk/csharp/high-level-sdk.mdx b/content/docs/sdk/csharp/high-level-sdk.mdx index f4297cc9fa..ad8fd875b5 100644 --- a/content/docs/sdk/csharp/high-level-sdk.mdx +++ b/content/docs/sdk/csharp/high-level-sdk.mdx @@ -1,5 +1,6 @@ --- title: High-level SDK +description: "IggyPublisher and IggyConsumer for C#, with background batching, retries and automatic offset commits." --- The per-call [`IIggyClient`](/docs/sdk/csharp/guide) API is explicit but verbose - you manage partitioning, batching, retries, and offsets yourself. The high-level `IggyPublisher` and `IggyConsumer` wrap that surface with fluent builders that handle: diff --git a/content/docs/sdk/csharp/intro.mdx b/content/docs/sdk/csharp/intro.mdx index 04b57df8cc..7e8ae6e302 100644 --- a/content/docs/sdk/csharp/intro.mdx +++ b/content/docs/sdk/csharp/intro.mdx @@ -1,5 +1,6 @@ --- title: C# SDK +description: "The async-first C# SDK for .NET applications over TCP and HTTP, from installation to creating a client." --- The Iggy SDK for C# is a modern, async-first client library for interacting with an Iggy message streaming server from your .NET applications. It supports TCP and HTTP transports. The package is available on [NuGet](https://www.nuget.org/packages/Apache.Iggy/) and the source code lives on [GitHub](https://github.com/apache/iggy/tree/master/foreign/csharp). diff --git a/content/docs/sdk/go/examples.mdx b/content/docs/sdk/go/examples.mdx index d8e400115b..b51e663e08 100644 --- a/content/docs/sdk/go/examples.mdx +++ b/content/docs/sdk/go/examples.mdx @@ -1,5 +1,6 @@ --- title: Examples +description: "Runnable Go examples from the core repository, and how to start a server they can connect to." --- A runnable getting-started example lives in the [examples/go](https://github.com/apache/iggy/tree/master/examples/go) directory of the core repository. Its `go.mod` carries a `replace` directive pointing at the in-repo SDK, so the example always builds against the SDK source in the same checkout. The examples are exercised in CI via `scripts/run-examples-from-readme.sh --language go`. diff --git a/content/docs/sdk/go/intro.mdx b/content/docs/sdk/go/intro.mdx index 04b8926845..e930662194 100644 --- a/content/docs/sdk/go/intro.mdx +++ b/content/docs/sdk/go/intro.mdx @@ -1,5 +1,6 @@ --- title: Go SDK +description: "The Go SDK, a blocking client speaking the VSR wire protocol over TCP, and which servers it pairs with." --- The Iggy Go SDK is a client library for interacting with the Iggy server from Go applications. It is a blocking client that speaks the VSR (Viewstamped Replication) wire protocol over TCP, with or without TLS. The package is available on [pkg.go.dev](https://pkg.go.dev/github.com/apache/iggy/foreign/go) and the source code lives in [foreign/go](https://github.com/apache/iggy/tree/master/foreign/go). diff --git a/content/docs/sdk/introduction.mdx b/content/docs/sdk/introduction.mdx index b2e6c1be64..3ed4a3879b 100644 --- a/content/docs/sdk/introduction.mdx +++ b/content/docs/sdk/introduction.mdx @@ -1,5 +1,6 @@ --- title: Introduction +description: "The official Iggy client SDKs, their packages and registries, and the transports each one supports." --- Iggy provides official client SDKs in multiple languages. The **Rust SDK** is the most feature-complete, offering both a high-level API (with auto-batching, consumer groups, offset management, retry logic) and a low-level API for direct protocol operations. diff --git a/content/docs/sdk/java/examples.mdx b/content/docs/sdk/java/examples.mdx index ee9accafd8..e56e6df773 100644 --- a/content/docs/sdk/java/examples.mdx +++ b/content/docs/sdk/java/examples.mdx @@ -1,5 +1,6 @@ --- title: Examples +description: "Runnable Java examples from the core repository, built with Gradle, and how to start a server for them." --- Runnable examples live in the [examples/java](https://github.com/apache/iggy/tree/master/examples/java) directory of the core repository as a standalone Gradle project. The project builds against the in-repo SDK (via an `includeBuild` substitution), so the examples always match the SDK source in the same checkout. Java 17 is recommended. The included `gradlew` wrapper downloads the pinned Gradle version on first run. diff --git a/content/docs/sdk/java/intro.mdx b/content/docs/sdk/java/intro.mdx index f647118cca..651b1c0c2a 100644 --- a/content/docs/sdk/java/intro.mdx +++ b/content/docs/sdk/java/intro.mdx @@ -1,5 +1,6 @@ --- title: Java SDK +description: "The Java SDK, with blocking and CompletableFuture clients over TCP, and which servers they pair with." --- The Iggy Java SDK is a client library for interacting with the Iggy server from Java applications. It provides a blocking client and an async client (based on `CompletableFuture`) over TCP, plus a blocking HTTP client. The artifact is published to [Maven Central](https://central.sonatype.com/artifact/org.apache.iggy/iggy) and the source code lives in [foreign/java](https://github.com/apache/iggy/tree/master/foreign/java). diff --git a/content/docs/sdk/node/examples.mdx b/content/docs/sdk/node/examples.mdx index 9adf48e9c1..fb5accd4b4 100644 --- a/content/docs/sdk/node/examples.mdx +++ b/content/docs/sdk/node/examples.mdx @@ -1,5 +1,6 @@ --- title: Examples +description: "TypeScript examples for the Node.js SDK, exercised by CI so they match the current SDK." --- Working examples are available in the [examples/node](https://github.com/apache/iggy/tree/master/examples/node) directory, written in TypeScript. CI exercises them, so they always match the current SDK: diff --git a/content/docs/sdk/node/intro.mdx b/content/docs/sdk/node/intro.mdx index bca8e7fd47..359dd62311 100644 --- a/content/docs/sdk/node/intro.mdx +++ b/content/docs/sdk/node/intro.mdx @@ -1,5 +1,6 @@ --- title: Node.js SDK +description: "The Node.js and TypeScript SDK, from installation to keeping client and server versions in step." --- The Iggy Node.js SDK is a client library that allows you to interact with the Iggy API from your Node.js and TypeScript applications. It communicates with the Iggy server over TCP or TLS using the binary protocol. The package is available on [npm](https://www.npmjs.com/package/apache-iggy) and the source code can be found on [GitHub](https://github.com/apache/iggy/tree/master/foreign/node). diff --git a/content/docs/sdk/php/intro.mdx b/content/docs/sdk/php/intro.mdx index 3b1e08ee14..aa3e120beb 100644 --- a/content/docs/sdk/php/intro.mdx +++ b/content/docs/sdk/php/intro.mdx @@ -1,5 +1,6 @@ --- title: PHP SDK +description: "The experimental PHP SDK, a native extension wrapping the Rust SDK, and how to build it from source." --- The Iggy PHP SDK is a native PHP extension built in Rust with [ext-php-rs](https://github.com/davidcole1340/ext-php-rs). It wraps the Rust SDK and exposes a synchronous `Iggy\Client`: each call drives the extension's internal Tokio runtime and blocks the calling PHP thread until the operation completes. The source code is available on [GitHub](https://github.com/apache/iggy/tree/master/foreign/php). diff --git a/content/docs/sdk/python/examples.mdx b/content/docs/sdk/python/examples.mdx index 02afce36da..d0fa72d87e 100644 --- a/content/docs/sdk/python/examples.mdx +++ b/content/docs/sdk/python/examples.mdx @@ -1,5 +1,6 @@ --- title: Examples +description: "Working Python examples from the core repository, covering connection strings, message headers and TLS." --- Working examples are available in the [examples/python](https://github.com/apache/iggy/tree/master/examples/python) directory. CI exercises them, so they always match the current SDK: diff --git a/content/docs/sdk/python/intro.mdx b/content/docs/sdk/python/intro.mdx index 8913702e86..a777c68944 100644 --- a/content/docs/sdk/python/intro.mdx +++ b/content/docs/sdk/python/intro.mdx @@ -1,5 +1,6 @@ --- title: Python SDK +description: "The Python SDK, a PyO3 wrapper around the Rust SDK, and how to install it." --- The Iggy Python SDK is a client library that allows you to interact with the Iggy API from your Python application. It is built as a PyO3 wrapper around the Rust SDK, which means it supports TCP, QUIC, HTTP, and WebSocket transports via connection strings. The package is available on [PyPI](https://pypi.org/project/apache-iggy/) and the source code can be found on [GitHub](https://github.com/apache/iggy/tree/master/foreign/python). diff --git a/content/docs/sdk/rust/examples.mdx b/content/docs/sdk/rust/examples.mdx index 71e0e1df59..b3d89278d3 100644 --- a/content/docs/sdk/rust/examples.mdx +++ b/content/docs/sdk/rust/examples.mdx @@ -1,5 +1,6 @@ --- title: Examples +description: "The Rust SDK examples in the core repository, and how to start a server to run them against." --- In the core repository, you can find the following [examples](https://github.com/apache/iggy/tree/master/examples/rust/src) using the Rust SDK. diff --git a/content/docs/sdk/rust/high-level-sdk.mdx b/content/docs/sdk/rust/high-level-sdk.mdx index 5ddce3e289..9ed00c0398 100644 --- a/content/docs/sdk/rust/high-level-sdk.mdx +++ b/content/docs/sdk/rust/high-level-sdk.mdx @@ -1,5 +1,6 @@ --- title: High-level SDK +description: "IggyProducer and IggyConsumer: batching, consumer groups and offset commits without the boilerplate." --- If you've read through the [getting started](/docs/introduction/getting-started) guide, you might have noticed that it's quite verbose and requires a lot of boilerplate code to get started. This is where the High-level SDK comes in, as it does provide a more user-friendly interface to interact with the Iggy API for both, producer and consumer. Let's consider the following features: diff --git a/content/docs/sdk/rust/intro.mdx b/content/docs/sdk/rust/intro.mdx index c3fa32a93e..20e46f7ae4 100644 --- a/content/docs/sdk/rust/intro.mdx +++ b/content/docs/sdk/rust/intro.mdx @@ -1,5 +1,6 @@ --- title: Intro +description: "The Rust SDK, the primary and most complete Iggy client, and where its source lives." --- The Rust SDK is the primary and most feature-complete client for Iggy. It is available on [crates.io](https://crates.io/crates/iggy) and the source code is part of the [core repository](https://github.com/apache/iggy/tree/master/core/sdk). diff --git a/content/docs/sdk/rust/stream-builder.mdx b/content/docs/sdk/rust/stream-builder.mdx index 37eed4599b..cfcf905fe3 100644 --- a/content/docs/sdk/rust/stream-builder.mdx +++ b/content/docs/sdk/rust/stream-builder.mdx @@ -1,5 +1,6 @@ --- title: Stream Builder +description: "The stream builder, for event-driven applications that would otherwise repeat producer and consumer setup." --- In the previous section, the high level SDK introduced the connection string to simplify the client configuration and diff --git a/content/docs/server/benchmarking.mdx b/content/docs/server/benchmarking.mdx index e1325341e6..a09ef288e0 100644 --- a/content/docs/server/benchmarking.mdx +++ b/content/docs/server/benchmarking.mdx @@ -1,5 +1,6 @@ --- title: Benchmarking +description: "How Iggy is benchmarked, the tooling that ships with it, and the public platform for comparing results." --- **Benchmarks should be the first-class citizens**. We believe that performance is crucial for any system, and we strive to provide the best possible performance for our users. Please check, why we believe that the **[transparent benchmarking](https://iggy.apache.org/blogs/2025/02/17/transparent-benchmarks)** is so important. diff --git a/content/docs/server/configuration.mdx b/content/docs/server/configuration.mdx index 94a95bf079..1f0c4f4997 100644 --- a/content/docs/server/configuration.mdx +++ b/content/docs/server/configuration.mdx @@ -1,5 +1,6 @@ --- title: Configuration +description: "The server's TOML configuration, how a file is merged over the embedded defaults, and what each section controls." --- The server reads a single TOML file. A copy of [`core/server/config.toml`](https://github.com/apache/iggy/blob/master/core/server/config.toml) is embedded into the binary and serves as the default configuration, so the server boots with working defaults even without any file on disk. The embedded file is commented throughout and is the authoritative reference for the exact version you're running. diff --git a/content/docs/server/docker.mdx b/content/docs/server/docker.mdx index 08d0759097..339b3cb06b 100644 --- a/content/docs/server/docker.mdx +++ b/content/docs/server/docker.mdx @@ -1,5 +1,6 @@ --- title: Docker & Helm +description: "Run the Iggy server from the official Docker images, and deploy it with the Helm chart." --- ## Docker diff --git a/content/docs/server/introduction.mdx b/content/docs/server/introduction.mdx index 38787ec30e..bdac48bdb9 100644 --- a/content/docs/server/introduction.mdx +++ b/content/docs/server/introduction.mdx @@ -1,5 +1,6 @@ --- title: Introduction +description: "What the Iggy server does, and where its releases and Docker images are published." --- Iggy server is the most important part of the system as it's responsible for handling all the incoming connections, managing the data and providing the API for the clients. The server is written in Rust and can be run on any platform that supports it. diff --git a/content/docs/server/networking.mdx b/content/docs/server/networking.mdx index 7948f9fdd5..9deb6b9199 100644 --- a/content/docs/server/networking.mdx +++ b/content/docs/server/networking.mdx @@ -1,5 +1,6 @@ --- title: Networking +description: "The four transports Iggy serves simultaneously, TCP, QUIC, WebSocket and HTTP, and what each is best suited to." --- Iggy supports four transport protocols simultaneously, each optimized for different use cases. All stateful protocols (TCP, QUIC, WebSocket) use the same custom binary wire protocol, while HTTP provides a standard REST API. diff --git a/content/docs/server/security.mdx b/content/docs/server/security.mdx index 1294b2c3b4..98fe47f31d 100644 --- a/content/docs/server/security.mdx +++ b/content/docs/server/security.mdx @@ -1,5 +1,6 @@ --- title: Security +description: "Authentication, authorization, transport encryption and encryption at rest in the Iggy server." --- Iggy provides multiple layers of security covering authentication, authorization, transport encryption, and data encryption at rest. diff --git a/content/docs/server/storage-engine.mdx b/content/docs/server/storage-engine.mdx index 7d003434d0..3deee01b1f 100644 --- a/content/docs/server/storage-engine.mdx +++ b/content/docs/server/storage-engine.mdx @@ -1,5 +1,6 @@ --- title: Storage Engine +description: "The segmented append-only log, and how streams, topics, partitions and segments map onto files on disk." --- Iggy's storage engine is built around the concept of a **segmented append-only log**. Every piece of data flows through a well-defined hierarchy: System -> Streams -> Topics -> Partitions -> Segments. This page covers how data is stored, indexed, flushed, recovered, and cleaned up on disk. diff --git a/content/docs/server/topic-options.mdx b/content/docs/server/topic-options.mdx index c1b1b36b4a..c75df0210c 100644 --- a/content/docs/server/topic-options.mdx +++ b/content/docs/server/topic-options.mdx @@ -1,5 +1,6 @@ --- title: Topic Options +description: "Per-topic retention, durability and segment layout options, set when the topic is created." --- Retention, durability, and segment layout are per-topic decisions, made when the topic is created. They used to be server-wide `[system.*]` keys, which gave every knob two homes. The server now [refuses to boot](/docs/server/configuration#relocated-configuration-keys) on the old keys, and each topic carries its own values instead. diff --git a/content/docs/web_ui/start.mdx b/content/docs/web_ui/start.mdx index c43cf7132d..2eac8f298c 100644 --- a/content/docs/web_ui/start.mdx +++ b/content/docs/web_ui/start.mdx @@ -1,5 +1,6 @@ --- title: Iggy Web UI +description: "The Iggy Web UI dashboard, for monitoring server health and browsing streams, topics, messages and users." --- Iggy Web UI provides a comprehensive dashboard for Iggy server. It allows you to monitor the server's health, streams, topics, browse the messages, users and more. The dashboard is built with SvelteKit and TypeScript and is available as [open-source](https://github.com/apache/iggy/tree/master/web) as well as the Docker image on [Docker Hub](https://hub.docker.com/r/apache/iggy-web-ui).