From 296fb7ec57f65114b1c80aa3e853e89021d26286 Mon Sep 17 00:00:00 2001 From: micheleRP Date: Thu, 16 Apr 2026 16:50:48 -0600 Subject: [PATCH 1/7] DOC-1907: Add Paginate Messages and Pipeline Events page Document Redpanda Console unlimited pagination for browsing topic messages and Redpanda Connect pipeline events. Uses single-source pattern with ifdef::env-cloud[] conditional xrefs so the page renders correctly in both the Self-Managed and Cloud docs sites. Co-Authored-By: Claude Opus 4.7 (1M context) --- modules/ROOT/nav.adoc | 1 + modules/console/pages/index.adoc | 2 +- .../pages/ui/paginate-messages-events.adoc | 65 +++++++++++++++++++ 3 files changed, 67 insertions(+), 1 deletion(-) create mode 100644 modules/console/pages/ui/paginate-messages-events.adoc diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index 8e30301d3a..f0e20337c2 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -49,6 +49,7 @@ *** xref:develop:consume-data/follower-fetching.adoc[Follower Fetching] *** xref:console:ui/programmable-push-filters.adoc[Filter Messages] *** xref:console:ui/record-deserialization.adoc[Deserialize Messages] +*** xref:console:ui/paginate-messages-events.adoc[Paginate Messages and Events] ** xref:develop:data-transforms/index.adoc[] *** xref:develop:data-transforms/how-transforms-work.adoc[Overview] *** xref:develop:data-transforms/run-transforms-index.adoc[Get Started] diff --git a/modules/console/pages/index.adoc b/modules/console/pages/index.adoc index c55aa249da..d6dd75943c 100644 --- a/modules/console/pages/index.adoc +++ b/modules/console/pages/index.adoc @@ -24,7 +24,7 @@ image::broker-overview.png[] Observe and debug your streaming data: -* *Message inspection*: Browse and filter messages within your topics, with options to search by key, timestamp, or custom filters. +* *Message inspection*: Browse and filter messages within your topics, with options to search by key, timestamp, or custom filters. See xref:console:ui/paginate-messages-events.adoc[] to inspect large result sets. * xref:console:ui/programmable-push-filters.adoc[*Programmable push filters*]: Write custom JavaScript filters to isolate specific messages, enabling deep inspection and debugging. * *Rewind and Replay*: Roll back consumer offsets to reprocess messages, allowing you to correct issues or replay data as needed. diff --git a/modules/console/pages/ui/paginate-messages-events.adoc b/modules/console/pages/ui/paginate-messages-events.adoc new file mode 100644 index 0000000000..98fb1f9068 --- /dev/null +++ b/modules/console/pages/ui/paginate-messages-events.adoc @@ -0,0 +1,65 @@ += Paginate Messages and Pipeline Events in {ui} +:description: Retrieve more than 500 messages or pipeline events in {ui} by paging through larger result sets. +:page-topic-type: how-to + + +// tag::single-source[] + +{description} Use pagination when you need to inspect large topics or debug +ifdef::env-cloud[] +xref:develop:connect/about.adoc[Redpanda Connect] +endif::[] +ifndef::env-cloud[] +xref:redpanda-connect:home:index.adoc[Redpanda Connect] +endif::[] +pipelines that emit more events than the initial result set includes. + +== Paginate topic messages + +When you browse a topic in {ui}, the *Messages* tab returns an initial batch of records. To retrieve additional records beyond this batch: + +. In the menu, go to *Topics* and select a topic. +. Open the *Messages* tab. +. Apply any filters or a starting offset to narrow the records you want to inspect. ++ +ifdef::env-cloud[] +See xref:manage:schema-reg/programmable-push-filters.adoc[] and xref:manage:schema-reg/record-deserialization.adoc[]. +endif::[] +ifndef::env-cloud[] +See xref:console:ui/programmable-push-filters.adoc[] and xref:console:ui/record-deserialization.adoc[]. +endif::[] +. After the initial results load, load the next batch to continue through the topic. + +You can continue paginating until you reach the end of the topic or the end of the offset range you specified. + +== Paginate Redpanda Connect pipeline events + +When you debug a Redpanda Connect pipeline, {ui} shows the events the pipeline emits, including logs at the configured log level. Pipelines that run with debug-level logging often produce more events than fit in a single result set. + +To retrieve additional pipeline events: + +. In the menu, go to *Connect* and select the pipeline you want to debug. +. Open the pipeline's events view. +. After the initial events load, load the next batch to continue inspecting events. + +ifdef::env-cloud[] +For guidance on configuring log levels and monitoring pipelines, see xref:develop:connect/configuration/monitor-connect.adoc[]. +endif::[] +ifndef::env-cloud[] +For guidance on configuring log levels and monitoring pipelines, see xref:redpanda-connect:guides:monitoring.adoc[]. +endif::[] + +== Performance considerations + +Retrieving large result sets increases load on the {ui} backend and the cluster. To keep responses fast: + +* Narrow the result set with filters or a bounded offset range before paginating. +ifdef::env-cloud[] +* Use xref:manage:schema-reg/programmable-push-filters.adoc[JavaScript push filters] to match only the records you need. +endif::[] +ifndef::env-cloud[] +* Use xref:console:ui/programmable-push-filters.adoc[JavaScript push filters] to match only the records you need. +endif::[] +* For pipeline debugging, raise the log level only as long as you need it. + +// end::single-source[] From 301daf595d1fa0ff1669c2c80ac2568b2b9821a6 Mon Sep 17 00:00:00 2001 From: micheleRP Date: Thu, 16 Apr 2026 16:53:11 -0600 Subject: [PATCH 2/7] [TEMP] Point cloud-docs source to DOC-1907 branch for preview Must be reverted before merge. Allows the cloud-docs stub page at modules/develop/pages/consume-data/paginate-messages-events.adoc to resolve against this docs PR branch when rendering the cloud-docs deploy preview. Co-Authored-By: Claude Opus 4.7 (1M context) --- local-antora-playbook.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/local-antora-playbook.yml b/local-antora-playbook.yml index 0590286e59..037ef2b4a2 100644 --- a/local-antora-playbook.yml +++ b/local-antora-playbook.yml @@ -18,7 +18,7 @@ content: - url: https://github.com/redpanda-data/docs branches: [v/*, shared, site-search,'!v-end-of-life/*'] - url: https://github.com/redpanda-data/cloud-docs - branches: 'main' + branches: 'DOC-1907-paginate-messages-events' - url: https://github.com/redpanda-data/redpanda-labs branches: main start_paths: [docs,'*/docs'] From 74668a2347429241c692478129f8d15b8ab22c0b Mon Sep 17 00:00:00 2001 From: micheleRP Date: Wed, 22 Apr 2026 17:49:38 -0600 Subject: [PATCH 3/7] DOC-1907: Narrow scope to topic messages only Pipeline-events pagination is covered in a separate PR, so remove the pipeline-events section, trim the intro/description/performance bullet, and switch the nav entry to empty brackets per CodeRabbit nitpick. The topic-messages section body is unchanged pending reviewer clarification on the continuous pagination toggle. Co-Authored-By: Claude Opus 4.7 (1M context) --- modules/ROOT/nav.adoc | 2 +- .../pages/ui/paginate-messages-events.adoc | 31 ++----------------- 2 files changed, 4 insertions(+), 29 deletions(-) diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index f0e20337c2..f4e9ef7c02 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -49,7 +49,7 @@ *** xref:develop:consume-data/follower-fetching.adoc[Follower Fetching] *** xref:console:ui/programmable-push-filters.adoc[Filter Messages] *** xref:console:ui/record-deserialization.adoc[Deserialize Messages] -*** xref:console:ui/paginate-messages-events.adoc[Paginate Messages and Events] +*** xref:console:ui/paginate-messages-events.adoc[] ** xref:develop:data-transforms/index.adoc[] *** xref:develop:data-transforms/how-transforms-work.adoc[Overview] *** xref:develop:data-transforms/run-transforms-index.adoc[Get Started] diff --git a/modules/console/pages/ui/paginate-messages-events.adoc b/modules/console/pages/ui/paginate-messages-events.adoc index 98fb1f9068..a95d333dc8 100644 --- a/modules/console/pages/ui/paginate-messages-events.adoc +++ b/modules/console/pages/ui/paginate-messages-events.adoc @@ -1,18 +1,11 @@ -= Paginate Messages and Pipeline Events in {ui} -:description: Retrieve more than 500 messages or pipeline events in {ui} by paging through larger result sets. += Paginate Messages in {ui} +:description: Retrieve more than the default batch of messages in {ui} by paging through larger result sets. :page-topic-type: how-to // tag::single-source[] -{description} Use pagination when you need to inspect large topics or debug -ifdef::env-cloud[] -xref:develop:connect/about.adoc[Redpanda Connect] -endif::[] -ifndef::env-cloud[] -xref:redpanda-connect:home:index.adoc[Redpanda Connect] -endif::[] -pipelines that emit more events than the initial result set includes. +{description} Use pagination when you need to inspect large topics without being limited to the initial result set. == Paginate topic messages @@ -32,23 +25,6 @@ endif::[] You can continue paginating until you reach the end of the topic or the end of the offset range you specified. -== Paginate Redpanda Connect pipeline events - -When you debug a Redpanda Connect pipeline, {ui} shows the events the pipeline emits, including logs at the configured log level. Pipelines that run with debug-level logging often produce more events than fit in a single result set. - -To retrieve additional pipeline events: - -. In the menu, go to *Connect* and select the pipeline you want to debug. -. Open the pipeline's events view. -. After the initial events load, load the next batch to continue inspecting events. - -ifdef::env-cloud[] -For guidance on configuring log levels and monitoring pipelines, see xref:develop:connect/configuration/monitor-connect.adoc[]. -endif::[] -ifndef::env-cloud[] -For guidance on configuring log levels and monitoring pipelines, see xref:redpanda-connect:guides:monitoring.adoc[]. -endif::[] - == Performance considerations Retrieving large result sets increases load on the {ui} backend and the cluster. To keep responses fast: @@ -60,6 +36,5 @@ endif::[] ifndef::env-cloud[] * Use xref:console:ui/programmable-push-filters.adoc[JavaScript push filters] to match only the records you need. endif::[] -* For pipeline debugging, raise the log level only as long as you need it. // end::single-source[] From 8eabc466557bced94b1f7f0c7b13232f6753da73 Mon Sep 17 00:00:00 2001 From: micheleRP Date: Wed, 29 Apr 2026 11:21:45 -0600 Subject: [PATCH 4/7] update with Julin's feedback --- .../pages/ui/paginate-messages-events.adoc | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/modules/console/pages/ui/paginate-messages-events.adoc b/modules/console/pages/ui/paginate-messages-events.adoc index a95d333dc8..66eb7fe62c 100644 --- a/modules/console/pages/ui/paginate-messages-events.adoc +++ b/modules/console/pages/ui/paginate-messages-events.adoc @@ -1,19 +1,17 @@ = Paginate Messages in {ui} -:description: Retrieve more than the default batch of messages in {ui} by paging through larger result sets. +:description: Enable Continuous Pagination on a topic's Messages tab to scroll through all records instead of being capped by Max results. :page-topic-type: how-to // tag::single-source[] -{description} Use pagination when you need to inspect large topics without being limited to the initial result set. +By default, the *Messages* tab on a topic returns up to the number of records set in *Max results*. Enable *Continuous Pagination* when you need to inspect a topic beyond that cap. -== Paginate topic messages - -When you browse a topic in {ui}, the *Messages* tab returns an initial batch of records. To retrieve additional records beyond this batch: +== Browse all messages in a topic . In the menu, go to *Topics* and select a topic. . Open the *Messages* tab. -. Apply any filters or a starting offset to narrow the records you want to inspect. +. (Optional) Set *Start offset* and *Max results*, or apply filters, to narrow the records you want to inspect. + ifdef::env-cloud[] See xref:manage:schema-reg/programmable-push-filters.adoc[] and xref:manage:schema-reg/record-deserialization.adoc[]. @@ -21,20 +19,22 @@ endif::[] ifndef::env-cloud[] See xref:console:ui/programmable-push-filters.adoc[] and xref:console:ui/record-deserialization.adoc[]. endif::[] -. After the initial results load, load the next batch to continue through the topic. +. Enable the *Continuous Pagination* toggle. +. Scroll the message list. {ui} keeps loading records until you reach the end of the topic. -You can continue paginating until you reach the end of the topic or the end of the offset range you specified. +When continuous pagination is on, the max results cap no longer limits the browsing session. == Performance considerations Retrieving large result sets increases load on the {ui} backend and the cluster. To keep responses fast: -* Narrow the result set with filters or a bounded offset range before paginating. +* Narrow the result set with filters or a bounded offset range before enabling continuous pagination*. ifdef::env-cloud[] * Use xref:manage:schema-reg/programmable-push-filters.adoc[JavaScript push filters] to match only the records you need. endif::[] ifndef::env-cloud[] * Use xref:console:ui/programmable-push-filters.adoc[JavaScript push filters] to match only the records you need. endif::[] +* Leave continuous pagination off and rely on max results when you only need a sample. // end::single-source[] From 6f26ee2f62b9c5bbc68122a55e02828c21ffe55d Mon Sep 17 00:00:00 2001 From: Michele Cyran Date: Thu, 30 Apr 2026 09:01:51 -0600 Subject: [PATCH 5/7] Update modules/console/pages/ui/paginate-messages-events.adoc Co-authored-by: Joyce Fee <102751339+Feediver1@users.noreply.github.com> --- modules/console/pages/ui/paginate-messages-events.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/console/pages/ui/paginate-messages-events.adoc b/modules/console/pages/ui/paginate-messages-events.adoc index 66eb7fe62c..cfc34b0f49 100644 --- a/modules/console/pages/ui/paginate-messages-events.adoc +++ b/modules/console/pages/ui/paginate-messages-events.adoc @@ -28,7 +28,7 @@ When continuous pagination is on, the max results cap no longer limits the brows Retrieving large result sets increases load on the {ui} backend and the cluster. To keep responses fast: -* Narrow the result set with filters or a bounded offset range before enabling continuous pagination*. +* Narrow the result set with filters or a bounded offset range before enabling continuous pagination. ifdef::env-cloud[] * Use xref:manage:schema-reg/programmable-push-filters.adoc[JavaScript push filters] to match only the records you need. endif::[] From f4855c8272fd8ebffcd79f9ac4ffca207e93ff7e Mon Sep 17 00:00:00 2001 From: micheleRP Date: Thu, 30 Apr 2026 09:05:19 -0600 Subject: [PATCH 6/7] incorporate review feedback --- modules/console/pages/ui/paginate-messages-events.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/modules/console/pages/ui/paginate-messages-events.adoc b/modules/console/pages/ui/paginate-messages-events.adoc index cfc34b0f49..2790fd67b7 100644 --- a/modules/console/pages/ui/paginate-messages-events.adoc +++ b/modules/console/pages/ui/paginate-messages-events.adoc @@ -5,11 +5,11 @@ // tag::single-source[] -By default, the *Messages* tab on a topic returns up to the number of records set in *Max results*. Enable *Continuous Pagination* when you need to inspect a topic beyond that cap. +By default, the *Messages* tab on a topic returns the number of records set in *Max results*. Enable *Continuous Pagination* when you need to inspect a topic beyond that cap. == Browse all messages in a topic -. In the menu, go to *Topics* and select a topic. +. Go to *Topics* and select a topic. . Open the *Messages* tab. . (Optional) Set *Start offset* and *Max results*, or apply filters, to narrow the records you want to inspect. + From 09f19c84fb121c351fe0ffc28973a894da546474 Mon Sep 17 00:00:00 2001 From: micheleRP Date: Thu, 30 Apr 2026 12:14:55 -0600 Subject: [PATCH 7/7] revert playbook --- local-antora-playbook.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/local-antora-playbook.yml b/local-antora-playbook.yml index 037ef2b4a2..0590286e59 100644 --- a/local-antora-playbook.yml +++ b/local-antora-playbook.yml @@ -18,7 +18,7 @@ content: - url: https://github.com/redpanda-data/docs branches: [v/*, shared, site-search,'!v-end-of-life/*'] - url: https://github.com/redpanda-data/cloud-docs - branches: 'DOC-1907-paginate-messages-events' + branches: 'main' - url: https://github.com/redpanda-data/redpanda-labs branches: main start_paths: [docs,'*/docs']