Skip to content

RDSC-5610: Multi-source documentation (conflict-resolved) - #4029

Merged
andy-stark-redis merged 39 commits into
mainfrom
RDSC-5610-resolved
Sep 17, 2026
Merged

andy-stark-redis merged 39 commits into
mainfrom
RDSC-5610-resolved

Conversation

@andy-stark-redis

@andy-stark-redis andy-stark-redis commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Resolves the merge conflicts on #4028 (itself a recreation of #3916 after its base branch was deleted following #3996's squash-merge) so this can land cleanly on main.

Both #3996 and #3916/#4028 independently edited the same 3 files after #3916 forked off #3996's branch. Conflicts resolved per-hunk on the actual content, not by picking one side wholesale:

Verified with a full hugo build: no new warnings, the merged release notes render all CVEs, and both installation pages render their respective sections once each (no duplication).

Closes #4028 in favor of this branch, since GitHub won't let a fork-branch PR be edited without maintainer-edit access.

🤖 Generated with Claude Code


Note

Low Risk
Documentation-only changes describing RDI 2.0 behavior; no application code or runtime config in this PR.

Overview
Documents RDI 2.0 capabilities across the Redis Data Integration docs, centered on multiple source databases in one pipeline and related operational changes.

Multi-source — Adds Multiple sources in one pipeline and updates architecture, pipelines, jobs (server_name), DLQ/stream naming, and observability (per-source collectors and metrics). “When to use” and the decision tree no longer treat a single source as required; API v1 is called out as single-source only.

Secrets and deploy — Documents per-database secrets via redis-di set-secret <KEY> --db <source|target> (e.g. MYSQL_DB_* env vars) with legacy SOURCE_DB_* / TARGET_DB_* noted for upgraded single-source pipelines. K8s examples use <source>-db secrets. Clearing a pipeline is redis-di deploy --empty instead of delete; start/stop/reset can target one source with --source.

Processor defaultFlink is documented as the default processors.type; classic is opt-in. Flink job matchers (lists / regex:) are documented for multi-table and multi-source jobs.

Upgrade / install — Expands upgrading to 2.0 (Flink default, clear vs delete, redeploy after clear, metrics key shape). Prepare-db guides, pipeline-config, FAQ, API migration, and install/HA pages are aligned with the above.

Reviewed by Cursor Bugbot for commit c1b014b. Bugbot is set up for automated code reviews on this repo. Configure here.

ZdravkoDonev-redis and others added 30 commits September 14, 2026 02:21
Co-authored-by: Stoyan Rachev <stoyanr@gmail.com>
Co-authored-by: Stoyan Rachev <stoyanr@gmail.com>
…vs-flink.md

Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com>
…e-dbs/_index.md

Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com>
…e-dbs/snowflake.md

Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com>
…e-dbs/snowflake.md

Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com>
…e-dbs/spanner.md

Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com>
…e-dbs/spanner.md

Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com>
Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com>
Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com>
Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com>
Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com>
Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com>
Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com>
Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com>
Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com>
Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com>
…e-dbs/mongodb.md

Co-authored-by: Zdravko Donev <zdravko.donev@redis.com>
@github-actions

Copy link
Copy Markdown
Contributor

Staging links:
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/1.19.1/data-pipelines/deploy
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/1.19.1/when-to-use
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/architecture/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/architecture/classic-vs-flink/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/data-pipelines/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/data-pipelines/deploy
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/data-pipelines/deploy/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/data-pipelines/multiple-sources/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/data-pipelines/pipeline-config/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/data-pipelines/prepare-dbs/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-mysql/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/data-pipelines/prepare-dbs/mongodb/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/data-pipelines/prepare-dbs/my-sql-mariadb/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/data-pipelines/prepare-dbs/oracle/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/data-pipelines/prepare-dbs/snowflake/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/data-pipelines/prepare-dbs/spanner/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/data-pipelines/prepare-dbs/sql-server/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/data-pipelines/prepare-dbs/supabase/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/data-pipelines/rejected-records/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/data-pipelines/supported-types/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/data-pipelines/transform-examples/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/data-pipelines/transform-examples/redis-json-example/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/faq/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/installation/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/installation/ha-test/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/installation/install-k8s/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/installation/install-vm/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/installation/migration-classic-to-flink/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/installation/upgrade/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/observability/
https://redis.io/docs/staging/RDSC-5610-resolved/integrate/redis-data-integration/when-to-use
https://redis.io/docs/staging/RDSC-5610-resolved/operate/rc/rdi/setup

@github-actions

Copy link
Copy Markdown
Contributor

🧠 Redis Memory

Found 5 related items from repository history (5 new this commit):

Memory updated at c1b014b

@andy-stark-redis
andy-stark-redis requested a review from a team September 17, 2026 10:21
@andy-stark-redis andy-stark-redis self-assigned this Sep 17, 2026
@andy-stark-redis
andy-stark-redis merged commit 31cd670 into main Sep 17, 2026
100 checks passed
@andy-stark-redis
andy-stark-redis deleted the RDSC-5610-resolved branch September 17, 2026 10:46
EliShteinman added a commit to EliShteinman/docs that referenced this pull request Sep 19, 2026
Documents RDI 2.0.0 multi-source pipelines, with a new multiple-sources page, per-database secret keys, and the RDI API reference moved to 2.0.0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants