DOC-6711 RDI config reference ideas

[andy-stark-redis]

WIP: don't merge yet!

This PR updates the way the RDI reference docs are formatted and delivered. The new pages are in folders with "v2" at the end of the title to allow comparison with the existing pages.

Two basic elements to this:

1. There's a corresponding branch on the RDI side that now exports the config and CLI info as JSON files (RDI currently exports these as finished Markdown files when the reference info gets updated).
2. There are new Hugo layouts for the CLI reference and config reference pages in the docs repo that use the JSON files directly.

This seems like a good separation of concerns - we can update the presentation of the reference on the docs side without a PR and rebuild on the RDI side.

The CLI ref layouts are largely the same as they were, but I've taken the opportunity to address another issue: finding RDI properties in the config file reference, which is long and complicated. The initial suggested improvements are:

• Properties as formatted list items rather than a table.
• Properties have individual page anchors, so you can link to them directly from other doc pages, blogs, etc.
• There is now a search filter to find properties in the list.

One particular feature of the filter to call out is the idea of providing preset collections of properties that might be helpful for particular tasks. The current ones are just placeholders (eg, "Flink processor tuning"), but of course we can have any collections we want here. A further refinement to this (fairly easy to add) is to allow the user to create their own property collections and add them to the menu. This could use local storage, but another option would be to encode the collection in the URL. This might be useful for sharing collections and also for links from elsewhere in the docs ("use this set of properties if you want to achieve xxx...")

Anyway, as usual, all feedback is welcome. If we decide to go ahead with this then I'll push the corresponding changes on the RDI side.

---

Note

Medium Risk
Large generated reference data and a new build gate affect published docs accuracy; scope is documentation and build tooling, not runtime product code.

Overview
Adds a data-driven v2 path for RDI reference docs alongside the existing hand-authored pages, using JSON under data/rdi-reference/ instead of Markdown exported from RDI.

Build / CI: New validate_rdi target runs build/validate_rdi_collections.py before hugo and serve_hugo, so every property path in collections.json must exist in config.json (schema drift fails the build).

Config reference (v2): config-yaml-reference-v2.md renders via the rdi-config-reference shortcode from config.json, with collections.json defining preset property groups (getting started, classic/Flink tuning).

CLI reference (v2): cli-v2/_content.gotmpl generates one Hugo page per redis-di command from cli.json (large Click-introspection dump), using rdi/cli-page-body.html for page bodies. Section index is cli-v2/_index.md (link title “CLI commands (v2)”).

Legacy reference pages are unchanged; v2 entries are for side-by-side comparison until cutover.

^{Reviewed by Cursor Bugbot for commit 4d03e85. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

DOC-6711

[github-actions]

Staging links:
https://redis.io/docs/staging/DOC-6711-rdi-allow-structure-format-of-reference-from-our-docs/integrate/redis-data-integration/reference/cli-v2/
https://redis.io/docs/staging/DOC-6711-rdi-allow-structure-format-of-reference-from-our-docs/integrate/redis-data-integration/reference/config-yaml-reference-v2/

[jit-ci]

🛡️ Jit Security Scan Results

✅ No security findings were detected in this PR

---

^{Security scan by Jit}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 4d03e85. Configure here.}

[cursor]

Dot paths split schema keys

Medium Severity

Property dot paths are built with real schema segment names (e.g. redis.batch.size, stream.name.pattern), but the UI splits displayPath on every .. That treats one key as several segments, so breadcrumb links and # anchors point at IDs that were never emitted for those fake segments.

 

^{Reviewed by Cursor Bugbot for commit 4d03e85. Configure here.}

[dwdougherty]

I haven't reviewed in detail all the new code, but I did skim it. I also looked at some of the preview content. As far as I can tell, it looks good to me. Approved.