Skip to content

feat: add experimental tree command with agent index - #3005

Draft
kanoru3101 wants to merge 91 commits into
mainfrom
feat/tree-v2
Draft

feat: add experimental tree command with agent index#3005
kanoru3101 wants to merge 91 commits into
mainfrom
feat/tree-v2

Conversation

@kanoru3101

@kanoru3101 kanoru3101 commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

What/Why/How?

Adds the experimental tree command — one command that shows an API description's structure to humans and serves it as a navigable index to LLM agents.
Humans get quick orientation and impact analysis in multi-file descriptions ("what breaks if I change this schema?"); agents get a way to work with descriptions too large for a context window, borrowing the retrieval loop from PageIndex: build a small hierarchical index, let the model reason over it, and fetch only what it needs — fully deterministic here, since an API description already carries its structure and summaries (no AI calls, no keys).

Everything is powered by one new api-graph module in @redocly/openapi-core: a single walk of the ORIGINAL resolved files (lint pattern, no bundling) produces a dependency graph plus index metadata, and the CLI renders views over it:

redocly tree openapi.yaml                                  # human structure view with real file attribution
redocly tree openapi.yaml --uses schemas/Order             # impact analysis: affected paths and operations
redocly tree openapi.yaml --format=json --level 2          # agent index: sections + tags, a few KB for any spec size
redocly tree openapi.yaml --node Tickets                   # drill into a branch: operations with summaries and line ranges
redocly tree openapi.yaml --node 'GET /orders' --with-deps # leaf retrieval: raw source lines + resolved $refs + dependency closure

Every node carries a stable semantic id (GET /orders, schemas/Order), a JSON pointer, the defining file, and start_line/end_line, so an agent can follow up with plain file reads.
Components the root document declares keep their canonical section/Name ids even when they live in their own file, so --uses input forms work the same for single-file and multi-file APIs; a component no root entry declares (as in redocly split output) is addressed by its file path.
--with-deps returns a self-contained slice (operation plus everything it transitively references, in dependency order, 64 KB cap with an explicit truncated marker).
Unresolvable $refs render as ❌ nodes with stderr warnings; stdout stays clean JSON in machine formats.
The dependency graph (stylish/mermaid/dot/--files/--uses) and the hierarchical index (--format=json/--node) are two projections of the same analyzeApi pass; raw line slices are the retrieval ground truth, with $refs kept intact and a resolved refs[] list as hyperlinks.
v1 scope: full OpenAPI 2.0–3.2; AsyncAPI/Arazzo render as a flat referenced-components list.

Architecture note for review: the engine lives in core on purpose — the CLI is one of several planned surfaces; an MCP server and portal-side index generation would consume the same module directly.

Measured end to end on GitHub's own 9.8 MB REST API description (github/rest-api-description, used unmodified), with a BPE tokenizer over real command output — task: "create a repository for the authenticated user":

Input Tokens
Whole file 1,946,991
Full index, unfiltered 306,494
Three-step chain: --level 2--node repos--with-deps 49,610

This is the case the hierarchy exists for: at 1.9M tokens the file is ten 200k windows, and even the complete index does not fit one — so bounded drill-down is the only way an agent can touch it, and the chain still lands at a quarter of the window with room to work.
Per-step numbers, real JSON output, and the split-layout run (2,842 files) are in the benchmark guide.

Docs

Reference

Testing

  • Core api-graph unit tests — graph building, index assembly, retrieval slices, deps closure, outside-cwd path normalization, callback exclusion, split-alias canonical ids.
  • CLI tree unit tests — structure graph behavior, index filters, affected matching, printers.
  • E2e — every mode and flag combination incl. the full agent loop on a split multi-file fixture, --uses+--format=json, error paths, and the webhooks warning; snapshots pin real output.
  • Docs examples are captured real CLI output.

Screenshots (optional)

Check yourself

  • This PR follows the contributing guide
  • All new/updated code is covered by tests
  • Core code changed? - Tested with other Redocly products (internal contributions only)
  • New package installed? - Tested in different environments (browser/node)
  • Documentation update has been considered

Security

  • The security impact of the change has been considered
  • Code follows company security practices and guidelines

kanoru3101 and others added 30 commits June 11, 2026 16:38
Dynamic imports inside the test body made vitest transform the whole
untransformed dependency subtree within the test, exceeding the 5000ms
per-test budget when istanbul coverage is enabled. Static top-level
imports move that cost to the file-load phase, which has no timeout.
Add the `treeview` language to the example code fences in tree.md
(matching eject.md / translate.md house style) to satisfy markdownlint
MD040.

Stop tracking the internal agentic planning/spec docs under
docs/superpowers/: they were accidentally committed into the published
documentation and caused all vale and linkcheck failures plus most
markdownlint errors. Nothing references them; they remain available
locally but are no longer published.
The absolute-ref to node-id rule, the codepoint sort comparator, and the
operation-method set each had duplicate copies across build-graph.ts and
build-structure.ts. Move them to node-id.ts as the single source so the
two graph builders cannot drift.

Also fix build-graph's edge-refs sort to use the codepoint comparator
(was the default .sort()), matching the determinism the module documents,
and drop a redundant narrating comment in build-structure.

No behavior change: 58 tree unit tests and 11 e2e snapshots pass unchanged.
@changeset-bot

changeset-bot Bot commented Aug 3, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: ad2690a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@redocly/openapi-core Minor
@redocly/cli Minor
@redocly/client-generator Patch
@redocly/respect-core Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Performance Benchmark (Lower is Faster)

CLI Version Bundle Lint Check Config
cli-latest ▓ 1.00x (Fastest) ▓ 1.00x (Fastest) ▓ 1.01x ± 0.02
cli-next ▓ 1.01x ± 0.02 ▓ 1.01x ± 0.02 ▓ 1.00x (Fastest)

@kanoru3101 kanoru3101 changed the title feat: add tree with map command feat: add experimental tree command with agent index Aug 3, 2026
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 77.45% (🎯 77%) 12171 / 15713
🔵 Statements 77.51% (🎯 77%) 13045 / 16829
🔵 Functions 81.92% (🎯 81%) 2483 / 3031
🔵 Branches 71.06% (🎯 70%) 8906 / 12533
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
packages/cli/src/types.ts 100% 100% 100% 100%
packages/cli/src/commands/lint.ts 94.33% 87.17% 100% 94.33% 76-78, 143, 177
packages/cli/src/commands/tree/build-graph.ts 100% 94.44% 100% 100%
packages/cli/src/commands/tree/filter-affected.ts 100% 90% 100% 100%
packages/cli/src/commands/tree/filter-index.ts 94.73% 87.5% 100% 94.44% 33
packages/cli/src/commands/tree/index.ts 0% 0% 0% 0% 63-376
packages/cli/src/commands/tree/match-affected-by.ts 100% 100% 100% 100%
packages/cli/src/commands/tree/print/dot.ts 100% 100% 100% 100%
packages/cli/src/commands/tree/print/index-json.ts 0% 100% 0% 0% 4
packages/cli/src/commands/tree/print/json.ts 100% 100% 100% 100%
packages/cli/src/commands/tree/print/mermaid.ts 100% 75% 100% 100%
packages/cli/src/commands/tree/print/stylish.ts 100% 100% 100% 100%
packages/core/src/api-graph/build-graph.ts 95.1% 83.33% 100% 98.24% 124, 183-187, 227, 326, 428, 432, 441, 448
packages/core/src/api-graph/build-index.ts 93.9% 72.58% 95.83% 95.71% 147, 149-151, 234
packages/core/src/api-graph/node-id.ts 97.5% 93.18% 100% 100% 14
packages/core/src/api-graph/slice.ts 95.45% 83.52% 100% 98.94% 70, 83, 155, 209, 249
Generated in workflow #11147 for commit ad2690a by the Vitest Coverage Report Action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants