[rest] Add database branch and tag APIs

[JingsongLi]

Purpose

Add database-level writable branches and immutable tags to the REST client and protocol. Callers manage references with RESTCatalog.treeManagement() and access table versions through ordinary database names such as training$branch_experiment and training$tag_train_v1.

For example, catalog.getTable(Identifier.create("training$branch_experiment", "features")) uses the existing /v1/{prefix}/databases/{database}/tables/{table} endpoint with the full database name encoded as one path segment. A single catalog can access main, branches and tags without changing client state.

Changes

• Add paged reference listing, get, create, delete, and merge under database-scoped /trees. Mutations use JSON request objects. References contain a name and type, without public hashes or reference IDs.
• Define branch/tag-to-branch merge with automatic fast-forward, whole-table three-way merge, and no-op behavior. NORMAL, FORCE, and DROP can be selected globally or per table. Merge conflicts preserve server details through MergeConflictException.
• Reuse existing table paths, request objects and responses for reference access, including listing, create/get/alter/drop, snapshot/schema access, commit, token, and query authorization. Database lookup also accepts a selector so engines can validate virtual namespaces.
• Add DatabaseIdentifier.parse for the reserved, case-sensitive $branch_ and $tag_ suffix grammar. Table identifiers retain the full database name through serialization, loaders, and cache keys. Storage commits remove internally supplied physical table branch suffixes while retaining the database selector.
• Reject malformed or mixed selectors and unsupported reference operations. CREATE, DROP and ALTER DATABASE do not accept reference suffixes; reference management continues to use physical database names. Missing references never fall back to the physical database.
• Document Java and SQL usage, frozen tag reads, reserved-name compatibility, and a server MVP using native table branches/tags. No reference header, catalog option, separate bound client, or extra table resource paths are introduced.

This PR provides client and protocol support. Production reference storage, virtual namespace and table-version resolution, retention, namespace changes, and database merge execution require a catalog server implementation. The stateful test fixture resolves fixed references to native branches and pinned snapshots; it is not a production reference-management server.

Validation

• 112 targeted tests passed with Checkstyle, Spotless, and Enforcer enabled:

  mvn -pl paimon-core -am -DwildcardSuites=none -DfailIfNoTests=false '-Dtest=DatabaseIdentifierTest,RESTCatalogReferenceTest,ResourcePathsTest,RESTCatalogTreeManagementTest,RESTApiDatabaseReferenceTest,RequestJacksonCompatibilityTest,RESTApiSemanticViewTest,RESTCatalogSemanticViewManagementTest,MockRESTCatalogTest#testHeaderOptions+testCreateTableDefaultOptions' test

• Tests cover selector grammar, virtual database lookup, database mutation guards, request/response reuse, authentication, serialized loaders/tables, physical-branch commits, and error propagation without fallback. A real-data-file test verifies independent branch writes, pinned tag reads after source writes, and tag write rejection through a single catalog.
• cd docs && COREPACK_ENABLE_AUTO_PIN=0 yarn build passed, including OpenAPI validation (67 catalog operations and 10 management operations) and Docusaurus rendering.
• Java usage examples compile with Java 8 source/target settings. Documentation JSON examples, rendered section anchors, and git diff --check pass.

[leaves12138]

Reviewed da8e1af, specifically against the basic versioned-catalog workflows in Nessie.

Conclusion: this is a reasonable experimental reference-management API foundation, with a deliberately restricted fixed-table/offline server design. It does not yet cover the basic Nessie-style versioned-catalog workflow end to end. I did not find a blocking implementation defect in the added HTTP client code. The design gaps below are not claims that this PR implements a broken server: there is no reference/merge server implementation in this PR, and the guide explicitly defers these guarantees. I am leaving design feedback rather than approving it as Nessie-equivalent or treating documented scope exclusions as code defects.

What is covered at the contract/design level

• Reference listing, inspection, branch/tag creation, deletion, and merging branches or tags within one database.
• Complete-table-version three-way merge, automatic fast-forward where compatible with the selected modes, independent changes to different tables, same-table conflicts, NORMAL/FORCE/DROP overrides, and ancestry tracking for repeated merges including DROP.
• An offline workflow using pre-existing tables, paused writers, explicit table branch names, and tagged training inputs, once the catalog server implements the described orchestration and retention protections.
• Correct recognition that existing table mergeBranch/fastForward cannot simply be looped over to implement these database semantics; source and target must remain independently writable and pinned schemas/empty-table state need explicit handling.

These are meaningful foundations. In particular, row-level merging is not required to match Nessie's catalog-level model, and a public multi-table write-transaction API is not necessary for the first WAP workflow.

What is still needed for basic Nessie-like workflows

1. Reference-aware table resolution and membership. A database branch cannot yet be selected for list/load/create/drop/rename operations, and getReference returns only name/type. A newly started consumer with only a database tag cannot discover its frozen tables, schemas, and snapshot mappings without out-of-band source-branch knowledge. Explicit addressing is fine; it does not need to be a header, but the selected reference/revision must be carried consistently through the existing table APIs, loaders, and caches.
2. Atomic publication and a pinned read view. The guide explicitly allows non-atomic reads during multi-table publication. Even with all writers paused, a running query can see new features with old labels while an A/B merge is installed sequentially. This misses the basic WAP guarantee: branch-local changes become visible together. Stage immutable table-version mappings, atomically publish a database root, and allow a reader to pin that root for all table resolutions. Merely changing a map of mutable table-branch names is not sufficient. This can be implemented without exposing a general multi-table transaction API.
3. Opaque revisions, conditional updates, and retry semantics. Source and target are identified only by names, with no expected target revision or fixed source revision. A reviewed source branch can advance before merge; an administrative FORCE merge can overwrite target changes made after review; after a lost response, retrying a live-source merge may publish a different source state. The documented pause/freeze-tag workflow is a workaround for the restricted MVP, not an online concurrency contract. An opaque revision/generation plus expected-revision checks is sufficient; a public Git-style hash is not essential. A merge operation/result identifier or equivalent deduplication contract should distinguish retry from a new merge.

History/diff, dry-run or structured per-table conflict reports, and reference reset/rollback are useful next steps. Cherry-pick, cross-database transactions, and SQL BEGIN/COMMIT can be deferred. Note also that this API versions a single database rather than all namespaces in a Nessie repository; that is a deliberate narrower scope, not a naming detail.

Suggested acceptance boundary

Before calling the result a basic Nessie-like MVP, use a stateful server with real Paimon snapshots to verify: create an isolated branch; write A/B; freeze a tag and read it by tag alone after later source writes; publish A/B while readers remain active; reject a stale target revision; return the same result after a lost-response retry; preserve both target tables after conflicts; merge the same DROP source twice; continue writing independently after merge; and keep tagged data/schema readable after deleting its source reference. Add branch-local create/drop/rename and empty/schema-only table versions when those scopes are enabled.

Validation: all 80 targeted API/catalog/Jackson/semantic-view compatibility tests passed locally with Checkstyle, Spotless, and Enforcer enabled. GitHub CI was green at the final check, with Python skipped. These tests validate the client/wire contract, not server isolation, atomicity, retention, or the merge algorithm.

Comparison sources: the Nessie repository at c1b0e31b94ae151bb0a9a539ad857606410b0260, especially site/docs/guides/transactions.md, api/model/src/main/java/org/projectnessie/api/v2/TreeApi.java, its params/Merge.java / params/BaseMergeTransplant.java, and model/MergeBehavior.java.

[leaves12138]

Design coverage note for the Nessie comparison: deferring this is reasonable for the fixed-table management-API milestone, but it excludes a basic versioned-catalog scenario. A consumer starting with only (database, tag) cannot list that tag's tables or resolve their frozen schemas/versions: getReference only returns name/type and the documented read path additionally requires the original table branch. Likewise, branch-local CREATE/DROP/RENAME has no reference-aware route. Please treat reference-aware membership/table resolution as a required next milestone before claiming basic Nessie-like coverage. Explicit addressing is fine; no particular header or catalog-option design is required.

[leaves12138]

For a Nessie-style WAP workflow, this is the principal semantic gap, not merely the absence of a public transaction API. Pausing writers does not pause readers: while a two-table merge installs A and then B, a query can observe new A with old B. Nessie's branch/merge workflow exposes the collected changes through an atomic reference update. Before supporting that use case, the design needs an immutable database table-version mapping, one atomic publication point, and a way to pin that revision across table lookups. The explicitly restricted offline MVP can defer this, but it should not be treated as providing Nessie's basic multi-table publication guarantee.

[leaves12138]

For the broader versioned-catalog workflow, name/type plus merge modes cannot express “merge the source revision I validated into the target revision I reviewed.” A live source can advance before processing, and FORCE can overwrite newly arrived target changes without a stale-state check. Retrying after a lost success response can also merge a newer source state. The documented writer pause and immutable source-tag discipline mitigate this in the restricted MVP, but do not provide the online contract. Please reserve a follow-up for opaque source/target revisions (including expected-target checks) and an operation/result identity or equivalent retry contract. Public Git-style hashes are not necessary.