[docs] Complete and reconcile the 0.9-to-1.0 upgrade notes

[wuchong]

Search before asking

• I searched in the issues and found no existing issue tracking this documentation update.

Description

The official 0.9-to-1.0 upgrade notes omit several configuration changes, compatibility constraints, and required operator actions documented in the release draft. Complete the upgrade-notes page using the draft as a reference, and reconcile its general upgrade guidance with the FIP-27 guidance already published.

References:

• Official upgrade notes from v0.9 to v1.0
• Google Docs release draft — Upgrade Notes, specifically the section between UPGRADE NOTE DRAFT START and UPGRADE NOTE DRAFT END.

The gaps below refer to the upgrade-notes page; some features may already have dedicated documentation elsewhere. Summarize upgrade impact and link to those pages where appropriate.

Missing changes and limitations

Authorization and configuration

• Additional DESCRIBE authorization checks. Explain that listOffsets requires table DESCRIBE permission, while databaseExists / tableExists return false when the principal lacks the relevant permission. Include the troubleshooting implication: an apparent missing database/table after upgrading may be a permission issue.
• Lower default bucket limit. Document the max.bucket.num default change from 128000 to 20000, enforcement per partition for partitioned tables, and TooManyBucketsException for creation requests exceeding the limit. Explain that existing tables are unaffected and when operators need to raise the limit.
• Remote storage configuration migration. Document remote.data.dirs taking precedence over remote.data.dir. When migrating an existing cluster, the old remote.data.dir value must be the first entry in the new list.
• Local multi-disk configuration compatibility. Document data.dirs taking precedence over data.dir when both are configured. Existing single-directory configurations continue to work; adopting multiple disks is optional.

Flink connector and client behavior

• Newly discovered partitions start from earliest. Explain that partitions present at streaming-job startup follow scan.startup.mode, while partitions discovered later always start from the earliest offset. Call out the behavior change for jobs that previously relied on latest to skip data in new partitions.
• Removal of bucket-level source-reader metrics. Document removal of the per-bucket currentOffset gauges under fluss.reader.table.<table>.bucket.<n> and the need to update dashboards/alerts. Mention the retained currentFetchEventTimeLag metric for monitoring read lag.
• JAAS configuration restricted to PlainLoginModule. Explain the restriction on client.security.sasl.jaas.config, the AuthenticationException for unsupported login modules, and the recommended client.security.sasl.username / client.security.sasl.password options.

Compatibility and operations

• Column statistics and the V1 log-batch format. Explain that enabling table.statistics.columns introduces batches that clients at v0.9 or earlier cannot decode. The default remains the compatible format. All servers and all clients reading the affected table must be upgraded to v1.0 before enabling the option.
• CoordinatorServer HA upgrade guidance. Document the server upgrade sequence (TabletServers one by one, then CoordinatorServer), adding standby coordinators after the cluster upgrade, and the absence of an additional HA feature flag. Align this with the lakehouse component ordering described below.
• Historical partition lookup limitations. Document that table.datalake.historical-partition.enabled supports only auto-partitioned Paimon primary-key tables with a single partition key, and that existing lookup jobs need a restart after changing the option.
• Column-statistics type limitations. List or link to the supported types: BOOLEAN, TINYINT, SMALLINT, INTEGER, BIGINT, FLOAT, DOUBLE, STRING, CHAR, DECIMAL, DATE, TIME, TIMESTAMP, and TIMESTAMP_LTZ. Explain the lack of statistics-based log filter pushdown for unsupported types such as ARRAY, MAP, and ROW.

Additional release and compatibility notes

• Mention that release artifacts no longer carry the -incubating suffix.
• Clarify which new capabilities require both v1.0 servers and v1.0 clients, including full KV scan, multi-table subscription, log filter pushdown, column statistics, and KV snapshot leases. Distinguish compatibility for existing workloads from enabling new features or table options.

Reconcile upgrade ordering and retain existing coverage

The draft ends with general advice to upgrade the cluster and then its clients/connectors. This needs qualification before publication: the official FIP-27 section already requires the following order for lakehouse deployments:

1. Lake-reading Flink connectors and their matching lake storage plugins.
2. Tiering service.
3. Fluss cluster.

• Rewrite the general compatibility guidance so it agrees with this component order and the server-internal rolling-upgrade sequence.
• Retain the FIP-27 clean/legacy schema compatibility matrix and rollback limitations, and the AWS SDK v2 credentials-provider migration guidance. These are already on the official page but absent from the draft.
• Reuse the existing ACL ALL permission and datalake.enabled sections, which already cover the corresponding draft material.

Completion criteria

Verify the draft-derived details against the final 1.0 behavior, keep the page focused on defaults, compatibility, scope of impact, and required actions, and ensure the resulting upgrade guidance is internally consistent.

Willingness to contribute

• I'm willing to submit a PR!