Search before asking
Motivation
Fluss exposes remoteLogCopyBytesPerSecond to measure remote log upload throughput, but there is no equivalent metric for KV snapshot uploads.
KV snapshots upload newly created SST files and other snapshot files to remote storage. Without upload throughput metrics, users cannot observe the remote write traffic generated by KV snapshots or assess its contribution to the overall remote storage write workload.
Solution
Expose the following metrics in the existing table-level KV metric group:
| Metric |
Type |
Description |
remoteKvCopyBytes |
Counter |
Cumulative bytes of successfully uploaded KV snapshot files. |
remoteKvCopyBytesPerSecond |
Meter |
KV snapshot upload throughput in bytes per second, derived from the same counter. |
Record the uploaded file size in KvSnapshotDataUploader after each file upload completes successfully.
The metrics should follow these semantics:
- Include newly uploaded SST files and other snapshot files handled by the uploader.
- Exclude SST files reused from previous snapshots without uploading.
- Count successful file uploads even if another file upload or the subsequent snapshot commit fails.
- Count successful reuploads again, since they generate additional write traffic.
- Support concurrent file uploads with a thread-safe counter.
Anything else?
No response
Willingness to contribute
Search before asking
Motivation
Fluss exposes
remoteLogCopyBytesPerSecondto measure remote log upload throughput, but there is no equivalent metric for KV snapshot uploads.KV snapshots upload newly created SST files and other snapshot files to remote storage. Without upload throughput metrics, users cannot observe the remote write traffic generated by KV snapshots or assess its contribution to the overall remote storage write workload.
Solution
Expose the following metrics in the existing table-level KV metric group:
remoteKvCopyBytesremoteKvCopyBytesPerSecondRecord the uploaded file size in
KvSnapshotDataUploaderafter each file upload completes successfully.The metrics should follow these semantics:
Anything else?
No response
Willingness to contribute