Document using Longhorn with self-replicating components - #387
Open
l4zy0n3 wants to merge 1 commit into
Open
Conversation
✅ Deploy Preview for suse-obs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
l4zy0n3
force-pushed
the
longhorn-ha-storage-docs
branch
from
August 20, 2026 10:41
c6823e8 to
b007ebf
Compare
l4zy0n3
changed the title
l4zy0n3
force-pushed
the
longhorn-ha-storage-docs
branch
from
August 20, 2026 10:46
b007ebf to
d9eeb3a
Compare
l4zy0n3
changed the title
l4zy0n3
force-pushed
the
longhorn-ha-storage-docs
branch
from
August 20, 2026 10:49
d9eeb3a to
cfbcf45
Compare
Kafka, ZooKeeper, ClickHouse, and HDFS DataNodes already replicate their data at the application layer. Longhorn's default StorageClass also replicates each volume, multiplying storage usage on top of that. Document a single-replica, data-local Longhorn StorageClass for these components, the per-component override, and the caveats: strict-local is incompatible with RWX, and node drains hang by default with only one replica unless the Node Drain Policy is set to always-allow.
l4zy0n3
force-pushed
the
longhorn-ha-storage-docs
branch
from
August 20, 2026 10:53
cfbcf45 to
1912e1e
Compare
akashraj4261
approved these changes
Aug 20, 2026
rb3ckers
requested changes
Aug 21, 2026
|
|
||
| == Using Longhorn with self-replicating components | ||
|
|
||
| Kafka, ZooKeeper, ClickHouse, and HDFS DataNodes already replicate their data across pods/nodes at the application layer. If you run https://longhorn.io/[Longhorn] as your storage backend, its default StorageClass also replicates each volume (3 copies by default), multiplying storage usage on top of the application's own copies. |
Contributor
There was a problem hiding this comment.
For elasticsearch we need a helm chart update I believe, right?
Contributor
There was a problem hiding this comment.
We should also include that this applies to an HA deployment. For non-ha deployments we do not recommend this strict-local setup but instead we might want to recommend a storage class with only the default replica count set to 2 instead of 3.
We could actually suggest that also for all other PVCs in HA: use another custom storage class with replicaCount: "2" (instead of 3) by setting it as the global.storageClass. WDTY?
Contributor
There was a problem hiding this comment.
And we can include a link to Longhorn best practices:
See also Longhorn [IO Performance best practices](https://longhorn.io/docs/1.12.1/best-practices/#io-performance)
|
|
||
| [NOTE] | ||
| ==== | ||
| `dataLocality: strict-local` is incompatible with `ReadWriteMany` volumes. Also, on Longhorn with `numberOfReplicas: 1`, draining that node hangs indefinitely by default -- set the https://longhorn.io/docs/latest/references/settings/#node-drain-policy[Node Drain Policy] to `always-allow` beforehand. |
Contributor
There was a problem hiding this comment.
We should include a little bit more I think:
Suggested change
| `dataLocality: strict-local` is incompatible with `ReadWriteMany` volumes. Also, on Longhorn with `numberOfReplicas: 1`, draining that node hangs indefinitely by default -- set the https://longhorn.io/docs/latest/references/settings/#node-drain-policy[Node Drain Policy] to `always-allow` beforehand. | |
| `dataLocality: strict-local` is incompatible with `ReadWriteMany` volumes. Also, on Longhorn with `numberOfReplicas: 1`, draining that node hangs indefinitely by default -- set the https://longhorn.io/docs/latest/references/settings/#node-drain-policy[Node Drain Policy] to `always-allow` beforehand. | |
| When removing or replacing a node follow the [Longhorn documentation](https://longhorn.io/docs/1.12.1/nodes-and-volumes/nodes/graceful-node-removal/) to do this 1 node at a time only, and wait for the data replication on the database level to have completed. Without waiting for replication to finish data is likely to be lost. |
Contributor
There was a problem hiding this comment.
They should also make sure to know what to do when a node fails (and the disk is lost). Also documented by Longhorn: https://longhorn.io/docs/1.12.1/high-availability/node-failure/
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Documents how to configure Longhorn for components that already replicate their own data at the application layer (Kafka, ZooKeeper, ClickHouse, HDFS DataNodes), to avoid doubling up on storage-level replication.
Why
These components already keep multiple copies of their data across nodes. Longhorn's default StorageClass also replicates each volume (3 copies by default), multiplying storage usage unnecessarily on top of the application's own copies.
Changes
Using Longhorn with self-replicating componentsinstorage.adocnumberOfReplicas: 1,dataLocality: strict-local)storageClassoverride example for the affected componentsstrict-localis incompatible with RWX, and node drains hang indefinitely by default with a single replica unless Node Drain Policy is set toalways-allow