Skip to content

Commit 1ef370b

Browse files
committed
List and table styles to kubernetes-secrets-external-secrets-operator-csi-vault article
1 parent d2226c7 commit 1ef370b

2 files changed

Lines changed: 55 additions & 24 deletions

File tree

  • src/content/articles/kubernetes-secrets-external-secrets-operator-csi-vault

‎src/content/articles/kubernetes-secrets-external-secrets-operator-csi-vault/index.mdx‎

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,16 +18,17 @@ import patternDecisionDiagram from "./diagrams/pattern-decision-tree.jpg"
1818

1919
It's 3 AM and Vault is down. Your on-call engineer gets paged because deployments are failing — pods stuck in ContainerCreating, blocking a critical hotfix. Meanwhile, another team's services keep humming along despite the same outage. The difference isn't luck. It's how secrets get into pods.
2020

21-
Both teams use Vault. Both followed the documentation. But one team chose External Secrets Operator, which syncs secrets periodically and caches them as native Kubernetes secrets. The other chose the Secrets Store CSI Driver, which fetches secrets on-demand when pods start. When Vault went down, ESO's cached secrets kept working. CSI's synchronous fetches failed, and pods couldn't start. This isn't about which tool is better — it's about understanding the failure mode you've chosen __before__ it matters.
21+
Both teams use Vault. Both followed the documentation. But one team chose External Secrets Operator, which syncs secrets periodically and caches them as native Kubernetes secrets. The other chose the Secrets Store CSI Driver, which fetches secrets on-demand when pods start. When Vault went down, ESO's cached secrets kept working. CSI's synchronous fetches failed, and pods couldn't start. This isn't about which tool is better — it's about understanding the failure mode you've chosen _before_ it matters.
2222

2323
## The Two Patterns That Matter
2424

25-
External secrets management in Kubernetes has consolidated around two dominant patterns. External Secrets Operator runs as a controller in your cluster, periodically syncing secrets from Vault (or AWS Secrets Manager, Azure Key Vault, etc.) into native Kubernetes Secret objects. The Secrets Store CSI Driver takes a different approach: it mounts secrets directly into pods as volumes, fetching them from the external manager when pods start.
25+
<Highlighter>External secrets management in Kubernetes has consolidated around two dominant patterns.</Highlighter> External Secrets Operator runs as a controller in your cluster, periodically syncing secrets from Vault (or AWS Secrets Manager, Azure Key Vault, etc.) into native Kubernetes Secret objects. The Secrets Store CSI Driver takes a different approach: it mounts secrets directly into pods as volumes, fetching them from the external manager when pods start.
2626

2727
Both work fine when your secret manager is healthy. The difference is what happens when it isn't. ESO decouples secret fetching from pod lifecycle — the controller syncs independently, and pods consume cached Kubernetes Secrets. CSI couples them tightly — pods can't start until secrets are fetched. This architectural difference determines everything about how your applications behave during an outage.
2828

2929
<Table
3030
variant="vertical-column-delineation-table"
31+
fullWidth={false}
3132
content={{
3233
thead: {
3334
th: ['Pattern', 'Existing Pods', 'New Pods', 'Recovery'],
@@ -77,7 +78,10 @@ The pod __cannot start__ until this volume mount succeeds. If Vault is unavailab
7778
This has cascading implications that aren't obvious until you experience them:
7879

7980
<List
80-
variant="check-icons-list"
81+
variant="timeline-list"
82+
classes={{
83+
wrapper: "sm:ml-6 sm:mr-12 mb-6",
84+
}}
8185
items={[
8286
{
8387
text: 'New deployments block entirely — no pods can start',
@@ -108,6 +112,7 @@ The decision between ESO and CSI comes down to two questions: Can your applicati
108112

109113
<Diagram
110114
src={patternDecisionDiagram}
115+
size="full"
111116
title="Pattern decision tree."
112117
description="Flowchart guiding the choice between Kubernetes secret injection patterns based on failure tolerance. The process starts at Choose Secret Pattern and asks whether the application can tolerate minutes of staleness. If yes, the recommendation is ESO. If no, the chart asks whether the team accepts pod failures when the secret manager is down. If yes, the recommendation is CSI Driver. If no, the chart asks whether custom fallback logic is needed. If yes, the recommendation is Init Container. If no, the recommendation is ESO plus a short refresh interval. The diagram highlights that the main decision criteria are tolerance for stale secrets, tolerance for blocked pod startup during secret-manager outages, and whether the team is willing to build custom fallback behavior."
113118
/>
@@ -130,8 +135,8 @@ For most organizations, ESO is the right default. It's operationally simpler, Gi
130135

131136
The mistake isn't choosing either pattern. It's not understanding which failure mode you've chosen. The team that slept through the 3 AM Vault outage didn't get lucky — they understood that ESO's cached secrets would keep their services running. The team that got paged made a valid choice too; for their payment system, blocking on fresh credentials was the right call. Their runbooks reflected it.
132137

133-
Whatever you choose, document it. When the next outage happens, your incident responders shouldn't be learning your secret injection architecture for the first time.
134-
135138
<Callout type="success">
136139
Start with ESO for the majority of workloads, with a 15-minute refresh interval, and monitor sync status with alerts on stale ExternalSecrets. Add CSI for specific high-security applications where staleness is unacceptable. This gives you operational simplicity with escape hatches for edge cases.
137140
</Callout>
141+
142+
Whatever you choose, document it. When the next outage happens, your incident responders shouldn't be learning your secret injection architecture for the first time.

‎src/content/articles/kubernetes-secrets-external-secrets-operator-csi-vault/pdf.mdx‎

Lines changed: 45 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -29,24 +29,27 @@ import threeSecretDiagram from "./diagrams/three-secret-injection-patterns.jpg"
2929

3030
It's 3 AM and Vault is down. Your on-call engineer gets paged because deployments are failing — pods stuck in ContainerCreating, blocking a critical hotfix. Meanwhile, another team's services keep humming along despite the same outage. The difference isn't luck. It's how secrets get into pods.
3131

32-
Kubernetes secrets have a fundamental problem: they're base64-encoded, not encrypted. They sit in etcd alongside your cluster state, readable by anyone with RBAC access to the namespace. External secret managers like Vault, AWS Secrets Manager, and Azure Key Vault solve the security problem by keeping secrets outside the cluster — but they create a new problem. Your pods now depend on an external service, and that dependency has failure modes you need to understand __before__ the 3 AM page.
32+
Kubernetes secrets have a fundamental problem: they're base64-encoded, not encrypted. They sit in etcd alongside your cluster state, readable by anyone with RBAC access to the namespace. External secret managers like Vault, AWS Secrets Manager, and Azure Key Vault solve the security problem by keeping secrets outside the cluster — but they create a new problem. Your pods now depend on an external service, and that dependency has failure modes you need to understand _before_ the 3 AM page.
3333

3434
Three patterns dominate secret injection:
3535

3636
<List
37-
variant="check-icons-list"
37+
variant="badge-list"
38+
classes={{
39+
titleClass: "block w-full text-center",
40+
}}
3841
items={[
3942
{
40-
lead: 'External Secrets Operator',
41-
text: 'syncs secrets on a schedule and caches them as native Kubernetes secrets.',
43+
title: 'External Secrets Operator',
44+
lead: 'Syncs secrets on a schedule and caches them as native Kubernetes secrets.',
4245
},
4346
{
44-
lead: 'Secrets Store CSI Driver',
45-
text: 'fetches secrets on-demand when pods start.',
47+
title: 'Secrets Store CSI Driver',
48+
lead: 'Fetches secrets on-demand when pods start.',
4649
},
4750
{
48-
lead: 'Init containers',
49-
text: 'give you complete control but require you to implement everything yourself.',
51+
title: 'Init containers',
52+
lead: 'Gives you complete control but require you to implement everything yourself.',
5053
},
5154
]}
5255
/>
@@ -55,6 +58,7 @@ Each pattern fails differently when the secret manager goes away. Consider a 30-
5558

5659
<Table
5760
variant="vertical-column-delineation-table"
61+
fullWidth={false}
5862
content={{
5963
thead: {
6064
th: ['Pattern', 'Existing Pods', 'New Pods', 'Recovery'],
@@ -83,6 +87,8 @@ ESO's pods keep running with their last-synced secrets, and new pods start succe
8387

8488
<Diagram
8589
src={threeSecretDiagram}
90+
magnifyButtonPosition="top-left"
91+
size="full"
8692
title="Three secret injection patterns."
8793
description="Flowchart comparing three ways Kubernetes workloads receive secrets from an external secret manager. At the top, an external group labeled External Secret Manager contains Vault, AWS Secrets Manager, or another provider. Inside a grouped Kubernetes Cluster section, the first pattern labeled Pattern 1: ESO shows ESO Controller writing to a Kubernetes Secret, which is then consumed by a pod through envFrom or volumeMount. The second pattern labeled Pattern 2: CSI Driver shows the CSI Driver DaemonSet providing a pod volume mount. The third pattern labeled Pattern 3: Init Container shows an Init Container writing secret data into a shared emptyDir, which the Main Container then reads. The external manager connects to ESO through a path labeled Periodic sync and connects to both the CSI Driver and Init Container through paths labeled On pod start. The diagram emphasizes that ESO caches into native Kubernetes Secrets, CSI fetches during pod startup, and init containers implement a custom startup-time retrieval path."
8894
/>
@@ -164,7 +170,11 @@ Even with encryption at rest, native secrets have operational limitations. There
164170
External Secrets Operator is a Kubernetes operator that syncs secrets from external managers — Vault, AWS Secrets Manager, Azure Key Vault, GCP Secret Manager — into native Kubernetes secrets. The operator runs in your cluster, periodically fetches secrets from the external source, and creates or updates Kubernetes Secret objects that your pods consume normally.
165171

166172
<Table
167-
variant="vertical-column-delineation-table"
173+
variant="grid-and-accent-header-table"
174+
fullWidth={false}
175+
classes={{
176+
tbody: '!text-content',
177+
}}
168178
content={{
169179
thead: {
170180
th: ['Aspect', 'Native Secrets', 'External Manager'],
@@ -338,6 +348,7 @@ The key insight with ESO is the decoupling between secret fetching and pod lifec
338348

339349
<Diagram
340350
src={esoSyncDiagram}
351+
size="full"
341352
title="ESO sync flow."
342353
description="Sequence diagram showing how External Secrets Operator synchronizes secret values into Kubernetes. The participants are ExternalSecret, ESO Controller, Secret Manager, Kubernetes Secret, and Pod. The flow begins when the ESO Controller watches an ExternalSecret resource. Inside a loop that repeats every refreshInterval, the controller fetches secret values from the external Secret Manager, receives the values in response, and then creates or updates a Kubernetes Secret. Separately, a Pod mounts that Kubernetes Secret and receives the secret data. A note over the ESO Controller and Secret Manager explains that if the Secret Manager is down, the Kubernetes Secret keeps its last value. The diagram highlights that ESO fetches asynchronously on a controller schedule and that pods consume cached Kubernetes Secret data rather than making direct startup-time calls to the external manager."
343354
/>
@@ -472,14 +483,15 @@ spec:
472483
```
473484
Code: Pod with CSI secret volume.
474485

475-
The pod __cannot start__ until the volume mounts successfully — which means until secrets are fetched from Vault. This is the key difference from ESO: the dependency is synchronous.
486+
The pod _cannot start_ until the volume mounts successfully — which means until secrets are fetched from Vault. This is the key difference from ESO: the dependency is synchronous.
476487

477488
The CSI driver optionally supports rotation. Enable it by setting `--enable-secret-rotation=true` on the driver DaemonSet and adding `rotationPollInterval` to your SecretProviderClass. When enabled, the driver periodically re-fetches secrets and updates the mounted files. Your application must detect file changes (using inotify or periodic re-reads) to pick up rotated values — the driver updates files, but it can't restart your process.
478489

479490
The flow differs fundamentally from ESO: secrets are fetched synchronously during pod startup, not asynchronously by a controller.
480491

481492
<Diagram
482493
src={csiDriverDiagram}
494+
size="full"
483495
title="CSI driver secret fetch flow."
484496
description="Sequence diagram showing how the Secrets Store CSI Driver fetches secrets during pod startup. The participants are Pod, CSI Driver, Vault Provider, and HashiCorp Vault. The flow begins when the Pod sends a mount volume request to the CSI Driver. The CSI Driver asks the Vault Provider to fetch secrets. The provider authenticates to Vault and reads the requested values. Vault returns the secret values to the provider, the provider returns secret data to the CSI Driver, and the CSI Driver mounts the secrets as files into the Pod. A note explains that the pod starts only after the secrets are mounted. An alternate failure path shows the provider attempting to authenticate to Vault when Vault is unavailable. Vault returns a connection failure, the provider returns an error to the CSI Driver, the CSI Driver reports that the mount fails, and a note states that the Pod stays Pending. The diagram emphasizes that CSI secret retrieval is synchronous and directly coupled to pod startup."
485497
/>
@@ -687,7 +699,8 @@ The __init container__ pattern makes sense when you need behavior that ESO and C
687699
With three patterns to choose from, the decision comes down to two questions: how fresh do your secrets need to be, and what failure mode can your application tolerate?
688700

689701
<Table
690-
variant="vertical-column-delineation-table"
702+
variant="numbered-review-table"
703+
fullWidth={false}
691704
content={{
692705
thead: {
693706
th: ['Criteria', 'ESO', 'CSI Driver', 'Init Container', 'Native'],
@@ -728,38 +741,44 @@ With three patterns to choose from, the decision comes down to two questions: ho
728741
}}
729742
/>
730743

744+
The table captures raw capabilities, but choosing a pattern requires weighting those capabilities against your operational reality. A team with strong GitOps practices and tolerance for minutes of staleness will land in a different place than a team running payment infrastructure where a stale database credential means failed transactions. The deciding factor is usually the failure mode: ESO fails silently with stale data, CSI fails loudly by blocking pod startup, and init containers fail however you code them to. Each is the right answer for different risk profiles.
745+
731746
<List
732-
variant="check-icons-list"
747+
variant="card-grid-list"
733748
items={[
734749
{
735750
lead: 'ESO',
736-
text: 'fits most workloads. It\'s the simplest operationally: install the operator, create ExternalSecrets, and secrets flow into your cluster. GitOps works naturally because ExternalSecrets are declarative resources. You accept potential staleness — your secrets are only as fresh as your `refreshInterval`. For web applications, APIs, and most microservices, minutes of staleness is fine.',
751+
text: 'Fits most workloads. It\'s the simplest operationally: install the operator, create ExternalSecrets, and secrets flow into your cluster. GitOps works naturally because ExternalSecrets are declarative resources. You accept potential staleness — your secrets are only as fresh as your `refreshInterval`. For web applications, APIs, and most microservices, minutes of staleness is fine.',
737752
},
738753
{
739754
lead: 'CSI Driver',
740-
text: 'fits workloads that need fresh secrets at startup and can tolerate blocking failures. Payment processing, credential rotation during incident response, or compliance requirements that prohibit caching secrets in Kubernetes — these are CSI use cases. You accept brittleness: when Vault is down, pods don\'t start.',
755+
text: 'Fits workloads that need fresh secrets at startup and can tolerate blocking failures. Payment processing, credential rotation during incident response, or compliance requirements that prohibit caching secrets in Kubernetes — these are CSI use cases. You accept brittleness: when Vault is down, pods don\'t start.',
741756
},
742757
{
743758
lead: 'Init containers',
744-
text: 'fit edge cases where ESO and CSI don\'t provide enough control. Legacy applications that need secrets in unusual formats, multi-source fallback logic, or integration with systems that don\'t have ESO/CSI providers. You\'re writing and maintaining the logic yourself.',
759+
text: 'Fits edge cases where ESO and CSI don\'t provide enough control. Legacy applications that need secrets in unusual formats, multi-source fallback logic, or integration with systems that don\'t have ESO/CSI providers. You\'re writing and maintaining the logic yourself.',
745760
},
746761
{
747762
lead: 'Native secrets',
748-
text: 'fit non-sensitive configuration and development environments. With encryption at rest configured, they\'re acceptable for data that doesn\'t require rotation or audit trails.',
763+
text: 'Fits non-sensitive configuration and development environments. With encryption at rest configured, they\'re acceptable for data that doesn\'t require rotation or audit trails.',
749764
},
750765
]}
751766
/>
752767

753768
<Diagram
754769
src={patternDecisionDiagram}
770+
size="full"
755771
title="Pattern decision tree."
756772
description="Flowchart guiding the choice between Kubernetes secret injection patterns based on failure tolerance. The process starts at Choose Secret Pattern and asks whether the application can tolerate minutes of staleness. If yes, the recommendation is ESO. If no, the chart asks whether the team accepts pod failures when the secret manager is down. If yes, the recommendation is CSI Driver. If no, the chart asks whether custom fallback logic is needed. If yes, the recommendation is Init Container. If no, the recommendation is ESO plus a short interval. The diagram highlights that the main decision points are tolerance for stale secrets, tolerance for blocked pod startup, and willingness to build custom fallback behavior."
757773
/>
758774

759775
### Recommendations by Use Case
760776

761777
<List
762-
variant="check-icons-list"
778+
variant="zebra-list"
779+
classes={{
780+
wrapper: "sm:ml-6 sm:mr-12 mb-6",
781+
}}
763782
items={[
764783
{
765784
lead: 'Web applications and APIs',
@@ -797,7 +816,7 @@ Whichever pattern you choose, getting secrets into pods is only half the problem
797816
Whichever pattern you choose, the security fundamentals remain the same: authenticate with short-lived credentials, scope access narrowly, and restrict network paths.
798817

799818
<List
800-
variant="check-icons-list"
819+
variant="numbered-with-background-list"
801820
items={[
802821
{
803822
lead: 'Vault configuration',
@@ -915,7 +934,14 @@ spec:
915934
Code: Secret injection alerts.
916935

917936
<Table
918-
variant="vertical-column-delineation-table"
937+
variant="numbered-review-table"
938+
fullWidth={false}
939+
classes={{
940+
thead:
941+
'[&_th:nth-child(3)]:!text-center [&_th:nth-child(4)]:text-center',
942+
tbody:
943+
'[&_td:nth-child(3)]:!text-warning-offset [&_td:nth-child(4)]:text-danger [&_td:nth-child(3)]:!text-center [&_td:nth-child(4)]:text-center',
944+
}}
919945
content={{
920946
thead: {
921947
th: ['Metric', 'Warning', 'Critical', 'Response'],

0 commit comments

Comments
 (0)