You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 8d989e3
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: src/content/articles/backpressure-load-shedding-admission-control-overload/index.mdx
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -52,6 +52,8 @@ Overload doesn't just make systems slow — it makes them _worse_. There's a vic
52
52
53
53
The diagram shows three paths to failure, and they often happen simultaneously. The retry loop is the most insidious — well-intentioned client retry logic turns a temporary overload into a sustained assault. Thread pool saturation means requests that _could_ be processed sit waiting for a worker. And memory pressure from growing queues triggers garbage collection pauses that reduce your effective capacity right when you need it most.
54
54
55
+
<Newsletter />
56
+
55
57
This is why overload tends to get worse rather than self-correcting. Every mechanism designed for reliability under normal conditions — retries, timeouts, connection pooling — becomes a liability under overload.
56
58
57
59
<Callouttype="info">
@@ -131,8 +133,6 @@ Backpressure is the propagation of flow control signals from consumers back to p
131
133
132
134
The key principle I keep coming back to: you can't process your way out of overload. The only thing that actually reduces load is getting clients to send fewer requests. Backpressure is the mechanism that makes that happen.
133
135
134
-
<Newsletter />
135
-
136
136
### Explicit vs. Implicit Backpressure
137
137
138
138
I've seen several ways to signal overload, each with different tradeoffs:
Copy file name to clipboardExpand all lines: src/content/articles/blameless-postmortem-incident-analysis-systemic-causes/index.mdx
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -84,6 +84,8 @@ The divergence compounds over time. Blame-focused organizations accumulate hidde
84
84
85
85
There's a comforting belief that most engineers are careful and competent, and incidents happen because a few "bad apples" are careless or incompetent. If we could just identify and remove these people, incidents would stop.
86
86
87
+
<Newsletter />
88
+
87
89
This theory has been tested extensively in aviation, healthcare, and nuclear power. It doesn't hold up. "Careless" people aren't a distinct population you can screen out — everyone makes errors under the right conditions. Fatigue, time pressure, confusing interfaces, incomplete information, and conflicting priorities create errors in even the most skilled practitioners.
88
90
89
91
> If you believe the problem is a "bad apple," you'll try to remove the bad apple. If you believe the problem is the barrel, you'll redesign the barrel. The evidence from every high-reliability industry is clear: it's always the barrel.
@@ -166,8 +168,6 @@ If the answer to the last question is "hundreds of times," then the action itsel
166
168
167
169
The distinction matters because different types of accountability call for different responses. An engineer who made an honest mistake while following normal practices needs system improvements. A manager who created deadline pressure that encouraged skipping safety checks needs to change their leadership approach. These are both accountability — just not the "find someone to blame" kind.
168
170
169
-
<Newsletter />
170
-
171
171
### Prerequisites for Blameless Culture
172
172
173
173
Blameless postmortems don't work in a vacuum. They require organizational conditions that make psychological safety real, not just a stated value. I've seen teams try to adopt blameless practices while the broader organization still punishes mistakes — it doesn't work. People quickly learn that "blameless" is just rhetoric.
Copy file name to clipboardExpand all lines: src/content/articles/container-vulnerability-scanning-ci-shift-left-security/index.mdx
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -37,6 +37,8 @@ This article covers how to integrate container scanning that developers will act
37
37
38
38
Container images are layered, and vulnerabilities can exist at every layer. Understanding where they come from helps you prioritize what to fix and what to accept.
39
39
40
+
<Newsletter />
41
+
40
42
Here's the uncomfortable reality for a typical Node.js application: about 60% of vulnerabilities are in layers developers don't directly control.
41
43
42
44
<Table
@@ -328,8 +330,6 @@ container_scan:
328
330
```
329
331
Code: CI pipeline with Trivy scanning.
330
332
331
-
<Newsletter />
332
-
333
333
The `ignore-unfixed: true` (GH Action) option and `--ignore-unfixed` flag (GitLab CI) are crucial. Without it, you'll fail builds on CVEs that have no available patch — frustrating developers with problems they can't solve.
Copy file name to clipboardExpand all lines: src/content/articles/ephemeral-preview-environments-cost-control-cleanup/index.mdx
+20-20Lines changed: 20 additions & 20 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -688,26 +688,6 @@ Preview environments are worth the investment — catching bugs before productio
688
688
Start with TTL enforcement and PR-close cleanup — they'll catch 80% of waste. Add hibernation and right-sizing once the basics are working. Orphan detection is your safety net for everything else. Build iteratively, measure continuously, and adjust based on what you learn about your team's actual usage patterns.
description="Get the complete cost-control playbook for preview environments, including cleanup automation and infrastructure-sharing strategies."
695
-
listItems={[
696
-
{
697
-
text: "TTL policy implementation templates",
698
-
},
699
-
{
700
-
text: "Shared infrastructure isolation patterns",
701
-
},
702
-
{
703
-
text: "Hibernation and wake workflows",
704
-
},
705
-
{
706
-
text: "Preview cost dashboard blueprint",
707
-
},
708
-
]}
709
-
/>
710
-
711
691
The strategies in this article let you have it both ways: environments that are always available when developers need them, costs that stay predictable and reasonable. The key principles:
712
692
713
693
<List
@@ -739,4 +719,24 @@ The strategies in this article let you have it both ways: environments that are
description="Get the complete cost-control playbook for preview environments, including cleanup automation and infrastructure-sharing strategies."
726
+
listItems={[
727
+
{
728
+
text: "TTL policy implementation templates",
729
+
},
730
+
{
731
+
text: "Shared infrastructure isolation patterns",
732
+
},
733
+
{
734
+
text: "Hibernation and wake workflows",
735
+
},
736
+
{
737
+
text: "Preview cost dashboard blueprint",
738
+
},
739
+
]}
740
+
/>
741
+
742
742
The goal isn't to minimize spending — it's to maximize value per dollar. A preview environment that catches a bug before production is worth far more than the compute cost. But an environment for an abandoned PR is pure waste. Automate the distinction, and you'll have preview environments that accelerate development without budget surprises.
Copy file name to clipboardExpand all lines: src/content/articles/flaky-test-diagnosis-race-conditions-e2e-stabilization/index.mdx
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -74,6 +74,8 @@ Race conditions are the dominant cause of flakiness. The test and application ar
74
74
75
75
The telltale symptoms: the test passes locally but fails in CI, passes when you attach a debugger (which slows things down), or behaves inconsistently across different machines. The underlying issue is almost always the same — the test is asserting before the application has finished doing something.
76
76
77
+
<Newsletter />
78
+
77
79
Common patterns include clicking a button and immediately checking the result (before the async handler completes), interacting with an element while it's still animating, or making assertions before an API response arrives. The fix is always the same principle: __wait for the specific condition you need, not an arbitrary amount of time__.
78
80
79
81
### Environment Issues (~25% of Flakes)
@@ -151,8 +153,6 @@ Fixes depend on the dependency. For third-party APIs, mock them in tests or use
151
153
152
154
You can't fix what you can't measure. Before diving into specific flakes, you need infrastructure to detect them systematically and track trends over time. This means collecting test results across runs and analyzing patterns.
153
155
154
-
<Newsletter />
155
-
156
156
### Building a Flake Detection System
157
157
158
158
The foundation is storing every test result with enough metadata to analyze patterns later. You need the test identifier, pass/fail status, duration, commit SHA, timestamp, and any error messages. With this data, you can calculate flake rates and identify patterns.
Copy file name to clipboardExpand all lines: src/content/articles/kubernetes-cluster-upgrade-playbook-risk-reduction/index.mdx
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -124,6 +124,8 @@ __Compatibility verification__ catches the issues that break workloads. Check th
124
124
125
125
Deprecated APIs are the most common source of upgrade failures. An API that works today returns 404 after upgrade, breaking deployments, controllers, and CI pipelines. Catching these before the upgrade is essential.
126
126
127
+
<Newsletter />
128
+
127
129
The API server tracks which deprecated APIs are being actively used. Query the metrics endpoint to see what's at risk:
128
130
129
131
```bash
@@ -163,8 +165,6 @@ Run deprecated API detection continuously, not just before upgrades. Pluto in CI
163
165
164
166
Kubernetes components have strict version compatibility requirements. The API server can be at most one minor version ahead of the controller-manager and scheduler, which can be at most one minor version ahead of the kubelet. This means you can't upgrade everything simultaneously - there's a required sequence, and during parts of the upgrade you'll have components at different versions running together.
165
167
166
-
<Newsletter />
167
-
168
168
The good news: this is by design and fully supported. Kubernetes explicitly allows version skew during upgrades. If you're upgrading from 1.28 to 1.29, you'll temporarily have 1.29 API servers serving requests alongside 1.28 kubelets on worker nodes - and that's fine. Worker nodes continue operating normally while the control plane upgrades - they just can't create new pods or update existing ones until the API server is back. For HA clusters with multiple control plane nodes, you upgrade them one at a time, so at least one API server is always available (though there may be brief gaps during leader election).
Copy file name to clipboardExpand all lines: src/content/articles/kubernetes-cost-optimization-resource-sizing-spot-instances/index.mdx
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -40,6 +40,8 @@ __Requests__ are what the scheduler uses for placement decisions. When you set `
40
40
41
41
__Limits__ are enforcement boundaries. CPU limits throttle - if your container tries to use more than its limit, it gets slowed down but keeps running. Memory limits kill - exceed your memory limit and the kernel OOM-kills your container. Limits can exceed what's actually available on a node (overcommit), but requests cannot.
42
42
43
+
<Newsletter />
44
+
43
45
This distinction matters for cost because __requests determine how many nodes you need__. If every pod requests 1 CPU but only uses 0.1, you're paying for 10x the capacity you need. The scheduler sees the cluster as full when it's actually 90% idle.
44
46
45
47
Kubernetes assigns a QoS class based on how you set resources. __Guaranteed__ pods (requests equal limits for all resources) get the highest priority and are evicted last under pressure. __Burstable__ pods (requests less than limits) are the common case. __BestEffort__ pods (no requests or limits) are evicted first - avoid these in production. To control your pod's QoS class, set both CPU and memory requests equal to their limits for Guaranteed, or set requests lower than limits (or omit limits) for Burstable.
@@ -103,8 +105,6 @@ The fix isn't complicated - it's just tedious. You need historical usage data, y
103
105
104
106
Right-sizing requires historical usage data. You need to know what containers actually use, not what developers guessed they'd need. If you're running Prometheus with kube-state-metrics (and you should be), you already have everything you need.
105
107
106
-
<Newsletter />
107
-
108
108
The key metrics are P95 or P99 usage over a meaningful time window - at least 7 days to capture weekly patterns, ideally 30 days for workloads with monthly cycles. Average usage tells you what's typical, but peak usage tells you what you need to handle without degradation.
Copy file name to clipboardExpand all lines: src/content/articles/kubernetes-dns-debugging-ndots-coredns-troubleshooting/index.mdx
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -40,6 +40,8 @@ This article covers the DNS internals you need to debug production issues: how `
40
40
41
41
Understanding the DNS stack is essential for effective debugging. Kubernetes doesn't use a single DNS server. It's a layered system where your pod's `resolv.conf` configuration determines how queries are constructed, CoreDNS handles the actual resolution, and upstream servers resolve anything outside the cluster. Each layer has its own configuration and failure modes.
42
42
43
+
<Newsletter />
44
+
43
45
### How DNS Resolution Works
44
46
45
47
The first diagram shows internal service discovery — the fast path. When you look up a service name, the search domain gets appended, CoreDNS finds the service in the Kubernetes API, and you get an IP back immediately.
@@ -133,8 +135,6 @@ The default `ndots:5` exists because Kubernetes service names can have up to 4 d
133
135
134
136
The previous section mentioned ndots several times — now let's dig into why this single option causes so much trouble. In my experience, `ndots` is the root cause of most external DNS latency in Kubernetes. Understanding exactly how it works transforms a mysterious performance problem into a straightforward configuration fix.
135
137
136
-
<Newsletter />
137
-
138
138
### Understanding ndots Behavior
139
139
140
140
The rule is simple: if a hostname has fewer dots than the `ndots` value, the resolver tries search domains first. With the default `ndots:5`, any hostname with 4 or fewer dots (which includes nearly every external hostname) gets the search domain treatment.
The 50% target might seem conservative, but it's intentional. You want headroom to absorb traffic increases while HPA scales up. A target of 80% means you're already near capacity when HPA decides to act — and by the time new pods are ready, you've been overloaded for minutes.
82
82
83
+
<Newsletter />
84
+
83
85
### The Delay Problem
84
86
85
87
The HPA loop sounds fast—15-second intervals — but the total time from "traffic spike begins" to "new capacity receives traffic" is much longer. Every step in the pipeline adds latency.
@@ -320,8 +322,6 @@ spec:
320
322
321
323
Code: HPA with pod metrics (RPS, latency) and external metrics (queue depth).
322
324
323
-
<Newsletter />
324
-
325
325
The Prometheus Adapter translates PromQL queries into the custom metrics API format. Each rule maps a Prometheus metric to a Kubernetes resource (namespace, pod) and defines how to query it:
Copy file name to clipboardExpand all lines: src/content/articles/kubernetes-ingress-gateway-api-comparison-migration/index.mdx
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -37,6 +37,8 @@ The lesson: evaluate based on your actual requirements, not industry hype. The c
37
37
38
38
Before diving into features and migration, it helps to understand how these two APIs think about routing. Ingress is flat — one resource type handles everything. Gateway API is layered — different resource types for different concerns, managed by different teams.
39
39
40
+
<Newsletter />
41
+
40
42
### Ingress Model
41
43
42
44
Ingress uses a single resource type that combines routing rules, TLS configuration, and backend references. Everything lives in one YAML file, which makes simple cases easy but complex cases awkward. The catch is that Ingress itself only defines the _interface_ — the actual behavior depends entirely on which controller you're running. NGINX Ingress Controller interprets annotations one way; Traefik interprets them differently; some controllers ignore certain annotations entirely.
@@ -285,8 +287,6 @@ spec:
285
287
286
288
Code: Header-based routing in Gateway API.
287
289
288
-
<Newsletter />
289
-
290
290
With Ingress, you'd need to check whether your controller supports a custom annotation for header routing — and if you ever switch controllers, that annotation likely won't transfer.
0 commit comments