Skip to content

Commit dafc77c

Browse files
committed
List and Table updates to flaky-test-diagnosis-race-conditions-e2e-stabilization article
1 parent 8b99bc3 commit dafc77c

2 files changed

Lines changed: 96 additions & 65 deletions

File tree

  • src/content/articles/flaky-test-diagnosis-race-conditions-e2e-stabilization

‎src/content/articles/flaky-test-diagnosis-race-conditions-e2e-stabilization/index.mdx‎

Lines changed: 23 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -9,13 +9,11 @@ tags: ["reliability-and-testing", "typescript"]
99
featured: true
1010
---
1111

12-
*[E2E]: End-to-End
13-
*[CI]: Continuous Integration
1412
*[DOM]: Document Object Model
15-
*[API]: Application Programming Interface
13+
*[E2E]: End-to-End
1614
*[SPA]: Single Page Application
1715

18-
A single test with a 5% flake rate will block CI once every 20 runs. That sounds manageable. But a suite of 100 tests where each has just a 1% flake rate? That suite will fail 63% of builds. The probability compounds: $$(1 - 0.01)^{100} \approx 0.37$$, meaning only 37% of builds pass cleanly.
16+
A single test with a 5% flake rate will block CI once every 20 runs. That sounds manageable. <Highlighter>A suite of 100 tests where each has just a 1% flake rate? That suite will fail 63% of builds.</Highlighter> The probability compounds: $$(1 - 0.01)^{100} \approx 0.37$$, meaning only 37% of builds pass cleanly.
1917

2018
I've watched this pattern destroy teams' ability to ship. A few random failures appear. The first response is always the same: "just retry and merge." Within a few months, nobody trusts red builds anymore. Developers stop investigating failures because it's faster to retry than debug. Then a real regression ships because the failure was dismissed as "probably flaky." That's when teams lose the ability to distinguish signal from noise.
2119

@@ -44,7 +42,7 @@ await expect(page.locator('.success-message')).toBeVisible();
4442
```
4543
Code: Form submission race — wait for the API response before asserting.
4644

47-
The flaky version assumes the success message will appear immediately after click. The stable version explicitly waits for the API response, __then__ checks the DOM. The key insight: wait for the __event__ that causes the state change, not the state change itself.
45+
The flaky version assumes the success message will appear immediately after click. The stable version explicitly waits for the API response, _then_ checks the DOM. The key insight: wait for the _event_ that causes the state change, not the state change itself.
4846

4947
### The Debounced Search Race
5048

@@ -94,10 +92,10 @@ Race conditions are the biggest category, but they're not the only one. The seco
9492

9593
Environment issues cause roughly 25% of flakes. The test itself is often correct — it's the environment that's unstable. Tests depend on external state that varies between runs: cookies from previous tests, database records that weren't cleaned up, system time that behaves differently in different timezones.
9694

97-
The solution is isolation: each test should run in a pristine environment with no pollution from previous tests or external factors.
98-
9995
<Newsletter />
10096

97+
The solution is isolation: each test should run in a pristine environment with no pollution from previous tests or external factors.
98+
10199
### Browser State and Animations
102100

103101
Cookies, localStorage, and session data can leak between tests if you're reusing browser contexts. Modern SPAs also introduce animation timing issues — clicking an element mid-transition causes flakes. The cleanest approach is fresh contexts for each test with animations disabled:
@@ -129,7 +127,12 @@ Starting fresh is more reliable than cleaning up. Cleanup can fail silently (a c
129127
Database pollution is trickier. Three common approaches, each with tradeoffs:
130128

131129
<List
132-
variant="check-icons-list"
130+
variant="zebra-list"
131+
classes={{
132+
wrapper: "sm:ml-6 sm:mr-12 mb-6",
133+
ul: "!divide-y-0 !border-0 !overflow-visible !rounded-none space-y-4",
134+
li: "!rounded-lg !border border-trim",
135+
}}
133136
items={[
134137
{
135138
lead: "Transaction rollback",
@@ -170,12 +173,12 @@ Tests that work on your machine but fail in CI at specific hours often have time
170173

171174
For unit and integration tests, use your test framework's built-in time mocking. Vitest provides `vi.useFakeTimers()` which lets you control Date.now(), setTimeout, and other time-dependent APIs. For E2E tests running in the browser, you'll need to inject a fixed date via `page.addInitScript()` or use a library that patches the browser's Date object.
172175

173-
The simpler alternative is making assertions timezone-agnostic: compare timestamps rather than formatted date strings, and avoid assertions that depend on "today" or "this month."
174-
175176
<Callout type="info">
176177
If your test fails at midnight UTC, around month boundaries, or in different timezones, you have a time-dependent test. Either freeze time or make assertions timezone-agnostic.
177178
</Callout>
178179

180+
The simpler alternative is making assertions timezone-agnostic: compare timestamps rather than formatted date strings, and avoid assertions that depend on "today" or "this month."
181+
179182
## The Path Forward
180183

181184
Race conditions and environment issues cause 85% of flakes. Fixing these two categories transforms your test suite from a liability into an asset.
@@ -203,21 +206,24 @@ Race conditions and environment issues cause 85% of flakes. Fixing these two cat
203206
The approach is systematic:
204207

205208
<List
206-
variant="check-icons-list"
209+
variant="chevron-list"
210+
classes={{
211+
wrapper: "sm:ml-6 sm:mr-18 mb-6",
212+
}}
207213
items={[
208214
{
209-
lead: "Replace them with waits for specific conditions — network responses, DOM elements, URL changes.",
210-
text: "For race conditions, find every place your test uses `waitForTimeout()` or assumes immediate state changes after actions.",
215+
lead: 'For race conditions, find every place your test uses <span class="font-mono">waitForTimeout()</span> or assumes immediate state changes after actions.',
216+
text: "Replace them with waits for specific conditions — network responses, DOM elements, URL changes.",
211217
},
212218
{
213-
lead: "Clear browser state between tests. Isolate database state through transactions, per-test databases, or seed scripts. Freeze time or use timezone-agnostic assertions. Disable animations.",
214-
text: "For environment issues, audit your test setup.",
219+
lead: "For environment issues, audit your test setup.",
220+
text: "Clear browser state between tests. Isolate database state through transactions, per-test databases, or seed scripts. Freeze time or use timezone-agnostic assertions. Disable animations.",
215221
},
216222
]}
217223
/>
218224

219-
The deeper lesson is that flaky tests are symptoms. They reveal race conditions in your tests, instability in your application, or inconsistency in your environments. Fixing flakes often uncovers real bugs — an API that's slower under load, a component that renders before its data arrives, a cleanup process that doesn't handle edge cases.
220-
221225
<Callout type="success">
222226
Start small: pick your three worst flakes (highest impact, not necessarily highest flake rate), fix them this sprint, and measure CI pass rate before and after. A 10% improvement in CI reliability often translates to hours saved per week across the team.
223227
</Callout>
228+
229+
The deeper lesson is that flaky tests are symptoms. They reveal race conditions in your tests, instability in your application, or inconsistency in your environments. Fixing flakes often uncovers real bugs — an API that's slower under load, a component that renders before its data arrives, a cleanup process that doesn't handle edge cases.

0 commit comments

Comments
 (0)