-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathmethodologies-comparison.html
More file actions
429 lines (409 loc) · 33.4 KB
/
methodologies-comparison.html
File metadata and controls
429 lines (409 loc) · 33.4 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>All Methodologies Compared — SaaS & Consumer side-by-side</title>
<style>
:root{
--ink:#1c2230; --ink-soft:#465065; --line:#e3ddd0; --paper:#faf7f0;
--card:#ffffff; --accent:#b5482e; --accent-soft:#fbeae4;
--gold:#9a7b2e; --gold-soft:#f6efdc; --teal:#1f6f6b; --teal-soft:#e2f0ee;
--blue:#2d4a7c; --blue-soft:#e6ecf6; --purple:#5b3b78; --purple-soft:#efe6f6;
--shadow:0 1px 2px rgba(28,34,48,.06),0 8px 24px rgba(28,34,48,.06);
}
*{box-sizing:border-box}
html{scroll-padding-top:56px}
body{margin:0;background:var(--paper);color:var(--ink);
font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Helvetica,Arial,sans-serif;
line-height:1.62;font-size:16px;-webkit-font-smoothing:antialiased}
.wrap{max-width:1200px;margin:0 auto;padding:0 24px 96px}
/* Jargon link-out — subtle, hover reveals accent. Mirror of framework.css a.j */
a.j{color:var(--ink);text-decoration:none;border-bottom:1px dotted var(--ink-soft)}
a.j:hover{color:var(--accent);border-bottom:1px solid var(--accent)}
header.masthead{border-bottom:3px solid var(--ink);margin-bottom:8px;padding:48px 0 22px}
.kicker{font-size:12px;letter-spacing:.18em;text-transform:uppercase;color:var(--ink-soft);font-weight:700;margin:0 0 10px}
h1{font-family:Georgia,"Times New Roman",serif;font-size:40px;line-height:1.08;margin:0 0 12px;letter-spacing:-.01em}
.sub{color:var(--ink-soft);margin:0;max-width:72ch}
.goal{margin-top:18px;background:var(--accent-soft);border:1px solid #f2c9bd;border-left:5px solid var(--accent);border-radius:10px;padding:13px 18px;font-size:15px}
.goal span{color:var(--accent);text-transform:uppercase;letter-spacing:.08em;font-size:11px;font-weight:800}
nav.toc{position:sticky;top:0;background:rgba(250,247,240,.92);backdrop-filter:blur(6px);
border-bottom:1px solid var(--line);padding:12px 0;margin-bottom:30px;z-index:5;font-size:14px}
nav.toc a{color:var(--ink-soft);text-decoration:none;margin-right:16px;white-space:nowrap}
nav.toc a:hover{color:var(--accent)}
.finding{background:var(--ink);color:#f3efe6;border-radius:14px;padding:24px 28px;margin:0 0 30px;box-shadow:var(--shadow)}
.finding h2{color:#fff;margin:0 0 8px;font-family:Georgia,serif;font-size:21px}
.finding p{margin:0 0 10px;color:#cfc9bb}
.finding p:last-child{margin:0}
.finding b{color:#f3d9a0}
h2.sec{font-family:Georgia,serif;font-size:26px;margin:36px 0 6px;padding-top:14px;border-top:2px solid var(--line)}
.secsub{color:var(--ink-soft);font-size:14.5px;margin:0 0 16px;max-width:75ch}
.legend{display:flex;flex-wrap:wrap;gap:8px;margin:6px 0 22px}
.tag{font-size:11px;font-weight:800;letter-spacing:.05em;text-transform:uppercase;padding:5px 11px;border-radius:6px;color:#fff;white-space:nowrap}
.tag.scoring{background:var(--accent)} .tag.input{background:var(--purple)}
.tag.experiment{background:var(--blue)} .tag.goal{background:var(--gold)} .tag.qual{background:var(--teal)} .tag.define{background:var(--gold)}
.tag.ailab{background:#3a6859}
.tag.saas{background:var(--ink);color:#f3efe6}
.tag.consumer{background:#7a8aa3;color:#fff}
/* The big table */
.tblwrap{overflow-x:auto;margin:14px 0;border:1px solid var(--line);border-radius:12px;box-shadow:var(--shadow);background:#fff}
table.comp{border-collapse:collapse;width:100%;font-size:13px;min-width:1100px}
table.comp thead th{background:var(--ink);color:#f3efe6;padding:12px 11px;text-align:left;font-size:11px;letter-spacing:.06em;text-transform:uppercase;position:sticky;top:0;vertical-align:bottom}
table.comp tbody td{padding:12px 11px;border-bottom:1px solid var(--line);vertical-align:top;font-size:13px;line-height:1.5}
table.comp tbody tr:hover{background:#f6f4ee}
table.comp tbody tr:last-child td{border-bottom:none}
table.comp td.method{font-weight:700;font-size:14px}
table.comp td.method a{color:var(--accent);text-decoration:none;border-bottom:1px solid var(--accent)}
table.comp td.method a:hover{color:var(--ink)}
table.comp td .from{font-size:11.5px;color:var(--ink-soft);display:block;margin-top:2px;font-weight:400}
table.comp td .badges{margin-top:4px}
table.comp td .badges .tag{font-size:9.5px;padding:2px 6px;margin-right:3px;letter-spacing:.03em}
table.comp td.not{color:#8a4f4f;font-style:italic}
table.comp td.cost{font-weight:700;text-align:center;font-family:Georgia,serif;font-size:14px}
.cost.low{color:var(--teal)} .cost.med{color:var(--gold)} .cost.high{color:var(--accent)}
/* Category sections */
.catsec{background:#fff;border:1px solid var(--line);border-radius:12px;padding:18px 22px;margin:14px 0;box-shadow:var(--shadow)}
.catsec.scoring{border-left:5px solid var(--accent)}
.catsec.input{border-left:5px solid var(--purple)}
.catsec.experiment{border-left:5px solid var(--blue)}
.catsec.goal{border-left:5px solid var(--gold)}
.catsec.qual{border-left:5px solid var(--teal)}
.catsec.ailab{border-left:5px solid #3a6859}
.catsec h3{margin:0 0 8px;font-family:Georgia,serif;font-size:18px}
.catsec p{margin:0 0 6px;font-size:14px;color:var(--ink-soft);line-height:1.55}
.catsec ul{margin:8px 0 0;padding-left:20px;font-size:13.5px;color:var(--ink-soft)}
.catsec li{margin:4px 0}
.catsec a{color:var(--teal);text-decoration:none;border-bottom:1px solid var(--teal)}
.catsec a:hover{color:var(--accent);border-color:var(--accent)}
/* our pick */
.ours{background:var(--teal-soft);border-left:5px solid var(--teal);border-radius:10px;padding:18px 22px;margin:18px 0}
.ours h3{margin:0 0 8px;color:var(--teal);font-size:18px;font-family:Georgia,serif}
.ours .draft{font-size:11px;background:var(--teal);color:#fff;padding:3px 9px;border-radius:5px;text-transform:uppercase;letter-spacing:.06em;font-weight:700}
.ours code{font-family:"SF Mono",ui-monospace,Menlo,Consolas,monospace;font-size:13px;background:#fff;border:1px solid #bcdcd8;border-radius:5px;padding:2px 7px}
a.cite{color:var(--teal);text-decoration:none;border-bottom:1px solid var(--teal)}
a.cite:hover{color:var(--accent);border-color:var(--accent)}
footer{border-top:1px solid var(--line);padding-top:20px;color:var(--ink-soft);font-size:13px;margin-top:44px}
footer a{color:var(--teal)}
@media(max-width:680px){h1{font-size:30px}}
</style>
</head>
<body>
<nav class="sitenav">
<style>
.sitenav{position:fixed;top:12px;right:16px;z-index:100;font-size:13px;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Helvetica,Arial,sans-serif}
.sitenav details{background:#1c2230;color:#f3efe6;border-radius:8px;box-shadow:0 4px 16px rgba(0,0,0,.22)}
.sitenav summary{list-style:none;cursor:pointer;padding:8px 14px;font-weight:700;letter-spacing:.04em;user-select:none}
.sitenav summary::-webkit-details-marker{display:none}
.sitenav summary::after{content:" \25BE";color:#cfc9bb;font-size:10px;margin-left:2px}
.sitenav details[open] summary::after{content:" \25B4"}
.sitenav .navmenu{position:absolute;top:calc(100% + 6px);right:0;background:#fff;color:#1c2230;border:1px solid #e3ddd0;border-radius:10px;padding:16px 18px;box-shadow:0 8px 28px rgba(28,34,48,.22);display:grid;grid-template-columns:repeat(2,minmax(180px,1fr));gap:14px 26px;min-width:400px;max-width:92vw;max-height:80vh;overflow-y:auto}
.sitenav .navgrp h4{margin:0 0 5px;font-size:10.5px;letter-spacing:.09em;text-transform:uppercase;color:#465065;font-weight:700;border-bottom:1px solid #e3ddd0;padding-bottom:4px}
.sitenav .navgrp a{display:block;color:#1c2230;text-decoration:none;font-size:13px;padding:2px 0;line-height:1.5}
.sitenav .navgrp a:hover{color:#b5482e}
.sitenav .navgrp a.cur{font-weight:700;color:#b5482e}
@media(max-width:600px){.sitenav .navmenu{grid-template-columns:1fr;min-width:240px;max-width:88vw}.sitenav{top:8px;right:8px}}
</style>
<details>
<summary>📑 Jump to</summary>
<div class="navmenu">
<div class="navgrp"><h4>Start here</h4>
<a href="index.html"><b>← Home (goal & map)</b></a>
<a href="impact-saas-companies.html">SaaS / B2B field study</a>
<a href="impact-consumer-companies.html">Consumer-tech field study</a>
<a class="cur" href="methodologies-comparison.html"><b>All methods compared →</b></a>
<a href="experiment-trustworthiness.html">How 40k tests actually work →</a>
<a href="jargon.html">Jargon (glossary)</a>
</div>
<div class="navgrp"><h4>Scoring</h4>
<a href="rice-framework.html">RICE (Intercom)</a>
</div>
<div class="navgrp"><h4>Define-first</h4>
<a href="north-star-framework.html">North Star (Amplitude / Slack)</a>
<a href="v2mom-framework.html">V2MOM (Salesforce)</a>
<a href="pyramid-of-clarity-framework.html">Pyramid of Clarity (Asana)</a>
<a href="heart-framework.html">HEART (Google)</a>
<a href="dibb-framework.html">DIBB (Spotify)</a>
</div>
<div class="navgrp"><h4>Written discipline</h4>
<a href="pr-faq-framework.html">PR-FAQ / Working Backwards (Amazon)</a>
<a href="stripe-shaping-framework.html">Stripe shaping</a>
</div>
<div class="navgrp"><h4>Experimentation (SaaS)</h4>
<a href="microsoft-exp-framework.html">Microsoft ExP / CUPED</a>
<a href="linkedin-xlnt-framework.html">LinkedIn T-REX</a>
</div>
<div class="navgrp"><h4>Experimentation (Consumer)</h4>
<a href="netflix-experimentation.html">Netflix · ABlaze</a>
<a href="booking-experimentation.html">Booking.com</a>
<a href="airbnb-erf-framework.html">Airbnb ERF</a>
<a href="uber-xp-framework.html">Uber XP</a>
<a href="doordash-switchback-framework.html">DoorDash switchback</a>
<a href="lyft-experimentation.html">Lyft</a>
<a href="pinterest-ab-framework.html">Pinterest</a>
</div>
<div class="navgrp"><h4>AI labs</h4>
<a href="anthropic-pm-on-ai-exponential.html">Anthropic · PM on AI exponential</a>
<a href="google-customer-zero-2026.html">Google · "Customer zero" 2026</a>
</div>
</div>
</details>
</nav>
<div class="wrap">
<header class="masthead">
<p class="kicker">Methods · Side-by-side comparison</p>
<h1>All Methodologies Compared — SaaS & Consumer, side-by-side</h1>
<p class="sub">Two tables. <strong>Prioritization frameworks</strong> answer the core question: <em>"100 features, 10 engineers — which 10 first?"</em> <strong>Experimentation</strong> answers a different question: <em>"of what we shipped, which actually worked?"</em> Each row links to its full deep-dive; each claim about <em>what a method does</em> traces back to that company's own published source.</p>
<div class="goal"><span>Goal</span><br>Decide features by data-backed expected impact — choose by outcome, not by to-do list or opinion.</div>
</header>
<nav class="toc">
<a href="#headline">How to read</a>
<a href="#table">Prioritization frameworks</a>
<a href="#exp-table">Experimentation</a>
<a href="#cats">Camps explained</a>
<a href="index.html" style="color:var(--blue);font-weight:700">← Home</a>
</nav>
<div class="finding" id="headline">
<h2>How to read this — and the one thing it makes obvious</h2>
<p>Methods sort into <b>two questions</b>. <b>Three camps answer "100 features, 10 engineers — which 10 first?"</b>: <span class="tag scoring">Scoring</span> <span class="tag define">Define-first</span> <span class="tag qual">Written discipline</span>. They're complementary — scoring without a goal is theatre; define-first without scoring is alignment but no priority; written discipline without either is taste.</p>
<p>The other group on this page, <span class="tag experiment">Experimentation</span>, answers a different question — <em>"of what we shipped, which actually worked?"</em> — and <span class="tag ailab">AI labs</span> publish build-side velocity claims, not prioritisation rubrics. Both are kept on this page for context; Experimentation lives in <a href="#exp-table" style="color:#9bc7ef">its own table</a> below.</p>
<p>The point of the tables is to make the gaps obvious — <em>which jobs does each method actually do, and which does it leave to a partner method?</em></p>
</div>
<div class="legend">
<span class="tag saas">SaaS / B2B</span>
<span class="tag consumer">Consumer-tech</span>
<span class="tag scoring">Scoring</span>
<span class="tag define">Define-first</span>
<span class="tag qual">Written discipline</span>
<span class="tag experiment">Experimentation</span>
<span class="tag ailab">AI lab</span>
</div>
<!-- PRIORITIZATION FRAMEWORKS TABLE -->
<h2 class="sec" id="table">Prioritization frameworks — pre-build "which to build first?"</h2>
<p class="secsub">The three camps that answer the core question, plus the AI-lab context rows. Scroll right on mobile. The <em>What it doesn't do</em> column is the most useful one — it reveals what you have to pair the method with.</p>
<div class="tblwrap">
<table class="comp">
<thead>
<tr>
<th>Method & source</th>
<th>Category</th>
<th>What it decides</th>
<th>What it does NOT do</th>
<th>Best for</th>
<th>Deep-dive</th>
</tr>
</thead>
<tbody>
<!-- SCORING -->
<tr>
<td class="method">RICE<div class="from">Intercom</div><div class="badges"><span class="tag saas">SaaS</span></div></td>
<td><span class="tag scoring">Scoring</span></td>
<td>Ranks candidate ideas with one formula: <code>R×I×C÷E</code>. Reveals weakest input as low Confidence.</td>
<td class="not">Pick the goal. Estimate the lift. Replace experiments.</td>
<td>Long candidate lists where every idea is "kind of good." Cheap weekly use.</td>
<td class="method"><a href="rice-framework.html">RICE →</a></td>
</tr>
<!-- DEFINE-FIRST: NORTH STAR (was Input modeling — merged) -->
<tr>
<td class="method">North Star Framework<div class="from">Amplitude (also Slack)</div><div class="badges"><span class="tag saas">SaaS</span></div></td>
<td><span class="tag define">Define-first</span></td>
<td>Picks <em>which metric</em> to move; forces feature → input mapping; identifies leverage points.</td>
<td class="not">Rank features. Estimate lifts. Provide a per-feature score.</td>
<td>Teams that ship many features but can't say which metric they're moving.</td>
<td class="method"><a href="north-star-framework.html">North Star →</a></td>
</tr>
<!-- DEFINE-FIRST (PR-FAQ, HEART, DIBB, V2MOM, Pyramid) -->
<tr>
<td class="method">PR-FAQ / <a href="pr-faq-framework.html">Working Backwards</a><div class="from">Amazon</div><div class="badges"><span class="tag consumer">Consumer</span></div></td>
<td><span class="tag define">Define-first</span></td>
<td>Mock press release + FAQ written <em>before</em> code. Forces customer value & success metrics into writing.</td>
<td class="not">Score against alternatives. Measure actual lift.</td>
<td>Big bets where the team can't agree on customer value.</td>
<td class="method"><a href="pr-faq-framework.html">PR-FAQ →</a></td>
</tr>
<tr>
<td class="method">HEART + Goals-Signals-Metrics<div class="from">Google</div><div class="badges"><span class="tag consumer">Consumer</span></div></td>
<td><span class="tag define">Define-first</span></td>
<td>Picks <em>which metrics</em> to watch (Happiness/Engagement/Adoption/Retention/Task-success) + ladders Goal → Signal → Metric.</td>
<td class="not">Estimate lift. Decide priority between features.</td>
<td>Any feature — cheap 30-min exercise to stop vague "engagement" arguments.</td>
<td class="method"><a href="heart-framework.html">HEART →</a></td>
</tr>
<tr>
<td class="method">DIBB + Bets Board<div class="from">Spotify</div><div class="badges"><span class="tag consumer">Consumer</span></div></td>
<td><span class="tag define">Define-first</span></td>
<td>Data → Insight → Belief → Bet chain. Each arrow must hold. Bets Board limits concurrent bets.</td>
<td class="not">Replace a scoring formula. Run experiments.</td>
<td>Killing under-evidenced bets before commitment.</td>
<td class="method"><a href="dibb-framework.html">DIBB →</a></td>
</tr>
<tr>
<td class="method">V2MOM<div class="from">Salesforce</div><div class="badges"><span class="tag saas">SaaS</span></div></td>
<td><span class="tag define">Define-first</span></td>
<td>One-page cascade: Vision/Values/Methods/Obstacles/Measures. Every leader writes one aligned to the CEO's.</td>
<td class="not">Pick which feature ships next. Estimate impact.</td>
<td>Setting direction when the team is misaligned on <em>what we're trying to do</em>.</td>
<td class="method"><a href="v2mom-framework.html">V2MOM →</a></td>
</tr>
<tr>
<td class="method">Pyramid of Clarity<div class="from">Asana</div><div class="badges"><span class="tag saas">SaaS</span></div></td>
<td><span class="tag define">Define-first</span></td>
<td>Mission → Strategy → Objectives → Key Results → Projects. Each layer must reference the one above.</td>
<td class="not">Score projects against each other within a KR.</td>
<td>Visibly catching projects that don't trace to any goal.</td>
<td class="method"><a href="pyramid-of-clarity-framework.html">Pyramid →</a></td>
</tr>
<!-- WRITTEN DISCIPLINE -->
<tr>
<td class="method"><a href="stripe-shaping-framework.html">Shaping</a> + Deep-dives + LT filter<div class="from">Stripe</div><div class="badges"><span class="tag saas">SaaS</span></div></td>
<td><span class="tag qual">Written discipline</span></td>
<td>Qualitative: shape the problem, write long-form, filter through multi-decade direction.</td>
<td class="not">Produce a number. Anything quantitative.</td>
<td>High-judgement bets at orgs that already have a writing culture.</td>
<td class="method"><a href="stripe-shaping-framework.html">Stripe →</a></td>
</tr>
<!-- AI LABS -->
<tr>
<td class="method"><a href="anthropic-pm-on-ai-exponential.html">Prototype-first habits</a><div class="from">Anthropic (Claude Code)</div><div class="badges"><span class="tag saas">SaaS</span></div></td>
<td><span class="tag ailab">AI lab</span></td>
<td>Five habits for working when the AI substrate improves week-over-week: short sprints, demos over docs, side quests alongside the roadmap, capability-first prototyping, daily-use the product.</td>
<td class="not">Rank features pre-build. Provide a score. Tell you how to verify post-launch. Handle non-<a class="j" href="jargon.html#dogfooding">dogfood</a>-able work.</td>
<td>AI-native product teams whose staff are the target user; fast-moving capability-led products.</td>
<td class="method"><a href="anthropic-pm-on-ai-exponential.html">Anthropic →</a></td>
</tr>
<tr>
<td class="method"><a href="google-customer-zero-2026.html">"Customer zero" velocity story</a><div class="from">Google (Sundar Pichai, 2026)</div><div class="badges"><span class="tag saas">SaaS</span><span class="tag consumer">Consumer</span></div></td>
<td><span class="tag ailab">AI lab</span></td>
<td>Verbatim 2026 internal-velocity claims: 75% AI-generated & engineer-approved code, 6× faster code migration, 6× more developer-tool tokens in 2 months, training in weeks-not-months, idea-to-Swift-prototype in days. "Customer zero" framing — Google builds with its own agents first.</td>
<td class="not">Rank features pre-build. Provide a score. Tell you what to ship, keep, or kill. Explain why Project Mariner was shut down (no public rationale).</td>
<td>Teams asking "what's the build-side leverage of leaning into agentic coding tooling?" — not teams asking "how should we decide what to build?"</td>
<td class="method"><a href="google-customer-zero-2026.html">Google →</a></td>
</tr>
</tbody>
</table>
</div>
<!-- EXPERIMENTATION TABLE -->
<h2 class="sec" id="exp-table">Experimentation — post-build "did it work?"</h2>
<p class="secsub">A different question from the table above. These methods presume the prioritization decision is already made; they produce causal answers to whether shipped features worked. Useful for the <em>next</em> round of picks, not the first one.</p>
<div class="tblwrap">
<table class="comp">
<thead>
<tr>
<th>Method & source</th>
<th>Category</th>
<th>What it decides</th>
<th>What it does NOT do</th>
<th>Best for</th>
<th>Deep-dive</th>
</tr>
</thead>
<tbody>
<tr>
<td class="method">ExP + OEC + <a class="j" href="jargon.html#cuped">CUPED</a><div class="from">Microsoft</div><div class="badges"><span class="tag saas">SaaS</span></div></td>
<td><span class="tag experiment">Experimentation</span></td>
<td>Measures causal impact via controlled tests; OEC = the one success metric; CUPED halves required sample.</td>
<td class="not">Decide what to test (needs a candidate list). Work without traffic.</td>
<td>High-traffic features; high cost-of-being-wrong decisions.</td>
<td class="method"><a href="microsoft-exp-framework.html">Microsoft ExP →</a></td>
</tr>
<tr>
<td class="method">XLNT / T-REX<div class="from">LinkedIn</div><div class="badges"><span class="tag saas">SaaS</span></div></td>
<td><span class="tag experiment">Experimentation</span></td>
<td>Same as Microsoft ExP, told as a platform-investment story; 40k tests/day, 20× engine speed-up.</td>
<td class="not">Add anything to Microsoft's discipline — it's the same rule at scale.</td>
<td>Demonstrating what "experimentation as default" looks like at maximum operational scale.</td>
<td class="method"><a href="linkedin-xlnt-framework.html">LinkedIn T-REX →</a></td>
</tr>
<tr>
<td class="method">ABlaze + Sequential + Causal<div class="from">Netflix</div><div class="badges"><span class="tag consumer">Consumer</span></div></td>
<td><span class="tag experiment">Experimentation</span></td>
<td>A/B as the default platform-wide; sequential tests for software canaries (play-delay regressions); causal inference as a research area for non-randomisable decisions.</td>
<td class="not">Help when traffic is tiny. Replace good metric design.</td>
<td>High-volume A/B as default; quasi-experimental designs when A/B isn't possible (specific applications like pricing/regional rollouts are widely cited but not documented in the 2016 ABlaze post).</td>
<td class="method"><a href="netflix-experimentation.html">Netflix →</a></td>
</tr>
<tr>
<td class="method">Democratised experimentation<div class="from">Booking.com</div><div class="badges"><span class="tag consumer">Consumer</span></div></td>
<td><span class="tag experiment">Experimentation</span></td>
<td>Any employee can launch a test (no manager gate); auto <a class="j" href="jargon.html#srm">SRM</a> & shared metric definitions enforce trust.</td>
<td class="not">Replace direction-setting. Pick the OEC for you.</td>
<td>Cultures where ideas are constrained by review bottlenecks, not by hypotheses.</td>
<td class="method"><a href="booking-experimentation.html">Booking →</a></td>
</tr>
<tr>
<td class="method">ERF (variance reduction)<div class="from">Airbnb</div><div class="badges"><span class="tag consumer">Consumer</span></div></td>
<td><span class="tag experiment">Experimentation</span></td>
<td>Standardised analysis: one metric <a class="j" href="jargon.html#dsl">DSL</a>, automatic variance reduction, templated reports → engineers read tests without analysts.</td>
<td class="not">Help if you have no metric library yet.</td>
<td>Scaling experiment volume past where analyst-per-test is feasible.</td>
<td class="method"><a href="airbnb-erf-framework.html">Airbnb ERF →</a></td>
</tr>
<tr>
<td class="method">XP + <a class="j" href="jargon.html#universal-holdout">universal holdout</a><div class="from">Uber</div><div class="badges"><span class="tag consumer">Consumer</span></div></td>
<td><span class="tag experiment">Experimentation</span></td>
<td>Staged ramp with auto-rollback; <strong>universal holdout</strong> measures cumulative quarterly impact (not just per-feature).</td>
<td class="not">Pick the OEC. Handle marketplace interference alone.</td>
<td>Detecting "many local wins, no net gain" — the failure mode roll-ups hide.</td>
<td class="method"><a href="uber-xp-framework.html">Uber XP →</a></td>
</tr>
<tr>
<td class="method">Switchback (Dash-AB)<div class="from">DoorDash</div><div class="badges"><span class="tag consumer">Consumer</span></div></td>
<td><span class="tag experiment">Experimentation</span></td>
<td>Marketplace-specific design: randomise <em>region × time window</em>, not users, to avoid interference (<a class="j" href="jargon.html#sutva">SUTVA</a> violation).</td>
<td class="not">Apply when there's no shared resource — overkill for UI tests.</td>
<td>Any feature that changes a shared pool (Dasher supply, pricing, slot inventory).</td>
<td class="method"><a href="doordash-switchback-framework.html">DoorDash →</a></td>
</tr>
<tr>
<td class="method">Time/region splits + bandits<div class="from">Lyft</div><div class="badges"><span class="tag consumer">Consumer</span></div></td>
<td><span class="tag experiment">Experimentation</span></td>
<td>Three-way routing: A/B for non-marketplace, time/region splits for marketplace, bandits for continuous decisions.</td>
<td class="not">Replace classical A/B; it complements it.</td>
<td>Always-on optimisation where the winner depends on context (Lyft's published bandit example: customer communications).</td>
<td class="method"><a href="lyft-experimentation.html">Lyft →</a></td>
</tr>
<tr>
<td class="method">A/B platform + real-time<div class="from">Pinterest</div><div class="badges"><span class="tag consumer">Consumer</span></div></td>
<td><span class="tag experiment">Experimentation</span></td>
<td>"Boring" but copyable: lightweight config UI + standard A/B + real-time regression monitoring.</td>
<td class="not">Innovate statistically — same OEC discipline as the others.</td>
<td>Small/mid teams building their first proper experimentation stack.</td>
<td class="method"><a href="pinterest-ab-framework.html">Pinterest →</a></td>
</tr>
</tbody>
</table>
</div>
<!-- CAMPS / CATEGORIES -->
<h2 class="sec" id="cats">The camps — what each does, what it leaves out</h2>
<p class="secsub">Re-organising the methods by camp, so you can see which jobs each kind of method actually does. A complete stack covers the three main camps; experimentation gets added on top once you're shipping enough to measure.</p>
<div class="catsec scoring">
<h3><span class="tag scoring">Scoring</span> — turn many candidates into a coarse ranking</h3>
<p>Method (deep-dived): <a href="rice-framework.html">RICE</a> (Intercom). <b>What it does:</b> lets you compare 50 ideas on the same scale and surface the top 5–10. The Confidence multiplier is the discipline — un-evidenced ideas are visibly weaker. Four additional scoring/prioritization methods are <em>surfaced for future deep-dive</em> on the <a href="index.html#scoring-group">home page Scoring group</a>: Kano, MoSCoW, Opportunity Scoring (ODI), Cost of Delay — each with a canonical external source pending dedicated deep-dive pages.</p>
<p><b>What they don't do:</b> tell you what goal to score against, or estimate lift. Pair with Define-first (North Star / V2MOM / Pyramid) for goals, and with Experimentation for lift.</p>
</div>
<div class="catsec goal">
<h3><span class="tag define">Define-first</span> — set the destination, force traceability</h3>
<p>Methods: <a href="north-star-framework.html">North Star Framework</a> (Amplitude / Slack), <a href="pr-faq-framework.html">Amazon PR-FAQ</a>, <a href="heart-framework.html">Google HEART/GSM</a>, <a href="dibb-framework.html">Spotify DIBB</a>, <a href="v2mom-framework.html">Salesforce V2MOM</a>, <a href="pyramid-of-clarity-framework.html">Asana Pyramid</a>. <b>What they do:</b> establish what success looks like — either as a company-wide outcome metric (North Star writes the causal model <code>NSM = f(inputs)</code> and picks the focus input) or as a cascaded goal (V2MOM, Pyramid, HEART, DIBB, PR-FAQ) — and require every piece of work to trace upward to it. The discipline that stops "feature-itis."</p>
<p><b>What they don't do:</b> rank features against each other, or measure outcomes. They're upstream of Scoring and Experimentation, not substitutes.</p>
</div>
<div class="catsec qual">
<h3><span class="tag qual">Written discipline</span> — rigour by prose, not by formula</h3>
<p>Method: <a href="stripe-shaping-framework.html">Stripe product shaping</a>. <b>What it does:</b> forces every bet into a deep-dive doc filtered by a multi-decade direction; quality of writing is the funding gate.</p>
<p><b>What it doesn't do:</b> produce a number, or operate without an existing writing culture. The honest "no scoring formula" exemplar — included to make explicit that successful companies sometimes skip scoring deliberately.</p>
</div>
<div class="catsec experiment">
<h3><span class="tag experiment">Experimentation</span> — post-build "did it work?" (a different question)</h3>
<p>Methods: <a href="microsoft-exp-framework.html">Microsoft ExP</a>, <a href="linkedin-xlnt-framework.html">LinkedIn T-REX</a>, <a href="netflix-experimentation.html">Netflix</a>, <a href="booking-experimentation.html">Booking</a>, <a href="airbnb-erf-framework.html">Airbnb ERF</a>, <a href="uber-xp-framework.html">Uber XP</a>, <a href="doordash-switchback-framework.html">DoorDash switchback</a>, <a href="lyft-experimentation.html">Lyft</a>, <a href="pinterest-ab-framework.html">Pinterest</a>. <b>What they do:</b> A/B and quasi-experimental designs that produce a causal estimate of lift on a pre-declared OEC. The only category that produces ground-truth numbers.</p>
<p><b>What they don't do:</b> answer "which 10 to build first?" — they presume the prioritization decision is already made. Pair with Define-first for what to test, Scoring for which to test first.</p>
</div>
<div class="catsec ailab">
<h3><span class="tag ailab">AI lab</span> — capability-led, no published pre-build score</h3>
<p>Methods: <a href="anthropic-pm-on-ai-exponential.html">Anthropic · PM on AI exponential</a> and <a href="google-customer-zero-2026.html">Google · "Customer zero" 2026</a>. <b>What they do:</b> the Anthropic essay describes five PM habits for working on an AI product whose substrate improves fast (short sprints, prototype-first, side quests, capability-first builds, daily use). The Google study records what Sundar Pichai has actually said in 2026 about build-side velocity ("customer zero" framing, verbatim numbers). <b>What they don't do:</b> rank features pre-build, or address post-launch verification of real users — neither lab has published a prioritisation rubric or kill criterion for the agentic era. Useful as <em>working discipline</em> and <em>build-side context</em>, not as prioritisation. Pair with Scoring or Define-first for the upstream "which one to build" decision.</p>
</div>
<footer>
From: <a href="impact-saas-companies.html">SaaS / B2B case studies</a> · <a href="impact-consumer-companies.html">Consumer-tech case studies</a><br>
All methods in the two tables link to a dedicated deep-dive page. Every "What it decides / doesn't do" claim traces to that company's published material; caveats noted on the deep-dive pages where sourcing is secondary. <em>Note: a 2026-05-26 cleanup removed the OpenAI "Iterative Deployment" row — its source documents (Planning for AGI and Beyond, Preparedness Framework v2) could not be reliably re-fetched from the editing environment to verify the row's content. A later 2026-05-26 addition introduced the Google "Customer zero" 2026 row — grounded entirely in two first-party Sundar Pichai blog posts (Cloud Next April 22, I/O May 19); it is documented as a velocity-side study, not a feature-decision methodology, because Google has not published one for the agentic era. A 2026-05-28 cleanup removed the Atlassian "Weighted Scoring" row entirely — both cited Atlassian sources (atlassian.com/agile/.../prioritization-framework and the Jira Product Discovery handbook) were verified to be general educational guides covering RICE/Kano/MoSCoW/Value-Effort/Opportunity-Scoring/Cost-of-Delay (URL 1) and RICE/Impact-Effort/$10-Game/RUF/3-Bucket/WSJF (URL 2); neither describes a method called "Weighted Scoring" with the "100 points across weighted criteria" mechanic our row attributed to Atlassian. The four newly-canonical methods from URL 1 — Kano, MoSCoW, Opportunity Scoring (ODI), Cost of Delay — are surfaced on the index for future deep-dives. Value vs Effort was skipped (no canonical originator).</em>
</footer>
</div>
</body>
</html>