Repository navigation
Expand file tree
/
Copy pathdemos.html
More file actions
388 lines (357 loc) · 19.6 KB
/
Copy pathdemos.html
File metadata and controls
388 lines (357 loc) · 19.6 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
---
layout: default
title: Demos & examples
description: Every demo in the reference, mapped to the data-mesh principles they exercise.
permalink: /demos/
---
<section class="hero hero--compact">
<div class="container">
<span class="hero__eyebrow">Demos & examples</span>
<h1>See every principle, live.</h1>
<p class="hero__lead">
23 scripts under <code>examples/lgtm-datamesh/demos/</code>
exercise the data-mesh reference end to end. Each one is mapped
to the principle it demonstrates — domain ownership, data as a
product, self-serve platform, or federated governance.
</p>
</div>
</section>
<!-- ── The five-act walkthrough ────────────────────────────── -->
<section class="section">
<div class="container" style="max-width: 42rem">
<h2 class="section-heading">The five-act walkthrough</h2>
<p>
<code>walkthrough.sh</code> orchestrates five acts — trace,
scale, canary, lineage, topology — each shelling out to an
existing demo script with narration between them. Press Enter
to advance between acts.
</p>
<pre><code>cd examples/lgtm-datamesh && ./demos/walkthrough.sh</code></pre>
</div>
</section>
<!-- ── Demos by principle ─────────────────────────────────── -->
<!-- ── 3a Domain ownership ──────────────────────────────── -->
<section class="section">
<div class="container">
<h2 class="section-heading">Demos by principle</h2>
<h3>Domain ownership</h3>
<p>
These demos verify that each domain service owns its data, its
API surface, and its operational lifecycle — independent build,
deploy, and verification.
</p>
<div class="cards">
<a class="card"
href="https://github.com/{{ site.github_username }}/{{ site.github_repo }}/blob/main/examples/lgtm-datamesh/demos/demo-order.sh">
<div class="card__eyebrow">Domain ownership · <strong>REST</strong></div>
<h3 class="card__title">Order-service walking skeleton</h3>
<p class="card__desc">
Full spine: image build, helm deploy, operator-managed
Postgres, REST round-trip. Creates an order and confirms
it in the database.
</p>
<div class="card__meta">§4 Services & data products · Source ↗</div>
</a>
<a class="card"
href="https://github.com/{{ site.github_username }}/{{ site.github_repo }}/blob/main/examples/lgtm-datamesh/demos/demo-grpc.sh">
<div class="card__eyebrow">Domain ownership · <strong>gRPC</strong></div>
<h3 class="card__title">Cross-service gRPC</h3>
<p class="card__desc">
order-service calls inventory-service CheckStock over
gRPC. Proves domain isolation with clear API boundaries
between data products.
</p>
<div class="card__meta">§4, §6 · Source ↗</div>
</a>
<a class="card"
href="https://github.com/{{ site.github_username }}/{{ site.github_repo }}/blob/main/examples/lgtm-datamesh/demos/demo-service.sh">
<div class="card__eyebrow">Domain ownership · <strong>scaffold</strong></div>
<h3 class="card__title">Generic service health</h3>
<p class="card__desc">
Build, deploy, and verify health for any scaffolded
service. Proves the common domain template works end to
end.
</p>
<div class="card__meta">§4 · Source ↗</div>
</a>
</div>
</div>
</section>
<!-- ── 3b Data as a product ─────────────────────────────── -->
<section class="section">
<div class="container">
<h3>Data as a product</h3>
<p>
These demos exercise the contract, catalog, and discovery
surfaces that make each domain's data discoverable, addressable,
and trustworthy.
</p>
<div class="cards">
<a class="card"
href="https://github.com/{{ site.github_username }}/{{ site.github_repo }}/blob/main/examples/lgtm-datamesh/demos/demo-avro.sh">
<div class="card__eyebrow">Data as a product · <strong>Avro</strong></div>
<h3 class="card__title">Avro contracts (schema registry)</h3>
<p class="card__desc">
order-service registers an Avro schema in Apicurio;
notification-service decodes by schema ID. The contract
is the schema, not a handshake.
</p>
<div class="card__meta">§5 Contracts & catalog · Source ↗</div>
</a>
<a class="card"
href="https://github.com/{{ site.github_username }}/{{ site.github_repo }}/blob/main/examples/lgtm-datamesh/demos/demo-discovery.sh">
<div class="card__eyebrow">Data as a product · <strong>registry</strong></div>
<h3 class="card__title">Discovery contracts published</h3>
<p class="card__desc">
Publishes OpenAPI, Protobuf, GraphQL SDL, and Avro to
Apicurio — the feedstock for the catalog.
</p>
<div class="card__meta">§5 · Source ↗</div>
</a>
<a class="card"
href="https://github.com/{{ site.github_username }}/{{ site.github_repo }}/blob/main/examples/lgtm-datamesh/demos/demo-graphql.sh">
<div class="card__eyebrow">Data as a product · <strong>GraphQL</strong></div>
<h3 class="card__title">Federated read layer (GraphQL)</h3>
<p class="card__desc">
One query to the gateway stitches order (REST) and stock
(gRPC) into a single response. The composed read surface
of the mesh.
</p>
<div class="card__meta">§6 Data planes · Source ↗</div>
</a>
<a class="card"
href="https://github.com/{{ site.github_username }}/{{ site.github_repo }}/blob/main/examples/lgtm-datamesh/demos/demo-reviews.sh">
<div class="card__eyebrow">Data as a product · <strong>REST</strong></div>
<h3 class="card__title">Review-service data product</h3>
<p class="card__desc">
Builds and verifies the review-service REST surface end
to end — a standalone data product with its own schema.
</p>
<div class="card__meta">§4 · Source ↗</div>
</a>
<a class="card"
href="https://github.com/{{ site.github_username }}/{{ site.github_repo }}/blob/main/examples/lgtm-datamesh/demos/demo-add-data-product.sh">
<div class="card__eyebrow">Data as a product · <strong>lifecycle</strong></div>
<h3 class="card__title">Add/remove a data product</h3>
<p class="card__desc">
Full add-to-mesh workflow: deploy, publish contract,
ingest catalog, then clean rollback. The mesh grows by
one product.
</p>
<div class="card__meta">§5 · Source ↗</div>
</a>
<a class="card"
href="https://github.com/{{ site.github_username }}/{{ site.github_repo }}/blob/main/examples/lgtm-datamesh/demos/demo-openmetadata.sh">
<div class="card__eyebrow">Data as a product · <strong>catalog</strong></div>
<h3 class="card__title">OpenMetadata catalog health</h3>
<p class="card__desc">
Verifies the catalog server is deployed, healthy, and
backed by Postgres — the discovery surface the mesh
depends on.
</p>
<div class="card__meta">§5 · Source ↗</div>
</a>
<a class="card"
href="https://github.com/{{ site.github_username }}/{{ site.github_repo }}/blob/main/examples/lgtm-datamesh/demos/demo-om-lineage.sh">
<div class="card__eyebrow">Data as a product · <strong>lineage</strong></div>
<h3 class="card__title">Cross-product lineage</h3>
<p class="card__desc">
Proves the lineage graph: orders table, order-placed
topic, notifications table. Data flows are visible, not
guesswork.
</p>
<div class="card__meta">§5 · Source ↗</div>
</a>
</div>
</div>
</section>
<!-- ── 3c Self-serve data platform ──────────────────────── -->
<section class="section">
<div class="container">
<h3>Self-serve data platform</h3>
<p>
These demos show the platform delivering capabilities by
declaration — operators provision databases, message buses, and
autoscalers; the observability stack surfaces metrics, traces,
and topology.
</p>
<div class="cards">
<a class="card"
href="https://github.com/{{ site.github_username }}/{{ site.github_repo }}/blob/main/examples/lgtm-datamesh/demos/demo-kafka.sh">
<div class="card__eyebrow">Self-serve platform · <strong>Kafka</strong></div>
<h3 class="card__title">Async spine (Kafka events)</h3>
<p class="card__desc">
order-service publishes order.placed;
notification-service consumes it. The Strimzi operator
delivers the Kafka cluster by declaration.
</p>
<div class="card__meta">§6 Data planes · Source ↗</div>
</a>
<a class="card"
href="https://github.com/{{ site.github_username }}/{{ site.github_repo }}/blob/main/examples/lgtm-datamesh/demos/demo-notifications.sh">
<div class="card__eyebrow">Self-serve platform · <strong>Postgres</strong></div>
<h3 class="card__title">Durable notifications (Postgres)</h3>
<p class="card__desc">
Alembic migration, persisted events, survives restart.
The CloudNativePG operator delivers the database by
declaration.
</p>
<div class="card__meta">§4, §6 · Source ↗</div>
</a>
<a class="card"
href="https://github.com/{{ site.github_username }}/{{ site.github_repo }}/blob/main/examples/lgtm-datamesh/demos/demo-keda-kafka.sh">
<div class="card__eyebrow">Self-serve platform · <strong>KEDA</strong></div>
<h3 class="card__title">Kafka consumer-lag scaling</h3>
<p class="card__desc">
notification-service scales from zero on Kafka lag, then
back to zero when caught up. Elastic data products by
declaration.
</p>
<div class="card__meta">§8 Elastic & resilient · Source ↗</div>
</a>
<a class="card"
href="https://github.com/{{ site.github_username }}/{{ site.github_repo }}/blob/main/examples/lgtm-datamesh/demos/demo-keda-http.sh">
<div class="card__eyebrow">Self-serve platform · <strong>KEDA HTTP</strong></div>
<h3 class="card__title">HTTP request scaling (KEDA)</h3>
<p class="card__desc">
graphql-gateway scales from zero on HTTP request
concurrency via the KEDA HTTP add-on. Zero cost when
idle, instant wake on demand.
</p>
<div class="card__meta">§8 · Source ↗</div>
</a>
<a class="card"
href="https://github.com/{{ site.github_username }}/{{ site.github_repo }}/blob/main/examples/lgtm-datamesh/demos/demo-observability.sh">
<div class="card__eyebrow">Self-serve platform · <strong>Prometheus</strong></div>
<h3 class="card__title">Metrics stack health</h3>
<p class="card__desc">
Prometheus scraping capstone targets, Grafana dashboard
provisioned with mesh and request-rate panels. The
platform delivers observability.
</p>
<div class="card__meta">§9 Observability · Source ↗</div>
</a>
<a class="card"
href="https://github.com/{{ site.github_username }}/{{ site.github_repo }}/blob/main/examples/lgtm-datamesh/demos/demo-tracing.sh">
<div class="card__eyebrow">Self-serve platform · <strong>Tempo</strong></div>
<h3 class="card__title">Trace backend health (Tempo)</h3>
<p class="card__desc">
Tempo up, Grafana datasource wired, synthetic span
ingested and queried back. The trace pipeline works
before you send real traffic.
</p>
<div class="card__meta">§9 · Source ↗</div>
</a>
<a class="card"
href="https://github.com/{{ site.github_username }}/{{ site.github_repo }}/blob/main/examples/lgtm-datamesh/demos/demo-trace-flow.sh">
<div class="card__eyebrow">Self-serve platform · <strong>traces</strong></div>
<h3 class="card__title">End-to-end trace across products</h3>
<p class="card__desc">
A GraphQL query fans out to three services; the trace
lands in Tempo with cross-product spans. Distributed
tracing as a platform feature.
</p>
<div class="card__meta">§9 · Source ↗</div>
</a>
</div>
</div>
</section>
<!-- ── 3d Federated computational governance ────────────── -->
<section class="section">
<div class="container">
<h3>Federated computational governance</h3>
<p>
These demos exercise the governance mechanisms — mTLS,
progressive delivery, and mesh topology — that keep
independently-owned products interoperable.
</p>
<div class="cards">
<a class="card"
href="https://github.com/{{ site.github_username }}/{{ site.github_repo }}/blob/main/examples/lgtm-datamesh/demos/demo-canary-verify.sh">
<div class="card__eyebrow">Federated governance · <strong>Istio</strong></div>
<h3 class="card__title">Canary contract evolution (verified)</h3>
<p class="card__desc">
v1 and v2 both meshed; traffic splits by VirtualService
weight. Asserts the 90/10 and 50/50 splits land in
expected bands.
</p>
<div class="card__meta">§7 Progressive delivery · Source ↗</div>
</a>
<a class="card"
href="https://github.com/{{ site.github_username }}/{{ site.github_repo }}/blob/main/examples/lgtm-datamesh/demos/demo-canary.sh">
<div class="card__eyebrow">Federated governance · <strong>canary</strong></div>
<h3 class="card__title">Canary demo (repeatable)</h3>
<p class="card__desc">
Presenter-facing: bring the canary up at a chosen
weight, shift it, tear it cleanly back to v1-only
baseline. Fully replayable.
</p>
<div class="card__meta">§7 · Source ↗</div>
</a>
<a class="card"
href="https://github.com/{{ site.github_username }}/{{ site.github_repo }}/blob/main/examples/lgtm-datamesh/demos/demo-kiali.sh">
<div class="card__eyebrow">Federated governance · <strong>topology</strong></div>
<h3 class="card__title">Kiali mesh topology</h3>
<p class="card__desc">
Kiali API answers, Prometheus reachable, capstone
namespace visible in the graph. The mesh is observable
as a first-class surface.
</p>
<div class="card__meta">§7, §9 · Source ↗</div>
</a>
</div>
</div>
</section>
<!-- ── Running a demo ─────────────────────────────────────── -->
<section class="section-tight">
<div class="container" style="max-width: 42rem">
<h2 class="section-heading">Running a demo</h2>
<p>
All demos assume you are in the
<code>examples/lgtm-datamesh/</code> directory with kubectl
context set to <code>capstone</code>. See the
<a href="{{ '/setup/' | relative_url }}">setup & prerequisites</a>
page for installation and bootstrap.
</p>
<pre><code>cd examples/lgtm-datamesh
./demos/demo-order.sh</code></pre>
<h3>What to look at after running a demo</h3>
<p>
The tools are NodePorts published on <code>127.0.0.1</code> when
the <code>capstone</code> profile is created, so the URLs below
are available as soon as the stack is up. Run
<code>./scripts/show-endpoints.sh</code> to see which are
reachable. Ports are fixed at creation; recreate an older profile
with <code>./scripts/setup-capstone-profile.sh --replace</code>.
<code>--replace</code> deletes and recreates the cluster, so run
<code>./scripts/bootstrap-capstone.sh</code> again afterwards.
</p>
<ul>
<li>
<strong>Grafana</strong> — dashboards for request rate,
canary weight, KEDA replicas:
<code>http://127.0.0.1:3000</code>
</li>
<li>
<strong>Kiali</strong> — live mesh topology graph:
<code>http://127.0.0.1:20001/kiali</code>
</li>
<li>
<strong>Tempo</strong> — distributed traces: open Grafana
→ Explore → select the Tempo datasource
(or direct: <code>http://127.0.0.1:3200</code>)
</li>
<li>
<strong>OpenMetadata</strong> — catalog and lineage:
<code>http://127.0.0.1:8585</code>
</li>
</ul>
<h3>Restoring baseline</h3>
<p>
The demo scripts clean up their own releases on success. After
running several, shared services may be missing. To put the
workloads back in bootstrap state:
</p>
<pre><code>./scripts/restore-baseline.sh</code></pre>
</div>
</section>