Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .github/workflows/cache-cleanup.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: Cache cleanup

# Pull request runs only restore caches, but the BuildKit and tool caches of a closed PR
# ref are dead weight against the 10 GB repository quota, so drop them when the PR closes.
on:
pull_request:
types: [closed]

permissions:
actions: write
contents: read

jobs:
delete-pr-caches:
# The token of a pull request from a fork is read-only, so the delete would fail; such a
# run only restores caches and has nothing of its own to remove.
if: github.event.pull_request.head.repo.full_name == github.repository
runs-on: ubuntu-latest
steps:
- name: Delete the caches of this pull request
env:
GH_TOKEN: ${{ github.token }}
GH_REPO: ${{ github.repository }}
REF: refs/pull/${{ github.event.pull_request.number }}/merge
run: |
ids="$(gh cache list --ref "$REF" --limit 100 --json id --jq '.[].id')"
if [ -z "$ids" ]; then echo "no caches for $REF"; exit 0; fi
for id in $ids; do gh cache delete "$id" || true; done
273 changes: 256 additions & 17 deletions .github/workflows/examples.yml

Large diffs are not rendered by default.

42 changes: 42 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,48 @@ Five services exchanging the `order.placed` fact: **order, payment, inventory,
notification, shipping**. Local = Docker; prod = plain Kubernetes (no managed cloud) with
CloudNativePG, Strimzi/Kafka, Apicurio, Istio, KEDA, Debezium, and OTel → the LGTM stack.

## Infrastructure images (pinned, fully qualified)

All shared infra lives in `examples/_infra/` and uses fully qualified names, never `:latest`:
`docker.io/apache/kafka:4.3.1` (KRaft), `docker.io/library/postgres:18.6-alpine` (the volume mounts at
`/var/lib/postgresql`, not `/data`: PGDATA moved to `/var/lib/postgresql/18/docker`),
`docker.io/library/redis:8.10.2-alpine`, `quay.io/debezium/connect:3.7.0.Final`,
`docker.io/apicurio/apicurio-registry:3.3.3`, `ghcr.io/open-feature/flagd:v0.17.0`,
`docker.io/grafana/otel-lgtm:0.36.0`, `docker.io/envoyproxy/envoy:v1.39.3`. kafka-ui is gone: inspect topics
with `kcat` or `kafka-console-consumer.sh` inside the `kafka` container. Test tools: newman 6.2.3 +
newman-reporter-htmlextra 1.23.1, hey v0.1.5, ghz v0.121.0, conftest 0.71.1.
Redis 8 is tri-licensed (RSALv2 / SSPLv1 / AGPLv3); we take the free AGPLv3 open-source option, no license key needed.

## CI cache budget

The repo gets 10 GB of Actions cache. Policy in `examples.yml`: every run restores caches, only a push to
`main` writes them, and each ecosystem keeps a fixed number of entries. Maven and Go are one shared entry each
(`maven-Linux-<hash of all pom.xml>`, `go-Linux-<hash of all go.sum>`), written by the main-only `warm-maven` /
`warm-go` jobs, which resolve every project and delete the superseded entry. Conan is one entry per
distinct `conan.lock` (22), pruned of sources, build trees and downloads before it is saved; on a main miss the
builder stage is forced to run so the cache mount is never empty. BuildKit layers are `type=gha,mode=min`.
Steady state on main: typical about 3.7 GB (22 x 82 MB Conan + Maven 1.0 + Go 0.6 + layers 0.25 + Pages 0.05),
bound 4.9 GB, peak about 8 GB when a Conan key bump or bulk lock change leaves superseded per-lock entries and
layer blobs age out after 7 days unused. Measured: unpruned Conan entries, BuildKit layers (27 blobs = 200 MB).
Partly measured: prune ratio (one project). Estimated: Maven <=1.0 GB, Go <=0.6 GB, Pages. After the first merge,
check the real saved Maven and Go entry sizes and correct the numbers (arithmetic in the `examples.yml` comment).
`cache-cleanup.yml` drops a PR ref's caches when the PR closes (same-repo PRs only).

Check usage with `gh api repos/patterncatalyst/cloud-native-design-patterns/actions/cache/usage`. Drop one PR's
caches with
`gh cache list --ref refs/pull/<N>/merge --limit 100 --json id --jq '.[].id' | xargs -r -n1 gh cache delete`.
The pre-S8 entries use other key prefixes than the current ones, so deleting them is safe at any time, even
after the first main run with the new keys: `conan-Linux-` (old Conan), `setup-java-Linux-x64-maven-` and
`setup-go-Linux-x64-ubuntu24-go-` (old Maven and Go). Current keys start `Linux-conan2-`, `maven-Linux-`, `go-Linux-`.

```bash
for k in conan-Linux- setup-java-Linux-x64-maven- setup-go-Linux-x64-ubuntu24-go-; do
gh cache list --key "$k" --limit 100 --json id --jq '.[].id' | xargs -r -n1 gh cache delete
done
```

Do not use `gh cache delete --all`: it also removes the entries the current policy just wrote.

## Locked language stacks (do not substitute)

- **Go** 1.27: `net/http` stdlib (1.22 ServeMux, no chi/gin/echo); `franz-go` (Kafka); pgx v5;
Expand Down
30 changes: 15 additions & 15 deletions _docs/09-api-registry.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ curl -sf -X POST \
curl -s -o /dev/null -w "%{http_code}" -X POST \
http://apicurio.registry.svc/apis/registry/v3/groups/orders/artifacts/order-placed/versions \
-H "Content-Type: application/json" \
--data-binary @order-placed.avsc # → 200 if compatible, 409 if not
--data-binary @order-placed.avsc # → 200 if compatible, 400 if not
```

The same call works from application or pipeline code in any language, returning
Expand All @@ -72,7 +72,7 @@ public class RegistryClient {
.body(schema)
.retrieve()
.toBodilessEntity();
return resp.getStatusCode().value(); // 200 if compatible, 409 if rejected
return resp.getStatusCode().value(); // 200 if compatible, 400 if rejected
}
}
```
Expand All @@ -87,7 +87,7 @@ public interface RegistryClient {
Response publish(@PathParam("group") String group,
@PathParam("artifactId") String artifactId,
byte[] schema);
// caller checks response.getStatus(): 200 = compatible, 409 = rejected
// caller checks response.getStatus(): 200 = compatible, 400 = rejected
}
```

Expand All @@ -104,7 +104,7 @@ public class RegistryClient(HttpClient http)
content.Headers.ContentType = new("application/json");
var resp = await http.PostAsync(
$"{Registry}/groups/{group}/artifacts/{artifactId}/versions", content);
return (int)resp.StatusCode; // 200 if compatible, 409 if rejected
return (int)resp.StatusCode; // 200 if compatible, 400 if rejected
}
}
```
Expand All @@ -121,7 +121,7 @@ def publish(group: str, artifact_id: str, schema: bytes) -> int:
headers={"Content-Type": "application/json"},
data=schema,
)
return r.status_code # 200 if compatible, 409 if the rule rejects it
return r.status_code # 200 if compatible, 400 if the rule rejects it
```

{% raw %}
Expand All @@ -137,7 +137,7 @@ inline int publish(std::string_view group, std::string_view artifact_id,
auto r = cpr::Post(cpr::Url{url},
cpr::Header{{"Content-Type", "application/json"}},
cpr::Body{std::string(schema)});
return r.status_code; // 200 if compatible, 409 if rejected
return r.status_code; // 200 if compatible, 400 if rejected
}
```
{% endraw %}
Expand All @@ -153,21 +153,21 @@ func publish(group, artifactID string, schema []byte) (int, error) {
return 0, err
}
defer resp.Body.Close()
return resp.StatusCode, nil // 200 if compatible, 409 if rejected
return resp.StatusCode, nil // 200 if compatible, 400 if rejected
}
```

## Gate breaking changes in CI

This is the guardrail that makes the whole contract story real. The CI pipeline
posts the new schema version to the registry; if it violates the artifact's
compatibility rule, Apicurio returns **`409 Conflict`** and the step fails the
compatibility rule, Apicurio returns **`400 Bad Request`** and the step fails the
build, blocking the merge. The contract is enforced *before* code ships — not
discovered broken in production by a downstream team three days later.

{% include excalidraw.html
file="09-registry-gate"
alt="A pull request with a schema change posts the new version to the Apicurio registry; the registry checks it against the compatibility rule and either allows merge and publish, or returns 409 and blocks the merge; consumers then fetch the published schema and generate stubs"
alt="A pull request with a schema change posts the new version to the Apicurio registry; the registry checks it against the compatibility rule and either allows merge and publish, or returns 400 and blocks the merge; consumers then fetch the published schema and generate stubs"
caption="Figure 9.2 — The registry checks compatibility at publish time and blocks the merge on a breaking change" %}

```yaml
Expand All @@ -177,7 +177,7 @@ discovered broken in production by a downstream team three days later.
code=$(curl -s -o /dev/null -w "%{http_code}" -X POST \
"$REGISTRY/groups/orders/artifacts/order-placed/versions" \
-H "Content-Type: application/json" --data-binary @order-placed.avsc)
if [ "$code" = "409" ]; then
if [ "$code" = "400" ]; then
echo "::error::Incompatible schema change — blocking the merge"
exit 1
fi
Expand All @@ -187,7 +187,7 @@ The rule is the contract about the contract. `BACKWARD` (new consumers can read
old data) is the common default for events, because consumers upgrade on their own
schedule and must tolerate data already on the topic. `FORWARD` flips it for cases
where old consumers must read new data, and `FULL` demands both. Picking the rule
is a design decision; once picked, the registry enforces it for you and the `409`
is a design decision; once picked, the registry enforces it for you and the `400`
is non-negotiable.

## Schema evolution strategies
Expand Down Expand Up @@ -219,7 +219,7 @@ order of increasing cost:

The registry enforces the *boundary* (compatible or not), but the strategy determines
how often you approach that boundary. Teams that evolve additively rarely trigger a
`409`; teams that rename or restructure hit it constantly.
`400`; teams that rename or restructure hit it constantly.

## Validate before you publish

Expand Down Expand Up @@ -255,7 +255,7 @@ public interface DryRunValidator {
Response validate(@PathParam("group") String group,
@PathParam("artifactId") String artifactId,
byte[] schema);
// 200 = compatible (no version created); 409 = incompatible
// 200 = compatible (no version created); 400 = incompatible
}
```

Expand Down Expand Up @@ -326,11 +326,11 @@ func isCompatible(group, artifactID string, schema []byte) (bool, error) {

Prove the gate bites. Register `order-placed.avsc`, then make a genuinely breaking
change — rename a required field — and `curl` the new version: you should get a
`409`, and the CI step should exit non-zero. Then make an additive change — a new
`400`, and the CI step should exit non-zero. Then make an additive change — a new
*optional* field — and confirm it returns `200` and publishes a new version. The
breaking change failing the build and the additive change passing it is the entire
point of the registry.

---
*Verification status: verified — [`examples/09-api-registry/`](https://github.com/patterncatalyst/cloud-native-design-patterns/tree/main/examples/09-api-registry/) passes 8/8 checks on Docker Engine (2026-10-10)
(schema registration, BACKWARD compatibility rule, rejection of a breaking change with 409, acceptance of an additive change, versioned retrieval).*
(schema registration, BACKWARD compatibility rule, rejection of a breaking change with 400, acceptance of an additive change, versioned retrieval).*
2 changes: 1 addition & 1 deletion _docs/15-appendix-b-versioning.md
Original file line number Diff line number Diff line change
Expand Up @@ -330,7 +330,7 @@ The protocol-specific rules are all faces of four ideas:

Prove the additive/breaking line with the registry from **API Registry**. Add an
optional field to a schema and confirm the `BACKWARD` check passes; change a field's
type or reuse a protobuf tag and confirm it fails with `409`. The mechanical check
type or reuse a protobuf tag and confirm it fails with `400`. The mechanical check
catching the breaking change — instead of a reviewer's memory — is the principle
made real.

Expand Down
4 changes: 2 additions & 2 deletions _docs/28-appendix-o-newman.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ audience. `cli` is for a human watching; `junit` emits the XML that CI systems t
reports; `htmlextra` (a community reporter) produces a shareable page.

```bash
# install once (Node.js): npm i -g newman newman-reporter-htmlextra
# install once (Node.js): npm install -g --prefix ~/.local newman@6.2.3 newman-reporter-htmlextra@1.23.1
newman run orders.postman_collection.json \
--environment local.postman_environment.json \
--reporters cli,junit,htmlextra \
Expand Down Expand Up @@ -209,7 +209,7 @@ jobs:
- uses: actions/checkout@v4
- run: docker compose up -d --build --wait # order, payment, inventory, …
- run: ./scripts/wait-for-ready.sh # poll /readyz before testing
- run: npm i -g newman
- run: npm i -g newman@6.2.3 newman-reporter-htmlextra@1.23.1
- run: |
newman run orders.postman_collection.json \
-e ci.postman_environment.json \
Expand Down
33 changes: 28 additions & 5 deletions _example_pages/prerequisites.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,17 @@ conftest --version

`~/.local/bin` is on the default `PATH` for Fedora and RHEL login shells.

### hey (HTTP load testing)
### kcat (Kafka inspection)

Lists topics and reads or produces Kafka messages from the host (example 04 uses it); it
replaces the removed kafka-ui. Optional, since `kafka-console-consumer.sh` inside the
`kafka` container does the same:

```bash
sudo dnf install -y kcat
```

### hey v0.1.5 (HTTP load testing)

Used by examples that demonstrate scaling or performance patterns. Download a binary from
the [hey releases](https://github.com/rakyll/hey/releases) page, or build it with Go:
Expand All @@ -168,10 +178,22 @@ the [hey releases](https://github.com/rakyll/hey/releases) page, or build it wit
go install github.com/rakyll/hey@v0.1.5
```

### ghz (gRPC load testing)
### ghz v0.121.0 (gRPC load testing)

Used by examples with gRPC endpoints. Download a binary from the
[ghz releases](https://github.com/bojand/ghz/releases) page.
Used by examples with gRPC endpoints. Download the pinned binary from the
[ghz releases](https://github.com/bojand/ghz/releases) page:

```bash
GHZ_VERSION=0.121.0
BASE="https://github.com/bojand/ghz/releases/download/v${GHZ_VERSION}"
cd "$(mktemp -d)"
curl -fsSLO "${BASE}/ghz-linux-x86_64.tar.gz"
curl -fsSLO "${BASE}/ghz-linux-x86_64.tar.gz.sha256"
echo "$(cat ghz-linux-x86_64.tar.gz.sha256) ghz-linux-x86_64.tar.gz" | sha256sum -c -
mkdir -p ~/.local/bin
tar -xzf ghz-linux-x86_64.tar.gz -C ~/.local/bin ghz
ghz --version
```

### newman 6.2.3 (Postman CLI, required for example 28)

Expand All @@ -180,14 +202,15 @@ Used by example 28 (Newman API testing). Requires Node.js. Install into the user

```bash
sudo dnf install -y nodejs npm
npm install -g --prefix ~/.local newman@6.2.3
npm install -g --prefix ~/.local newman@6.2.3 newman-reporter-htmlextra@1.23.1
```

Verify:

```bash
curl --version
jq --version
kcat -V 2>&1 | head -1 # optional
hey -h 2>&1 | head -1 # optional
ghz --version # optional
newman --version # required for example 28
Expand Down
Loading
Loading