Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,16 @@ source beside it. Edit the `.d2`, run `just diagrams`, and commit both. The
committed SVG comes from d2 0.7.1; a different version re-renders every file
and produces a large diff that is not a real change.

**The social preview card is generated.** `site/public/social-card.png` is
rendered from `docs/media/social-card.typ` by `just social-card`, which needs Inter on a
font path — set `FANOUT_FONT_PATH` to a directory holding it. CI has no such
directory, so nothing re-renders the card to compare it; what `site-build` does
assert is that the committed PNG exists and still matches the dimensions the
site's head advertises. If you edit the source, run the recipe and commit the
PNG in the same change. The recipe fails when the
font is missing rather than shipping a card in a substitute face, which typst
would otherwise do without complaint.

**Runtime configuration is strict.** [`fanout.example.yaml`](fanout.example.yaml)
is the complete schema. New settings need a YAML key, a `FANOUT_` environment
name, a test, and matching documentation; unknown names deliberately fail
Expand Down
19 changes: 19 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,25 @@ optional dashboard read load against your hardware. Fanout does not publish a
throughput headline until the raw reports and exact driver revision can ship
with it; see [the benchmark publication standard](docs/benchmarking.md).

## How it compares

Fanout is a single node holding traces, logs, and metrics for a system you can
reason about from one place. That premise, rather than any single feature, is
what separates it from its neighbours.

| If you use | Where Fanout differs |
| --- | --- |
| **Grafana with Loki, Tempo, and Mimir** | That stack keeps a service and a query language per signal, plus object storage underneath. Fanout keeps one process, one data directory, and one typed contract across all three signals, at the cost of the horizontal scale those components are built for. |
| **SigNoz** | Both are OTLP-native and self-hosted. SigNoz composes a collector, ClickHouse, and query services; Fanout compiles ingest, storage, query, alerting, and the browser client into one binary, with DuckLake/Parquet on local disk instead of a database cluster. |
| **Jaeger** | Jaeger covers traces and expects a storage backend you run separately. Fanout ingests traces, logs, and metrics into the same store, with nothing else to deploy. |
| **Prometheus with Grafana** | Prometheus pulls metrics and is excellent at them. Fanout accepts pushed OTLP for all three signals and is built around investigating a specific incident rather than maintaining long-range metric series. |
| **Datadog**, **Honeycomb**, **Grafana Cloud** | Those are managed services: someone else runs the storage, the scaling, and the upgrades, and your telemetry leaves your network to get there. Fanout is a binary you run, on data that stays on your disk. |
| **An OpenTelemetry Collector piped into ClickHouse** | The same shape, assembled by hand: collector, database, dashboards, and the glue between them. Fanout is that assembly as one program, with an agent and an MCP server already wired to the same query contract. |

Fanout is a single node. It has no clustering, no replication, and no object
tier; a deployment that outgrows one machine's disk and CPU has outgrown
Fanout.

## Requirements

- **Go and a C compiler** with `CGO_ENABLED=1` — DuckDB is a cgo dependency
Expand Down
55 changes: 55 additions & 0 deletions docs/media/social-card.typ
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
// The GitHub social preview card, which the documentation site also serves as
// its og:image at /social-card.png.
//
// Render it with `just social-card`, which pins the typst version, requires
// Inter on a font path, and fails when the face is missing. Running typst by
// hand does not: it warns about an unknown font family, exits 0, and writes a
// card set in whatever it found instead. The recipe also passes `--root .`,
// without which typst refuses to read the mark below — it sits outside this
// file's directory, and typst sandboxes an input to its own root.
//
// The page is 960pt x 480pt, which is exactly 1280 x 640 pixels at 96 ppi: the
// size GitHub expects for a social preview, and large enough that a link unfurl
// does not resample it.
//
// The mark is read from ui/host/public/favicon.svg — the canonical asset that
// internal/brand tracks — rather than copied here, so a revised mark reaches
// the card the next time it is rendered.
#set page(width: 960pt, height: 480pt, margin: (x: 60pt, top: 44pt, bottom: 34pt), fill: rgb("#0b0f14"))
#set text(font: "Inter", fill: rgb("#f2f5f8"))

#place(top + left, dx: -60pt, dy: -44pt, rect(width: 960pt, height: 5pt,
fill: gradient.linear(rgb("#5FE8CE"), rgb("#41B6F8"), rgb("#9A50F4"))))

#grid(
columns: (86pt, 1fr),
column-gutter: 16pt,
align: horizon,
image("../../ui/host/public/favicon.svg", width: 84pt),
text(size: 56pt, weight: 700, "Fanout"),
)

#v(30pt)
#text(size: 34pt, weight: 700, fill: rgb("#5FE8CE"))[
Single-binary, agent-native \
OpenTelemetry investigation.
]

#v(18pt)
#text(size: 20pt, fill: rgb("#8a94a6"))[
Ingest OTLP, store as Parquet, query with DuckDB, \
and ask an agent about it. One Go process.
]

// The footer is pushed down by flexible space rather than placed at the bottom
// out of flow. Placed, it drew at a fixed offset regardless of how tall the
// text above had grown, so one extra headline line put the divider through the
// body copy and typst still exited 0. In flow, the same overflow spills onto a
// second page, and a two-page render fails the PNG export outright.
#v(1fr)
#line(length: 100%, stroke: 0.75pt + rgb("#1b2430"))
#v(12pt)
#grid(columns: (1fr, auto),
text(size: 19pt, fill: rgb("#8a94a6"))[No collector fleet. No object store. One data directory.],
text(size: 19pt, fill: rgb("#8a94a6"))[github.com/labstack/fanout],
)
58 changes: 58 additions & 0 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,64 @@ site: docs-generate
site-build: docs-generate-check site-deps
cd site && npm run build

# Renders the social preview card from docs/media/social-card.typ into
# site/public, where GitHub takes it as the repository preview and the site
# serves it as its og:image.
#
# Outside `build` and `check` for the same reason as the diagrams above: the PNG
# is committed, and CI has no font path to re-render it with. `site-build`
# asserts the committed PNG is present and correctly sized, which is the part a
# gate can honestly do.
#
# The card needs Inter, which is not vendored here — the browser bundle carries
# its own copy for the UI, and a second copy in the tree to draw one image is a
# poor trade. Typst warns about an unknown font family and still exits 0, so a
# missing face does not fail the render; it quietly produces a card set in
# whatever typst found instead. This reads the warning back and fails on it.
#
# The version is pinned for the same reason d2 is: typst re-lays out text
# between releases, so a different one rewrites every glyph position and
# produces a diff that is not a real change.
#
# The mark comes from ui/host/public/favicon.svg, the canonical asset that
# internal/brand tracks, so the card cannot drift from the product logo.
social-card:
#!/usr/bin/env bash
set -euo pipefail
want_typst="0.15.1"
if ! command -v typst >/dev/null; then
echo "typst not found — install with: brew install typst" >&2
exit 1
fi
have_typst=$(typst --version | awk '{print $2}')
if [ "$have_typst" != "$want_typst" ]; then
echo "typst $have_typst is installed; the committed card was rendered with $want_typst" >&2
echo "another version re-lays out every glyph, so the diff would not be a real change" >&2
exit 1
fi
if [ -z "${FANOUT_FONT_PATH:-}" ]; then
echo "set FANOUT_FONT_PATH to a directory holding Inter" >&2
echo "the OFL original is at https://github.com/google/fonts/tree/main/ofl/inter" >&2
exit 1
fi
# Rendered aside and moved into place only once it is known good, so a run
# that fell back to a substitute face cannot leave that card in the tree.
# A directory, not `mktemp -t <name>`: that reserves a name without the .png
# suffix typst needs, so appending one both leaves the reserved file behind
# and writes to a path nothing reserved.
staged_dir=$(mktemp -d)
trap 'rm -rf "$staged_dir"' EXIT
staged="$staged_dir/social-card.png"
render=$(typst compile --root . --font-path "$FANOUT_FONT_PATH" --ppi 96 --format png \
docs/media/social-card.typ "$staged" 2>&1)
if [ -n "$render" ]; then echo "$render"; fi
if echo "$render" | grep -qi "unknown font family"; then
echo "typst could not find a font it was asked for — the card above is set in a fallback face" >&2
exit 1
fi
mv "$staged" site/public/social-card.png
echo "rendered site/public/social-card.png"

# ── Gate ─────────────────────────────────────────────────────────────────────

# lefthook's pre-push hook and CI both run this.
Expand Down
2 changes: 1 addition & 1 deletion site/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"scripts": {
"dev": "astro dev",
"start": "astro dev",
"build": "astro check && astro build && node scripts/check-tables.mjs",
"build": "astro check && astro build && node scripts/check-tables.mjs && node scripts/check-social-card.mjs",
"preview": "astro preview",
"astro": "astro"
},
Expand Down
Binary file added site/public/social-card.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
114 changes: 114 additions & 0 deletions site/scripts/check-social-card.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
// Assert that the og:image every page advertises is actually in the build, and
// is the size the same head claims it is.
//
// Starlight emits `twitter:card: summary_large_image` and no image to pair with
// it, so `src/components/Head.astro` supplies one. That tag is written by hand
// against a file in `public/`, and nothing else connects the two: rename or drop
// `social-card.png` and the build still succeeds while every page points a
// crawler at a 404 — a failure that shows up in other people's link previews
// long before it shows up in anything this repository builds or serves.
//
// The dimensions are checked too, because `og:image:width` and `og:image:height`
// are hand-typed. A card re-rendered at another size with the numbers left
// behind makes consumers reserve the wrong box, which is the whole reason those
// tags exist.
import { readdir, readFile } from "node:fs/promises";
import { join } from "node:path";

const dist = new URL("../dist/", import.meta.url).pathname;

async function htmlFiles(dir) {
const found = [];
for (const entry of await readdir(dir, { withFileTypes: true })) {
const path = join(dir, entry.name);
if (entry.isDirectory()) found.push(...(await htmlFiles(path)));
else if (entry.name.endsWith(".html")) found.push(path);
}
return found;
}

function meta(html, attribute, name) {
const pattern = new RegExp(
`<meta[^>]+${attribute}="${name}"[^>]+content="([^"]+)"`,
"i",
);
return html.match(pattern)?.[1] ?? null;
}

// The IHDR chunk is the first one in every PNG and holds the dimensions as two
// big-endian 32-bit integers, so the header alone answers this without a decoder.
function pngSize(bytes) {
const signature = "89504e470d0a1a0a";
if (bytes.subarray(0, 8).toString("hex") !== signature) return null;
if (bytes.subarray(12, 16).toString("ascii") !== "IHDR") return null;
return { width: bytes.readUInt32BE(16), height: bytes.readUInt32BE(20) };
}

let pages = [];
try {
pages = await htmlFiles(dist);
} catch (error) {
console.error(`check-social-card: cannot read ${dist} — run \`npm run build\` first`);
console.error(String(error));
process.exit(1);
}

const missingTag = [];
const declared = new Map();
for (const page of pages) {
const html = await readFile(page, "utf8");
const href = meta(html, "property", "og:image");
if (href === null) {
missingTag.push(page.replace(dist, ""));
continue;
}
const width = meta(html, "property", "og:image:width");
const height = meta(html, "property", "og:image:height");
declared.set(href, { width, height });
}

if (missingTag.length > 0) {
console.error("check-social-card: pages built without an og:image, so they unfurl as an empty large card:");
for (const page of missingTag) console.error(` ${page}`);
process.exit(1);
}

if (declared.size === 0) {
console.error("check-social-card: the build produced no HTML, which cannot be right");
process.exit(1);
}

for (const [href, size] of declared) {
// The tag is written by hand, so an author could reasonably make it relative.
// Open Graph consumers do not resolve those, and neither does this check —
// saying so beats throwing a URL parse error at whoever runs the build.
let path;
try {
path = new URL(href).pathname;
} catch {
console.error(`check-social-card: og:image is "${href}", which is not an absolute URL — consumers cannot resolve it`);
process.exit(1);
}
let bytes;
try {
bytes = await readFile(join(dist, path));
} catch {
console.error(`check-social-card: ${href} is advertised by every page but ${path} is not in the build`);
process.exit(1);
}

const actual = pngSize(bytes);
if (actual === null) {
console.error(`check-social-card: ${path} is not a PNG, so consumers that trust the tag will show nothing`);
process.exit(1);
}
if (String(actual.width) !== size.width || String(actual.height) !== size.height) {
console.error(
`check-social-card: ${path} is ${actual.width}x${actual.height}, but the head declares ${size.width}x${size.height}`,
);
console.error("check-social-card: re-render with `just social-card`, or correct the tags in src/components/Head.astro");
process.exit(1);
}
}

console.log(`check-social-card: ${declared.size} og:image target(s), present and correctly sized`);
20 changes: 20 additions & 0 deletions site/src/components/Head.astro
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,29 @@ const id = Astro.locals.starlightRoute.entry.id;
const markdownHref = ids.has(id)
? new URL(`/${id}.md`, Astro.site ?? Astro.url).href
: null;

// Starlight emits `twitter:card: summary_large_image` but never an image to go
// with it, so every link to the site unfurled as an empty large card. One image
// for the whole site rather than a card per page: a per-page render would make
// the build depend on a font the site only loads in the browser, and the card
// is the same one GitHub shows for the repository.
//
// Absolute URLs are required here: Open Graph consumers do not resolve relative
// hrefs. `astro.config.mjs` always sets `site`, so the fallback is not a
// configuration this repository ships — Astro types `Astro.site` as optional
// regardless, and the line above resolves its URL the same way.
const socialCardHref = new URL("/social-card.png", Astro.site ?? Astro.url).href;
---

<Default><slot /></Default>
<meta property="og:image" content={socialCardHref} />
<meta property="og:image:width" content="1280" />
<meta property="og:image:height" content="640" />
<meta
property="og:image:alt"
content="Fanout: single-binary, agent-native OpenTelemetry investigation."
/>
<meta name="twitter:image" content={socialCardHref} />
{
markdownHref && (
<>
Expand Down
Loading