docs(onboarding): make .env.example the canonical dev template (closes #260)

[dchaudhari7177]

Addresses #260 (auto-close keyword kept in the title only, per CLAUDE.md).

Documentation and one template file. No code, no compose change.

.env.example becomes the dev template

It now carries DEBUG, DUCKDB_SERVICE_URL and a commented HIGHFIVE_API_KEY, each annotated with which service reads it and the fact that all three are optional. Gone: the "Production Environment Variables" heading, NODE_ENV=production, PORT=3001, the production VITE_API_URL, and HIGHFIVE_API_KEY=your_secure_production_key_here — the placeholder that was becoming the admin password of every fresh dev box.

Production wording now points exclusively at .env.production.example, which is what docker-compose.prod.yml actually consumes, and the header says so explicitly so nobody copies the wrong one.

Four restatements become four pointers

File	Was	Now
CLAUDE.md	inline "Required .env" block	cp .env.example .env + pointer
CONTRIBUTING.md	its own inline block	same
docs/07-deployment-view/docker-compose.md	a third inline block	same, plus the correct service list
docs/troubleshooting.md	a fourth inline block	same, plus a correction (below)

Also fixed: docker-compose.md said cd hivehive after cloning highfive.git.

Two things I found beyond the issue

troubleshooting.md was actively wrong. It said the .env "must contain at minimum" DEBUG and DUCKDB_SERVICE_URL. Both are optional — image-service/app.py:512 and duckdb-service/app.py:123 are os.getenv("DEBUG", "false"), and image-service/app.py:159 defaults DUCKDB_SERVICE_URL to the service name. So the one doc a stuck contributor reaches for was sending them to fix a non-problem. It now says to restore from the template, and that a service dying on a malformed .env is failing on a parse error (docker compose config shows it), not a missing value.

A fifth restatement exists that I did not touch: .claude/skills/esp32-onboarding/SKILL.md:32 tells the agent to confirm .env contains DEBUG=true and DUCKDB_SERVICE_URL=.... Same drift, but it is agent tooling rather than contributor docs, so I left the call to you — happy to fold it in.

Claims verified against the tree

• env_file: - .env appears at docker-compose.yml:10 (backend), :78 (image-service), :131 (duckdb-service); the homepage service has no env_file, so VITE_API_URL never reached it. The docs now say exactly this — the old CONTRIBUTING wording ("used by image-service and duckdb-service") omitted backend.
• The optionality claims are read off the os.getenv defaults cited above, not assumed.

Docs gate

docs/11-risks-and-technical-debt/README.md gains a Lessons-learned entry in the required format. The general lesson: a file that ships as a template is the canonical description of itself — docs link to it, they do not restate it. Five copies meant no single one was wrong enough to notice, and the compose environment: overrides hid the mismatch, which is structurally the same failure as the inert-security-control incident already recorded in that chapter.

make check-citations → 7 OK, 0 problems.

Note: my other open PR (#263) also appends to the Lessons-learned section. I inserted at a different anchor so the two should not textually conflict, but if they do it is a one-line rebase — say the word.

One deviation to flag

CLAUDE.md's end-of-implementation gate asks for the senior-reviewer subagent. I could not run it in my environment, so this has not been through that gate. Everything else — make check-citations, the docs update, branch and commit conventions, the no-auto-close-keyword-in-bodies rule — has been followed.

[cofade]

Code review

Found 1 issue:

1. The template's DUCKDB_SERVICE_URL comment is wrong about the backend. It says the variable is optional for "the backend and image-service" because "it defaults to this exact value, which is the Docker service name". That holds for image-service (image-service/app.py's os.getenv("DUCKDB_SERVICE_URL", "http://duckdb-service:8000")), but the backend's code default is DEFAULT_DUCKDB_URL = 'http://127.0.0.1:8002' — deliberately the host-port mapping for a backend running on the host via npm run dev. The compose stack works regardless because docker-compose.yml's backend environment: block pins DUCKDB_SERVICE_URL: http://duckdb-service:8000. Since this file now declares itself the one description of the dev env, it should not restate the backend's default incorrectly — that is the PR's own thesis.

highfive/.env.example

Lines 18 to 23 in ef43e1a

	# Where the backend and image-service reach the DuckDB service. Optional —
	# it defaults to this exact value, which is the Docker service name. Only
	# change it if you are running the services outside compose.
	DUCKDB_SERVICE_URL=http://duckdb-service:8000

highfive/backend/src/duckdbClient.ts

Lines 1 to 8 in 205bcb7

	// `DUCKDB_SERVICE_URL` is the in-compose address (http://duckdb-service:8000).
	// The default below is the docker host-port mapping (8002:8000) for a backend
	// running on the host against a composed duckdb-service. A bare-metal/pm2
	// deploy (duckdb-service on :8000 directly) MUST set the env var explicitly —
	// see the ecosystem.config.js template in
	// docs/07-deployment-view/production-runbook.md.
	export const DEFAULT_DUCKDB_URL = 'http://127.0.0.1:8002';

highfive/docker-compose.yml

Lines 10 to 23 in 205bcb7

	env_file:
	- .env
	environment:
	NODE_ENV: development
	# PORT is set explicitly even though backend/src/port.ts's
	# resolvePort() now defaults to 3002 — it documents the dev-stack
	# intent and silences the unset-fallback warning that server.ts
	# emits otherwise. Dropping this line is a no-op for the dashboard
	# (default matches the host:container mapping above) but will
	# trigger the warning; treat that as the signal to ask whether
	# the drop was intentional.
	PORT: '3002'
	DUCKDB_SERVICE_URL: http://duckdb-service:8000
	IMAGE_SERVICE_URL: http://image-service:4444

Suggested wording:

# Where image-service reaches the DuckDB service. Optional — image-service
# defaults to this exact value (the Docker service name), and in compose the
# backend gets it from docker-compose.yml's `environment:` block. (The
# backend's own code default is the host-side http://127.0.0.1:8002, for
# `npm run dev` outside compose.) Only change it if you run the services
# outside compose.

Nits, non-blocking:

• DISCORD_WEBHOOK_URL is also read from the root .env in dev, via compose interpolation (${DISCORD_WEBHOOK_URL:-} in docker-compose.yml), and is not in the template. A commented # DISCORD_WEBHOOK_URL= line would make the "one description" claim fully true.
• The old "VITE_API_KEY was removed (#142)" tripwire comment was dropped. ADR-019 covers it, but it is free to keep.
• Yes, please fold in .claude/skills/esp32-onboarding/SKILL.md:32 — it is the fifth restatement and drifts the same way.
• README.md has no production pointer at all; a one-liner under the quick start ("Deploying? See .env.production.example and docs/07-deployment-view/production-deployment.md") would fit here. That is the one useful idea in the duplicate #266, which points at the PM2 runbook by mistake.

Everything else verified against the tree: the env_file wiring (backend / image-service / duckdb-service; homepage has none), the os.getenv defaults, the cd hivehive correction, and the troubleshooting.md correction. Test-merges cleanly with #263 and #265; scripts/check-doc-citations.sh reports 7 OK on the merged tree.

🤖 Generated with Claude Code

[dchaudhari7177]

Good catch on the backend default — that was the one thing in the file that was actually wrong, and wrong in exactly the way the file exists to prevent. All five points are in.

The blocker. DUCKDB_SERVICE_URL now says what each consumer really does: image-service defaults to the compose service name (image-service/app.py:159), the backend does not — its code default is DEFAULT_DUCKDB_URL = 'http://127.0.0.1:8002' (backend/src/duckdbClient.ts), the host-port mapping for npm run dev outside compose — and inside compose the backend gets the value from docker-compose.yml's environment: block rather than from this file at all.

Nits, all taken:

• # DISCORD_WEBHOOK_URL= added, commented, noting compose interpolates it as ${DISCORD_WEBHOOK_URL:-} so unset disables the notification rather than failing.
• VITE_ tripwire restored, and made a rule rather than a history note: there is deliberately no VITE_ form, because that prefix inlines into the browser bundle, which is how it leaked in security: production API key is baked into the public JS bundle; admin login gate is health-only #142.
• .claude/skills/esp32-onboarding/SKILL.md folded in — it now says cp .env.example .env and points at the template instead of restating the two values.
• README production pointer added under the quick start, aimed at .env.production.example and docs/07-deployment-view/production-deployment.md (not the PM2 runbook).

scripts/check-doc-citations.sh: 7 OK, 0 problems. Prettier via the pre-commit hook, clean.

[cofade]

senior-reviewer — good change, one P1 wording correction before merge

You flagged that you couldn't run the senior-reviewer gate in your environment. Consider this that gate. MERGEABLE / CLEAN, docs + one template only.

Claims verified against the tree, not the PR body

claim	verdict
image-service defaults DUCKDB_SERVICE_URL to the compose service name	✅ image-service/app.py's module-level DUCKDB_SERVICE_URL
backend's code default is http://127.0.0.1:8002	✅ backend/src/duckdbClient.ts's DEFAULT_DUCKDB_URL; pinned in compose by the backend service's environment: block
DEBUG optional in both Flask services	✅ os.getenv("DEBUG", "false") in both image-service/app.py and duckdb-service/app.py
env_file: .env on backend, image-service, duckdb-service — not homepage	✅ three env_file: keys in docker-compose.yml; the homepage service has none
DISCORD_WEBHOOK_URL passes through as ${DISCORD_WEBHOOK_URL:-}	✅ both Python services
dropping VITE_API_URL from .env.example is safe	✅ production supplies it as a build arg in docker-compose.prod.yml, never from .env — so it genuinely never reached homepage

Your second commit's correction of the backend's DuckDB default is the right call, and the distinction it draws (code default vs. what compose pins) is the part that would otherwise have drifted again.

Removing HIGHFIVE_API_KEY=your_secure_production_key_here from a file the README tells you to cp is a real security improvement, not tidying — that placeholder was becoming the admin password of every fresh checkout. Same category as the incident in ADR-019.

P1 — the rewritten troubleshooting.md entry can now mislead

The entry says "The most common cause is a missing or malformed .env" and then "no variable in it is actually required." Read together, that reads as though the file is optional. It is not. With no .env at the repo root:

$ docker compose config -q
env file C:\...\highfive\.env not found: GetFileAttributesEx C:\...\highfive\.env: The system cannot find the file specified.

That hard failure is exactly why scripts/check-duckdb-bind-claims.sh manufactures a throwaway .env and removes it again in its EXIT trap. Please state the string compose actually prints — it is the symptom the section exists to resolve, and it is the one thing a stuck contributor can grep for. The same gap is in the CLAUDE.md hunk, which drops the word "Required" from the heading while the file remains mandatory.

P2 — unrelated table churn in the skill file

Nine lines of column-padding realignment in the gotchas table of .claude/skills/esp32-onboarding/SKILL.md, unrelated to the .env change. Cosmetic, inflates the diff; not blocking.

P2 — a tradeoff worth naming, not an objection

CLAUDE.md is auto-loaded into every agent session; .env.example is not. Trading the inline block for a pointer is right for drift, but costs a file read to learn the same three variables. Your wording ("every variable in it is optional; the stack boots on the defaults") mostly compensates, so I'd keep it as-is — flagging the tradeoff so it's a decision rather than an accident.

Chapter 11 entry

In the required format, and the abstraction is the right one: a file that ships as a template is the canonical description of itself — docs link to it, they do not restate it. Good.

Verdict

Approve once the P1 wording is fixed. Please also rebase onto main first — #272 landed new ts-quality, python-lint and repo-guards CI jobs after this branch's last run, so the current green checks don't cover them.

To answer the open question in your description: yes, folding in .claude/skills/esp32-onboarding/SKILL.md was the right call — a fifth restatement in agent tooling drifts the same way, and agents are exactly the readers who will not notice.

[cofade]

For context: my review above was written against main at 669f6c3, which landed #272 after this branch's last CI run. The rebase is only about picking up the new ts-quality / python-lint / repo-guards jobs — no content conflict with your diff that I could find.

[dchaudhari7177]

Thanks — P1 addressed and rebased onto main (now at 669f6c3+), so the new ts-quality / python-lint / repo-guards jobs run against this branch.

P1 — .env is required, its variables are not. You are right that the two sentences read together made the file look optional. Reworded to state the distinction directly and to quote the string compose actually prints, since that is the greppable symptom:

$ docker compose config -q
env file /path/to/highfive/.env not found: ...

I reproduced it here on Docker 27.5.1 and got CreateFile …: The system cannot find the file specified. rather than your GetFileAttributesEx … — same failure, different stat call, so the tail varies by Docker version as well as by OS. I quoted only the stable env file … not found prefix and said so explicitly, rather than pinning a tail that is already inconsistent between two Windows machines.

The entry now also notes that an empty .env is enough to boot the stack, and points at scripts/check-duckdb-bind-claims.sh as the in-repo proof — that is what makes "a service that exits with a .env present is failing on a syntax error" a safe thing to conclude.

CLAUDE.md keeps Required on the heading and draws the same file-vs-variables distinction, so the two files no longer disagree.

P2 — table churn. Reverted. The esp32-onboarding skill file is back to a one-line diff; the nine lines of column realignment were prettier reflowing the table after my edit, and they add nothing.

P2 — CLAUDE.md auto-loads, .env.example does not. Agreed it is a real cost, and I would rather pay it than keep a fifth copy that drifts. Leaving as-is per your read.

make check-citations reports 7 OK on the rebased tree; check-python-twins and the pre-commit hooks are green.

[cofade]

Senior-reviewer re-review -- needs changes before merge

The earlier P1 about the production-shaped .env.example and the distinction between the required .env file and its optional variables is resolved. The current diff is still not mergeable as-is because it introduces three P1 documentation defects.

P1 findings

1. docs/troubleshooting.md -- overbroad diagnosis after .env exists

   The section says that a service which exits while .env is present is failing on a syntax error. That is false: the same document immediately covers runtime failures such as ImportError: libgomp.so.1. A successful docker compose config only proves that Compose can parse and render the configuration; it does not prove that the services can start. Narrow the statement so that syntax errors are the conclusion only when Compose validation fails, and direct contributors to docker compose logs <service-name> for failures after configuration succeeds.

2. docs/11-risks-and-technical-debt/README.md -- incorrect code attribution

   The lessons-learned entry attributes the DUCKDB_SERVICE_URL default to both image-service/app.py and duckdb-service/app.py. The default is read by image-service/app.py; duckdb-service/app.py does not read DUCKDB_SERVICE_URL in that area and only reads DEBUG. Correct the attribution so the lesson points to the files that actually own the behavior.

3. CONTRIBUTING.md -- incomplete guidance about editing .env

   It says editing .env is only needed when choosing a custom admin password. .env.example also documents the optional DISCORD_WEBHOOK_URL for development alerts. Reword this to say that no edits are needed for the defaults, but edits are appropriate for optional custom credentials or notifications.

Architectural smell

The PR declares .env.example to be the single source of truth, but other documents still repeat policy about which services load the file and when it should be edited. The new contradictions show that this invariant is not enforced. Also, docker-compose.yml loads DUCKDB_SERVICE_URL from .env but hard-codes the backend value in its environment: block, leaving a split configuration path that should either be removed or explicitly documented as intentional.

Requested next step

Address the three P1 findings, then rerun the documentation-link and Compose validation checks and request another review. The dev-only .env.example separation and the current Compose parsing are otherwise sound.