Skip to content

test: keep .env out of the suite; report the real interpreter in notebooks - #12

Merged
jqnatividad merged 2 commits into
dathere:mainfrom
jqnatividad:tests-hermetic-env
Sep 16, 2026
Merged

jqnatividad merged 2 commits into
dathere:mainfrom
jqnatividad:tests-hermetic-env

Conversation

@jqnatividad

Copy link
Copy Markdown
Collaborator

Two related bits of drift, both found the hard way while setting up a local instance.

1. The suite was not actually hermetic

CONTRIBUTING.md, the CI comment and the suggested_commands memory all stated
the suite needs no .env. True — but core/config.py reads one for every
field and builds its settings singleton at import time, so a developer's local
.env silently changed what the suite asserted.

Concretely: following the README's own troubleshooting advice and setting
NOTEBOOK_VERIFICATION_ENABLED=false turned the suite red. That is how this was
found.

  • tests/conftest.py sets DATA_CONCIERGE_ENV_FILE="" before anything imports
    config; config.py honours it, empty meaning no env file.
  • Three tests asserted a shipped default (or a security guard's default posture)
    by reading the live singleton, which also reflects the process environment.
    They now assert Settings.model_fields[...].default, or pin the value with
    monkeypatch, so an exported variable cannot reach them either:
    test_ships_enabled, test_ships_locked_down,
    test_model_validator_rejects_on_update_construction.
  • The three texts now describe the real guarantee, including the part the
    conftest guard does not cover.

2. Generated notebooks claimed an impossible interpreter

notebook_generator.py hardcoded language_info.version = "3.11" in two places.
That has been wrong since requires-python became >=3.12. The notebook is the
deliverable — reproducible, citable, independently verifiable — so its provenance
has to be true. Both sites now report platform.python_version().

Verified

  • 802 passed with a hostile .env (both flags flipped) and the same two
    variables exported in the shell
  • 802 passed clean; ruff check clean
  • A generated notebook confirmed to carry the running interpreter
    (language_info.version == platform.python_version())

Note

MCP_ALLOW_PRIVATE_URLS was the second latent instance of this bug — the
variable exists precisely so local development can point at localhost MCP
servers, which makes the developer most likely to run the suite the one most
likely to have it set.

🤖 Generated with Claude Code

jqnatividad and others added 2 commits September 15, 2026 23:26
…books

The suite was documented as hermetic in three places, but core/config.py reads
.env for every field and builds its settings singleton at import, so a local
.env silently changed what the suite asserted. Following the README's own advice
to set NOTEBOOK_VERIFICATION_ENABLED=false turned the suite red, which is how
this was found.

conftest.py now sets DATA_CONCIERGE_ENV_FILE="" before anything imports config,
and config.py honours it (empty means no env file). Three tests asserted shipped
defaults or a security guard's default posture by reading the live singleton,
which also reflects the process environment; they now assert
Settings.model_fields[...] or pin the value with monkeypatch, so an exported
variable cannot reach them either.

Separately, generated notebooks hardcoded language_info.version = "3.11" in two
places — impossible since requires-python became >=3.12. The notebook is the
deliverable and its provenance has to be true, so both now report
platform.python_version().

Verified: 802 passed with a hostile .env (both flags flipped) AND the same two
variables exported in the shell; 802 passed clean; ruff clean. Generated
notebook metadata confirmed to carry the running interpreter.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both README sections still described the bug the previous commit fixed. They
claimed core/config.py hardcodes env_file=".env" and that a .env entry fails
test_ships_enabled — neither is true now: env_file is conditional on
DATA_CONCIERGE_ENV_FILE, which tests/conftest.py sets to "", and
test_ships_enabled asserts the class default rather than the live singleton.
Confirmed by appending the flag to .env and running the file: 33 passed.

Setting it in .env is now a fine way to keep a local default, so both sections
say so. The behaviour-flags section at :264 carried the same advice plus a
cross-reference to the reason being removed, so it is updated too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jqnatividad
jqnatividad merged commit 3ca1b08 into dathere:main Sep 16, 2026
6 checks passed
@jqnatividad
jqnatividad deleted the tests-hermetic-env branch September 16, 2026 10:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant