Add CI workflow and document it in the README#14
Merged
Conversation
Run on push and pull_request to main: set up Python 3.11, install dev requirements, run ruff, and run an offline pytest suite. - ruff.toml pins a minimal correctness-focused rule set (pyflakes plus the import, statement, syntax, and warning subsets of pycodestyle) so linting flags real problems without rewriting deliberate style. Fixes the one real finding, a missing final newline in six files. - requirements-dev.txt adds ruff and pytest on top of the runtime requirements. - tests/ covers the schema contracts, source-label helpers, eval grading, and the Critic evidence wiring, all with no OpenAI, Pinecone, or Tavily key. - The live eval harness and the Critic's groundedness judgment need real keys and are intentionally left out of CI. - permissions is set to contents: read and the job needs no secrets.
- Add the CI workflow status badge under the title. - Add a Mermaid architecture diagram of the Researcher and Critic graph plus the ingestion path under the Architecture heading. - Note the demo corpus (three public 10-K filings across retail, airline, and industrial sectors) and that the app is corpus-neutral, in the Usage section. - Correct one sentence: the eval harness verifies groundedness offline, not in CI, since that check needs live keys.
Importing config.py constructs the Pinecone client, which raises when no API key is set, so test collection failed on a keyless runner. Set clearly-fake placeholder keys in tests/conftest.py before the modules import, and only when the environment does not already supply real ones. No test contacts a provider.
Document the GitHub Actions workflow in a new Continuous integration section between Evaluation and Observability, and add a matching bullet to the Stack list.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a GitHub Actions CI workflow and documents it in the README.
CI (Task 1)
.github/workflows/ci.ymlruns on push and pull_request to main: Python 3.11, installrequirements-dev.txt,ruff check ., thenpytest tests/ -q.ruff.tomlpins a minimal correctness-focused rule set (pyflakes plus the import, statement, syntax, and warning subsets of pycodestyle). The one real finding, a missing final newline in six files, is fixed rather than silenced.requirements-dev.txtadds ruff and pytest on top of the runtime requirements.tests/is an offline suite covering the schema contracts, source-label helpers, eval grading, and the Critic evidence wiring.permissionsiscontents: readand the job needs no secrets.What runs without keys, and what is gated
The whole suite passes with no
OPENAI_API_KEY,PINECONE_API_KEY, orTAVILY_API_KEYset. Two things need live keys and are intentionally left out of CI:evals/eval.pyharness, which stands up the live agents, andeval.pyfor local runs with keys. CI instead verifies the deterministic logic around it, including that the Critic is handed the exact retrieved chunks as evidence and returns the verdict faithfully.README (Task 2)
Verification
Clean Python 3.11.5 venv: install,
ruff check .clean, and 22 tests pass in a few seconds, well under two minutes.