Add Vertex AI backend for orgs that disallow raw API keys - #3083
Add Vertex AI backend for orgs that disallow raw API keys#3083eliorerz wants to merge 4 commits into
Conversation
The existing "gemini" backend only talks to the API-key-only Generative Language API, which some GCP orgs disallow outright via policy. Adds a "vertex" backend using google-genai's Vertex AI mode, authenticated via Application Default Credentials (service account, Workload Identity Federation, or `gcloud auth application-default login`) instead of a static key -- no env_key/env_keys, following the same keyless pattern already used for the "bedrock" backend's AWS credential chain. Configured via GOOGLE_CLOUD_PROJECT (required) and GOOGLE_CLOUD_LOCATION (defaults to us-central1), with GRAPHIFY_VERTEX_MODEL to override the default gemini-2.5-flash. Thinking is disabled by default (thinking_budget=0) -- confirmed live that a small max_output_tokens can otherwise be silently consumed entirely by (billed, never-returned) thinking tokens before any extraction JSON is emitted, and thinking traces carry no value for a fixed-schema extraction task anyway. Verified end-to-end against a real GCP project: auto-detection via GOOGLE_CLOUD_PROJECT, the plain-text call path, and the full JSON extraction path (including token accounting) all confirmed working.
![graphify-labs[bot]](https://avatars.githubusercontent.com/in/4356270?s=60&v=4){kind=small}
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 1 advisory finding(s) below merit a look before merge.
Formal verification. 1 change(s) tested, no difference found (not proven).
Graphify review — findings
Adds a vertex backend that calls Vertex AI's Gemini models through google-genai with Application Default Credentials, requiring only GOOGLE_CLOUD_PROJECT (location defaults to us-central1) so orgs that forbid raw API keys can extract without one. Wires it through extract_files_direct and _call_llm, exempts it from the API-key requirement alongside bedrock/claude-cli, and adds it to detect_backend as a fallback ranked below a static Gemini key. Defaults thinking_budget to 0 since thinking tokens are billed but never surface in the extraction JSON and can exhaust the output budget, and treats a hollow response as truncation so adaptive retry bisects the chunk.
Worth a look
- Vertex autodetection shadows explicit Ollama configuration —
graphify/llm.py:3252· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 770 functions depend on the 178 functions this change touches.
Health — this change adds coupling hotspots:
- new:
deduplicate_entities()— 63 callers, 21 callees - new:
build_merge()— 46 callers, 14 callees - new:
extract_files_direct()— 17 callers, 21 callees - new:
extract_corpus_parallel()— 26 callers, 11 callees - new:
_call_claude_cli()— 31 callers, 9 callees - new:
dispatch_command()— 2 callers, 122 callees - new:
_extract_with_adaptive_retry()— 22 callers, 10 callees - new:
_call_llm()— 11 callers, 19 callees - …and 16 more — each is listed as a finding
Verification — 770 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 491 function(s) in the blast radius were not formally verified this run
Formal verification
Could not verify: Could not verify \_call\_llm.
The verifier did not have enough to check \_call\_llm, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly TypeError — names the real obstacle, not a sampling gap)
No difference found (not proven): No behavior difference found in detect\_backend (not a proof).
The verifier ran both versions of detect\_backend on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
Could not verify: Could not verify extract\_files\_direct.
The verifier did not have enough to check extract\_files\_direct, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_format\_backend\_env\_keys.
The verifier did not have enough to check \_format\_backend\_env\_keys, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly KeyError — names the real obstacle, not a sampling gap)
· 2 grounded finding(s) anchored inline below; 22 more finding(s) on lines outside this diff (see the check run).
| return result | ||
|
|
||
|
|
||
| def extract_files_direct( |
There was a problem hiding this comment.
extract_files_direct()
fans out to 21 callees (efferent coupling); 17 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| @@ -3103,7 +3230,8 @@ def _validate_ollama_base_url(url: str, *, warn: bool = True) -> None: | |||
| def detect_backend() -> str | None: | |||
There was a problem hiding this comment.
detect_backend()
16 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
GOOGLE_CLOUD_PROJECT is often set globally for other GCP tools (gcloud, terraform, etc.), not specifically for graphify. An explicit Ollama configuration (OLLAMA_BASE_URL/OLLAMA_HOST) should not be shadowed by an ambient GCP environment variable. Fixes the finding: 'Vertex autodetection shadows explicit Ollama configuration'
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 5 advisory finding(s) below merit a look before merge.
Formal verification. 1 change(s) tested, no difference found (not proven).
Graphify review — findings
Adds a vertex backend that reaches Gemini models through Vertex AI over Application Default Credentials, so orgs whose policy forbids raw API keys can run graphify with a working gcloud/service-account/WIF environment instead. It authenticates keylessly like bedrock (needs GOOGLE_CLOUD_PROJECT, defaults location to us-central1), zeroes the thinking budget by default to stop reasoning tokens silently eating the output budget, and treats a hollow response as truncation so adaptive retry can bisect the chunk. detect_backend picks it up only when GOOGLE_CLOUD_PROJECT is set, ranked after Ollama and all paid keys so an ambient GCP env var never shadows an explicit or paid backend.
Worth a look
- Vertex direct extraction leaks google-genai clients —
graphify/llm.py:1954· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Vertex missing-dependency hint is bypassed —
graphify/llm.py:1955· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Vertex helper call path leaks google-genai clients —
graphify/llm.py:3071· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Ambient GOOGLE_CLOUD_PROJECT silently selects cloud Vertex backend —
graphify/llm.py:3262· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- detect_backend() now returns 'vertex' when GOOGLE_CLOUD_PROJECT is ambiently set, silently changing selection —
graphify/llm.py:3266· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 770 functions depend on the 178 functions this change touches.
Health — this change adds coupling hotspots:
- new:
deduplicate_entities()— 63 callers, 21 callees - new:
build_merge()— 46 callers, 14 callees - new:
extract_files_direct()— 17 callers, 21 callees - new:
extract_corpus_parallel()— 26 callers, 11 callees - new:
_call_claude_cli()— 31 callers, 9 callees - new:
dispatch_command()— 2 callers, 122 callees - new:
_extract_with_adaptive_retry()— 22 callers, 10 callees - new:
_call_llm()— 11 callers, 19 callees - …and 16 more — each is listed as a finding
Verification — 770 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 491 function(s) in the blast radius were not formally verified this run
Formal verification
Could not verify: Could not verify \_call\_llm.
The verifier did not have enough to check \_call\_llm, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly TypeError — names the real obstacle, not a sampling gap)
No difference found (not proven): No behavior difference found in detect\_backend (not a proof).
The verifier ran both versions of detect\_backend on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
Could not verify: Could not verify extract\_files\_direct.
The verifier did not have enough to check extract\_files\_direct, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_format\_backend\_env\_keys.
The verifier did not have enough to check \_format\_backend\_env\_keys, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly KeyError — names the real obstacle, not a sampling gap)
· 2 grounded finding(s) anchored inline below; 22 more finding(s) on lines outside this diff (see the check run).
| return result | ||
|
|
||
|
|
||
| def extract_files_direct( |
There was a problem hiding this comment.
extract_files_direct()
fans out to 21 callees (efferent coupling); 17 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| @@ -3103,13 +3230,15 @@ def _validate_ollama_base_url(url: str, *, warn: bool = True) -> None: | |||
| def detect_backend() -> str | None: | |||
There was a problem hiding this comment.
detect_backend()
16 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
- Vertex AI backend using Application Default Credentials - No API key required (service account/Workload Identity Federation) - Fixes Ollama shadowing issue PR: Graphify-Labs#3083 (upstream)
…tion (#498) ## Summary Enable semantic extraction (docs, images, etc.) in the graphify brain refresh workflow by integrating Vertex AI backend with Workload Identity Federation authentication. ## Changes ### 1. Workload Identity Federation Authentication - Added `google-github-actions/auth@v2` step to authenticate to `osac-ci` GCP project - Added `id-token: write` permission for OIDC token generation - Service account: `osac-ci@osac-ci.iam.gserviceaccount.com` - WIF provider: `projects/1008600636152/locations/global/workloadIdentityPools/github-actions-pool/providers/github-actions-provider` ### 2. Vertex AI Backend Configuration - Set `GOOGLE_CLOUD_PROJECT=osac-ci` - Set `GOOGLE_CLOUD_LOCATION=us-central1` - Backend auto-detection works via these env vars (no `--backend vertex` flag needed) ### 3. Install graphify from Fork (Temporary) **Before:** `pip install --user "graphifyy[sql]==0.9.41"` **After:** `pip install --user "git+https://github.com/eliorerz/graphify@v8#egg=graphifyy[sql,vertex]"` This is temporary until upstream PRs merge: - Graphify-Labs/graphify#3083 (Vertex AI backend) - Graphify-Labs/graphify#3087 (ci-select + fixes) Once merged and a new `graphifyy` release is published, revert to: `pip install --user "graphifyy[sql,vertex]==${GRAPHIFY_VERSION}"` ### 4. Enable Semantic Extraction - **Removed** `--code-only` flag from full extraction (`graphify extract .`) - Now extracts semantic content from docs, images, markdown, etc. via Gemini 2.5 Flash - Incremental updates (`graphify update .`) unchanged (still code-only) ## Rationale Red Hat's GCP org policy **disallows raw API keys entirely** (confirmed live against the osac-ci project). This blocks the existing `gemini` backend (requires `GEMINI_API_KEY`). Vertex AI uses **Application Default Credentials** (service account via Workload Identity Federation) — no static secrets, aligns with security policy. ## Infrastructure Already Deployed - ✅ GCP project: `osac-ci` (project number `1008600636152`) - ✅ Service account with `roles/aiplatform.user` - ✅ WIF pool/provider configured, org-scoped to `osac-project` repos, main branch only - ✅ Vertex backend tested live end-to-end with real extraction ## Testing - [x] YAML validates (`python3 -c "import yaml; yaml.safe_load(...)"`) - [ ] Will test on first scheduled run or manual workflow_dispatch trigger - [ ] Confirm graph.json bundle includes semantic nodes from docs/images ## Rollback Plan If issues occur: 1. Revert this PR 2. Falls back to code-only extraction (AST-only graph, no semantic content) 3. No data loss — graph.json just won't have doc/image nodes <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Improvements** * Improved automated knowledge refresh reliability with more secure cloud authentication. * Refreshes now perform complete content extraction when existing semantic data is incomplete, while continuing to use incremental updates when appropriate. * Enhanced semantic processing delivers more complete and meaningful results. * **Documentation** * Clarified that knowledge reflects committed project content and may not reflect current repository or workflow status. * Added guidance to verify live repository and workflow details directly. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Vertex backend failed at runtime with 'backend vertex requires
GOOGLE_CLOUD_PROJECT to be set' even when GOOGLE_CLOUD_PROJECT was
genuinely set in the environment (confirmed in osac run 32878010795).
Root cause: cli.py lines 3497-3535 pre-flight check for backends
without API keys has special-case exemptions for ollama (localhost),
bedrock (AWS env vars), and claude-cli (binary check), but no
exemption for vertex. Since _get_backend_api_key('vertex') always
returns empty (vertex has no env_key in BACKENDS, uses ADC instead),
allow_no_key stayed False and triggered the error regardless of
GOOGLE_CLOUD_PROJECT being set.
Fix: add vertex branch mirroring bedrock pattern, checking for
GOOGLE_CLOUD_PROJECT env var (same check _vertex_client() does).
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 4 advisory finding(s) below merit a look before merge.
Formal verification. 1 change(s) tested, no difference found (not proven).
Graphify review — findings
Adds a vertex LLM backend that talks to Vertex AI Gemini models through google-genai with Application Default Credentials, so orgs whose policy forbids raw API keys can run extraction with just GOOGLE_CLOUD_PROJECT (and optional GOOGLE_CLOUD_LOCATION) set. Wires it through _call_vertex/_call_llm, key-optional validation, and the "no backend" error message, defaulting to gemini-2.5-flash with thinking_budget=0 so billed thinking tokens can't silently drain the output budget, and treating hollow responses as truncation to trigger adaptive retry. Slots vertex last in detect_backend priority — after Ollama and all paid keys — so an ambient GOOGLE_CLOUD_PROJECT never shadows an explicit local or paid backend.
Worth a look
- detect_backend now returns a backend without any API key configured —
graphify/llm.py:3230· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Ambient GOOGLE_CLOUD_PROJECT auto-selects Vertex backend and can send repo data to GCP without explicit graphify opt-in —
graphify/llm.py:3262· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Vertex auto-detection shadows configured custom providers —
graphify/llm.py:3262· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Whitespace-only GOOGLE_CLOUD_PROJECT is detected as Vertex but later rejected —
graphify/llm.py:3262· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 854 functions depend on the 228 functions this change touches.
Health — this change adds coupling hotspots:
- new:
deduplicate_entities()— 63 callers, 21 callees - new:
build_merge()— 46 callers, 14 callees - new:
extract_files_direct()— 17 callers, 21 callees - new:
extract_corpus_parallel()— 26 callers, 11 callees - new:
_call_claude_cli()— 31 callers, 9 callees - new:
dispatch_command()— 2 callers, 122 callees - new:
_extract_with_adaptive_retry()— 22 callers, 10 callees - new:
_call_llm()— 11 callers, 19 callees - …and 18 more — each is listed as a finding
Verification — 854 functions in the blast radius were not formally verified this run (proofs are advisory here).
Health delta baseline: last indexed commit 43d54ac (diverged from this PR's base — delta is approximate).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 743 function(s) in the blast radius were not formally verified this run
Formal verification
Could not verify: Could not verify dispatch\_command.
The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)
Could not verify: Could not verify \_call\_llm.
The verifier did not have enough to check \_call\_llm, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly TypeError — names the real obstacle, not a sampling gap)
No difference found (not proven): No behavior difference found in detect\_backend (not a proof).
The verifier ran both versions of detect\_backend on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
Could not verify: Could not verify extract\_files\_direct.
The verifier did not have enough to check extract\_files\_direct, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_format\_backend\_env\_keys.
The verifier did not have enough to check \_format\_backend\_env\_keys, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly KeyError — names the real obstacle, not a sampling gap)
· 2 grounded finding(s) anchored inline below; 24 more finding(s) on lines outside this diff (see the check run).
| return result | ||
|
|
||
|
|
||
| def extract_files_direct( |
There was a problem hiding this comment.
extract_files_direct()
fans out to 21 callees (efferent coupling); 17 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| @@ -3103,13 +3230,15 @@ def _validate_ollama_base_url(url: str, *, warn: bool = True) -> None: | |||
| def detect_backend() -> str | None: | |||
There was a problem hiding this comment.
detect_backend()
16 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
Fixes graphify-brain-refresh workflow failure (run 32878010795) where vertex backend rejected GOOGLE_CLOUD_PROJECT even when set. **Problem**: graphify cli.py's pre-flight check had no allow-no-key exemption for vertex backend (unlike ollama/bedrock/claude-cli), so it always rejected vertex regardless of GOOGLE_CLOUD_PROJECT being present. **Fix**: Install graphify from eliorerz/graphify@1a122e1 (feat/vertex-ai-backend branch) which adds the missing vertex check. Will revert to PyPI install once the next graphify release includes this fix. **Related**: - graphify fix: eliorerz/graphify#4 - Upstream PR: Graphify-Labs/graphify#3083 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated the automated brain refresh process to use a pinned Graphify source revision. * Improved consistency and reproducibility of refresh runs. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Every other backend in this module wires _resolve_max_retries() into its SDK client so a burst of 429s during a large parallel run gets absorbed instead of dropping the chunk -- google-genai needs this set explicitly via http_options.retry_options. Confirmed live that an unconfigured client does NOT retry 429s at all (every field of an empty HttpRetryOptions() defaults to None), which is exactly why a large first-run corpus (~1100 docs across the osac mono-repo) surfaced "Resource exhausted" as hard chunk failures instead of transient delays (9 of 16 semantic chunks failed outright, producing a smaller graph that graphify's own shrink guard correctly refused to publish).
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 5 advisory finding(s) below merit a look before merge.
Formal verification. 1 change(s) tested, no difference found (not proven).
Graphify review — findings
Adds a vertex backend that talks to Vertex AI's Gemini models through google-genai using Application Default Credentials, so orgs whose policy forbids raw API keys can run graphify with just GOOGLE_CLOUD_PROJECT set. Wires it into extract_files_direct and _call_llm, treats it as keyless (like bedrock) throughout key-validation and CLI dispatch, configures the client to retry 429/5xx up to _resolve_max_retries() so large parallel runs absorb "Resource exhausted" bursts, and zeroes the thinking budget by default since thinking tokens are billed but can silently consume the whole output budget. Places vertex last in detect_backend priority and reclassifies hollow responses as truncation so adaptive retry can bisect the chunk.
Worth a look
- Vertex clients are never closed after per-call construction —
graphify/llm.py:1951· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- _rec called with possibly-None token counts from Vertex usage_metadata —
graphify/llm.py:3103· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Ambient GOOGLE_CLOUD_PROJECT now changes default backend selection —
graphify/llm.py:3250· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Ambient GOOGLE_CLOUD_PROJECT can auto-route source files to Vertex AI —
graphify/llm.py:3275· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- detect_backend never returns 'vertex' despite docstring claim —
graphify/llm.py:3275· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 854 functions depend on the 228 functions this change touches.
Health — this change adds coupling hotspots:
- new:
deduplicate_entities()— 63 callers, 21 callees - new:
build_merge()— 46 callers, 14 callees - new:
extract_files_direct()— 17 callers, 21 callees - new:
extract_corpus_parallel()— 26 callers, 11 callees - new:
_call_claude_cli()— 31 callers, 9 callees - new:
dispatch_command()— 2 callers, 122 callees - new:
_extract_with_adaptive_retry()— 22 callers, 10 callees - new:
_call_llm()— 11 callers, 19 callees - …and 18 more — each is listed as a finding
Verification — 854 functions in the blast radius were not formally verified this run (proofs are advisory here).
Health delta baseline: last indexed commit 43d54ac (diverged from this PR's base — delta is approximate).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 743 function(s) in the blast radius were not formally verified this run
Formal verification
Could not verify: Could not verify dispatch\_command.
The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)
Could not verify: Could not verify \_call\_llm.
The verifier did not have enough to check \_call\_llm, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly TypeError — names the real obstacle, not a sampling gap)
No difference found (not proven): No behavior difference found in detect\_backend (not a proof).
The verifier ran both versions of detect\_backend on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
Could not verify: Could not verify extract\_files\_direct.
The verifier did not have enough to check extract\_files\_direct, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_format\_backend\_env\_keys.
The verifier did not have enough to check \_format\_backend\_env\_keys, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly KeyError — names the real obstacle, not a sampling gap)
· 2 grounded finding(s) anchored inline below; 24 more finding(s) on lines outside this diff (see the check run).
| return result | ||
|
|
||
|
|
||
| def extract_files_direct( |
There was a problem hiding this comment.
extract_files_direct()
fans out to 21 callees (efferent coupling); 17 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| @@ -3103,13 +3250,15 @@ def _validate_ollama_base_url(url: str, *, warn: bool = True) -> None: | |||
| def detect_backend() -> str | None: | |||
There was a problem hiding this comment.
detect_backend()
16 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
1 participant
Summary
geminibackend only talks to the API-key-only Generative Language API. Some GCP orgs disallow raw API keys entirely via policy (confirmed live against a real org policy), leaving no way to use Gemini through graphify at all.vertexbackend usinggoogle-genai's Vertex AI mode, authenticated via Application Default Credentials (service account, Workload Identity Federation, orgcloud auth application-default login) — no API key, following the same keyless pattern already used forbedrock's AWS credential chain.GOOGLE_CLOUD_PROJECT(required) andGOOGLE_CLOUD_LOCATION(defaults tous-central1), withGRAPHIFY_VERTEX_MODELto override the defaultgemini-2.5-flash.thinking_budget=0) — confirmed live that a smallmax_output_tokenscan otherwise be silently consumed entirely by billed, never-returned thinking tokens before any extraction JSON is emitted. Thinking traces carry no value for a fixed-schema extraction task.extract_files_direct's JSON extraction and_call_llm's plain-text path),detect_backend()auto-detection, vision/image support, and thevertexpip extra (google-genai).Test plan
python3 -m py_compile graphify/llm.pydetect_backend()correctly returnsvertexwhen onlyGOOGLE_CLOUD_PROJECTis set_call_llm(..., backend="vertex")) returns correct text + usage accountingextract_files_direct(..., backend="vertex")) against a real file produces correct nodes/edges/token counts, no truncationGOOGLE_CLOUD_PROJECTraises a clearValueErrorinstead of an opaque SDK error