Skip to content

fix(ci): make the PR checks pass again on main - #269

Merged
google-oss-prow[bot] merged 1 commit into
kubeflow:mainfrom
kmr-rohit:fix/ci-checks
Oct 13, 2026
Merged

google-oss-prow[bot] merged 1 commit into
kubeflow:mainfrom
kmr-rohit:fix/ci-checks

Conversation

@kmr-rohit

Copy link
Copy Markdown
Contributor

Every check on main has failed since #249 and #261 (Pytest, Lint and format, Compile and Test), so every open PR shows red. This PR fixes the causes. It is independent of the GitOps PRs and can merge first.

Causes and fixes

Check Cause Fix
Pytest tests/test_code_utils.py, test_issues_pipeline.py, test_pipeline_utils.py import helpers that #249 moved to pipelines/utils/ add pipelines/utils to sys.path, as the pipelines do
Pytest 3 widget tests expect <a> links in the answer bubble; #249 intentionally moved links to the Sources panel tests updated to the current contract (label only, escaped; non-http links inert; inline code untouched)
Pytest 63 errors in test_mcp_server.py: it tests pre-#249 internals (server.client, _init, _rerank_hits, ...) that no longer exist module skipped with the reason; needs a rewrite against milvus_search / intent_router (good follow-up issue)
Compile and Test kubeflow-pipeline.py imports ingest libraries not installed in CI (python-frontmatter, html-table-rescuer, ...) compile the pipelines in their own virtualenv from pipelines/requirements.txt
(latent) pipelines/requirements.txt was not installable: kfp==2.17.0 pins requests==2.33.0, the file pinned 2.34.2 pin requests==2.33.0
Lint and format milvus_search.py, server.py not formatted ruff format (no logic change)

Tested locally

  • requirements-test.txt only (PR Safety job): 112 passed, 69 skipped.
  • Plus mcp-server/requirements.txt (Compile and Test job): 112 passed, 69 skipped.
  • All three pipelines compile in a clean virtualenv from pipelines/requirements.txt.
  • ruff check . and ruff format --check docs-agent-mcp/mcp-server tests pass.

Every check on main has failed since kubeflow#249/kubeflow#261:

- Pytest: the pipeline tests imported code_utils, issues_utils and utils
  from pipelines/, but kubeflow#249 moved them to pipelines/utils/. Add that
  directory to sys.path, as the pipelines themselves do.
- Widget tests: kubeflow#249 intentionally stopped rendering links in the answer
  bubble (sources go to the Sources panel). Update the three tests to
  that contract: http(s) links render as their escaped label, other
  links stay inert text, inline code is untouched.
- test_mcp_server.py targets pre-kubeflow#249 internals (server.client, _init,
  _rerank_hits, ...) that no longer exist. Skip the module with the
  reason; it needs a rewrite against milvus_search/intent_router.
- Compile: kubeflow-pipeline.py needs the ingest libraries
  (python-frontmatter, html-table-rescuer, ...). Compile the pipelines
  in their own virtualenv from pipelines/requirements.txt.
- pipelines/requirements.txt could not be installed at all: kfp 2.17.0
  pins requests==2.33.0 but the file pinned 2.34.2.
- ruff format on milvus_search.py and server.py.

Locally, with requirements-test.txt alone and with the MCP server
requirements added: 112 passed, 69 skipped. All three pipelines compile
in a clean virtualenv. ruff check and format pass.

Signed-off-by: Rohit Kumar <rr7433446@gmail.com>
@tarekabouzeid

Copy link
Copy Markdown
Member

Thanks @kmr-rohit
/lgtm
/approve

@google-oss-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tarekabouzeid

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@google-oss-prow
google-oss-prow Bot merged commit 0e49ebd into kubeflow:main Oct 13, 2026
8 checks passed
kmr-rohit added a commit to kmr-rohit/docs-agent that referenced this pull request Oct 13, 2026
Merging kubeflow#269 made the tests pass again, and the oke-cicd deploy job ran
for the first time since September: it helm-upgraded the docs-agent
release on the demo cluster with chart defaults (only the MCP image
changed; production stayed healthy). With Argo CD managing the cluster,
two writers would keep overwriting each other.

The workflow now tests every PR and, on main, only builds and pushes the
MCP image. The kubectl/helm/OCI steps and the cluster credentials are
gone; the job is renamed "Build and push MCP image" and still gated on
ENABLE_OKE_DEPLOY. Rolling out an image is a PR to
deploy/envs/<env>/docs-agent.yaml plus a sync in Argo CD.

deploy/envs/prod pins ghcr.io/kubeflow/mcp-kubeflow-docs:0e49ebd, the
image that CI deployed, so Git matches the cluster instead of pointing
back at the old fork image.

Signed-off-by: Rohit Kumar <rr7433446@gmail.com>
google-oss-prow Bot pushed a commit that referenced this pull request Oct 15, 2026
* feat(deploy): Argo CD observe-only setup and Qwen judge release

Add Argo CD install values and Applications for docs-agent,
gateway-guardrails and llm-runtime, tracking main with no syncPolicy so
Argo reports drift without changing the cluster.

Deployment-specific values (domain, ACME email, CORS origins, image
overrides) come from a private config repo as a second source, so no
personal or environment data lives in this repo. Secrets stay as
Kubernetes Secrets referenced by name.

Add llm-judge: a second llm-runtime release using the existing qwen
profile on its own GPU, served as flo-judge, for LLM-as-a-judge evals.
Gemma keeps answering; a different model family judges. An Istio
AuthorizationPolicy limits callers to eval, triage and Langfuse.

Signed-off-by: Rohit Kumar <rr7433446@gmail.com>

* feat(deploy): keep env values in deploy/envs, ACME issuer optional

Per-environment values move to deploy/envs/prod/ in this repo and are
read by Argo as a second source; no private config repo is needed.

gateway-guardrails gains gateway.tls.acme.createIssuer (default true).
With false, the Certificate uses an existing ClusterIssuer, so the ACME
contact email lives only in the cluster and not in public values.

deploy/secrets/ lists the Secrets the stack needs and adds a script that
backs them up to a private local directory outside the repo.

Signed-off-by: Rohit Kumar <rr7433446@gmail.com>

* docs(deploy): list required Secrets in the Argo README instead of a secrets folder

Signed-off-by: Rohit Kumar <rr7433446@gmail.com>

* feat(deploy): Langfuse values in Git and Argo CD read-only account

Capture the live Langfuse release values in deploy/envs/prod/langfuse,
including the settings that were applied by hand (public NextAuth URL,
sign-up disabled, image pull policy). Pin salt, encryption key and
NextAuth secret to the existing langfuse-app Secret so an upgrade cannot
rotate them. Allow LLM connections to llm-judge and llm-stable.

Add an observe-only Argo Application for Langfuse, and a viewer account
bound to role:readonly. Password hashes are passed at install time.

Signed-off-by: Rohit Kumar <rr7433446@gmail.com>

* docs(deploy): record that in-cluster Istio policies are not enforced without sidecars

Signed-off-by: Rohit Kumar <rr7433446@gmail.com>

* feat(chart): docs-agent mcp.extraEnv for tracing keys

The live MCP deployment carried LANGFUSE_PUBLIC_KEY/SECRET_KEY added with
kubectl set env. Neither kubectl diff nor Argo flags fields added outside
Git, so a clean redeploy would have silently dropped tracing. Add a
generic mcp.extraEnv list (default empty) and set the two Secret
references in deploy/envs/prod/docs-agent.yaml.

Signed-off-by: Rohit Kumar <rr7433446@gmail.com>

* feat(mcp): Langfuse tracing for MCP tools, retrieval and embeddings

The live MCP server runs a fork image (santhoshtoorpu/...:otel-flo) whose
tracing code was never upstreamed, so any image built from main loses
the TOOL / RETRIEVER / EMBEDDING spans. Port it onto current main:

- flo_tracing.py (from the live image): no-op unless LANGFUSE_PUBLIC_KEY
  is set; tracing errors never break a search. Adds a traced() decorator
  so each function gets one line instead of a re-indented body;
  functools.wraps keeps the signatures FastMCP builds tool schemas from.
- Traced: the three MCP tools, search_docs_auto, search_collection,
  embed_query. Outputs record hit count, retrieval mode, intent and top
  scores.
- Fix: top_scores read the score from entity, but Milvus puts distance
  on the hit, so it was always empty in production.
- langfuse==4.15.2 (the version in the live image).

Only the tracing is taken from the image; it was built from an older
snapshot, and main's later fixes (argument validation, release dates,
repo filter on code search) are kept.

Verified in-cluster with the real SDK and the evaluation project: one
search_kubeflow_docs call produced TOOL, RETRIEVER and EMBEDDING
observations with environment=staging. tests/test_flo_tracing.py covers
the decorator, the output summary and the tool schemas.

Co-authored-by: SanthoshToorpu <90833739+SanthoshToorpu@users.noreply.github.com>
Signed-off-by: Rohit Kumar <rr7433446@gmail.com>

* ci: stop deploying from CI; Argo CD is the only deployer

Merging #269 made the tests pass again, and the oke-cicd deploy job ran
for the first time since September: it helm-upgraded the docs-agent
release on the demo cluster with chart defaults (only the MCP image
changed; production stayed healthy). With Argo CD managing the cluster,
two writers would keep overwriting each other.

The workflow now tests every PR and, on main, only builds and pushes the
MCP image. The kubectl/helm/OCI steps and the cluster credentials are
gone; the job is renamed "Build and push MCP image" and still gated on
ENABLE_OKE_DEPLOY. Rolling out an image is a PR to
deploy/envs/<env>/docs-agent.yaml plus a sync in Argo CD.

deploy/envs/prod pins ghcr.io/kubeflow/mcp-kubeflow-docs:0e49ebd, the
image that CI deployed, so Git matches the cluster instead of pointing
back at the old fork image.

Signed-off-by: Rohit Kumar <rr7433446@gmail.com>

* fix(deploy): drop node-IP anti-affinity from Langfuse values

GPU nodes carry the nvidia.com/gpu NoSchedule taint, so Langfuse pods
never schedule there; the per-hostname rule was redundant.

Signed-off-by: Rohit Kumar <rr7433446@gmail.com>

* ci: build and push the session-issuer image to ghcr.io/kubeflow

The gateway-guardrails chart defaults to ghcr.io/kubeflow/kubeflow-session-issuer,
but nothing built it. Push it on main tagged by short SHA; prod moves to it in
a follow-up once the first image exists.

Signed-off-by: Rohit Kumar <rr7433446@gmail.com>

---------

Signed-off-by: Rohit Kumar <rr7433446@gmail.com>
Co-authored-by: SanthoshToorpu <90833739+SanthoshToorpu@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants