Agentic APIOps

Production-style API testing and diagnosis platform with Java execution authority, Python Agent workflows, a guarded Tool Gateway, tracing/evaluation, and a React console.

Why this project is different

• Java owns deterministic execution, security, and authoritative TestReport facts.
• Python AgentLab owns TestCase generation, diagnosis, tool orchestration, tracing, and runtime evaluation.
• Agents cannot bypass Java-owned resources; resource access crosses the guarded Java Tool Gateway.
• The Java ↔ Python boundary is verified with real cross-process E2E tests.

Architecture

Java owns deterministic execution and security boundaries. Python owns Agent orchestration, diagnosis, tracing, and evaluation.

Core flow

OpenAPI → Metadata → TestCase DSL → Async Runner → Assertion → TestReport
→ Diagnosis → Evidence / Tool → DiagnosisReport → Trace / Evaluation

LLM-generated TestCases are candidates. Java validates the DSL, runs the test, and publishes the authoritative TestReport; AgentLab consumes that report and controlled evidence to produce diagnosis and evaluation outputs.

Authority boundary

Fact / Responsibility	Authority
OpenAPI metadata	Java Platform
DSL validation	Java Platform
HTTP execution	Java Runner
Assertions	Java Platform
TestReport	Java Platform
Tool authorization	Java Tool Gateway
Agent workflow	Python AgentLab
Diagnosis reasoning	Python AgentLab
Agent trace	Python AgentLab
Runtime evaluation	Python AgentLab

LLM output is a candidate or inference. Java TestReport is execution truth. Python ALLOW is not Java ALLOW, and Human Approval is not Java Authorization.

Implemented & Verified

Java Platform

• OpenAPI metadata and project-scoped API access
• TestCase DSL validation
• HTTP Runner and asynchronous batch execution
• Assertion Engine and authoritative TestReport
• JWT/project authorization
• RAG retrieval and Context Pack construction
• Structured Agent generation and diagnosis integration
• Tool Gateway, guards, sanitization, and audit

Python AgentLab

• TestCase Generation workflow
• Diagnosis workflow with structured outputs
• Context engineering and Java evidence consumption
• Tool planning and guarded tool-use workflow
• HITL approval flow
• Trace recording and redaction
• Runtime evaluator, metrics, judge, and reporting

Console

• Overview
• API Studio
• Runs
• Diagnosis Studio and Diagnosis execution
• Traces
• Evaluation
• Settings

Agent Safety Boundary

Python tool selection is orchestration, not authorization. Python Guard is defense-in-depth; HITL is workflow consent, not Java resource authorization. Java Tool Gateway performs final authorization, guarding, execution, sanitization, and audit. Java DENY is terminal; Python has no raw-resource fallback.

Implementation Evidence

Capability	Implementation	Verification
OpenAPI metadata	OpenAPI metadata controller	Controller test
TestCase DSL validation	DSL validator	Validation tests
HTTP execution	Run execution service	Runner service tests
Assertions	Assertion Engine	Assertion tests
Async test batches	Batch controller	Batch controller tests
TestReport	Report controller	Report controller tests
Tool Gateway	Tool Gateway	Gateway tests
RAG / evidence	Context Pack builder	Context Pack tests
Python generation	Generation workflow	Generation workflow tests
Python diagnosis	Diagnosis workflow	Diagnosis workflow tests
Tracing	Trace recorder	Trace recorder tests
Runtime evaluation	Evaluator	Evaluator tests
Java ↔ Python integration	Java API client	Cross-process E2E test
Console context trail	Console shell	Context-trail verification

REST integration

The public integration surface is project-scoped:

Capability	Method	Path
OpenAPI metadata	GET	/api/v1/projects/{projectId}/openapi/apis/{apiId}
Submit test batch	POST	/api/v1/projects/{projectId}/test-batches
Read test report	GET	/api/v1/projects/{projectId}/test-runs/{runId}/report
Call a guarded tool	POST	/api/v1/projects/{projectId}/tool-calls

TestCase DSL validation is enforced inside Java Agent and Runner service boundaries; there is no standalone validation REST endpoint in the current public implementation.

Quick start

Run each verification block from the repository root.

Verify the repository

# Java Platform
cd java-apiops-platform
./mvnw clean verify

# Python AgentLab
cd python-apiops-agentlab
uv run ruff check .
uv run pytest

# React Console
cd apiops-console
npm ci
npm run lint
npm run test:context
npm run build

On Windows, use ./mvnw.cmd clean verify for the Java command.

Local development

# Python API
cd python-apiops-agentlab
uv run uvicorn app.main:app --reload

# Console
cd apiops-console
npm run dev

The Console and AgentLab connect to their configured Java/Python services. docker-compose.dev.yml provides local MySQL, Redis, and RabbitMQ infrastructure; it is not a one-command application deployment.

Verification

The figures below come from the current public release verification; this README/assets-only pass did not rerun the full suites:

• Java: 11-module Maven reactor, BUILD SUCCESS
• Python: Ruff PASS; 740 passed, 4 warnings
• Console: lint PASS, context test PASS, build PASS
• GitHub Actions: Java CI PASS

Repository structure

agentic-apiops-public
├── .github
│   └── assets
├── java-apiops-platform
├── python-apiops-agentlab
├── apiops-console
├── shared-schemas
├── examples
├── scripts
└── README.md

Java is responsible for execution, security, audit, and report truth. Python is responsible for Agent workflow, tool orchestration, tracing, and runtime evaluation. The projects meet through shared schemas, REST contracts, and the Java Tool Gateway.

Public scope

This public portfolio intentionally excludes private/internal documentation, runtime artifacts, and credentials. It includes executable production source, ordinary tests, shared schemas, and public-safe configuration.

Benchmark materials are not included in this public release.

Configuration templates are provided as .env.example files; real credentials are never committed.