AlphaDiana: A Measurement System for Agents in Verifiable Reasoning Tasks
-
Updated
Nov 2, 2026 - Python
#
agent-evaluations
Here are 17 public repositories matching this topic...
Acceptance testing for LLM systems via role subagents (Claude Code plugin) · Приёмочное тестирование LLM-систем ролевыми субагентами
qa acceptance-testing sandbox quality-assurance agents testing-framework ai-agents red-teaming llm evals llm-agents prompt-evaluation llm-testing llm-evals ai-agent-evaluation claude-code agent-evaluation subagents agent-testing agent-evaluations
-
Updated
Nov 1, 2026 - Python
Capability testing for agent harnesses: turn expert knowledge into executable specifications, run the real agent, get a binary pass/fail verdict with tests, traces, and deliverables.
typescript self-hosted tracing developer-tools testing-framework ai-agents e2e-testing ai-sdk fastapi opentelemetry llm vercel-ai-sdk llm-evaluation agent-evaluation agent-testing agent-evaluations
-
Updated
Nov 2, 2026 - TypeScript
Official Seclai Go SDK
go sdk ai serverless managed ai-agents llm ai-governance agent-observability jailbreak-protection prompt-injection-protection agent-evaluations
-
Updated
Nov 1, 2026 - Go
Official Seclai Python SDK
python sdk ai serverless managed ai-agents llm ai-governance agent-observability jailbreak-protection prompt-injection-protection agent-evaluations
-
Updated
Nov 1, 2026 - Python
Official Seclai JavaScript SDK
javascript typescript sdk ai serverless managed ai-agents llm ai-governance agent-observability jailbreak-protection prompt-injection-protection agent-evaluations
-
Updated
Nov 1, 2026 - TypeScript
Official Seclai C# SDK
sdk ai csharp serverless managed ai-agents llm ai-governance agent-observability jailbreak-protection prompt-injection-protection agent-evaluations
-
Updated
Nov 1, 2026 - C#
Official Seclai Command Line Interface
cli ai serverless skills managed ai-agents llm ai-governance agent-observability jailbreak-protection prompt-injection-protection agent-evaluations
-
Updated
Nov 1, 2026 - TypeScript
Live evaluation harness for AI agents. Measure success rate and consistency. Lightweight local-first Python tool for testing Claude, GPT, Grok and custom agents.
python local ai-agents local-first ai-evaluator ai-evaluation local-llm local-ai llm-evaluation llm-evaluation-toolkit llm-evaluation-metrics ai-agents-automation ai-evaluation-tools ai-agent-tools ai-evaluation-metrics agent-evaluations ai-evaluations agent-evaluation-tools llm-evaluation-benchmark-health-ai-calibration ohovo
-
Updated
Oct 17, 2026 - Python
Connect AI coding tools to Seclai via Model Context Protocol (MCP) — manage agents, knowledge bases, and content sources from Claude, Cursor, Windsurf, and more
serverless mcp cursor managed ai-agents claude windsurf ai-governance model-context-protocol mcp-server agent-observability jailbreak-protection prompt-injection-protection agent-evaluations
-
Updated
Oct 22, 2026
Cognitive AutoGen (CoAG) extends Microsoft AutoGen with a CoALA-inspired cognitive layer — structured working memory, RAG long-term memory and a 6-phase deliberative cycle
python coala multi-agent-system rag cognitive-architectures autogen-framework language-agents agent-evaluations agent-oriented-software-enginerring
-
Updated
Feb 24, 2026 - Python
Turn agent telemetry into eval jobs — outcome, quality, and spend — correlated with business outcomes per agent run.
business-intelligence business-analytics opentelemetry agent-framework opentelemetry-collector langsmith langfuse llm-as-judge llm-as-a-judge agentic-workflow agentic-ai agent-evals deepeval agent-evaluation kagent agentgateway agent-eval agent-evaluations agent-evaluation-tools byo-eval
-
Updated
Sep 1, 2026 - Python
Open-source agent assurance: turn a PRD and agent URL into frozen test suites with evidence-backed PASS/FAIL/UNVERIFIABLE verdicts. Web console, REST API, SQLite. Domain packs for regulated teams (fintech, insurance, health).
evaluation-framework ai-agents evals agent-evaluation agent-evaluations agent-evaluation-tools agent-evaluation-eval
-
Updated
Oct 17, 2026 - Python
Provider-neutral Python library for LLM and AI evaluation, regression testing, statistical release gates, judge calibration, and auditable evaluation evidence.
python machine-learning regression-testing evaluation-datasets online-experiments model-evaluation-metrics llmops trajectory-evaluation llm-evaluation-framework llm-as-a-judge ai-security-testing ai-testing-tool ai-evaluation-framework statistical-evaluation evaluation-harness agent-evaluations evaluation-engineering judge-calibration evaluation-rubrics ai-quality-gates
-
Updated
Oct 26, 2026 - Python
typescript typescript-sdk reproducibility typescript-action reproducibility-test agent-evaluation ai-reliability agent-evaluations failure-testing seacs
-
Updated
Sep 23, 2026 - JavaScript
Rogue AI detection, agent behavior analysis, FIST INDEX™ scoring, and AI governance. Join free at the-fist.com.
cybersecurity ai-safety ai-agents responsible-ai ai-governance responsible-ai-techniques agentic-ai agent-evaluation responsible-ai-evaluation-tools rogue-ai agent-evaluations agent-evaluation-tools rogue-agents rogue-agents-security
-
Updated
Oct 30, 2026
五款 AI Agent 的 Harness 竞品评测：Codex / Workbuddy / DSH / 元宝专家模式 / 豆包办公 ｜ 三层归因定责（模型/SKILL/Harness）+ 多题题库 + 全量现场记录 + 可复核报告
-
Updated
Oct 31, 2026 - Python
Add this topic to your repo
To associate your repository with the agent-evaluations topic, visit your repo's landing page and select "manage topics."