Reference frameworks for browser, API, performance, accessibility, visual, and LLM-evaluation quality engineering — built by Prayag Vyas, Senior Quality Engineer (7+ years). This is not a collection of sample tests: each repository is a governed system with explicit architecture, controlled targets, failure evidence, and CI you can audit.
This portfolio exists to support Staff SDET, Lead SDET, QA Architect, and Quality Engineering Lead conversations. Reach out on LinkedIn or through the personal profile.
Start here: five reference frameworks demonstrate deterministic feedback, diagnosable failure, and governed delivery across UI, API contracts, performance, JVM browser execution, and LLM evaluation. A separate shared product target lets those frameworks exercise the same persisted policy-and-claims domain without turning the app into a sixth framework. Open the dashboard, choose a capability below, and verify the current evidence record before reading the deeper architecture.
Live evidence status: use the deployed dashboard
as the source of truth. Check its generatedAt, schedule health, per-repository
freshness, execution result, and publication state, then follow the linked
source runs. This profile intentionally does not copy volatile status claims
from a dated dashboard snapshot. A successful publication does not turn a
failed test scope into a pass.
The frameworks · Choose by capability · Quality model · Operating model · Review paths · Field results
| Framework | What it demonstrates | Evidence |
|---|---|---|
| Verity Policy Coverage Eval Python · pytest · RAG + tool use |
LLM evaluation engineered like software: a three-tier eval pyramid (deterministic gate → semantic evals → adversarial), cassette replay, seeded defects, and judge calibration with bias controls. The hermetic tier runs without provider keys. |  Eval report · v0.1.0 · Review guide |
| Playwright TypeScript TypeScript · Playwright · Zod · Allure |
Modern browser coverage against a repo-owned target: strict typing, reusable fixtures, typed API clients, visual baselines, Axe accessibility checks, sharded Linux/Windows CI, and a governed retry budget with expiring quarantine. |  Allure report · v1.0.0 · Review guide |
| k6 Performance k6 · TypeScript · Grafana · OpenTelemetry |
Performance testing as a governed system: six workload models, SLO-based gates, reviewed regression baselines, Grafana/InfluxDB observability, and load-safety rules that keep public targets read-only. |  Performance reports · v0.4.0 · Evidence guide |
| ARIA API Java 21 · Pact · WireMock · OpenAPI |
API testing above the endpoint-script level: layered clients and services, a deterministic provider, Pact consumer contracts with provider verification, JSON-schema assertions, OpenAPI endpoint coverage, and redacted diagnostics. |  Allure report · v1.0.0 · Review guide |
| Selenium TestNG Java Java 21 · Selenium 4 · TestNG · Docker Grid |
JVM browser-execution patterns: thread-local drivers, explicit-wait-only synchronization, Docker Grid with capacity guidance, multi-browser CI, and redaction-aware Allure diagnostics. |  Allure report · v1.0.0 · Review guide |
Quality Reference Application is a separate, fictional policy-and-claims product under test. Its current main source is 0f08620; this is a documentation-only follow-up to product release e90067b, which persists synthetic policy and claim state, enforces member ownership and reviewer transitions, and exposes a versioned API plus accessible browser UI. Default-branch verification and multi-platform image publication passed; the current source-tagged image is ghcr.io/qa-test-automation-frameworks/quality-reference-application:sha-0f08620b8a1fecad41c4fe02055a03622b77aef0 (sha256:f1c6b85e71be9127fd22f602f1c2bec27a6d4041e21aba1ab2ead1581fd7f1f3). The product-code digest sha256:0de7875e9c69d3e6f8954456b6f7c90160032395f87aee4d3d64e33307b062a9 passed the merged Playwright main compatibility and rollback run: five of five shared-app UI cases passed and the previous immutable image preserved the v1 limit boundary. The A07 reviewer guide records the risk map, red control, compatible release, and rollback proof. This is a reference target, not an insurance platform or a production-scale claim system.
The shared-target adapters are merged across all five frameworks, and the reference application demonstrates versioned compatibility and rollback. For current run freshness, publication state, schedule health, and per-repository status, use the live evidence dashboard rather than dated profile claims. The A07 guide records the compatibility and rollback evidence.
| If you are evaluating | Start here | What to inspect first |
|---|---|---|
| AI/LLM quality | Verity | Calibration, replay, adversarial evidence, and explicit live limitations. |
| Modern UI, API, accessibility, and visual testing | Playwright | Controlled target, typed clients, cleanup, and flake governance. |
| Performance governance | k6 | Safety guard, SLOs, baseline provenance, and regression comparison. |
| API and contract quality | ARIA | Owned provider, Pact, OpenAPI, and redacted diagnostics. |
| JVM browser execution | Selenium | Grid lifecycle, waits, parallelism, and failure diagnostics. |
Each repository owns one surface of a layered quality model, and they are meant to be read together: Playwright proves modern browser, visual, and accessibility coverage; ARIA proves API behavior at the contract level; k6 proves performance evidence under governed load; Selenium proves the JVM execution patterns large teams still depend on; and Verity applies the same determinism and evidence standards to the newest surface — LLM evaluation. One quality strategy, demonstrated five ways.
| Time | Path | The question it answers |
|---|---|---|
| 3 minutes | Open the evidence dashboard, check its generatedAt, then inspect source-run links. |
Which runs are current, stale, failed, skipped, or published? |
| 12 minutes | Follow the independent portfolio review packet, then inspect the Verity reviewer guide and Playwright review guide. | Can a reviewer reproduce a safe success and an intentional failure, and understand the limits? |
| Deep review | Work through all five in order — Verity → Playwright → k6 → ARIA → Selenium — each with a documented, time-boxed review path. | Does the same engineering standard hold across every quality surface? |
The documentation workflow in this repository rejects a small, explicit list of unsupported self-assessment terms. A separate evidence contract validates each repository's default-branch run, SHA, verification record, and freshness. These are repository-local and portfolio-contract checks; they are not semantic validation of every organization claim. See the portfolio evidence standard.
These numbers come from one client platform migration, documented in the anonymized platform scaling case study — including the decision record behind them: stop optimizing for raw test count, keep a small set of business-critical E2E journeys, and push repeatable validations down to API level. The fixture, data-builder, and reporting conventions behind those numbers are implemented publicly in the Playwright, ARIA, and k6 repositories.
Every repository was built over a focused, multi-week stretch, working with
AI-assisted engineering tools rather than around them — the same practice
this portfolio treats as a core quality-engineering skill rather than a
shortcut. The frameworks, evidence contract, and governance patterns were
directed and reviewed by hand: architecture decisions, contract boundaries,
threshold placement, and the honesty rules (labeled limitations, quarantined
gates, NOT_REPRODUCED as a distinct status) are engineering judgment calls,
not generated defaults. Treat the commit history and CI runs, not the
calendar, as the evidence of what was actually built and verified.
Reviewing this portfolio? Open the evidence dashboard, pick a framework, inspect its latest CI run and failure evidence, then run its documented local quality gate. Everything above is designed to be challenged.
