Playwright TypeScript Framework for Conduit RealWorld

Modern browser automation as an engineered product: a repo-owned controlled target, strict TypeScript, typed API clients, Zod contracts, reusable fixtures, visual baselines, Axe accessibility checks, sharded CI, observability, and report-grade failure diagnostics.

Reviewer Proof

Evidence	Link
Live report	Allure report
Release	v1.0.0
CI
Repository activity	Default-branch commits · Pull requests
Docs and assets	Documentation · Screenshots
Best failure evidence	Failure example
Reliability experiment	Controlled faults and repeated-run sample

What This Framework Proves

Engineering question	Implemented answer
How can broad browser coverage stay deterministic?	A repo-owned Conduit target, stable seed data, explicit project boundaries, and pinned browser images.
How are API and UI layers kept coherent?	Typed API clients, Zod validation, API-driven setup, and page objects share controlled contracts.
How are failures made actionable?	Screenshots, traces, HTML and Allure reports, redacted logs, runtime metrics, and documented triage.
How is flakiness governed?	Project-specific retry rules, a retry budget, expiring quarantine metadata, and scheduled analytics.
How does CI scale?	Linux and Windows jobs, UI and visual shards, browser smoke projects, and machine-readable metrics.
How do I exercise the shared portfolio app?	See the standalone pinned Playwright pilot; Conduit projects remain independent.

Release Notes Summary

Release v1.0.0 includes cross-platform line-ending enforcement, project-specific retry policy, machine-readable runtime and retry metrics, expiring quarantine governance, OpenTelemetry support, and explicit controlled-target limitations. See CHANGELOG.md.

Quality Gates

Run the full local verification gate:

npm run verify:target

The gate runs:

• npm run format:check
• npm run check:runtime
• npm run lint
• npm run type-check
• npm run check:secrets
• npm run check:anti-patterns
• npm run check:visual-snapshots
• npm run check:openapi-contract
• npm run with:target -- npm run check:env
• npm run test:api
• npm run test:e2e
• npm run test:visual
• npm run test:accessibility
• npm run test:contracts
• npm run test:cross-browser
• scheduled npm run test:cross-browser:full

CI enforces formatting, linting, type checking, secret scanning, anti-pattern scanning, visual snapshot hygiene, OpenAPI/Zod contract alignment, environment readiness, API tests in regular and fixed-seed shuffled order, sharded authenticated and anonymous UI tests, sharded visual tests, Axe accessibility tests, selector-contract tests, cross-browser smoke tests, and five intentional fault controls plus a 20-run clean-target sample. Linux jobs run inside the pinned mcr.microsoft.com/playwright:v1.64.0-noble image. Report artifacts include Allure results, JUnit XML, JSON results, Playwright HTML output, and reliability sample summaries. The visual job runs on Windows to match the committed win32 Chromium baselines.

Controlled Target Workflow

The local target lives in test-target/ and implements the Conduit API/UI contract used by the tests. It starts with deterministic seed credentials:

npm run target:start
npm run with:target -- npm run check:env
npm run with:target -- npm run test:api
npm run verify:target

Default target values are injected only by the target wrapper and are fixture-only values:

• base URL: http://127.0.0.1:4300
• API URL: http://127.0.0.1:4300/api
• seed user email: seed.user@example.test
• seed user password: fixture value generated by scripts/target-runner.js
• seed username: seeduser

Use npm run target:seed while the target is running to reset data to the known baseline.

Test Strategy

• API tests validate authentication, article, comment, profile, and feed contracts through typed API clients.
• E2E tests cover focused user workflows through page objects and explicit navigation.
• Visual tests compare stable Chromium screenshots against committed baselines under tests/visual.
• Accessibility tests scan critical authenticated and unauthenticated pages with Axe.
• Authenticated and anonymous browser projects are separated in Playwright config so login tests do not inherit saved user state.

Architecture

flowchart TD
    Config["Zod config"] --> BaseApi["Base API client"]
    Logger["Redacted logger"] --> BaseApi
    Builders["Test data builders"] --> ApiClients["API clients"]
    BaseApi --> ApiClients
    ApiClients --> Fixtures["Playwright fixtures"]
    Pages["Page objects"] --> Fixtures
    Fixtures --> Specs["API, E2E, visual specs"]
    Specs --> Reports["HTML and Allure reports"]

Loading

Project Structure

src/api/          Typed API clients and Zod models
src/builders/     Deterministic test data builders
src/fixtures/     Playwright fixtures for auth, API clients, scenarios, observability, and page objects
src/observability/ Optional OpenTelemetry helpers and Playwright step wrappers
src/pages/        Page objects and reusable components
src/utils/        Config, logging, and custom matchers
tests/api/        API contract checks
tests/accessibility/ Axe accessibility checks
tests/contracts/  Selector contract checks for controlled UI test IDs
tests/e2e/        Browser workflows
tests/setup/      Report-visible setup and teardown projects
tests/visual/     Visual regression checks and baselines
test-target/      Repo-owned Conduit-compatible API and UI target
docs/adr/         Architecture decision records
docs/openapi/     Controlled target OpenAPI contract source

Setup

Use Node24.21.0 (pinned in .nvmrc and .node-version) and npm10.9.4. See runtime policy for the supported release range.

npx --yes npm@10.9.4 ci
npm run test:unit
npm run lint
npm run type-check

# Browser checks are a separate path:
npx playwright install chromium

For normal local verification, prefer npm run verify:target instead of hand-maintaining .env. For exploratory external runs, update .env with credentials for a test account and explicit target URLs. Do not commit .env, .auth, reports, traces, screenshots, or generated browser artifacts.

Selector Contract

The controlled UI exposes stable data-testid values for repeated or test-critical surfaces such as article cards, article body, article author, article title, article description, article date, sidebar tag lists, comments, validation errors, profile links, and profile fields. Page objects centralize these IDs in src/pages/test-ids.ts and otherwise prefer roles, labels, placeholders, and scoped locators.

Fixture boundaries (anonymous vs. authenticated API clients) are documented in Writing tests.

Observability

OpenTelemetry is opt-in for local runs:

npm run observability:up
npm run test:otel
npm run observability:down

OTEL_ENABLED=true starts per-test root spans, observed Playwright step spans, API request spans, and cleanup spans. Each test result includes an otel-trace-context.txt attachment with the trace and span identifiers. Jaeger is available at http://127.0.0.1:16686 when the observability profile is running.

Flakiness Analytics

Run npm run flake:report after a Playwright JSON result is available to write test-results/flake-report.md and test-results/portfolio-metrics-v1.json. CI enforces the retry budget with npm run flake:check on pull requests and scheduled runs. See Flakiness policy.

Reports

• Playwright HTML: playwright-report/
• Allure results: allure-results/
• JUnit XML: test-results/junit.xml
• JSON results: test-results/results.json
• Portfolio metrics: test-results/portfolio-metrics-v1.json

CI Matrix Summary

Surface	Matrix or shards	Retry policy
API	1 Linux job	0
Authenticated and anonymous Chromium	2 Linux shards	1 UI retry
Visual Chromium	2 Windows shards	0
Accessibility and selector contracts	1 Linux job each	0
Browser smoke	Firefox, WebKit, mobile Chromium	1 UI retry

Current workflow duration and sample size are published on the portfolio dashboard.

The CI workflow validates all eight required artifact groups, renders Allure with the lockfile-pinned CLI, and retains the rendered report as a CI artifact. Default-branch runs deploy that complete report through GitHub Pages. Pull requests render without deployment. source.json identifies the tested revision and run. Report generation failure leaves the previously deployed report intact; native Playwright HTML remains in each test job's artifacts.

For local rendering, install JDK 21 and set JAVA_HOME, then run npm run allure:generate after a test profile. The optional renderer uses Allure 2 for compatibility with the existing report format; adopting Allure 3 is a separate migration. Hosted long-term history retention remains a separate evidence workstream.

The manual Allure Report Deploy workflow republishes an existing rendered artifact from a successful default-branch CI run. Supply that run's numeric ID. It verifies source identity and never reruns tests or presents historical evidence as a fresh test execution.

License And Attribution

This project is licensed under the MIT License. When cloning, forking, redistributing, or deriving work from this repository, preserve the license notice and attribute the original source to Prayag Vyas (prayagv) as described in NOTICE.

Portfolio Review Path

• Latest CI workflow: GitHub Actions CI
• Published report: Allure report
• High-signal tests: tests/api, tests/e2e/article-lifecycle.spec.ts, tests/contracts/selectors/controlled-ui.selectors.spec.ts, tests/visual, and tests/accessibility
• Local proof command: npm run verify:target
• Verification evidence: Verification evidence

Public Readiness Checklist

• No untracked publishable files; every accepted tests/visual/**/*-snapshots/*.png baseline is listed in tests/visual/visual-snapshots.manifest.json.
• No ignored reports, traces, screenshots, videos, .auth, or local environment files committed.
• Verify npm run verify:target on the supported runtime before claiming a complete current run.
• GitHub Actions is green against the controlled target on the public default branch.
• CI and Allure badges point to the repository workflow and GitHub Pages report URLs.
• Confirm at least one meaningful initial commit exists before publishing.

For deterministic CI, use the repo-owned controlled target or point BASE_URL and API_URL at a controlled RealWorld-compatible deployment, not a public demo service. See Controlled Test Environment. The checked-in Docker Compose harness runs the repo-owned Conduit-compatible target on port 4300; set TEST_USER_PASSWORD explicitly before starting it.

See Commands for the full npm script reference and Writing tests for fixture and test-authoring design notes.

Known Limitations

• Some external RealWorld-compatible APIs do not expose account deletion. Those deployments use unique generated users; the repository-owned target instead uses disposable seeded credentials. Article/comment cleanup failures are reported rather than silently suppressed.
• Generated reports are intentionally ignored by Git; publish only clean CI artifacts from green runs.
• The Docker Compose harness runs the repo-owned Conduit-compatible target for deterministic local checks. External RealWorld-compatible deployments remain supported through explicit BASE_URL, API_URL, and seeded-user environment variables.
• User retention and cleanup rules are documented in Data isolation and retention.

Documentation

• Architecture
• Commands
• Configuration guide
• Debugging test failures
• Dos and don'ts
• Execution guide
• Observability
• Writing tests
• Portfolio review guide
• Engineering history
• Flakiness policy
• Flake report example
• Failure example and triage
• Offline failure evidence bundle
• Seeded defect examples
• Contributing guide
• Controlled test environment
• Data isolation and retention
• Test strategy matrix
• Verification evidence
• Controlled target OpenAPI contract
• ADR-001: Playwright over Cypress
• ADR-002: Conduit as Target
• ADR-003: Hybrid API + UI Strategy
• ADR index