Skip to content

TEST-46 docs: align guides with voice testing best practices - #1206

Open
scott-lowe-vapi wants to merge 1 commit into
mainfrom
codex/align-testing-guidance
Open

TEST-46 docs: align guides with voice testing best practices#1206
scott-lowe-vapi wants to merge 1 commit into
mainfrom
codex/align-testing-guidance

Conversation

@scott-lowe-vapi

@scott-lowe-vapi scott-lowe-vapi commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Description

  • Align Evals and Simulations guides with the testing best practices introduced in Rewrite voice agent testing best practices #1193.
  • Clarify tool, audio, and external-state boundaries; replace brittle examples with focused decision checks.
  • Replace incomplete CI snippets with a workflow that waits for completion and blocks on critical failures.
  • Add risk-based coverage guidance to tutorials, monitoring, and scorecards; correct stale Eval-suite API descriptions through Fern overrides.
  • Follow-up for TEST-46.

Testing Steps

  • Run the app locally using fern docs dev or navigate to preview deployment. Local preview stalled during startup; visual review remains pending.
  • Ensure that the changed pages and code snippets work. Static checks passed; no live calls or Eval runs were made.
  • fern check: 0 errors, 12 existing API-schema warnings.
  • MDX/YAML syntax checks for all 13 changed files, changed cURL JSON payloads, new internal links, and git diff --check.
  • CI example validated with 12 mocked cases covering passing results, failures, cancellation, missing/empty results, timeout, HTTP/JSON errors, and missing configuration.

## Description

- Align Evals and Simulations guides with the testing best practices introduced in #1193.
- Clarify tool, audio, and external-state boundaries; replace brittle examples with focused decision checks.
- Replace incomplete CI snippets with a workflow that waits for completion and blocks on critical failures.
- Add risk-based coverage guidance to tutorials, monitoring, and scorecards; correct stale Eval-suite API descriptions through Fern overrides.
- Follow-up for [TEST-46](https://linear.app/vapi/issue/TEST-46/update-docs-for-testing-best-practices).

## Testing Steps

- [ ] Run the app locally using `fern docs dev` or navigate to preview deployment. Local preview stalled during startup; visual review remains pending.
- [ ] Ensure that the changed pages and code snippets work. Static checks passed; no live calls or Eval runs were made.
- [x] `fern check`: 0 errors, 12 existing API-schema warnings.
- [x] MDX/YAML syntax checks for all 13 changed files, changed cURL JSON payloads, new internal links, and `git diff --check`.
- [x] CI example validated with 12 mocked cases covering passing results, failures, cancellation, missing/empty results, timeout, HTTP/JSON errors, and missing configuration.

Copy link
Copy Markdown
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@scott-lowe-vapi
scott-lowe-vapi marked this pull request as ready for review September 9, 2026 22:33
@lightsage-app

lightsage-app Bot commented Sep 9, 2026

Copy link
Copy Markdown

Lightsage docs evals

Result: passed
Staging docs: https://vapi-preview-01a08872-d1b4-7740-bdae-406995015707.docs.buildwithfern.com
Commit: 07cf989

Average score: 100/100
Passed: 3/3

Eval ID Status Score Model Tools Docs 404
08e24c18-82a0-45de-abdc-d237bd12bc0f Pass 100 cursor/auto 12 0
08e24c18-82a0-45de-abdc-d237bd12bc0f Pass 100 codex/gpt-5.4 30 0
08e24c18-82a0-45de-abdc-d237bd12bc0f Pass 100 claude-code/global.anthropic.claude... 11 0

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant