fix(self-driving): don't report benign user aborts as exceptions#795
Draft
posthog[bot] wants to merge 1 commit into
Draft
fix(self-driving): don't report benign user aborts as exceptions#795posthog[bot] wants to merge 1 commit into
posthog[bot] wants to merge 1 commit into
Conversation
Declining the GitHub App connection during self-driving setup emitted `[ABORT] github connection declined`, which the linear runner wraps in a WizardError and routes through `wizardAbort`. Because `wizardAbort` called `analytics.captureException` unconditionally, an expected user-driven cancellation was reported to error tracking even though the wizard already renders a friendly "GitHub connection required" outro and leaves nothing half-configured. Add an `expected` flag to `AbortCase` and `wizardAbort`. Expected aborts still render their outro and fire the `agent aborted` analytics event, but skip `captureException` (and shut analytics down as `cancelled`, not `error`). Mark the benign self-driving cases — github connection declined, requires-interactive-mode, requirements-incomplete — as expected. The genuine "self-driving is not available" (Signals unreachable) case stays unexpected and is still reported. Generated-By: PostHog Code Task-Id: ca4902cd-8c3f-44a2-a228-cb21617ed3c8
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
This affects the wizard team, not users. When someone declines the PostHog GitHub App connection during self-driving setup (STEP 3), the skill emits
[ABORT] github connection declined. The linear runner wraps that in aWizardErrorand routes it throughwizardAbort, which calledanalytics.captureExceptionunconditionally — so an expected, user-driven cancellation was filed as an exception in error tracking. The wizard already renders a friendly "GitHub connection required" outro and leaves nothing half-configured, so this was pure error-tracking noise (dozens of occurrences over the last 30 days, risking misdirected triage).Changes
expectedflag toAbortCaseand towizardAbort's options. When an abort isexpected, the outro still renders and theagent abortedanalytics event still fires, butcaptureExceptionis skipped and analytics shut down ascancelledrather thanerror.expectedflag intowizardAbort.expected:github connection declined,requires-interactive-mode, andrequirements-incomplete. The genuineself-driving is not availablecase (Signals unreachable in open beta) stays unexpected and is still reported.Test plan
wizardAbortskipscaptureExceptionfor expected aborts (and still captures unexpected ones); self-driving detect asserts which abort cases are flaggedexpected.pnpm build && pnpm test(1160 tests pass) andpnpm lint(0 errors).LLM context
Authored by PostHog Code (Claude) from an inbox report about error-tracking noise from benign GitHub-connection declines.
Created with PostHog Code from an inbox report.