feat(tests): add step timing tracking to E2E tests#48
Open
Conversation
Add timing instrumentation to E2E test framework:
**Test Framework Changes:**
- Add e2e_step_start() to record step start time
- Add e2e_step_end() to record step end time and calculate duration
- Update e2e_report_steps() to display duration for each step
- Remove e2e_step_pass/fail() - replaced by e2e_step_start/end pattern
**React Native iOS E2E Updates:**
- Instrument build-node, install-pods, build-ios, deploy, verify-app steps
- Source test-framework.sh in each instrumented step
- Use trap to record failures with timing
**Output Format:**
- Pass: ✓ PASS: step-name (42s)
- Fail: ✗ FAIL: step-name (15s)
Reason: error message
**Benefits:**
- CI logs now show exactly how long each step takes
- Easier to identify performance bottlenecks
- Better debugging when tests fail
- Foundation for performance regression detection
Example output:
✓ PASS: build-node (23s)
✓ PASS: install-pods (45s)
✓ PASS: build-ios (312s)
✓ PASS: deploy (8s)
✓ PASS: verify-app (14s)
When steps like install-pods and build-ios use 'cd ios', the relative 'reports/steps' path writes to the wrong location (ios/reports/steps). Fixed by: - Make e2e_step_start/end use absolute paths based on REPO_ROOT - Remove unnecessary _step_dir variables from YAML - Status files now written to correct location even after cd This fixes the 'Step never executed' failures for install-pods and build-ios.
The warning 'IOS_DEVELOPER_DIR not set' was showing even when auto-detection succeeded (by setting DEVELOPER_DIR). Changed to only warn when BOTH variables are unset, meaning auto-detection actually failed. Before:⚠️ [WARN] IOS_DEVELOPER_DIR not set (always shown) ✅ [OK] iOS environment ready After (auto-detection succeeds): ✅ [OK] iOS environment ready (no warning) After (auto-detection fails):⚠️ [WARN] Could not detect Xcode developer directory Set IOS_DEVELOPER_DIR or run: xcode-select -p
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds timing instrumentation to the E2E test framework so we can see exactly how long each step takes in CI logs.
Problem
Currently, E2E tests show pass/fail but no timing information. When a test takes 37 minutes (React Native iOS), we don't know where the time is being spent without manual analysis of logs.
Solution
Test Framework Changes
New Functions:
e2e_step_start(step_name)- Records start timestampe2e_step_end(step_name, status, [reason])- Records end timestamp, calculates durationEnhanced Reporting:
e2e_report_steps()now displays duration for each step:Removed Functions:
e2e_step_pass()/e2e_step_fail()- Replaced by start/end pattern for timingReact Native iOS E2E Instrumented Steps
Updated 5 critical steps in
test-suite-ios-e2e.yaml:build-node- npm install timinginstall-pods- CocoaPods install timingbuild-ios- xcodebuild timing (longest step)deploy- App deployment timingverify-app- Liveness check + soak period timingUsage Pattern
Benefits
Example CI Output
Before:
After:
Next Steps
Testing
Will validate in CI when this PR runs. Timing should appear in test output logs.
🤖 Generated with Claude Code