Skip to content

fix(vfsevents): fix CI-observed flake in s3events TestEnhancedMetadata - #342

Closed
funkyshu wants to merge 1 commit into
mainfrom
fix/s3events-testenhancedmetadata-race
Closed

fix(vfsevents): fix CI-observed flake in s3events TestEnhancedMetadata#342
funkyshu wants to merge 1 commit into
mainfrom
fix/s3events-testenhancedmetadata-race

Conversation

@funkyshu

Copy link
Copy Markdown
Member

Summary

Fixes a CI failure observed on ubuntu + go1.26 after #339 merged:

--- FAIL: TestS3WatcherTestSuite/TestEnhancedMetadata
    SqsClient.go:23: FAIL: DeleteMessage(string,string)
    SqsClient.go:23: FAIL: 1 out of 2 expectation(s) were met.
        The code you are testing needs to make 1 more call(s).

Root cause

TestEnhancedMetadata only synchronized on the event handler firing (via an eventReceived channel), not on pollOnce's own completion. This meant the subtest could reach its assertions and finish (triggering mock-expectation teardown) before the background pollOnce goroutine made its DeleteMessage call — a timing gap that's tight enough to usually pass locally but wide enough to occasionally lose the race on CI runners.

This is exactly the residual race risk flagged (but intentionally not fixed at the time, since it wasn't the primary target) during review of #339's fix for the sibling TestNonVersionedBucketMetadata test.

Fix

  • TestEnhancedMetadata now uses the same completion-based synchronization as TestNonVersionedBucketMetadata — waits for pollOnce to fully return via an error channel, rather than only waiting for the handler to fire.
  • Extracted the shared wait/cancel/grace-period logic (previously only in TestNonVersionedBucketMetadata) into a waitForPoll suite helper, now used by both tests, mirroring the equivalent waitForReceive helper added to gcsevents_test.go in fix(vfsevents): remove sleep-based races in vfsevents watcher tests #341.
  • Uses time.NewTimer + defer timer.Stop() throughout (rather than time.After), proactively addressing the un-stopped-timer issue raised in review of fix(vfsevents): remove sleep-based races in vfsevents watcher tests #341.
  • Updated contrib/vfsevents/CHANGELOG.md.

Test plan

  • go test -race -run TestS3WatcherTestSuite -count=300 ./contrib/vfsevents/watchers/s3events/... — clean.
  • go test -run TestS3WatcherTestSuite -count=1000 ./contrib/vfsevents/watchers/s3events/... (no -race, to stress scheduling/timing rather than just data races) — clean.
  • go build ./... && go vet ./... for contrib/vfsevents — clean.
  • golangci-lint run ./watchers/s3events/... — only a pre-existing, unrelated nolintlint finding in s3events.go (not touched by this PR).

Made with Cursor

TestEnhancedMetadata only synchronized on the event handler firing
(via an eventReceived channel), not on pollOnce's own completion. On
CI (ubuntu, go1.26), the subtest could finish and its mock-expectation
teardown could run before the background pollOnce goroutine made its
DeleteMessage call, producing:

  FAIL: DeleteMessage(string,string)
  FAIL: 1 out of 2 expectation(s) were met.

This is exactly the residual race risk flagged (but not fixed at the
time, since it wasn't the primary target) during review of #339's fix
for TestNonVersionedBucketMetadata.

Fixed by switching TestEnhancedMetadata to the same completion-based
synchronization (wait for pollOnce to fully return via an error
channel) already used by TestNonVersionedBucketMetadata, extracting
the shared wait/cancel/grace-period logic into a waitForPoll suite
helper (mirroring the equivalent gcsevents helper from #341). Uses
time.NewTimer + defer Stop() to proactively avoid the un-stopped
time.After timer issue raised in review of #341.

Verified:
- go test -race -run TestS3WatcherTestSuite -count=300 ./watchers/s3events/...
- go test -run TestS3WatcherTestSuite -count=1000 ./watchers/s3events/... (no -race, to
  stress test scheduling/timing rather than just data races)
both pass cleanly with 0 failures.
@c2fo-cibot c2fo-cibot Bot added the size/M Denotes a PR that changes 30-99 lines label Jul 16, 2026
@funkyshu

Copy link
Copy Markdown
Member Author

Folding this into #341 per discussion — same underlying pattern, and consolidating avoids two overlapping open PRs against the same test files.

@funkyshu funkyshu closed this Jul 16, 2026
@funkyshu
funkyshu deleted the fix/s3events-testenhancedmetadata-race branch July 16, 2026 08:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/M Denotes a PR that changes 30-99 lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant