Skip to content

add async hydration performance fixes#22

Open
elithrar wants to merge 5 commits into
mainfrom
opencode/async-hydration-performance
Open

add async hydration performance fixes#22
elithrar wants to merge 5 commits into
mainfrom
opencode/async-hydration-performance

Conversation

@elithrar

@elithrar elithrar commented Jun 13, 2026

Copy link
Copy Markdown
Collaborator

This improves async hydration and prefetch performance while preserving safe clone/auth behavior.

Async repo setup and directory prefetch could do avoidable git, snapshot, and hydration-queue work. Cached reads also reopened hydrated blobs repeatedly, but naive file-handle caching risks stale reads when overlays or snapshot generations change.

  • Add async repo prepare/mount flow with gated FUSE readiness and prepared-gitdir safeguards.
  • Improve clone and fetch setup: skip redundant fresh async branch fetches, normalize equivalent branch refs, validate ambient credentials before clone, and use --no-tags for clone/fetch/refresh.
  • Improve async hydration: dedupe queued prefetches, upgrade queued priority, re-check cache before fetch, wake workers for queued backlog, and batch snapshot size backfills.
  • Improve tree indexing: stream ls-tree and cat-file --batch-check output, preserving GIT_NO_LAZY_FETCH=1 for size resolution.
  • Batch directory prefetch enqueue by carrying base object and size metadata through ReaddirTyped instead of doing per-entry snapshot lookups.
  • Cap speculative prefetch for very large directories at 256 prioritized file tasks.
  • Cache hydrated base-file reads per FUSE file handle, with generation checks and pre/post mutation invalidation for writes, truncate, mtime updates, unlink, and rename.
  • Add regression coverage for prefetch batching/capping, cached base reads, generation changes, and overlapping write invalidation.
Code path Benchmark / signal Before After Gain Notes
Hydrator queued prefetch worker wake/ramp BenchmarkQueuedPrefetchWorkerRamp median 42.35 ms/op 11.84 ms/op 72.0% faster Workers now wake for queued backlog instead of waiting on slower ramp behavior.
Hydrator cached queued prefetch BenchmarkQueuedCachedPrefetch median 82.42 ms/op 62.81 ms/op 23.8% faster Cache re-checks avoid redundant fetches; BlobToCache calls dropped from 512/op to 0/op.
Git tree indexing synthetic BenchmarkBuildTreeIndex median 99.18 ms/op 84.17 ms/op 15.1% faster Streams ls-tree and cat-file --batch-check; memory dropped from 6.05 MB/op to 4.42 MB/op.
Hydrator duplicate queued prefetch BenchmarkQueuedDuplicatePrefetch median 50.37 ms/op 48.95 ms/op 2.8% faster Queue dedupe and priority upgrade keep duplicate prefetch work bounded.
Hydrator explicit duplicate reads BenchmarkAsyncHydrationDuplicateReads median 47.25 ms/op 47.74 ms/op 1.0% slower Effectively flat; left as a future optimization target rather than adding riskier waiter refactors.
Latest branch spot-check targeted hydrator benchmarks n/a duplicate 35.23 ms/op, cached 48.70 ms/op, ramp 11.84 ms/op n/a Re-run after follow-up prefetch/read changes and merge from main.
  • Validation: go build ./cmd/artifact-fs; go vet ./...; go test ./...; go test -race ./internal/hydrator; go test -race ./internal/fusefs; git diff --check.

@elithrar elithrar self-assigned this Jun 13, 2026
@elithrar elithrar marked this pull request as ready for review June 16, 2026 19:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant