feat: add grpc socket and flattn tx batches to allow for lower allocations#3297
feat: add grpc socket and flattn tx batches to allow for lower allocations#3297tac0turtle wants to merge 3 commits intomainfrom
Conversation
|
Warning Rate limit exceeded
To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📝 WalkthroughWalkthroughThis PR extends the gRPC execution service with Unix domain socket support and introduces transaction batch encoding to optimize message transmission. Changes span protobuf definitions, client/server implementations, socket handling utilities, and comprehensive test coverage. Changes
Sequence Diagram(s)sequenceDiagram
participant Client as gRPC Client
participant HTTP2 as HTTP/2 h2c Client
participant UnixSocket as Unix Socket
participant Server as gRPC Server
Client->>Client: Detect unix:// URL
Client->>HTTP2: Create h2c HTTP/2 client<br/>with unix socket dialer
Client->>UnixSocket: Dial unix socket path
UnixSocket->>Server: Establish connection
Server-->>UnixSocket: Accept connection
UnixSocket-->>HTTP2: Ready
HTTP2-->>Client: Client ready
Client->>Server: Send gRPC request<br/>(over HTTP/2)
Server->>Server: Handle request
Server-->>Client: Return gRPC response
sequenceDiagram
participant Caller as Caller
participant Client as gRPC Client
participant Encoder as TxBatch Encoder
participant Server as gRPC Server
participant Decoder as TxBatch Decoder
participant Executor as Executor
Caller->>Client: Call ExecuteTxs(txs [][]byte)
Client->>Encoder: encodeTxBatch(txs)
Encoder->>Encoder: Concatenate tx data<br/>Track per-tx sizes
Encoder-->>Client: TxBatch{data, tx_sizes}
Client->>Server: ExecuteTxsRequest{TxBatch}
Server->>Decoder: decodeTxBatchOrTxs(TxBatch)
Decoder->>Decoder: Validate sizes & data<br/>Reconstruct txs
Decoder-->>Server: [][]byte txs
Server->>Executor: Execute transactions
Executor-->>Server: Result
Server-->>Client: Response
Client-->>Caller: Result
Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Claude encountered an error —— View job I'll analyze this and get back to you. |
|
The latest Buf updates on your PR. Results from workflow CI / buf-check (pull_request).
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #3297 +/- ##
==========================================
+ Coverage 62.48% 62.68% +0.20%
==========================================
Files 122 124 +2
Lines 13047 13163 +116
==========================================
+ Hits 8152 8251 +99
- Misses 4009 4013 +4
- Partials 886 899 +13
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (3)
proto/evnode/v1/execution.proto (1)
63-67: Clarify precedence when bothtxsandtx_batchare populated.Please document a single authoritative rule (e.g., "
tx_batchtakes precedence;txsis legacy fallback") to avoid cross-client ambiguity during migration.Also applies to: 84-85, 142-143
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@proto/evnode/v1/execution.proto` around lines 63 - 67, The proto comments for the repeated bytes field txs and the TxBatch field tx_batch are ambiguous about which to use when both are populated; update the comments for txs and tx_batch (including the other occurrences at the ranges noted) to state a single authoritative rule: tx_batch takes precedence and txs is a legacy fallback (i.e., when tx_batch is present, implementations MUST ignore txs; when tx_batch is absent, clients MAY use txs for backward compatibility). Ensure the same wording is applied to every message where txs and tx_batch appear to avoid cross-client ambiguity.execution/grpc/unix.go (1)
33-47: Consider documenting thathttp.Serveblocks until error.The function works correctly, but callers should understand this blocks. The defer order is correct (LIFO): socket cleanup runs after listener close.
Minor note: ignoring errors from
listener.Close()andremoveStaleUnixSocketin defers is acceptable for cleanup, but logging could help debugging shutdown issues.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@execution/grpc/unix.go` around lines 33 - 47, Update the ListenAndServeUnix doc comment to explicitly state that http.Serve blocks until it returns an error and that the deferred listener.Close() and removeStaleUnixSocket() run when Serve returns (LIFO order); mention callers must expect blocking behavior and handle cancellation accordingly. While optional, add a brief note that errors from listener.Close() and removeStaleUnixSocket() are currently ignored in the defers and could be logged for shutdown diagnostics; reference ListenAndServeUnix, http.Serve, listener.Close, removeStaleUnixSocket, and NewExecutorServiceHandler when making this documentation change.execution/grpc/client.go (1)
50-64: Consider validating socket path at creation time.The empty socket path check inside
DialTLSContext(line 56-58) will only trigger on the first dial attempt rather than at client creation. SinceclientTransportForTargetalready extracts the path, validation could happen there to fail fast.That said, this is defensive coding within the closure, and the path extraction in
unixSocketPath(line 74-79) already ensures a non-empty path when the prefix is present. The check here guards against misuse ifnewUnixHTTP2Client("")is called directly.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@execution/grpc/client.go` around lines 50 - 64, The client builder newUnixHTTP2Client currently defers an empty-socket check until DialTLSContext; change it to validate socketPath up-front and return an error instead of creating a client for an empty path. Update newUnixHTTP2Client signature to return (*http.Client, error), check if socketPath == "" at the start and return a clear error, and then construct the http.Client as before. Propagate the change to callers such as clientTransportForTarget (which calls newUnixHTTP2Client) so they handle the error and fail fast when unixSocketPath extraction yields an empty string.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@apps/grpc/README.md`:
- Line 3: Update the README intro sentence to use the correct indefinite
article: replace the phrase "a Evolve node" with "an Evolve node" in the README
content so the sentence reads "This application runs an Evolve node with a
single sequencer..." (look for the README sentence containing "a Evolve node" to
make the change).
In `@execution/grpc/server.go`:
- Around line 85-87: GetTxsResponse currently only sets TxBatch and drops the
legacy txs field; update the server handler that returns GetTxsResponse (the
GetTxs implementation in grpc/server.go) to also populate the legacy txs field
with an equivalent flattened slice of transactions derived from txBatch (or a
converted form matching the old txs element type) so older clients continue to
see the expected list; ensure both fields are kept in sync (construct txs from
txBatch before returning) and preserve existing types/names used in
GetTxsResponse, tx_batch, and txs.
---
Nitpick comments:
In `@execution/grpc/client.go`:
- Around line 50-64: The client builder newUnixHTTP2Client currently defers an
empty-socket check until DialTLSContext; change it to validate socketPath
up-front and return an error instead of creating a client for an empty path.
Update newUnixHTTP2Client signature to return (*http.Client, error), check if
socketPath == "" at the start and return a clear error, and then construct the
http.Client as before. Propagate the change to callers such as
clientTransportForTarget (which calls newUnixHTTP2Client) so they handle the
error and fail fast when unixSocketPath extraction yields an empty string.
In `@execution/grpc/unix.go`:
- Around line 33-47: Update the ListenAndServeUnix doc comment to explicitly
state that http.Serve blocks until it returns an error and that the deferred
listener.Close() and removeStaleUnixSocket() run when Serve returns (LIFO
order); mention callers must expect blocking behavior and handle cancellation
accordingly. While optional, add a brief note that errors from listener.Close()
and removeStaleUnixSocket() are currently ignored in the defers and could be
logged for shutdown diagnostics; reference ListenAndServeUnix, http.Serve,
listener.Close, removeStaleUnixSocket, and NewExecutorServiceHandler when making
this documentation change.
In `@proto/evnode/v1/execution.proto`:
- Around line 63-67: The proto comments for the repeated bytes field txs and the
TxBatch field tx_batch are ambiguous about which to use when both are populated;
update the comments for txs and tx_batch (including the other occurrences at the
ranges noted) to state a single authoritative rule: tx_batch takes precedence
and txs is a legacy fallback (i.e., when tx_batch is present, implementations
MUST ignore txs; when tx_batch is absent, clients MAY use txs for backward
compatibility). Ensure the same wording is applied to every message where txs
and tx_batch appear to avoid cross-client ambiguity.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: acfb26c3-ba2a-423a-832a-163fd34eb7d3
⛔ Files ignored due to path filters (1)
types/pb/evnode/v1/execution.pb.gois excluded by!**/*.pb.go
📒 Files selected for processing (13)
apps/grpc/README.mdapps/grpc/cmd/run.goexecution/grpc/README.mdexecution/grpc/client.goexecution/grpc/client_test.goexecution/grpc/go.modexecution/grpc/server.goexecution/grpc/server_test.goexecution/grpc/tx_batch.goexecution/grpc/tx_batch_test.goexecution/grpc/unix.goexecution/grpc/unix_test.goproto/evnode/v1/execution.proto
| # gRPC Single Sequencer App | ||
|
|
||
| This application runs a Evolve node with a single sequencer that connects to a remote execution client via gRPC. It allows you to use any execution layer that implements the Evolve execution gRPC interface. | ||
| This application runs a Evolve node with a single sequencer that connects to an execution client via gRPC. It allows you to use any execution layer that implements the Evolve execution gRPC interface. |
There was a problem hiding this comment.
Fix article in intro sentence.
Use “an Evolve node” instead of “a Evolve node” for grammatical correctness.
✏️ Proposed doc fix
-This application runs a Evolve node with a single sequencer that connects to an execution client via gRPC. It allows you to use any execution layer that implements the Evolve execution gRPC interface.
+This application runs an Evolve node with a single sequencer that connects to an execution client via gRPC. It allows you to use any execution layer that implements the Evolve execution gRPC interface.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| This application runs a Evolve node with a single sequencer that connects to an execution client via gRPC. It allows you to use any execution layer that implements the Evolve execution gRPC interface. | |
| This application runs an Evolve node with a single sequencer that connects to an execution client via gRPC. It allows you to use any execution layer that implements the Evolve execution gRPC interface. |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@apps/grpc/README.md` at line 3, Update the README intro sentence to use the
correct indefinite article: replace the phrase "a Evolve node" with "an Evolve
node" in the README content so the sentence reads "This application runs an
Evolve node with a single sequencer..." (look for the README sentence containing
"a Evolve node" to make the change).
| return connect.NewResponse(&pb.GetTxsResponse{ | ||
| Txs: txs, | ||
| TxBatch: txBatch, | ||
| }), nil |
There was a problem hiding this comment.
GetTxs currently drops legacy txs payload and can break older consumers.
GetTxsResponse still defines txs, but this response now only sets tx_batch. Any client not yet upgraded to decode tx_batch will observe an empty tx list.
🔁 Compatibility-safe response shape
return connect.NewResponse(&pb.GetTxsResponse{
+ Txs: txs, // legacy compatibility during migration
TxBatch: txBatch,
}), nil🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@execution/grpc/server.go` around lines 85 - 87, GetTxsResponse currently only
sets TxBatch and drops the legacy txs field; update the server handler that
returns GetTxsResponse (the GetTxs implementation in grpc/server.go) to also
populate the legacy txs field with an equivalent flattened slice of transactions
derived from txBatch (or a converted form matching the old txs element type) so
older clients continue to see the expected list; ensure both fields are kept in
sync (construct txs from txBatch before returning) and preserve existing
types/names used in GetTxsResponse, tx_batch, and txs.
julienrbrt
left a comment
julienrbrt
left a comment
There was a problem hiding this comment.
ACK.
We should rename the grpc service specifically (the service and the file imho), as it is only used there and because the execution/grpc is an adaption of the execution interface.
We could even move the proto definition under execution/grpc but that's more a nit that is annoying to do.
| option go_package = "github.com/evstack/ev-node/types/pb/evnode/v1"; | ||
|
|
||
| // ExecutorService defines the execution layer interface for EVNode | ||
| service ExecutorService { |
There was a problem hiding this comment.
| service ExecutionGRPCService { |
| message FilterTxsRequest { | ||
| // All transactions (force-included + mempool) | ||
| // All transactions (force-included + mempool). | ||
| repeated bytes txs = 1; |
There was a problem hiding this comment.
Let's kill the ones we don't use.
| ### Changes | ||
|
|
||
| - Optimization of mutex usage in cache for reaper [#3286](https://github.com/evstack/ev-node/pull/3286) | ||
| - Add Unix domain socket support for gRPC execution endpoints via `unix:///path/to/socket`. |
2 participants
Overview
Flatten tx batch to avoid allocations and add in a unix socket over grpc to avoid tcp overhead
Summary by CodeRabbit
New Features
Documentation
Tests