Describe the bug
When a brpc server is started with SOCKET_MODE_RDMA, the server-side RDMA socket's _read_buf can be concurrently accessed by two independent bthreads, causing IOBuf internal state corruption and intermittent core dumps.
There are three race scenarios, all stemming from the same root cause: PollCq(CQ socket bthread) and OnNewMessages(main socket bthread) can run concurrently and both operate on the same Socket::_read_buf, which is not thread-safe.
Scenario 1: After RDMA handshake is established
RdmaTransport::Init sets the server-side edge trigger to InputMessenger::OnNewMessages (to drive the handshake via the standard InputMessenger path). After the handshake reaches ESTABLISHED, the edge trigger is never switched. If the TCP fd becomes readable (e.g., TCP keepalive, peer close), OnNewMessages is called and reads from the TCP fd into _read_buf. At the same time, PollCq processes RDMA completions and writes RDMA data into _read_buf via HandleCompletion. Both bthreads modify _read_buf concurrently → IOBuf corruption → core dump in IOBuf::cutn or IOBuf::clear.
Bthread A: PollCq (CQ socket) Bthread B: OnNewMessages (main socket)
┌─────────────────────────────┐ ┌──────────────────────────────────┐
│ ibv_poll_cq → got completion │ │ TCP fd readable (keepalive etc) │
│ HandleCompletion: │ │ DoRead: │
│ _rbuf.cutn( │ RACE! │ _read_buf.append_from_fd(...) │
│ &_socket->_read_buf) ★──┼──────────┼──> ★ writes _read_buf │
│ ★ writes _read_buf │ │ ProcessNewMessage: │
│ │ │ cutn → reads _read_buf ★ │
│ │ │ │
│ IOBuf internal state corrupted → core dump in cutn() or clear() │
└─────────────────────────────┘ └──────────────────────────────────┘
Scenario 2: During RDMA handshake (after QP is brought up)
In ExecuteServerHandshake Phase 1, BringUpQpputs the QP into RTS state. The client may complete its handshake and start sending RDMA data before the server's handshake finishes. PollCq receives the RDMA completion and HandleCompletion writes to _read_buf. Meanwhile, the server's OnNewMessages is still driving the handshake (Phase 2: draining the 4-byte ACK) and accessing _read_buf via source->cutn(). Concurrent access → core dump in IOBuf::cutn.
Client Server
───── ──────
│ │
│── RDMA hello ──TCP──────────────→ │ Phase 1: parse hello, BringUpQp (QP→RTS)
│←── server hello ──TCP──────────── │ SendLocalHello, state=S_ACK_WAIT
│ │
│── ACK (4B) ──TCP────────────────→ │ Phase 2: OnNewMessages reads ACK...
│ │
│══ RDMA data ════════════════════→ │ PollCq: HandleCompletion writes _read_buf ★
│ │ OnNewMessages: source->cutn(&flags, 4) ★
│ │ → CONCURRENT _read_buf access → CRASH
Scenario 3: Handshake ACK rejection
ExecuteServerHandshake Phase 2 checks source->size() > HELLO_ACK_LEN and rejects the connection. When a client falls back to TCP, it sends the 4-byte ACK followed immediately by the first baidu_std RPC request. If both arrive before the server's readv(), they are read into _read_buf in one call. The server sees source->size() > 4 and drops the connection. This causes mass handshake failures for all TCP-fallback clients.
Client Server
───── ──────
│── ACK (4B) ──┐ │
│── RPC (NB) ──┤ │
│ ↓ (same readv) │ source->size() = 4 + N > 4
│══════════════════════════════→ │ "Too many bytes in handshake ACK"
│ │ → PARSE_ERROR_ABSOLUTELY_WRONG → drop!
│ │ (should drain 4B ACK, let N bytes
│ │ be parsed by baidu_std parser)
Scenario 4: PollCq calling ProcessNewMessage with bytes == 0 during handshake
PollCq unconditionally calls ProcessNewMessage after processing completions, even when bytes == 0 (e.g., IBV_WC_SEND completions, or IBV_WC_RECV dropped during handshake). ProcessNewMessage calls CutInputMessage, which invokes the handshake parser (ExecuteServerHandshake) and calls cutn() on _read_buf. Meanwhile, OnNewMessages is also calling ProcessNewMessage → CutInputMessage → ExecuteServerHandshake → cutn() on the same _read_buf. Both bthreads call cutn() on the same IOBuf concurrently → corruption → core dump.
Bthread A: PollCq (CQ socket) Bthread B: OnNewMessages (main socket)
┌─────────────────────────────┐ ┌──────────────────────────────────┐
│ ibv_poll_cq → IBV_WC_SEND │ │ DoRead → reads ACK to _read_buf │
│ HandleCompletion → return 0 │ │ ProcessNewMessage: │
│ bytes = 0 │ │ CutInputMessage: │
│ ProcessNewMessage(bytes=0): │ RACE! │ ExecuteServerHandshake: │
│ CutInputMessage: │ │ source->cutn(&flags, 4) ★ │
│ ExecuteServerHandshake: │ │ │
│ source->cutn(&flags,4)★┼───────────┼──> │
│ │ │ │
│ Both bthreads cutn() the same _read_buf → IOBuf corruption → CRASH │
└─────────────────────────────┘ └──────────────────────────────────┘
Stack trace from production:
#0 butil::IOBuf::cutn
#1 brpc::rdma::RdmaEndpoint::ExecuteServerHandshake
#2 brpc::rdma::ExecuteServerHandshake
#3 brpc::policy::ParseRdmaHandshake
#4 brpc::InputMessenger::CutInputMessage
#5 brpc::InputMessenger::ProcessNewMessage
#6 brpc::rdma::RdmaEndpoint::PollCq ← PollCq, not OnNewMessages
#7 brpc::Transport::OnEdge
#8 bthread::TaskGroup::task_runner
To Reproduce
- Start a brpc server with ServerOptions.socket_mode =
SOCKET_MODE_RDMA
- Have RDMA clients connect; some will negotiate RDMA successfully, others may fall back to TCP
- After RDMA is established, if the TCP fd becomes readable (keepalive, peer close, etc.), the race triggers
- Core dump occurs intermittently (~20% probability at startup, sporadically during runtime)
- Stack trace shows crash in
IOBuf::clear() or IOBuf::cutn() during Socket::BeforeRecycled(), ExecuteServerHandshake, or PollCq
Expected behavior
No core dump. _read_buf should not be accessed concurrently by PollCq and OnNewMessages. Handshake should not fail when ACK and RPC data arrive in the same readv() call.
Versions
OS: Rocky 9.2
Compiler: GCC 13.3
brpc: master 2026.08.23
protobuf: -
Additional context/screenshots
The client-side RDMA socket does not have this issue because its edge trigger is RdmaEndpoint::OnNewDataFromTcp, which checks the RDMA state and only reads 1 byte for EOF detection in ESTABLISHED state (without touching _read_buf).
Describe the bug
When a brpc server is started with
SOCKET_MODE_RDMA, the server-side RDMA socket's _read_buf can be concurrently accessed by two independent bthreads, causing IOBuf internal state corruption and intermittent core dumps.There are three race scenarios, all stemming from the same root cause:
PollCq(CQ socket bthread) andOnNewMessages(main socket bthread) can run concurrently and both operate on the same Socket::_read_buf, which is not thread-safe.Scenario 1: After RDMA handshake is established
RdmaTransport::Init sets the server-side edge trigger to
InputMessenger::OnNewMessages(to drive the handshake via the standard InputMessenger path). After the handshake reaches ESTABLISHED, the edge trigger is never switched. If the TCP fd becomes readable (e.g., TCP keepalive, peer close), OnNewMessages is called and reads from the TCP fd into _read_buf. At the same time, PollCq processes RDMA completions and writes RDMA data into _read_buf via HandleCompletion. Both bthreads modify _read_buf concurrently → IOBuf corruption → core dump in IOBuf::cutn or IOBuf::clear.Scenario 2: During RDMA handshake (after QP is brought up)
In ExecuteServerHandshake Phase 1,
BringUpQpputs the QP into RTS state. The client may complete its handshake and start sending RDMA data before the server's handshake finishes. PollCq receives the RDMA completion and HandleCompletion writes to _read_buf. Meanwhile, the server's OnNewMessages is still driving the handshake (Phase 2: draining the 4-byte ACK) and accessing _read_buf via source->cutn(). Concurrent access → core dump in IOBuf::cutn.Scenario 3: Handshake ACK rejection
ExecuteServerHandshake Phase 2 checks source->size() > HELLO_ACK_LEN and rejects the connection. When a client falls back to TCP, it sends the 4-byte ACK followed immediately by the first baidu_std RPC request. If both arrive before the server's readv(), they are read into _read_buf in one call. The server sees source->size() > 4 and drops the connection. This causes mass handshake failures for all TCP-fallback clients.
Scenario 4: PollCq calling ProcessNewMessage with bytes == 0 during handshake
PollCq unconditionally calls ProcessNewMessage after processing completions, even when bytes == 0 (e.g., IBV_WC_SEND completions, or IBV_WC_RECV dropped during handshake). ProcessNewMessage calls CutInputMessage, which invokes the handshake parser (ExecuteServerHandshake) and calls cutn() on _read_buf. Meanwhile, OnNewMessages is also calling ProcessNewMessage → CutInputMessage → ExecuteServerHandshake → cutn() on the same _read_buf. Both bthreads call cutn() on the same IOBuf concurrently → corruption → core dump.
Stack trace from production:
To Reproduce
SOCKET_MODE_RDMAIOBuf::clear()orIOBuf::cutn()duringSocket::BeforeRecycled(),ExecuteServerHandshake, orPollCqExpected behavior
No core dump. _read_buf should not be accessed concurrently by PollCq and OnNewMessages. Handshake should not fail when ACK and RPC data arrive in the same readv() call.
Versions
OS: Rocky 9.2
Compiler: GCC 13.3
brpc: master 2026.08.23
protobuf: -
Additional context/screenshots
The client-side RDMA socket does not have this issue because its edge trigger is RdmaEndpoint::OnNewDataFromTcp, which checks the RDMA state and only reads 1 byte for EOF detection in ESTABLISHED state (without touching _read_buf).