Skip to content

[Enhancement] Reduce Remoting header encoding allocation via FastCodesHeader/RocketMQSerializable optimizations #10522

@wang-jiahua

Description

@wang-jiahua

Before Creating the Enhancement Request

  • I have confirmed that this should be classified as an enhancement rather than a bug/feature.

Summary

Enhance the FastCodesHeader interface and RocketMQSerializable utility to reduce allocation and CPU overhead during Remoting header encoding/decoding.

Motivation

In high-throughput scenarios, every RPC request/response goes through the Remoting header encode/decode path. Current implementation has several inefficiencies:

  1. FastCodesHeader.writeIfNotNull always converts numeric values (Long/Integer) to String via value.toString(), creating transient String objects.
  2. RocketMQSerializable.readStr allocates a new String even for single-byte ASCII values.
  3. RocketMQSerializable.mapDeserialize pre-allocates a HashMap with capacity 128, which is oversized for typical headers and wastes memory.
  4. There is no direct way to write primitive long/int values to the ByteBuf without first boxing them into Long/Integer.

These small overheads are amplified by millions of RPCs per second in production.

Describe the Solution You'd Like

  1. Add writeDecimalLong(ByteBuf, long) and writeDecimalInt(ByteBuf, int) to RocketMQSerializable to write primitive decimal numbers directly to ByteBuf without String allocation.
  2. Add writeLong(ByteBuf, String, long) and writeInt(ByteBuf, String, int) helpers to FastCodesHeader.
  3. Optimize FastCodesHeader.writeIfNotNull to route Long/Integer values through the new primitive writers.
  4. Add a SINGLE_BYTE_STRINGS cache for ASCII single-byte strings in RocketMQSerializable.readStr.
  5. Reduce the initial capacity of HashMap in mapDeserialize from 128 to 24.

All changes are additive and keep existing method signatures backward compatible.

Describe Alternatives You've Considered

  • Use String.valueOf + writeStr: still allocates String objects.
  • Use ByteBuf.writeAscii for numbers: Netty's API does not provide direct decimal writing with length prefix required by RocketMQ protocol.
  • Keep HashMap(128) in mapDeserialize: wastes memory for small maps; 24 is sufficient for typical headers.

Additional Context

This is part of a larger effort to reduce Broker allocation rate. Related previous PRs: #10443, #10491, #10485, #10514, #10513, #10517.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions