fix(jsonrpc): enforce log filter cap and improve match efficiency by 317787106 · Pull Request #6732 · tronprotocol/java-tron

317787106 · 2026-04-29T05:27:10Z

Problem

Three issues exist in the JSON-RPC log filter subsystem (eth_newFilter / eth_getLogs), reported in #6510:

Unbounded memory growth — eth_newFilter imposes no limit on the number of active filters held in memory. A client can register filters indefinitely and eventually exhaust heap on the node.
Correctness bug — LogMatch.matchBlockOneByOne evaluated the MAX_RESULT guard after addAll, so the result list could transiently contain more than MAX_RESULT entries before the exception was thrown.
Performance bottleneck under high filter count — handleLogsFilter iterated the filter map with an Iterator and called result.add() once per element, which is slow and contention-prone when thousands of filters are active.

Changes

1. Enforce a configurable cap on active log filters

Adds node.jsonrpc.maxLogFilterNum (default: 20000). When the cap is reached, eth_newFilter immediately returns JsonRpcExceedLimitException (JSON-RPC code -32005) instead of growing without bound.

node {
  jsonrpc {
    # Maximum number of concurrent eth_newFilter registrations (0 = unlimited)
    maxLogFilterNum = 20000
  }
}

2. Fix the MAX_RESULT boundary check in `LogMatch.matchBlockOneByOne` (correctness)

Moves the size + matchedLog.size() > MAX_RESULT guard before addAll. The result list now never exceeds MAX_RESULT entries regardless of how many logs a single block contributes.

3. Optimize `handleLogsFilter` for large filter maps

Condition	Before	After
filter count ≤ 10,000 (common case)	`Iterator` loop, `result.add()` per element	`ConcurrentHashMap.forEach` with local list + single `addAll`
filter count > 10,000	same slow path	bounded `ForkJoinPool(2)` + `parallelStream`

Key improvements:

Reduced lock contention: elements matched per filter are collected into a local list first, then inserted with a single addAll on the shared LinkedBlockingQueue.
Bounded parallelism: the ForkJoinPool is created once and reused (named logs-filter-pool for thread-dump visibility), avoiding unbounded thread creation under spike load.
Observability: a logger.debug timing line records dispatch cost and filter-map size.

Testing

Unit tests (new):
- HandleLogsFilterTest — 7 cases covering event dispatch, block-range filtering, expired-filter removal, and solidified/non-solidified routing
- LogMatchOverLimitTest — 4 cases covering under-limit, exact-limit, exceed-limit (verifies exception is thrown before addAll), and empty-block skip
- WalletCursorTest.testNewFilter_exceedsCapThrowsException — verifies the cap throws the correct exception with the limit value in the message
Manual testing

Closes #6510

…e addAll matchedLog instead after

alan-eth · 2026-04-29T06:06:00Z

@@ -400,6 +400,9 @@ node {

    # Maximum number for blockFilter
    maxBlockFilterNum = 50000


Really nice to see a default-on safeguard added — and using 0 as a sentinel for "unlimited" is a clean pattern.

Minor doc accuracy: the description here says "Maximum number of log entries returned by eth_getLogs / eth_newFilter", but the value actually caps the number of concurrent filter registrations, not the result size. The framework config.conf describes it correctly ("Allowed maximum number for newFilter, >0 otherwise no limit"). Could we align the wording in reference.conf to avoid operators misreading it as a per-query cap?

Suggestion:

# Maximum number of concurrent eth_newFilter registrations (>0; 0 means unlimited) maxLogFilterNum = 20000

Thanks for your suggestion, I fix it.

0xbigapple · 2026-04-29T10:33:07Z


  private static final String ERROR_SELECTOR = "08c379a0"; // Function selector for Error(string)
+  private static final int FILTER_PARALLEL_THRESHOLD = 10000;
+  private static final ForkJoinPool LOGS_FILTER_POOL = new ForkJoinPool(2);


[SHOULD]LOGS_FILTER_POOL is declared as a static (class-level) ForkJoinPool, meaning it is shared across all instances within the JVM. However, it is being shut down inside an instance-level close() method. Once close() is invoked, the shared pool transitions to a terminated state and cannot accept new tasks. Any subsequent usage (including from other instances or test cases) will result in RejectedExecutionException.
This is especially problematic in unit tests, where one test invoking close() can unintentionally affect others due to the shared static resource.

0xbigapple · 2026-04-29T10:55:56Z

      blockFilter2Result = blockFilter2ResultSolidity;
    }
-    if (blockFilter2Result.size() >= maxBlockFilterNum) {
+    if (maxBlockFilterNum > 0 && blockFilter2Result.size() >= maxBlockFilterNum) {


newBlockFilter semantics changed silently.
Old: maxBlockFilterNum=0 blocked all calls.
New: means "unlimited".
Worth calling out in the PR description; also update the reference.conf comment for maxBlockFilterNum — currently only maxLogFilterNum documents the new semantics.

0xbigapple · 2026-04-29T11:02:31Z

+   * The result list must contain only the logs from block 1 (not the partial block-2 logs).
+   */
+  @Test
+  public void testExceedsLimit_throwsBeforeAddAll()


[NIT] testExceedsLimit_throwsBeforeAddAll doesn't actually verify "before addAll". The pre-fix code also threw JsonRpcTooManyResultException

0xbigapple · 2026-04-29T11:02:47Z

+
+    try {
+      logMatch.matchBlockOneByOne();
+      Assert.fail("Expected JsonRpcTooManyResultException");


[NIT] Use assertThrows instead of try/fail/catch for expected-exception tests

0xbigapple · 2026-04-29T11:05:51Z

  private static final String FILTER_NOT_FOUND = "filter not found";
  public static final int EXPIRE_SECONDS = 5 * 60;
  private static final int maxBlockFilterNum = Args.getInstance().getJsonRpcMaxBlockFilterNum();
+  private static final int maxLogFilterNum = Args.getInstance().getJsonRpcMaxLogFilterNum();


[NIT] maxLogFilterNum is snapshotted into static final at class load. That's why the cap test has to populate 20K entries — Args.set...()
from tests has no effect. Reading via Args.getInstance() on each call (singleton, cheap) would let tests inject cap=5.

0xbigapple · 2026-04-29T11:13:24Z

+import org.tron.protos.Protocol.TransactionInfo.Log;
+
+/**
+ * Verifies the over-limit check in {@link LogMatch#matchBlockOneByOne()} introduced in PR #71.


[NIT] Javadoc reference to "PR #71" in LogMatchOverLimitTest is fork-local, should drop it.

0xbigapple · 2026-04-29T11:21:15Z

+import org.tron.core.services.jsonrpc.filters.LogFilterAndResult;
+import org.tron.protos.Protocol.TransactionInfo;
+
+public class HandleLogsFilterTest {


[SHOULD] Parallel branch has zero test coverage.

0xbigapple · 2026-04-29T11:35:51Z

 @Slf4j(topic = "API")
 public class JsonRpcApiUtil {

+  static SecureRandom random = new SecureRandom();


[NIT] Hoisting random to a static field is a nice perf win👍. One small follow-up: consider tightening it to private static final:
private static final SecureRandom random = new SecureRandom();
As written (package-private, non-final), anything in org.tron.core.services.jsonrpc can reassign it. A well-meaning test or perf experiment could quietly do: JsonRpcApiUtil.random = new Random(42);// compiles, no warning
…and filter IDs silently become predictable, which breaks the unguessability assumption clients rely on. private final closes that path at compile time, and also gives JMM safe-publication guarantees for free.

317787106 added 17 commits April 28, 2026 23:44

add log for LogsFilterCapsule

32e5f46

use parallelStream for handleLogsFilter

2a41b0d

add number of log filter limit

edf1190

add map size

30fcda3

use serial map

b176603

use parallel

5277f0c

set parallelism as 3

c32d5e2

add config item node.jsonrpc.maxLogFilterNum

3c54ff4

add missing EventBloomException for switchFork; throw exception befor…

96964c0

…e addAll matchedLog instead after

use ForkJoinPool as necessary

4923b2d

set handleLogsFilter to debug

515ffb4

give name to LOGS_FILTER_POOL

632740a

remove EventBloomException

2551c5a

extract SecureRandom

8b79d84

format code

361c53e

add testcase HandleLogsFilterTest,LogMatchOverLimitTest

f1d0e9a

add comment of config file

9d028c6

github-actions Bot requested review from 0xbigapple and bladehan1 April 29, 2026 05:27

alan-eth reviewed Apr 29, 2026

View reviewed changes

halibobo1205 added the topic:json-rpc label Apr 29, 2026

halibobo1205 added this to the GreatVoyage-v4.8.2 milestone Apr 29, 2026

fix the comment of maxLogFilterNum in reference.conf

e6354d0

0xbigapple reviewed Apr 29, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(jsonrpc): enforce log filter cap and improve match efficiency#6732

fix(jsonrpc): enforce log filter cap and improve match efficiency#6732
317787106 wants to merge 18 commits intotronprotocol:developfrom
317787106:hotfix/fix_newFilter

317787106 commented Apr 29, 2026

Uh oh!

alan-eth Apr 29, 2026

Uh oh!

317787106 Apr 29, 2026

Uh oh!

0xbigapple Apr 29, 2026

Uh oh!

0xbigapple Apr 29, 2026

Uh oh!

0xbigapple Apr 29, 2026

Uh oh!

0xbigapple Apr 29, 2026

Uh oh!

0xbigapple Apr 29, 2026

Uh oh!

0xbigapple Apr 29, 2026

Uh oh!

0xbigapple Apr 29, 2026

Uh oh!

0xbigapple Apr 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		@@ -400,6 +400,9 @@ node {

		# Maximum number for blockFilter
		maxBlockFilterNum = 50000

Conversation

317787106 commented Apr 29, 2026

Problem

Changes

1. Enforce a configurable cap on active log filters

2. Fix the MAX_RESULT boundary check in LogMatch.matchBlockOneByOne (correctness)

3. Optimize handleLogsFilter for large filter maps

Testing

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

2. Fix the MAX_RESULT boundary check in `LogMatch.matchBlockOneByOne` (correctness)

3. Optimize `handleLogsFilter` for large filter maps