feat(api): support int64_as_string parameter for GET requests #6699
Open
waynercheung wants to merge 1 commit intotronprotocol:developfrom
Open
feat(api): support int64_as_string parameter for GET requests #6699waynercheung wants to merge 1 commit intotronprotocol:developfrom
waynercheung wants to merge 1 commit intotronprotocol:developfrom
Conversation
b2ce599 to
0587166
Compare
…otocol#6568) Add an opt-in `int64_as_string` query parameter on TRON HTTP GET endpoints. When set, int64/uint64 protobuf fields in the response are serialized as quoted JSON strings to avoid precision loss in clients whose native number type cannot safely represent integers above 2^53 - 1 (e.g. JavaScript). Scope: GET only. POST is intentionally unsupported because reading the request body in a centralized location (RateLimiterServlet.service or a Filter) would consume request.getReader() and break downstream servlets that read the body themselves. Most TRON query endpoints support both GET and POST, so clients that need precision can use the GET form. POST- only write endpoints return Transaction proto whose int64 fields would break round-trip JsonFormat.merge if quoted, so they should not enable this flag in the first place. - JsonFormat: add INT64_AS_STRING ThreadLocal + setInt64AsString / clearInt64AsString / isInt64AsString helpers; split printFieldValue INT64/SINT64/SFIXED64 and UINT64/FIXED64 branches so they emit quoted strings only when the flag is set. - Util: add INT64_AS_STRING constant + getInt64AsString (URL query, mirrors getVisible). - RateLimiterServlet.service: set ThreadLocal from URL query on GET only; clear in finally so reused Tomcat threads do not leak state across requests. - GetBurnTrx / GetPendingSize / GetTransactionCountByBlockNum: emit quoted int64 in their hand-built JSON responses when isInt64AsString is true. - JsonFormatInt64AsStringTest: covers default behavior, int64 / uint64 quoting, non-int64 fields unaffected, nested / map / boundary values (2^53 +/- 1, Long.MAX/MIN, -1), state cleanup (normal close, after exception, explicit clear), thread isolation, thread-reuse anti-pollution. Backward compatibility: requests without int64_as_string=true produce byte-identical responses to develop -- the new code paths are gated entirely on the new flag. Closes tronprotocol#6568.
0587166 to
cb77f36
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds an opt-in
int64_as_stringquery parameter on TRON HTTP GET endpoints. When set, int64 / uint64 protobuf fields in the response are serialized as quoted JSON strings (e.g."123456789012345"instead of123456789012345), preventing precision loss in clients whose native number type cannot safely represent integers above 2^53 - 1 (e.g. JavaScript).Scope: GET only, by design.
POST is intentionally unsupported. Reading the body in a centralized location (
RateLimiterServlet.serviceor a Servlet Filter) would consumerequest.getReader(), which TRON's existing 25 servlets already consume themselves for business logic. Centralizing POST would require introducing aCachingHttpServletRequestWrapperto make the body re-readable -- significant infrastructure (~150 lines), runtime overhead (every POST body byte-copied to memory), and edge cases (multipart, encoding, large bodies). The simpler and lower-risk approach is to limit this flag to GET.Most TRON query endpoints support both GET and POST. JS clients that need precision can use the GET form. POST-only write endpoints (Transfer / Trigger / Proposal*) return Transaction proto whose int64 fields would break round-trip
JsonFormat.mergeif quoted -- those clients should not enable this flag in the first place.Backward compatibility: when the parameter is absent or false, every response is byte-identical to current
develop-- same JSON shape, same error messages, same log output, same exception paths.Why are these changes required?
JavaScript
Numberis IEEE 754 double-precision with 53 bits of mantissa. Any int64 / uint64 value larger than2^53 - 1loses precision when consumed as a JS number -- for example, an accountbalanceof9007199254740993becomes9007199254740992. Many TRON HTTP fields routinely exceed this threshold (balance,frozen.amount, energy/bandwidth limits, transactionfee_limit, blocktimestampin microseconds, etc.).The standard mitigation across protobuf JSON ecosystems (proto3 JSON spec, Google's
protobuf-java JsonFormat) is to emit 64-bit integers as quoted strings. This PR brings TRON's HTTP API in line with that convention while keeping the new behavior fully opt-in.Closes #6568.
This PR has been tested by:
Unit Tests -- 19 tests across 2 files, all passing locally:
JsonFormatInt64AsStringTest(13 tests): default behavior, int64 / uint64 quoting, non-int64 fields unaffected, nested / map / boundary values (2^53 ± 1,Long.MAX/MIN, -1), state cleanup (normal close, after exception, explicit clear), thread isolation, thread-reuse anti-pollution.RateLimiterServletInt64Test(6 tests): end-to-end integration pinning four contracts:?int64_as_string=trueproduces quoted int64 fields.service()'sfinallyblock clears the ThreadLocal (defends against silent leak across reused Tomcat threads).isInt64AsString() ? quoted : unquotedternary -- mutation-tested against ternary inversion.Manual Testing -- recommended pre-merge: hit a few endpoints with and without the flag, diff the JSON shape, confirm no regression on the default path.
Follow up
Transaction/Block/TransactionExtention/TransactionSignWeight/TransactionApprovedListproto structures cannot be re-parsed byJsonFormat.merge(the parser does not accept quoted int64 input). Clients that need precision and only consume the response should enable the flag; clients that need round-trip should leave it off.CachingHttpServletRequestWrapper. The trade-offs of that approach (body memory caching, multipart edge cases) deserve their own scoped review.Extra details
Files changed (9 total).
Core machinery:
JsonFormat.java:INT64_AS_STRINGThreadLocal +setInt64AsString/clearInt64AsString/isInt64AsStringhelpers; split INT64 and UINT64 branches inprintFieldValueto emit quoted strings only when the flag is set. When the flag is unset, the code path is byte-identical todevelop.Util.java:INT64_AS_STRINGconstant +getInt64AsString(URL query, mirrorsgetVisible).RateLimiterServlet.service: set ThreadLocal from URL query on GET only; clear infinallyso reused Tomcat threads do not leak state across requests. Thefinallyclear has a defensive comment because removing it silently breaks request isolation.Hand-built JSON responses (4 files) -- these emit JSON literals manually instead of going through
JsonFormat.printToString, so they readJsonFormat.isInt64AsString()to choose between quoted and unquoted format:GetBurnTrxServlet:burnTrxAmountfieldGetPendingSizeServlet:pendingSizefieldGetTransactionCountByBlockNumServlet:countfieldGetRewardServlet:rewardfieldTest coverage:
JsonFormatInt64AsStringTest(mechanism level)RateLimiterServletInt64Test(end-to-end integration, including critical defense against ThreadLocal leakage)~50 other read-only query servlets automatically pick up the flag through the centralized
RateLimiterServlet.serviceGET path with zero changes at the call site.Sample contract for clients