Arc support - #140
Closed
PikaZ76 wants to merge 291 commits into
Closed
Arc support#140PikaZ76 wants to merge 291 commits into
PikaZ76 wants to merge 291 commits into
Conversation
Support story
Add nft header
support ZSTD compression
generate the python code from proto
transform Hash and Account Address into Tronscan format
add test case for Tron address convertion;
Fix/add tron fields
INF-59 Integrate with CircleCI
Merged latest upstream changes from Coinbase
Enable support for launching multiple backfillers concurrently
Resolve CircleCI build failures
use python grpcio-tools to generate python code
Endpoint, EndpointGroup, EndpointConfig and StickySessionConfig only
carried json tags, which cover the path where a group arrives as a JSON
string (the CHAINSTORAGE_..._ENDPOINT_GROUP env var, decoded by
EndpointGroup.UnmarshalText). A group written as a YAML mapping is
decoded by mapstructure instead, which without a tag falls back to a
case-insensitive field-name match and therefore silently dropped every
key containing an underscore: rps_count_batch, provider_id, extra_urls,
endpoints_failover, use_failover and endpoint_config.
The visible symptom was an endpoint configured with
rps: 100
rps_count_batch: true
still tripping the provider's rate limit, because the limiter kept
charging one token per HTTP request instead of one per call inside a
JSON-RPC batch, i.e. up to rps * tx_batch_size provider-side calls per
second.
Add the mapstructure tags so both decoders agree, and log the endpoint
settings that are actually in effect at startup (host only, since
providers embed the API key in the url path), including a warning for an
empty group. Note the rate limiter is per endpoint per group, so the
same url configured in master, slave and validator gets an independent
budget in each.
TestEndpointGroupParsedFromYAML loads a YAML config end to end through
config.New to lock the behavior; TestDumpEndpoints is an opt-in
diagnostic that prints the effective endpoint settings for a given
environment.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…rror (#209) The syncer logged the per-block fetch error and dropped it: reprocessChannel carried only the block metadata, so the aggregate error could name which blocks failed but not why. During the 2026-08-31 robinhood-mainnet incident that left Temporal with "failed to get 10 blocks after attempting twice: [<metadata>]" while the actual cause -- 25,200 HTTP 429s and a provider-side loss of the debug_* namespace -- existed only in the worker pod stdout, which sent the incident response after a block-data gap that was never the problem. - reprocessChannel now carries the {metadata, err} pair. - The aggregate error summarizes the distinct causes, deduplicated by error class and endpoint so the message stays bounded when many blocks fail for one reason, e.g. "10/10 RateLimitError (HTTPError 429) on endpoint=nownodes-jsonrpc-slave; heights=[50881296..50881305]". - The jsonrpc client counts every throttled attempt as jsonrpc.rate_limited+endpoint=<name>, including the ones absorbed by a retry, so an endpoint approaching its quota alerts while the poller is merely slow rather than after it fails. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…t in bounded chunks (#210)
Move ownership of the chainstorage image into this repo so the monorepo can consume a prebuilt, sha-pinned tag instead of git-cloning and rebuilding on every deploy. Dockerfile: adopt the monorepo's semantics minus the git clone. - ARG BASE_REGISTRY so CI pulls base images through the ECR docker-hub pull-through cache; local builds keep using Docker Hub. - Build on golang:1.26-trixie with CGO_ENABLED=0 (static binaries), with a separate go mod download layer. - Deploy on ubuntu:22.04 with ca-certificates; ship /app/bin/* plus the postgres migrations at /app/migrations; CMD /app/worker. - Add .dockerignore so .git, bin/, local compose data and secrets never enter the build context. CircleCI: keep build_and_test untouched and add - orb circleci/aws-cli@5.4.2 - command build_image: OIDC assume-role via CIRCLECI_OIDC_ARN, ECR login, docker build --platform linux/amd64 tagged with the 7-char short sha, no_output_timeout 30m (the Go build is silent for minutes). - job build-image: every branch, build only, no push. - job publish-image: master only, requires build_and_test, pushes the single sha tag and prints the pushed ref. No latest/master tags. Auth needs no new secrets: cipherowl_build_context already carries CIRCLECI_OIDC_ARN, the circleci_oidc role trust has no project subject condition, and the ECR repo policy grants that role push. Claude-Session: https://claude.ai/code/session_01RvJf1meAams4fNAFtpyLB1 Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Collaborator
🟡 Heimdall Review Status
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed? Why?
How did you test the change?