Skip to content

Add nft media type fetcher #129

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nikitosing wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add nft media type fetcher #129

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions setup/env-variables/backend-env-variables.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@
| `HACKNEY_DEFAULT_POOL_SIZE` | Size of `default` hackney pool. Implemented in [#12406](https://github.com/blockscout/blockscout/pull/12406). | Version: v8.1.0\+ <br />Default: `1000` <br />Applications: API, Indexer |
| `UNIVERSAL_PROXY_CONFIG_URL` | Config URL for universal proxy. Implemented in [#12442](https://github.com/blockscout/blockscout/pull/12442). | Version: v8.1.0\+ <br />Default: `https://raw.githubusercontent.com/blockscout/backend-configs/refs/heads/main/universal-proxy-config.json` <br />Applications: API |
| `UNIVERSAL_PROXY_CONFIG` | JSON-encoded configuration string used to define settings for the universal proxy. Implemented in [#13787](https://github.com/blockscout/blockscout/pull/13787). | Version: v10.0.0\+ <br />Default: (empty) <br />Applications: API |
| `K8S_SERVICE` | Kubernetes service name for Elixir nodes clusterization, more info on how to configure it can be found here https://hexdocs.pm/libcluster/Cluster.Strategy.Kubernetes.DNS.html. Implemented in [#13080](https://github.com/blockscout/blockscout/pull/13080). | Version: v9.2.0\+ <br />Default: (empty) <br />Applications: API, Indexer |

Check warning on line 144 in setup/env-variables/backend-env-variables.mdx

View check run for this annotation

Mintlify / Mintlify Validation (blockscout) - vale-spellcheck

setup/env-variables/backend-env-variables.mdx#L144 

Did you really mean 'clusterization'?

## Database Management

Expand Down Expand Up @@ -236,6 +236,7 @@
| `INDEXER_DISABLE_REPLACED_TRANSACTION_FETCHER` | If `true`, `Indexer.Fetcher.ReplacedTransaction` fetcher doesn't run. | Version: v6.8.0\+ <br />Default: `false` <br />Applications: Indexer |
| `INDEXER_DISABLE_TOKEN_INSTANCE_ERC_1155_SANITIZE_FETCHER` | If `true`, `erc-1155-sanitize` token instance fetcher doesn't run. Implemented in [#9226](https://github.com/blockscout/blockscout/pull/9226). | Version: v6.2.0\+ <br />Default: `false` <br />Applications: Indexer |
| `INDEXER_DISABLE_TOKEN_INSTANCE_ERC_721_SANITIZE_FETCHER` | If `true`, `erc-721-sanitize` token instance fetcher doesn't run. Implemented in [#9226](https://github.com/blockscout/blockscout/pull/9226). | Version: v6.2.0\+ <br />Default: `false` <br />Applications: Indexer |
| `INDEXER_DISABLE_TOKEN_INSTANCE_MEDIA_TYPE_FETCHER` | If `true`, token instance media type backfill fetcher is disabled. When disabled, `image_type` and `animation_type` fields will not be populated for existing token instances. Implemented in [#14365](https://github.com/blockscout/blockscout/pull/14365). | Version: master <br />Default: `false` <br />Applications: Indexer |
| `INDEXER_DISABLE_TOKEN_INSTANCE_REALTIME_FETCHER` | If `true`, `realtime` token instance fetcher doesn't run. | Version: v5.1.4\+ <br />Default: `false` <br />Applications: Indexer |
| `INDEXER_DISABLE_TOKEN_INSTANCE_REFETCH_FETCHER` | If `true`, the Token instance fetcher, which re-fetches NFT collections marked to refetch, doesn't run. Implemented in [#10263](https://github.com/blockscout/blockscout/pull/10263). | Version: v7.0.0\+ <br />Default: `false` <br />Applications: Indexer |
| `INDEXER_DISABLE_TOKEN_INSTANCE_RETRY_FETCHER` | If `true`, `retry` token instance fetcher doesn't run. | Version: v5.1.4\+ <br />Default: `false` <br />Applications: Indexer |
Expand Down Expand Up @@ -276,7 +277,7 @@
| `INDEXER_RECEIPTS_BATCH_SIZE` | Batch size for transaction receipts fetcher. Implemented in [#6454](https://github.com/blockscout/blockscout/pull/6454). | Version: v5.0.0\+ <br />Default: `250` <br />Applications: Indexer |
| `INDEXER_RECEIPTS_CONCURRENCY` | Concurrency for transaction receipts fetcher. Implemented in [#6454](https://github.com/blockscout/blockscout/pull/6454). | Version: v5.0.0\+ <br />Default: `10` <br />Applications: Indexer |
| `INDEXER_SIGNED_AUTHORIZATION_STATUS_BATCH_SIZE` | Batch size (number of blocks) for EIP7702 authorizations status fetcher. Implemented in [#12451](https://github.com/blockscout/blockscout/pull/12451). | Version: v9.0.0\+ <br />Default: `10` <br />Applications: Indexer |
| `INDEXER_FHE_OPERATIONS_ENABLED` | Flag to enable parsing of Fully Homomorphic Encryption (FHE) operations from transactions. Implemented in [#13742](https://github.com/blockscout/blockscout/pull/13742). | Version: v11.0.0\+ <br />Default: `false` <br />Applications: Indexer |

Check warning on line 280 in setup/env-variables/backend-env-variables.mdx

View check run for this annotation

Mintlify / Mintlify Validation (blockscout) - vale-spellcheck

setup/env-variables/backend-env-variables.mdx#L280 

Did you really mean 'Homomorphic'?
| `INDEXER_SYSTEM_MEMORY_PERCENTAGE` | Percentage of total memory available to the VM that an application can use if `INDEXER_MEMORY_LIMIT` is not set. Implemented in [#10697](https://github.com/blockscout/blockscout/pull/10697). | Version: v6.9.0\+ <br />Default: `60` <br />Applications: Indexer |
| `INDEXER_ARCHIVAL_TOKEN_BALANCES_BATCH_SIZE` | Batch size for archival token balances fetcher. Implemented in [#7439](https://github.com/blockscout/blockscout/pull/7439). | Version: v10.0.0\+ <br />Default: `100` <br />Applications: Indexer |
| `INDEXER_ARCHIVAL_TOKEN_BALANCES_CONCURRENCY` | Concurrency for archival token balances fetcher. Implemented in [#8167](https://github.com/blockscout/blockscout/pull/8167). | Version: v10.0.0\+ <br />Default: `10` <br />Applications: Indexer |
Expand All @@ -287,6 +288,8 @@
| `INDEXER_TOKEN_INSTANCE_ALLOWED_URI_PROTOCOLS` | List of allowed URI protocols (schemes) for requests when fetching token instance metadata. Implemented in [#12102](https://github.com/blockscout/blockscout/pull/12102). | Version: v8.0.0\+ <br />Default: `http,https` <br />Applications: Indexer |
| `INDEXER_TOKEN_INSTANCE_CIDR_BLACKLIST` | List of IP addresses in CIDR format to block when fetching token instance metadata. Example: `"0.0.0.0/32,192.168.0.0/16"`. Implemented in [#12102](https://github.com/blockscout/blockscout/pull/12102). | Version: v8.0.0\+ <br />Default: (empty) <br />Applications: Indexer |
| `INDEXER_TOKEN_INSTANCE_HOST_FILTERING_ENABLED` | If `false`, the URL from which metadata is fetched will not be resolved to an IP address, and the IP address will not be checked against the blacklist. Implemented in [#12102](https://github.com/blockscout/blockscout/pull/12102). | Version: v8.0.0\+ <br />Default: `true` <br />Applications: Indexer |
| `INDEXER_TOKEN_INSTANCE_MEDIA_TYPE_BATCH_SIZE` | Number of token instances processed per batch by the media type fetcher, which backfills MIME types (`image_type`, `animation_type`) for NFT instances via HTTP HEAD requests to image/animation URLs from metadata. Implemented in [#14365](https://github.com/blockscout/blockscout/pull/14365). | Version: master <br />Default: `10` <br />Applications: Indexer |

Check warning on line 291 in setup/env-variables/backend-env-variables.mdx

View check run for this annotation

Mintlify / Mintlify Validation (blockscout) - vale-spellcheck

setup/env-variables/backend-env-variables.mdx#L291 

Did you really mean 'backfills'?
| `INDEXER_TOKEN_INSTANCE_MEDIA_TYPE_CONCURRENCY` | Max number of concurrent workers resolving media types for token instances. Implemented in [#14365](https://github.com/blockscout/blockscout/pull/14365). | Version: master <br />Default: `5` <br />Applications: Indexer |
| `INDEXER_TOKEN_INSTANCE_REALTIME_BATCH_SIZE` | Batch size for `realtime` token instance fetcher. Implemented in [#8313](https://github.com/blockscout/blockscout/pull/8313). | Version: v5.2.3\+ <br />Default: `1` <br />Applications: Indexer |
| `INDEXER_TOKEN_INSTANCE_REALTIME_CONCURRENCY` | Concurrency for `realtime` token instance fetcher. Implemented in [#7286](https://github.com/blockscout/blockscout/pull/7286). | Version: v5.1.4\+ <br />Default: `10` <br />Applications: Indexer |
| `INDEXER_TOKEN_INSTANCE_REALTIME_RETRY_ENABLED` | If `true`, `realtime` token instance fetcher will retry once on 404 and 500 error. Implemented in [#10036](https://github.com/blockscout/blockscout/pull/10036). | Version: v6.6.0\+ <br />Default: `false` <br />Applications: Indexer |
Expand Down Expand Up @@ -529,21 +532,21 @@
## CSV Export

<Info>
When `CSV_EXPORT_ASYNC_ENABLED` is `true`, supported CSV export endpoints create a tracked request, enqueue a background job, write the CSV to a temporary file, upload it to Gokapi, and return a `request_id` for polling instead of streaming the CSV directly in the HTTP response.

Check warning on line 535 in setup/env-variables/backend-env-variables.mdx

View check run for this annotation

Mintlify / Mintlify Validation (blockscout) - vale-spellcheck

setup/env-variables/backend-env-variables.mdx#L535 

Did you really mean 'Gokapi'?
</Info>

| Variable | Description | Parameters |
| ------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| `CSV_EXPORT_LIMIT` | Changes the max number of rows pulled by the CSV export logic. Applies to both sync and async CSV export paths. Implemented in [#10497](https://github.com/blockscout/blockscout/pull/10497) | Version: v6.8.0\+ <br />Default: `10000` <br />Applications: API |
| `CSV_EXPORT_ASYNC_ENABLED` | Enables async CSV export for supported endpoints. When enabled, the API returns `202 Accepted` with a `request_id` and processes exports through Oban instead of streaming them directly. Implemented in [#14028](https://github.com/blockscout/blockscout/pull/14028) | Version: v11.0.0\+ Required: No <br />Default: `false` <br />Applications: API |

Check warning on line 541 in setup/env-variables/backend-env-variables.mdx

View check run for this annotation

Mintlify / Mintlify Validation (blockscout) - vale-spellcheck

setup/env-variables/backend-env-variables.mdx#L541 

Did you really mean 'Oban'?
| `CSV_EXPORT_ASYNC_OBAN_CONCURRENCY` | Sets Oban concurrency for the `csv_export` queue used by async CSV exports. Implemented in [#14028](https://github.com/blockscout/blockscout/pull/14028) | Version: v11.0.0\+ Required: No <br />Default: `10` <br />Applications: API |

Check warning on line 542 in setup/env-variables/backend-env-variables.mdx

View check run for this annotation

Mintlify / Mintlify Validation (blockscout) - vale-spellcheck

setup/env-variables/backend-env-variables.mdx#L542 

Did you really mean 'Oban'?
| `CSV_EXPORT_ASYNC_GOKAPI_URL` | Base URL of the Gokapi instance used to store completed async CSV exports. Trailing slash is stripped during validation. Implemented in [#14028](https://github.com/blockscout/blockscout/pull/14028) | Version: v11.0.0\+ Required: Yes, if async export is enabled <br />Default: (empty) <br />Applications: API |

Check warning on line 543 in setup/env-variables/backend-env-variables.mdx

View check run for this annotation

Mintlify / Mintlify Validation (blockscout) - vale-spellcheck

setup/env-variables/backend-env-variables.mdx#L543 

Did you really mean 'Gokapi'?
| `CSV_EXPORT_ASYNC_GOKAPI_API_KEY` | API key sent to Gokapi in the `apikey` header for async CSV export uploads. Implemented in [#14028](https://github.com/blockscout/blockscout/pull/14028) | Version: v11.0.0\+ Required: Yes, if async export is enabled <br />Default: (empty) <br />Applications: API |

Check warning on line 544 in setup/env-variables/backend-env-variables.mdx

View check run for this annotation

Mintlify / Mintlify Validation (blockscout) - vale-spellcheck

setup/env-variables/backend-env-variables.mdx#L544 

Did you really mean 'Gokapi'?
| `CSV_EXPORT_ASYNC_MAX_PENDING_TASKS_PER_IP` | Maximum number of pending async CSV export requests allowed per client IP at once. Implemented in [#14028](https://github.com/blockscout/blockscout/pull/14028) | Version: v11.0.0\+ Required: No <br />Default: `3` <br />Applications: API |
| `CSV_EXPORT_ASYNC_UPLOAD_CHUNK_SIZE` | Chunk size in bytes for reading the generated CSV file and uploading it to Gokapi. Should be synchronized with Gokapi settings. Implemented in [#14028](https://github.com/blockscout/blockscout/pull/14028) | Version: v11.0.0\+ Required: No <br />Default: `47185920` <br />Applications: API |

Check warning on line 546 in setup/env-variables/backend-env-variables.mdx

View check run for this annotation

Mintlify / Mintlify Validation (blockscout) - vale-spellcheck

setup/env-variables/backend-env-variables.mdx#L546 

Did you really mean 'Gokapi'?

Check warning on line 546 in setup/env-variables/backend-env-variables.mdx

View check run for this annotation

Mintlify / Mintlify Validation (blockscout) - vale-spellcheck

setup/env-variables/backend-env-variables.mdx#L546 

Did you really mean 'Gokapi'?
| `CSV_EXPORT_DB_TIMEOUT` | Timeout for CSV export database work. Follows the [time format](/setup/env-variables/backend-env-variables#time-format). Implemented in [#14028](https://github.com/blockscout/blockscout/pull/14028) | Version: v11.0.0\+ Required: No <br />Default: `1h` if async export is enabled, otherwise `5m` <br />Applications: API |
| `CSV_EXPORT_ASYNC_TMP_DIR` | Directory used for in-progress async CSV export files before they are uploaded to Gokapi. Implemented in [#14028](https://github.com/blockscout/blockscout/pull/14028) | Version: v11.0.0\+ Required: No <br />Default: `/tmp/csv_export` <br />Applications: API |

Check warning on line 548 in setup/env-variables/backend-env-variables.mdx

View check run for this annotation

Mintlify / Mintlify Validation (blockscout) - vale-spellcheck

setup/env-variables/backend-env-variables.mdx#L548 

Did you really mean 'Gokapi'?
| `CSV_EXPORT_ASYNC_GOKAPI_TIMEOUT` | HTTP timeout and `recv_timeout` used for Gokapi requests during async CSV export. Follows the [time format](/setup/env-variables/backend-env-variables#time-format). Implemented in [#14028](https://github.com/blockscout/blockscout/pull/14028) | Version: v11.0.0\+ Required: No <br />Default: `60s` <br />Applications: API |

Check warning on line 549 in setup/env-variables/backend-env-variables.mdx

View check run for this annotation

Mintlify / Mintlify Validation (blockscout) - vale-spellcheck

setup/env-variables/backend-env-variables.mdx#L549 

Did you really mean 'Gokapi'?
| `CSV_EXPORT_ASYNC_GOKAPI_UPLOAD_EXPIRY_DAYS` | Sets Gokapi `expiryDays` for completed async CSV export uploads. Implemented in [#14028](https://github.com/blockscout/blockscout/pull/14028) | Version: v11.0.0\+ Required: No <br />Default: `1` <br />Applications: API |
| `CSV_EXPORT_ASYNC_GOKAPI_UPLOAD_ALLOWED_DOWNLOADS` | Sets Gokapi `allowedDownloads` for completed async CSV export uploads. Implemented in [#14028](https://github.com/blockscout/blockscout/pull/14028) | Version: v11.0.0\+ Required: No <br />Default: `1` <br />Applications: API |

Expand Down Expand Up @@ -590,7 +593,7 @@
| `API_RATE_LIMIT_REMOTE_IP_KNOWN_PROXIES` | Comma separated list of trusted proxy IP addresses or CIDR ranges that are allowed to set the client IP headers for rate limiting. Implemented in [#12386](https://github.com/blockscout/blockscout/pull/13082) | Version: v9.1.0\+ <br />Default: `(empty)` <br />Applications: API |
| `ETH_JSON_RPC_MAX_BATCH_SIZE` | Max batch size for Eth JSON RPC requests. Implemented in [#9409](https://github.com/blockscout/blockscout/pull/9409) | Version: v6.3.0\+ <br />Default: `5` <br />Applications: API |
| `API_RATE_LIMIT_HAMMER_REDIS_SSL_ENABLED` | If `true` Blockscout will use SSL to connect to Redis DB for Hammer rate limit library. Implemented in [#13901](https://github.com/blockscout/blockscout/pull/13901) | Version: master <br />Default: (empty) <br />Applications: API |
| `API_RATE_LIMIT_HAMMER_REDIS_SENTINEL_URLS` | List of Redis sentinel urls for Hammer rate limit library. Implemented in [#13901](https://github.com/blockscout/blockscout/pull/13901) | Version: master <br />Default: (empty) <br />Applications: API |

Check warning on line 596 in setup/env-variables/backend-env-variables.mdx

View check run for this annotation

Mintlify / Mintlify Validation (blockscout) - vale-spellcheck

setup/env-variables/backend-env-variables.mdx#L596 

Did you really mean 'urls'?
| `API_RATE_LIMIT_HAMMER_REDIS_SENTINEL_MASTER_NAME`| Sentinel master name for Hammer rate limit library. Implemented in [#13901](https://github.com/blockscout/blockscout/pull/13901) | Version: master <br />Default: (empty) <br />Applications: API |

## General application rate Limits
Expand All @@ -604,7 +607,7 @@
| `RATE_LIMITER_ON_DEMAND_MAX_BAN_INTERVAL` | Max time an IP address can be banned from on-demand fetcher calls. Implemented in [#12218](https://github.com/blockscout/blockscout/pull/12218) | Version: v8.0.0\+ <br />Default: `1h` <br />Applications: API |
| `RATE_LIMITER_ON_DEMAND_LIMITATION_PERIOD` | Time after which the number of bans for the IP address will be reset. Implemented in [#12218](https://github.com/blockscout/blockscout/pull/12218) | Version: v8.0.0\+ <br />Default: `1h` <br />Applications: API |
| `RATE_LIMITER_REDIS_SSL_ENABLED` | If `true` Blockscout will use SSL to connect to Redis DB for rate limiter. Implemented in [#13901](https://github.com/blockscout/blockscout/pull/13901) | Version: master <br />Default: (empty) <br />Applications: API |
| `RATE_LIMITER_REDIS_SENTINEL_URLS` | List of Redis sentinel urls for rate limiter. Implemented in [#13901](https://github.com/blockscout/blockscout/pull/13901) | Version: master <br />Default: (empty) <br />Applications: API |

Check warning on line 610 in setup/env-variables/backend-env-variables.mdx

View check run for this annotation

Mintlify / Mintlify Validation (blockscout) - vale-spellcheck

setup/env-variables/backend-env-variables.mdx#L610 

Did you really mean 'urls'?
| `RATE_LIMITER_REDIS_SENTINEL_MASTER_NAME` | Sentinel master name for rate limiter. Implemented in [#13901](https://github.com/blockscout/blockscout/pull/13901) | Version: master <br />Default: (empty) <br />Applications: API |

## GraphQL API
Expand Down Expand Up @@ -653,7 +656,7 @@
| `ACCOUNT_KEYCLOAK_CLIENT_SECRET` | [Keycloak](https://www.keycloak.org/) client secret. Implemented in [#14068](https://github.com/blockscout/blockscout/pull/14068). | Version: v10.1.0\+, v9.4.0\+ <br />Default: (empty) <br />Applications: API |
| `ACCOUNT_KEYCLOAK_EMAIL_WEBHOOK_URL` | URL address where new email users are reported. Implemented in [#14068](https://github.com/blockscout/blockscout/pull/14068). | Version: v10.1.0\+, v9.4.0\+ <br />Default: (empty) <br />Applications: API |
| `ACCOUNT_REDIS_SSL_ENABLED` | If `true` Blockscout will use SSL to connect to Account Redis DB. Implemented in [#13901](https://github.com/blockscout/blockscout/pull/13901). | Version: master <br />Default: (empty) <br />Applications: API |
| `ACCOUNT_REDIS_SENTINEL_URLS` | List of Account Redis sentinel urls. Implemented in [#13901](https://github.com/blockscout/blockscout/pull/13901). | Version: master <br />Default: (empty) <br />Applications: API |

Check warning on line 659 in setup/env-variables/backend-env-variables.mdx

View check run for this annotation

Mintlify / Mintlify Validation (blockscout) - vale-spellcheck

setup/env-variables/backend-env-variables.mdx#L659 

Did you really mean 'urls'?
| `ACCOUNT_REDIS_SENTINEL_MASTER_NAME` | Account Sentinel master name. Implemented in [#13901](https://github.com/blockscout/blockscout/pull/13901). | Version: master <br />Default: (empty) <br />Applications: API |

## Specific Smart Contracts
Expand Down