-
Notifications
You must be signed in to change notification settings - Fork 16
feat(docs): add planned integrations for ENSDb snapshots, CLI, and webhooks #2107
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
1c2aafb
feat(docs): add planned integrations for ENSDb snapshots, CLI, and we…
djstrong 2d3d4e8
refactor(docs): update ENSDb snapshots and CLI references for clarity
djstrong bdaca3f
Apply suggestions from code review
djstrong 80b2463
Apply suggestions from code review
djstrong f9dc571
refactor(docs): update integration documentation for ENSDb and ENSEngine
djstrong 49dce66
Merge branch 'main' into coming-soon-docs
djstrong 3149616
docs: add new documentation for ensdb-cli and ENSDb snapshots
djstrong f138deb
docs: update ENSNode documentation for clarity and accuracy
djstrong 810d297
docs: update integration sidebar to include new topics
djstrong b750732
docs: refine ENSEngine documentation for clarity
djstrong 43c28f5
Merge branch 'main' into coming-soon-docs
djstrong 215cbef
docs: update integration documentation for ensdb-cli and ENSEngine
djstrong 2830e7f
fix: correct import path for EnsDbSnapshotsTeaser in ensdb-cli docume…
djstrong 36f8f8a
Merge branch 'main' into coming-soon-docs
djstrong 4ac8ad4
Merge branch 'main' into coming-soon-docs
shrugs File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
24 changes: 24 additions & 0 deletions
24
docs/ensnode.io/src/components/molecules/EnsDbSnapshotsTeaser.astro
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| --- | ||
| import { Card, CardGrid } from "@astrojs/starlight/components"; | ||
| --- | ||
|
|
||
| <CardGrid> | ||
| <Card title="Cut your RPC bill" icon="approve-check-circle"> | ||
| Skip the historical RPC fanout entirely. Pull down a published ENSDb snapshot instead of replaying 9+ years of ENS history through your own RPC provider - saving real money on every bootstrap, every CI run, every fresh environment. | ||
| </Card> | ||
| <Card title="Spin up in minutes, not days" icon="rocket"> | ||
| Skip the long historical backfill too. A nearly up-to-date ENS index is ready as fast as you can download it - no specially configured Postgres required, no days of waiting. Perfect for hackathons, demos, prototypes, and any other time you need a fresh ENSDb fast. | ||
| </Card> | ||
| <Card title="Repeatable CI environments" icon="seti:shell"> | ||
| Use snapshots as deterministic fixtures for tests and CI pipelines. Every run starts from the same well-known ENS state, with zero RPC dependency. | ||
| </Card> | ||
| <Card title="Confident release reviews" icon="approve-check"> | ||
| Diff snapshots across ENSNode versions to catch unexpected changes in indexed data - a powerful way to validate indexing-logic refactors before a release ships. | ||
| </Card> | ||
| <Card title="Easier self-hosting" icon="seti:db"> | ||
| Bootstrap your own ENSNode instance from a recent snapshot, then catch up to realtime from there. Less time waiting, more time building. | ||
| </Card> | ||
| <Card title="ensdb-cli: purpose-built ops tooling" icon="seti:config"> | ||
| `ensdb-cli` surfaces the operations you actually need day-to-day: inspect indexer instances, manage snapshots, clean up unused schemas, and more. | ||
| </Card> | ||
| </CardGrid> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
72 changes: 72 additions & 0 deletions
72
docs/ensnode.io/src/content/docs/docs/integrate/integration-options/ensdb-cli.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| --- | ||
| title: ensdb-cli & ENSDb snapshots | ||
| description: Coming soon - publish, download, and diff snapshots of the live ENS state with the planned ensdb-cli. | ||
| --- | ||
|
|
||
| import { LinkCard, Aside } from "@astrojs/starlight/components"; | ||
| import EnsDbSnapshotsTeaser from "../../../../../components/molecules/EnsDbSnapshotsTeaser.astro"; | ||
|
djstrong marked this conversation as resolved.
|
||
|
|
||
|
djstrong marked this conversation as resolved.
|
||
| <Aside type="caution" title="Coming soon"> | ||
| **`ensdb-cli`** and **ENSDb snapshots** are planned capabilities that are coming soon. | ||
| </Aside> | ||
|
|
||
| ## The problem | ||
|
|
||
| Completing a full historical indexing backfill of ENS from zero is **expensive on two axes: time and money**. A full historical indexing backfill of ENS across mainnet, Basenames, Lineanames, 3DNS, ENSv1, ENSv2, etc generally means days of indexing *and* a heavy RPC bill - every request to your RPC provider, multiplied by every chain, multiplied by every block of 9+ years of ENS history you have to index. That tax is painful for anyone building, prototyping, demoing, or testing. | ||
|
|
||
| [ENSDb](/docs/services/ensdb) already gives you the live, onchain state of ENS in a plain PostgreSQL database that any language with a Postgres driver can read. The next step is making it **trivial - and cheap - to bootstrap and operate** that database. That's what ENSDb snapshots and `ensdb-cli` are for. | ||
|
|
||
| ## What we're building | ||
|
|
||
| ENSDb snapshots are **portable, versioned packages of an ENSDb instance** that anyone can publish or consume. Pull one down, restore it into Postgres, and start querying ENS in minutes instead of waiting days to complete 9-years+ of historical indexing backfill from scratch - and **without** being required to send hundreds of thousands of pricy RPC-calls yourself. When you start from a fresh ENSDb snapshot, it's cheap and fast to catch up to realtime indexing. | ||
|
djstrong marked this conversation as resolved.
djstrong marked this conversation as resolved.
djstrong marked this conversation as resolved.
djstrong marked this conversation as resolved.
djstrong marked this conversation as resolved.
|
||
|
|
||
| `ensdb-cli` will be the operator-facing tool around them - the surface where you list indexer instances and their indexing status, generate fresh snapshots, inspect what configuration a snapshot was built with, and clean up schemas you no longer need. | ||
|
|
||
| <EnsDbSnapshotsTeaser /> | ||
|
|
||
| ## Who this is for | ||
|
|
||
| - **Hackathon teams and prototypers** who want a full ENSDb right now and don't have time or money to wait days on 9+ years of an ENS historical indexing backfill. | ||
| - **Self-hosters** standing up new ENSNode instances who'd rather start from a recent snapshot than backfill from genesis. | ||
| - **CI and testing pipelines** that need deterministic ENS state to run repeatable assertions against. | ||
| - **Maintainers** running release reviews who want to diff indexed data across versions to catch unintended regressions before they ship. | ||
| - **Anyone building specialized ENS services on top of ENSDb** who needs faster iteration loops and lower operational cost. | ||
|
|
||
| ## How this complements ENSDb | ||
|
|
||
| Snapshots and the CLI sit *on top of* the [ENSDb open standard](/docs/services/ensdb). Because ENSDb is just Postgres governed by a well-defined schema, snapshots are portable: any [ENSDb Writer](/docs/services/ensdb/concepts/glossary#ensdb-writer) implementation can produce one, and any [ENSDb Reader](/docs/services/ensdb/concepts/glossary#ensdb-reader) (or your own code) can consume one. | ||
|
|
||
| This is the same foundation that powers ENSNode's other planned ENSDb capabilities - including [ENSEngine](/docs/integrate/integration-options/ensengine), enabling you to react to changes in ENSDb through webhooks and other event-driven strategies without need for polling. | ||
|
|
||
|
|
||
| ## Related | ||
|
|
||
| <LinkCard | ||
| title="ENSDb Integration Quickstart" | ||
| description="The integration point these snapshots build on - the live ENS state, ready for SQL or the ENSDb SDK." | ||
| href="/docs/integrate/integration-options/ensdb" | ||
| /> | ||
|
|
||
| <LinkCard | ||
| title="ENSDb overview" | ||
| description="Vision, philosophy, and how an ENSDb instance works." | ||
| href="/docs/services/ensdb" | ||
| /> | ||
|
|
||
| <LinkCard | ||
| title="ENSEngine Integration Quickstart" | ||
| description="React to ENS state changes with webhooks & events rather than polling." | ||
| href="/docs/integrate/integration-options/ensengine" | ||
| /> | ||
|
|
||
| <LinkCard | ||
| title="ENSEngine overview" | ||
| description="React to changes in ENSDb through webhooks and other reactive / event-based development strategies." | ||
| href="/docs/services/ensengine" | ||
| /> | ||
|
|
||
| <LinkCard | ||
| title="Self-host ENSNode" | ||
| description="Operate your own ENSNode instance to build and publish your own ENSDb snapshots." | ||
| href="/docs/self-host" | ||
| /> | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
84 changes: 84 additions & 0 deletions
84
docs/ensnode.io/src/content/docs/docs/integrate/integration-options/ensengine.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| --- | ||
|
vercel[bot] marked this conversation as resolved.
|
||
| title: ENSEngine | ||
|
djstrong marked this conversation as resolved.
|
||
| description: Coming soon - ENS-native webhooks and push-based events that fire when ENS state changes, so apps can stop polling and start reacting. | ||
| --- | ||
|
|
||
| import { LinkCard, Aside } from "@astrojs/starlight/components"; | ||
|
|
||
| <Aside type="caution" title="Coming soon"> | ||
| ENS-native event-driven reactive development (such as webhooks) is coming soon with [ENSEngine](/docs/services/ensengine) that builds on top of [ENSDb](/docs/services/ensdb). | ||
| </Aside> | ||
|
|
||
| ## The problem | ||
|
|
||
| Today, if you want your app to react to ENS state changes, you have two unappealing options: | ||
|
|
||
| - **Poll constantly.** Hammer API endpoints or RPCs every few seconds and discard most responses. Wasteful, inefficient, and scales poorly. | ||
| - **Build your own complex indexer and data pipeline.** Stand up infrastructure to listen to chain events, decode them, and turn them into the ENS-level signals you actually care about. Way too much work for what should be a primitive. | ||
|
|
||
| ## What we're building | ||
|
|
||
| ENSEngine will be an **ENS-native event-driven reactive development service** (for webhooks and more) so your app can subscribe to ENS state changes in ENSDb the same way you'd subscribe to a Stripe or GitHub webhook - except the events speak the language of discrete meaningful ENS actions, not the language of bare-metal raw onchain event transaction logs. | ||
|
|
||
| ```http | ||
| POST https://your-app.example.com/webhooks/ens | ||
| Content-Type: application/json | ||
|
|
||
| { | ||
| "type": "domain.transferred", | ||
| "name": "vitalik.eth", | ||
| ... | ||
| } | ||
| ``` | ||
|
|
||
| The shape above is illustrative only. Full details coming soon. | ||
|
|
||
| ## Example events | ||
|
|
||
| Events will represent meaningful ENS events, such as: | ||
|
|
||
| - `domain.registered`, `domain.renewed`, `domain.transferred` | ||
| - `resolver.address_changed`, `resolver.text_changed`, `resolver.contenthash_changed` | ||
| - `subname.created` | ||
| - ... and many more. | ||
|
|
||
| ## What this unlocks | ||
|
|
||
| ### Cache invalidation | ||
|
|
||
| Build aggressive, edge-friendly caches for ENS data. The goal is to make it practical to cache profiles, avatars, text records, and resolver responses for much longer than polling-based integrations allow, then invalidate them when relevant onchain state changes. That's the foundation for web2-grade UX on a web3 protocol - fast page loads, low RPC fanout, globally distributed reads - without giving up freshness. | ||
|
|
||
| Of course, ENSNode will also include metadata letting you know key attributes about ENS data you fetch so that you can wisely implement the best possible caching strategies. For example: If a particular data record is coming from purely onchain data (that will receive a guaranteed ENSEngine event notification as soon as it changes) or if a particular data record is coming from offchain data (such as a resolver record sourced through CCIP-read) and that therefore still requires polling to identify if it has changed. | ||
|
djstrong marked this conversation as resolved.
|
||
|
|
||
|
djstrong marked this conversation as resolved.
|
||
| Where this matters: | ||
|
|
||
| - Frontend apps caching ENS profiles | ||
| - CDNs serving ENS-driven content | ||
| - API gateways with ENS lookups | ||
| - Any service that caches resolver records or other ENS data. | ||
|
|
||
| ### Additional key use cases | ||
|
|
||
| Once a reliable change feed exists, a lot of follow-on use cases get dramatically easier to build: | ||
|
|
||
| - **Notification services** - expiry reminders, watchlists, security alerts on unexpected transfers, activity bots in Discord/Telegram/Twitter. | ||
| - **Sync to alternative databases** - want to store alternative representations of ENSDb in databases that provide specialized benefits that Postgres doesn't offer, such as Elasticsearch, Typesense, Meilisearch, Redis, Kafka, Google BigQuery, and more? ENSEngine will ensure 100% delivery of database changes to sinks with strict ordering and exactly-once processing. | ||
|
djstrong marked this conversation as resolved.
djstrong marked this conversation as resolved.
|
||
| - **Realtime dashboards** - surface ENS analytics and activity data as it happens. | ||
|
djstrong marked this conversation as resolved.
|
||
|
|
||
| ## How it works (high level) | ||
|
|
||
| ENSEngine builds on solid foundations: | ||
|
|
||
| 1. **[ENSIndexer](/docs/services/ensindexer)** keeps **[ENSDb](/docs/services/ensdb)** up to date with the live, onchain state of ENS - including automatic chain reorg healing. | ||
| 2. **[ENSEngine](/docs/services/ensengine)** watches that database for changes and turns them into enriched, filtered, ENS-aware events. | ||
| 3. Those events get delivered to **your sinks** - webhooks, cache invalidation hooks, database sync targets, notification systems, and more. | ||
|
|
||
| For the architecture overview, see [ENSEngine Overview](/docs/services/ensengine). | ||
|
|
||
| ## Related | ||
|
|
||
| <LinkCard | ||
| title="ENSEngine Overview" | ||
| description="Learn more about ENSEngine." | ||
| href="/docs/services/ensengine" | ||
| /> | ||
|
djstrong marked this conversation as resolved.
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.