Resource Managers: durable metering for the CRE and other products #2158
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,157 @@ | ||
| package resourcemanager | ||
|
|
||
| import ( | ||
| "context" | ||
|
|
||
| meteringpb "github.com/smartcontractkit/chainlink-protos/metering/go" | ||
| ) | ||
|
|
||
| // Meterable is implemented by producers that manage durable billable | ||
| // resources (trigger registrations, workflow specs, log filters). A producer | ||
| // registers itself with a ResourceManager (see ResourceManager.Register) so it | ||
| // is polled once per snapshot tick for the absolute state of its currently | ||
| // active resources, in addition to emitting lifecycle edges inline via | ||
| // EmitMeterRecord. | ||
| type Meterable interface { | ||
| // ResourceIdentity returns the producer's base identity: the coarse | ||
| // dimensions (product, tenant, numeric_tenant_id, environment, zone, don, service) plus | ||
| // the service-level resource_pool / resource_pool_id. Per-resource billing | ||
| // fields (resource_type/resource_id/org_id/event_id/value) are carried by | ||
| // Utilizations on MeterRecord and MeterSnapshot. | ||
| ResourceIdentity() ResourceIdentity | ||
|
|
||
| // GetUtilization returns the current level of the producer's currently | ||
| // active resources, one SnapshotEntry per resource. The manager emits one | ||
| // MeterSnapshot per entry. | ||
| // | ||
| // It is called on the snapshot tick and MUST be a cheap, non-blocking | ||
| // read-snapshot of in-memory state: no network, no disk, no lock held | ||
| // across I/O. It must tolerate ctx cancellation (returning promptly, and | ||
| // nil/empty is acceptable) and tolerate concurrent registration of new | ||
| // resources. An empty or nil return is valid and means nothing is currently | ||
| // active: no snapshots are emitted, and billing zeroes the resource out by | ||
| // its absence from subsequent snapshots. | ||
| GetUtilization(ctx context.Context) []SnapshotEntry | ||
| } | ||
|
|
||
| // SnapshotEntry is the current level of one active resource at a snapshot tick. | ||
| // Identity is the base resource identity, and Utilizations carries one or more | ||
| // billed dimensions for that resource (resource_type/resource_id/org_id/event_id/value). | ||
| type SnapshotEntry struct { | ||
| Identity ResourceIdentity | ||
| Utilizations []*meteringpb.Utilization | ||
| } | ||
|
|
||
| // DeploymentIdentity carries the static deployment + node identity dimensions | ||
| // that are fixed for a LOOP plugin process. They are resolved once from node | ||
| // config by the host and delivered to every LOOP plugin over the environment | ||
| // (loop.EnvConfig), not the standard-capabilities boundary, so any LOOP plugin | ||
| // can populate the coarse metering rollup dimensions. Any field may be empty if | ||
| // the host did not provide it. | ||
| type DeploymentIdentity struct { | ||
|
Contributor
There was a problem hiding this comment. Where does this get used? Are the env vars meant to plug into this struct?
Contributor
Author
There was a problem hiding this comment. yes, the env vars plug into, passed over to the loop via env. They should then be set on the ResourceManagers config so that they can be sourced into the emission of each event. |
||
| // Product is the deployment product, e.g. "cre". | ||
| Product string | ||
| // Tenant is the human-readable deployment tenant name, e.g. "mainline" or | ||
| // "enterprise". | ||
| Tenant string | ||
| // NumericTenantID is the numbered tenant identifier as a string. | ||
| NumericTenantID string | ||
| // Environment is the deployment environment, e.g. "production", "staging". | ||
| Environment string | ||
| // Zone is the deployment zone, e.g. "wf-zone-a". | ||
| Zone string | ||
| // NodeID is the node's logical name, e.g. "clp-cre-wf-zone-a-1". It is NOT | ||
| // the CSA public key; it is a stable name the billing service can use to | ||
| // look up the node's CSA key in the workflow registry. The CSA key itself is | ||
| // carried separately as the node_csa_key event attribute. | ||
| NodeID string | ||
| } | ||
|
|
||
| // DonIdentity captures DON-specific identity dimensions as one unit. | ||
| type DonIdentity struct { | ||
| // DonID is the DON ID the emitting service belongs to. | ||
| DonID string | ||
| // NodeID is the node's logical name within the scope of the DON, e.g. | ||
| // "clp-cre-wf-zone-a-1". It is a human-readable ID, NOT the CSA | ||
| // public key. The prefix can be redundant with other fully-qualified | ||
| // dimensions, but helps readability. The CSA key is emitted separately via | ||
| // the node_csa_key attribute. | ||
| NodeID string | ||
| } | ||
|
|
||
| // ResourceIdentity is the structured, first-class identity of a durable | ||
| // resource. Its fields map 1:1 to metering.v1.ResourceIdentity so every | ||
| // emitted record carries each dimension as a discrete column rather than a | ||
| // parsed dotted string or out-of-band telemetry attribute. | ||
| type ResourceIdentity struct { | ||
| // Product is the deployment product, e.g. "cre". | ||
| Product string | ||
|
|
||
| // Tenant is the human-readable deployment tenant name, e.g. "mainline" or | ||
| // "enterprise". | ||
| Tenant string | ||
|
|
||
| // NumericTenantID is the numbered tenant identifier as a string. | ||
| NumericTenantID string | ||
|
|
||
| // Environment is the deployment environment, e.g. "production", | ||
| // "staging". | ||
| Environment string | ||
|
|
||
| // Zone is the deployment zone, e.g. "wf-zone-a". | ||
| Zone string | ||
|
|
||
| // Don groups DON-specific identity dimensions so consumers can | ||
| // branch on one struct instead of handling don/node permutations. | ||
| Don *DonIdentity | ||
|
|
||
| // Service is the stable service constant identifying the emitting service, | ||
| // e.g. "cron-trigger", "http-trigger", "evm-log-trigger", | ||
| // "workflow-syncer-v2". It must not | ||
| // encode deployment environment or zone. | ||
| Service string | ||
|
|
||
| // ResourcePool is the service-level resource pool the record applies to, | ||
| // e.g. "trigger_registrations", "log_filters", "workflow_storage". | ||
| ResourcePool string | ||
|
|
||
| // ResourcePoolID optionally scopes identity further within the resource pool. | ||
| ResourcePoolID string | ||
| } | ||
|
|
||
| // toProto converts r to its wire form. Field order mirrors the proto. | ||
| func (r ResourceIdentity) toProto() *meteringpb.ResourceIdentity { | ||
| pb := &meteringpb.ResourceIdentity{ | ||
| Product: r.Product, | ||
| Tenant: r.Tenant, | ||
| NumericTenantId: r.NumericTenantID, | ||
| Environment: r.Environment, | ||
| Zone: r.Zone, | ||
| Service: r.Service, | ||
| ResourcePool: r.ResourcePool, | ||
| ResourcePoolId: r.ResourcePoolID, | ||
| } | ||
| if r.Don != nil { | ||
| pb.Don = &meteringpb.DonIdentity{ | ||
| DonId: r.Don.DonID, | ||
| NodeId: r.Don.NodeID, | ||
| } | ||
| } | ||
| return pb | ||
| } | ||
|
|
||
| // DonID returns the DON ID when present. | ||
| func (r ResourceIdentity) DonID() string { | ||
| if r.Don == nil { | ||
| return "" | ||
| } | ||
| return r.Don.DonID | ||
| } | ||
|
|
||
| // NodeID returns the node ID when present. | ||
| func (r ResourceIdentity) NodeID() string { | ||
| if r.Don == nil { | ||
| return "" | ||
| } | ||
| return r.Don.NodeID | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note to reader, DonID isn't here b/c a node can belong to many DONs; it is the responsibility of the loop to properly receive its DonID from
Initialisein the standard cap interface (or for non-CRE use cases, elsewhere)