Skip to content

Add cluster metrics listener for observability (node count, connections)#18

Merged
zuchmanski merged 1 commit into
v5-developfrom
feature/cluster-metrics-listener
Jul 2, 2026
Merged

Add cluster metrics listener for observability (node count, connections)#18
zuchmanski merged 1 commit into
v5-developfrom
feature/cluster-metrics-listener

Conversation

@zuchmanski

Copy link
Copy Markdown

What

Adds an optional, dependency-free metrics hook to the client so applications can export cluster/node statistics (node count, per-node pooled connections, failures) to Datadog/StatsD/Prometheus/etc.

The client emits a fresh snapshot once per tend interval (default 1s) — including on cycles where the node set did not change — so downstream gauges stay alive between changes. No more scraping "Tend finished. N nodes have joined the cluster" log lines.

How to integrate

  1. Implement a reporter — any object responding to report(cluster_stats):
class AerospikeDatadogReporter
  def initialize(statsd)
    @statsd = statsd
  end

  # Called from the tend thread, once per tend interval.
  def report(stats)
    @statsd.gauge("aerospike.cluster.nodes", stats.node_count)
    @statsd.gauge("aerospike.cluster.open_connections", stats.open_connections)

    stats.nodes.each do |node|
      tags = ["node:#{node.name}"]
      @statsd.gauge("aerospike.node.available_connections", node.available_connections, tags: tags)
      @statsd.gauge("aerospike.node.failures", node.failures, tags: tags)
      @statsd.gauge("aerospike.node.active", node.active? ? 1 : 0, tags: tags)
    end
  end
end
  1. Pass it via the :metrics_listener client policy option:
require "datadog/statsd"

client = Aerospike::Client.new(
  hosts,
  policy: { metrics_listener: AerospikeDatadogReporter.new(Datadog::Statsd.new) }
)

That's it. #report will be called every tend interval with a ClusterStats snapshot.

Snapshot API

ClusterStats:

  • node_count → Integer
  • node_names → Array
  • open_connections → Integer (sum of idle pooled connections across nodes)
  • nodes → Array<NodeStats>
  • to_h → Hash (handy for JSON logging)

NodeStats:

  • name, available_connections, failures, active?

Notes / tradeoffs

  • No new gem dependencies. The app supplies the reporter; the client stays free of dogstatsd. This is the conventional way client libs expose observability.
  • available_connections = idle pooled connections per node (node.connections.length). The pool doesn't track in-use count, so this is the closest signal without deeper changes.
  • #report runs inline on the tend thread — keep it fast/non-blocking (a StatsD UDP send is fine). If heavy work is needed, push it to your own queue/thread inside the reporter.
  • Listener exceptions are caught and logged ("Exception in metrics listener: ..."); a broken reporter cannot disrupt cluster tending.
  • Duck-typed: implementing Aerospike::MetricsListener is optional and only documents the contract.

Changes

  • lib/aerospike/cluster_stats.rb — new ClusterStats / NodeStats value objects
  • lib/aerospike/metrics_listener.rb — new listener contract (docs)
  • lib/aerospike/policy/client_policy.rb — new metrics_listener option
  • lib/aerospike/cluster.rb#report_metrics / #cluster_stats, invoked each tend cycle
  • lib/aerospike.rb — require the new files
  • Specs: spec/aerospike/cluster_stats_spec.rb + #report_metrics coverage in cluster_spec.rb

Testing

bundle exec rspec spec/aerospike/cluster_stats_spec.rb spec/aerospike/cluster_spec.rb → 30 examples, 0 failures. (The suite-level after(:suite) error requires a running Aerospike server and is unrelated to this change.)

🤖 Generated with Claude Code

Expose cluster/node statistics (node count, per-node pooled connection
and failure counts) via an optional, dependency-free metrics listener.

Register any object responding to `report(cluster_stats)` through the new
`ClientPolicy#metrics_listener` option. Its `#report` is invoked from the
tend thread once per tend interval with a fresh `ClusterStats` snapshot --
including on unchanged cycles -- so downstream gauges (e.g. Datadog) stay
alive. Listener exceptions are caught and logged so a broken reporter
cannot disrupt tending.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@zuchmanski
zuchmanski requested a review from madejejej July 2, 2026 12:08
@zuchmanski
zuchmanski marked this pull request as ready for review July 2, 2026 12:08
@zuchmanski
zuchmanski merged commit df85a5d into v5-develop Jul 2, 2026
4 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants