Skip to content

ADP-1490: add new Prometheus client for Flink scrape compatibility and wire the existing collector to that #149

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
varkey98 merged 14 commits into from
Aug 3, 2026
Merged

ADP-1490: add new Prometheus client for Flink scrape compatibility and wire the existing collector to that #149

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
dac2b03
update to use new prometheus client
varkey98 Jul 30, 2026
b196d1b
adding metrics wrappers
varkey98 Jul 30, 2026
596aff3
updating readmes
varkey98 Jul 30, 2026
91cc9dd
remove metric wrappers
varkey98 Jul 31, 2026
8da6316
update more deprecated annotations
varkey98 Aug 2, 2026
20c41f6
update package name
varkey98 Aug 2, 2026
3409880
add a note
varkey98 Aug 2, 2026
add586f
update name
varkey98 Aug 2, 2026
0825fbf
revert unwanted changes
varkey98 Aug 2, 2026
c2f15f1
revert upgrade
varkey98 Aug 2, 2026
7112200
update comment
varkey98 Aug 2, 2026
eef5372
add a fixme comment
varkey98 Aug 2, 2026
2d1dd11
empty commit to retrigger workflows
varkey98 Aug 2, 2026
66589cc
merge from main
varkey98 Aug 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,8 @@

# Ignore Gradle build output directory
build
bin
.DS_Store
.idea

settings-gradle.lockfile
17 changes: 15 additions & 2 deletions docstore-metrics/gradle.lockfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,26 @@ io.micrometer:micrometer-commons:1.14.4=compileClasspath,runtimeClasspath,testCo
io.micrometer:micrometer-core:1.14.4=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.micrometer:micrometer-observation:1.14.4=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.micrometer:micrometer-registry-prometheus-simpleclient:1.14.4=runtimeClasspath,testRuntimeClasspath
io.micrometer:micrometer-registry-prometheus:1.14.4=runtimeClasspath,testRuntimeClasspath
io.netty:netty-bom:4.1.136.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-config:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-core:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exporter-common:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exporter-httpserver:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exposition-formats:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exposition-textformats:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-instrumentation-dropwizard5:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-instrumentation-dropwizard:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-model:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-simpleclient-bridge:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-tracer-common:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-tracer-initializer:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-tracer-otel-agent:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-tracer-otel:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient:0.16.0=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient_common:0.16.0=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient_dropwizard:0.16.0=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient_pushgateway:0.16.0=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient_servlet_common:0.16.0=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient_servlet_jakarta:0.16.0=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient_tracer_common:0.16.0=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient_tracer_otel:0.16.0=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient_tracer_otel_agent:0.16.0=runtimeClasspath,testRuntimeClasspath
Expand Down
9 changes: 8 additions & 1 deletion gradle/libs.versions.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
dropwizard-metrics = "4.2.25"
micrometer = "1.14.4"
prometheus-simpleclient = "0.16.0"
# Aligns with the io.prometheus:* transitives pulled in by micrometer-registry-prometheus.
prometheus = "1.3.5"
caffeine = "3.1.8"

[libraries]
Expand All @@ -12,10 +14,15 @@ dropwizard-metrics-jakarta-servlets = { module = "io.dropwizard.metrics:metrics-
dropwizard-metrics-jvm = { module = "io.dropwizard.metrics:metrics-jvm", version.ref = "dropwizard-metrics" }
micrometer-core = { module = "io.micrometer:micrometer-core", version.ref = "micrometer" }
micrometer-registry-prometheus-simpleclient = { module = "io.micrometer:micrometer-registry-prometheus-simpleclient", version.ref = "micrometer" }
micrometer-registry-prometheus = { module = "io.micrometer:micrometer-registry-prometheus", version.ref = "micrometer" }
prometheus-exporter-httpserver = { module = "io.prometheus:prometheus-metrics-exporter-httpserver", version.ref = "prometheus" }
prometheus-exporter-servlet-jakarta = { module = "io.prometheus:prometheus-metrics-exporter-servlet-jakarta", version.ref = "prometheus" }
prometheus-instrumentation-dropwizard = { module = "io.prometheus:prometheus-metrics-instrumentation-dropwizard", version.ref = "prometheus" }
prometheus-simpleclient-bridge = { module = "io.prometheus:prometheus-metrics-simpleclient-bridge", version.ref = "prometheus" }
micrometer-jvm-extras = { module = "io.github.mweirauch:micrometer-jvm-extras", version = "0.2.2" }
prometheus-simpleclient-dropwizard = { module = "io.prometheus:simpleclient_dropwizard", version.ref = "prometheus-simpleclient" }
prometheus-simpleclient-servlet-jakarta = { module = "io.prometheus:simpleclient_servlet_jakarta", version.ref = "prometheus-simpleclient" }
prometheus-simpleclient-pushgateway = { module = "io.prometheus:simpleclient_pushgateway", version.ref = "prometheus-simpleclient" }
prometheus-simpleclient-dropwizard = { module = "io.prometheus:simpleclient_dropwizard", version.ref = "prometheus-simpleclient" }
caffeine = { module = "com.github.ben-manes.caffeine:caffeine", version.ref = "caffeine" }
apache-httpcomponents-httpclient = { module = "org.apache.httpcomponents:httpclient", version = "4.5.13" }

Expand Down
16 changes: 16 additions & 0 deletions integrationtest-service-framework/gradle.lockfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,23 @@ io.micrometer:micrometer-commons:1.14.4=runtimeClasspath,testRuntimeClasspath
io.micrometer:micrometer-core:1.14.4=runtimeClasspath,testRuntimeClasspath
io.micrometer:micrometer-observation:1.14.4=runtimeClasspath,testRuntimeClasspath
io.micrometer:micrometer-registry-prometheus-simpleclient:1.14.4=runtimeClasspath,testRuntimeClasspath
io.micrometer:micrometer-registry-prometheus:1.14.4=runtimeClasspath,testRuntimeClasspath
io.netty:netty-bom:4.1.136.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-config:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-core:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exporter-common:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exporter-httpserver:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exporter-servlet-jakarta:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exposition-formats:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exposition-textformats:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-instrumentation-dropwizard5:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-instrumentation-dropwizard:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-model:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-simpleclient-bridge:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-tracer-common:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-tracer-initializer:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-tracer-otel-agent:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-tracer-otel:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient:0.16.0=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient_common:0.16.0=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient_dropwizard:0.16.0=runtimeClasspath,testRuntimeClasspath
Expand Down
16 changes: 16 additions & 0 deletions platform-grpc-service-framework/gradle.lockfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ io.micrometer:micrometer-commons:1.14.4=compileClasspath,runtimeClasspath,testCo
io.micrometer:micrometer-core:1.14.4=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.micrometer:micrometer-observation:1.14.4=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.micrometer:micrometer-registry-prometheus-simpleclient:1.14.4=runtimeClasspath,testRuntimeClasspath
io.micrometer:micrometer-registry-prometheus:1.14.4=runtimeClasspath,testRuntimeClasspath
io.netty:netty-bom:4.1.136.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-buffer:4.1.136.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-codec-http2:4.1.136.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
Expand All @@ -59,6 +60,21 @@ io.netty:netty-resolver:4.1.136.Final=compileClasspath,runtimeClasspath,testComp
io.netty:netty-transport-native-unix-common:4.1.136.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-transport:4.1.136.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.perfmark:perfmark-api:0.27.0=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-config:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-core:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exporter-common:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exporter-httpserver:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exporter-servlet-jakarta:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exposition-formats:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exposition-textformats:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-instrumentation-dropwizard5:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-instrumentation-dropwizard:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-model:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-simpleclient-bridge:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-tracer-common:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-tracer-initializer:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-tracer-otel-agent:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-tracer-otel:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient:0.16.0=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient_common:0.16.0=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient_dropwizard:0.16.0=runtimeClasspath,testRuntimeClasspath
Expand Down
16 changes: 16 additions & 0 deletions platform-http-service-framework/gradle.lockfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,24 @@ io.micrometer:micrometer-commons:1.14.4=compileClasspath,runtimeClasspath,testCo
io.micrometer:micrometer-core:1.14.4=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.micrometer:micrometer-observation:1.14.4=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.micrometer:micrometer-registry-prometheus-simpleclient:1.14.4=runtimeClasspath,testRuntimeClasspath
io.micrometer:micrometer-registry-prometheus:1.14.4=runtimeClasspath,testRuntimeClasspath
io.netty:netty-bom:4.1.136.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.perfmark:perfmark-api:0.27.0=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-config:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-core:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exporter-common:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exporter-httpserver:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exporter-servlet-jakarta:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exposition-formats:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exposition-textformats:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-instrumentation-dropwizard5:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-instrumentation-dropwizard:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-model:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-simpleclient-bridge:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-tracer-common:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-tracer-initializer:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-tracer-otel-agent:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-tracer-otel:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient:0.16.0=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient_common:0.16.0=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient_dropwizard:0.16.0=runtimeClasspath,testRuntimeClasspath
Expand Down
16 changes: 16 additions & 0 deletions platform-hybrid-service-framework/gradle.lockfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ io.micrometer:micrometer-commons:1.14.4=runtimeClasspath,testRuntimeClasspath
io.micrometer:micrometer-core:1.14.4=runtimeClasspath,testRuntimeClasspath
io.micrometer:micrometer-observation:1.14.4=runtimeClasspath,testRuntimeClasspath
io.micrometer:micrometer-registry-prometheus-simpleclient:1.14.4=runtimeClasspath,testRuntimeClasspath
io.micrometer:micrometer-registry-prometheus:1.14.4=runtimeClasspath,testRuntimeClasspath
io.netty:netty-bom:4.1.136.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-buffer:4.1.136.Final=runtimeClasspath,testRuntimeClasspath
io.netty:netty-codec-http2:4.1.136.Final=runtimeClasspath,testRuntimeClasspath
Expand All @@ -63,6 +64,21 @@ io.netty:netty-resolver:4.1.136.Final=runtimeClasspath,testRuntimeClasspath
io.netty:netty-transport-native-unix-common:4.1.136.Final=runtimeClasspath,testRuntimeClasspath
io.netty:netty-transport:4.1.136.Final=runtimeClasspath,testRuntimeClasspath
io.perfmark:perfmark-api:0.27.0=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-config:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-core:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exporter-common:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exporter-httpserver:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exporter-servlet-jakarta:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exposition-formats:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-exposition-textformats:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-instrumentation-dropwizard5:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-instrumentation-dropwizard:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-model:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-simpleclient-bridge:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-tracer-common:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-tracer-initializer:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-tracer-otel-agent:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:prometheus-metrics-tracer-otel:1.3.5=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient:0.16.0=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient_common:0.16.0=runtimeClasspath,testRuntimeClasspath
io.prometheus:simpleclient_dropwizard:0.16.0=runtimeClasspath,testRuntimeClasspath
Expand Down
5 changes: 5 additions & 0 deletions platform-metrics/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,3 +42,8 @@ counter.increment();
## References
This library uses MicroMeter for metrics, and the API of MicroMeter is exposed directly
to leverage the power of it. See https://micrometer.io/docs for more details.

For Flink TaskManagers, `SharedMeterRegistry.getPrometheusRegistry(serviceName, port)`
binds the process scrape endpoint and returns the Prometheus registry so a Flink metric
reporter can bridge Flink-native metrics onto the same endpoint. Application meters still
go through `PlatformMetricsRegistry.getMeterRegistry()`.
21 changes: 18 additions & 3 deletions platform-metrics/build.gradle.kts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,18 +15,33 @@ dependencies {
api(localLibs.micrometer.core)
api(localLibs.jakarta.servlet.api)

// New Prometheus client stack (io.micrometer.prometheusmetrics / io.prometheus.metrics).
implementation(localLibs.micrometer.registry.prometheus)
// Standalone HTTP server that exposes the Prometheus scrape endpoint (SharedMeterRegistry).
implementation(localLibs.prometheus.exporter.httpserver)
// Bridges Dropwizard MetricRegistry into the new PrometheusRegistry.
implementation(localLibs.prometheus.instrumentation.dropwizard)
// Bridges the legacy simpleclient CollectorRegistry onto the new PrometheusRegistry.
implementation(localLibs.prometheus.simpleclient.bridge)

// Legacy simpleclient stack — still required by PrometheusPushMeterRegistry
// (io.micrometer.prometheus.PrometheusMeterRegistry + PushGateway).
implementation(localLibs.micrometer.registry.prometheus.simpleclient)
implementation(localLibs.prometheus.simpleclient.pushgateway)
// Bridges Dropwizard MetricRegistry into the legacy simpleclient CollectorRegistry.
implementation(localLibs.prometheus.simpleclient.dropwizard)

implementation(localLibs.micrometer.jvm.extras)
implementation(commonLibs.slf4j2.api)
implementation(localLibs.dropwizard.metrics.jvm)
implementation(localLibs.prometheus.simpleclient.dropwizard)
implementation(localLibs.prometheus.simpleclient.servlet.jakarta)
implementation(localLibs.prometheus.simpleclient.pushgateway)
implementation(commonLibs.jetty.servlet)
implementation(commonLibs.guava)

compileOnly(localLibs.caffeine)

annotationProcessor(commonLibs.lombok)
compileOnly(commonLibs.lombok)

testImplementation(commonLibs.junit.jupiter)
testImplementation(commonLibs.mockito.core)
testImplementation(localLibs.caffeine)
Expand Down
Loading
Loading