diff --git a/components-starter/camel-observability-services-starter/src/main/doc/intro.adoc b/components-starter/camel-observability-services-starter/src/main/doc/intro.adoc new file mode 100644 index 00000000000..3038cd5a6b9 --- /dev/null +++ b/components-starter/camel-observability-services-starter/src/main/doc/intro.adoc @@ -0,0 +1,8 @@ +Spring Boot auto-configuration bundling the Camel observability services: metrics, tracing and health. + +The starter pulls in `camel-micrometer`, `micrometer-registry-prometheus`, `camel-opentelemetry2` and +`camel-management`, and contributes a set of defaults so that a Camel application exposes a Prometheus +scrape endpoint and Kubernetes-shaped liveness and readiness probes without any configuration. + +The defaults are registered as the lowest precedence property source, so anything you set in +`application.properties`, in an environment variable or on the command line overrides them. diff --git a/components-starter/camel-observability-services-starter/src/main/doc/usage.adoc b/components-starter/camel-observability-services-starter/src/main/doc/usage.adoc new file mode 100644 index 00000000000..bba4aec5c42 --- /dev/null +++ b/components-starter/camel-observability-services-starter/src/main/doc/usage.adoc @@ -0,0 +1,89 @@ +Add the starter to the classpath and the endpoints below are available on a separate management listener, +bound to loopback on port `9876`: + +[width="100%",cols="3,5",options="header"] +|=== +| Endpoint | Purpose +| `http://127.0.0.1:9876/observe/health` | Aggregate health +| `http://127.0.0.1:9876/observe/health/live` | Kubernetes liveness probe +| `http://127.0.0.1:9876/observe/health/ready` | Kubernetes readiness probe +| `http://127.0.0.1:9876/observe/metrics` | Prometheus scrape endpoint +|=== + +=== Injected defaults + +This is the full set of properties the starter contributes. Every one of them can be overridden by your own +configuration: + +[width="100%",cols="4,2,4",options="header"] +|=== +| Property | Value | Notes +| `management.server.port` | `9876` | Management endpoints run on their own listener, separate from the application port. +| `management.server.address` | `127.0.0.1` | The listener binds to loopback. See <>. +| `management.endpoints.web.exposure.include` | `health,prometheus` | Only these two endpoints are exposed. +| `management.endpoints.web.base-path` | `/observe` | Replaces the `/actuator` base path. +| `management.endpoints.web.path-mapping.prometheus` | `metrics` | Serves the Prometheus endpoint at `/observe/metrics`. +| `camel.metrics.log-metrics-on-shutdown` | `true` | Dumps a metrics summary to the log when the application stops. +| `camel.metrics.log-metrics-on-shutdown-filters` | `app.info,camel.exchanges.*,process.cpu.usage,jvm.memory.max,jvm.memory.used` | Which metrics that summary contains. +| `camel.metrics.log-metrics-on-shutdown-format` | `prometheus` | Format of that summary. +| `camel.opentelemetry2.enabled` | `true` | Enables the OpenTelemetry tracing instrumentation. +| `management.endpoint.health.probes.enabled` | `true` | Enables the liveness and readiness health groups. +| `management.health.livenessState.enabled` | `true` | Registers the `livenessState` indicator. +| `management.health.readinessState.enabled` | `true` | Registers the `readinessState` indicator. +| `management.endpoint.health.show-details` | `when-authorized` | See <>. +| `management.endpoint.health.group.live.include` | `livenessState,camelLivenessState` | Contents of `/observe/health/live`. +| `management.endpoint.health.group.live.show-details` | `always` | See <>. +| `management.endpoint.health.group.ready.include` | `readinessState,camelReadinessState` | Contents of `/observe/health/ready`. +| `management.endpoint.health.group.ready.show-details` | `always` | See <>. +|=== + +=== Exposing the management listener + +The management listener binds to `127.0.0.1`, so out of the box the health and metrics endpoints are reachable +from the same host only. This matches the Spring Boot baseline, which ships no separate management listener at +all: opening one on every interface should be a decision you make, not one the starter makes for you. + +A Kubernetes deployment whose kubelet probes or Prometheus scrapers reach the pod over the network has to widen +the bind address: + +[source,properties] +---- +management.server.address = 0.0.0.0 +---- + +Pair that with a `NetworkPolicy`, or with authentication in front of the management port, so the endpoints are +only reachable from your probes and your scrapers. + +=== Health detail exposure + +`/observe/health` uses `when-authorized`, the setting that shows the individual health indicators to an +authenticated caller and the overall status alone to everybody else. Camel health checks report on the +resources a route talks to — broker connections, data sources, remote endpoints — and their detail can +identify those resources, so it is not something to hand to an unauthenticated caller. With no Spring Security +on the classpath there is no authenticated caller, and the endpoint behaves as `never`. + +To restore the previous behaviour of showing the details to anyone who asks: + +[source,properties] +---- +management.endpoint.health.show-details = always +---- + +The `live` and `ready` groups keep `always`. The kubelet reaches them unauthenticated, and it puts the response +body into the probe-failure event, so `kubectl describe pod` names the indicator that took the pod down. Both +groups contain availability-state indicators only — `livenessState`, `readinessState` and their Camel +counterparts report a status and carry no data — so there is nothing in those responses beyond the indicator +names and their state. + +=== Camel health exposure level + +`camel.health.exposure-level` is left at its Camel default of `default`, which filters health check metadata — +endpoint URIs, route and consumer identifiers — out of the health response while keeping the check names, +error messages and stack traces that tell you what failed. + +If you want the metadata as well, opt in: + +[source,properties] +---- +camel.health.exposure-level = full +---- diff --git a/components-starter/camel-observability-services-starter/src/main/java/org/apache/camel/observability/services/springboot/ObservabilityServicesEnvironmentPostProcessor.java b/components-starter/camel-observability-services-starter/src/main/java/org/apache/camel/observability/services/springboot/ObservabilityServicesEnvironmentPostProcessor.java index cea919ed3a8..ff0d1cfa9c3 100644 --- a/components-starter/camel-observability-services-starter/src/main/java/org/apache/camel/observability/services/springboot/ObservabilityServicesEnvironmentPostProcessor.java +++ b/components-starter/camel-observability-services-starter/src/main/java/org/apache/camel/observability/services/springboot/ObservabilityServicesEnvironmentPostProcessor.java @@ -29,6 +29,11 @@ * Contributes the opinionated observability defaults as the lowest precedence property source so that any * user-provided configuration (application.properties, environment variables, system properties, ...) overrides them * following the standard Spring Boot precedence rules. + *

+ * The defaults stay within the Spring Boot baseline: the management listener binds to loopback, and the aggregate + * health endpoint only shows its details to an authorized caller. The {@code live} and {@code ready} probe groups + * keep {@code show-details=always} because they are consumed unauthenticated by the kubelet, and the indicators + * they contain report an availability state and nothing else. */ public class ObservabilityServicesEnvironmentPostProcessor implements EnvironmentPostProcessor, Ordered { @@ -38,6 +43,8 @@ public class ObservabilityServicesEnvironmentPostProcessor implements Environmen public void postProcessEnvironment(ConfigurableEnvironment environment, SpringApplication application) { Map defaults = new LinkedHashMap<>(); defaults.put("management.server.port", "9876"); + // bind the management listener to loopback; exposing it beyond the host is a conscious step + defaults.put("management.server.address", "127.0.0.1"); defaults.put("management.endpoints.web.exposure.include", "health,prometheus"); defaults.put("management.endpoints.web.base-path", "/observe"); defaults.put("management.endpoints.web.path-mapping.prometheus", "metrics"); @@ -48,15 +55,15 @@ public void postProcessEnvironment(ConfigurableEnvironment environment, SpringAp // Opentelemetry defaults.put("camel.opentelemetry2.enabled", "true"); // Health - defaults.put("camel.health.exposure-level", "full"); defaults.put("management.endpoint.health.probes.enabled", "true"); defaults.put("management.health.readinessState.enabled", "true"); defaults.put("management.health.livenessState.enabled", "true"); - defaults.put("management.endpoint.health.show-details", "always"); - // /observe/health/live remap + defaults.put("management.endpoint.health.show-details", "when-authorized"); + // /observe/health/live remap; the group carries availability states only, and the kubelet reads it + // unauthenticated, so its details stay visible defaults.put("management.endpoint.health.group.live.include", "livenessState,camelLivenessState"); defaults.put("management.endpoint.health.group.live.show-details", "always"); - // /observe/health/ready remap + // /observe/health/ready remap; see the note on the live group above defaults.put("management.endpoint.health.group.ready.include", "readinessState,camelReadinessState"); defaults.put("management.endpoint.health.group.ready.show-details", "always"); diff --git a/components-starter/camel-observability-services-starter/src/test/java/org/apache/camel/observability/services/springboot/ObservabilityServicesEnvironmentPostProcessorTest.java b/components-starter/camel-observability-services-starter/src/test/java/org/apache/camel/observability/services/springboot/ObservabilityServicesEnvironmentPostProcessorTest.java index 77c4c623960..d27752f7736 100644 --- a/components-starter/camel-observability-services-starter/src/test/java/org/apache/camel/observability/services/springboot/ObservabilityServicesEnvironmentPostProcessorTest.java +++ b/components-starter/camel-observability-services-starter/src/test/java/org/apache/camel/observability/services/springboot/ObservabilityServicesEnvironmentPostProcessorTest.java @@ -16,12 +16,18 @@ */ package org.apache.camel.observability.services.springboot; +import java.util.Arrays; +import java.util.LinkedHashSet; +import java.util.Set; + import org.junit.jupiter.api.Test; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.core.env.ConfigurableEnvironment; +import org.springframework.core.env.EnumerablePropertySource; import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNull; import static org.junit.jupiter.api.Assertions.assertTrue; @SpringBootTest(classes = ObservabilityServicesTestApplication.class) @@ -42,19 +48,68 @@ public void defaultsAreApplied() { environment.getProperty("camel.metrics.log-metrics-on-shutdown-filters")); assertEquals("prometheus", environment.getProperty("camel.metrics.log-metrics-on-shutdown-format")); - assertEquals("full", environment.getProperty("camel.health.exposure-level")); assertEquals("true", environment.getProperty("management.endpoint.health.probes.enabled")); assertEquals("true", environment.getProperty("management.health.readinessState.enabled")); assertEquals("true", environment.getProperty("management.health.livenessState.enabled")); - assertEquals("always", environment.getProperty("management.endpoint.health.show-details")); assertEquals("livenessState,camelLivenessState", environment.getProperty("management.endpoint.health.group.live.include")); - assertEquals("always", environment.getProperty("management.endpoint.health.group.live.show-details")); assertEquals("readinessState,camelReadinessState", environment.getProperty("management.endpoint.health.group.ready.include")); + } + + @Test + public void managementListenerBindsToLoopback() { + // the port alone would put a second listener on every interface + assertEquals("127.0.0.1", environment.getProperty("management.server.address")); + } + + @Test + public void aggregateHealthDetailsRequireAuthorization() { + assertEquals("when-authorized", environment.getProperty("management.endpoint.health.show-details")); + } + + @Test + public void probeGroupsKeepTheirDetails() { + // the kubelet reads these unauthenticated and surfaces the body in a probe-failure event; the groups + // hold availability-state indicators only, which report a status and no data + assertEquals("always", environment.getProperty("management.endpoint.health.group.live.show-details")); assertEquals("always", environment.getProperty("management.endpoint.health.group.ready.show-details")); } + @Test + public void camelHealthExposureLevelIsNotForced() { + // 'full' is an opt-in; left alone, Camel's own 'default' level applies and filters health check metadata + assertNull(environment.getProperty("camel.health.exposure-level")); + } + + @Test + public void injectedPropertiesAreTheDocumentedSet() { + EnumerablePropertySource source = (EnumerablePropertySource) environment.getPropertySources() + .get(ObservabilityServicesEnvironmentPostProcessor.PROPERTY_SOURCE_NAME); + + Set expected = new LinkedHashSet<>(Arrays.asList( + "management.server.port", + "management.server.address", + "management.endpoints.web.exposure.include", + "management.endpoints.web.base-path", + "management.endpoints.web.path-mapping.prometheus", + "camel.metrics.log-metrics-on-shutdown", + "camel.metrics.log-metrics-on-shutdown-filters", + "camel.metrics.log-metrics-on-shutdown-format", + "camel.opentelemetry2.enabled", + "management.endpoint.health.probes.enabled", + "management.health.readinessState.enabled", + "management.health.livenessState.enabled", + "management.endpoint.health.show-details", + "management.endpoint.health.group.live.include", + "management.endpoint.health.group.live.show-details", + "management.endpoint.health.group.ready.include", + "management.endpoint.health.group.ready.show-details")); + + // keep this in sync with the table in src/main/doc/usage.adoc + assertEquals(expected, new LinkedHashSet<>(Arrays.asList(source.getPropertyNames()))); + } + @Test public void userApplicationPropertiesOverrideDefaults() { // these values are set in src/test/resources/application.properties and must win diff --git a/components-starter/camel-observability-services-starter/src/test/java/org/apache/camel/observability/services/springboot/ObservabilityServicesOptInOverridesTest.java b/components-starter/camel-observability-services-starter/src/test/java/org/apache/camel/observability/services/springboot/ObservabilityServicesOptInOverridesTest.java new file mode 100644 index 00000000000..ea51c4a3530 --- /dev/null +++ b/components-starter/camel-observability-services-starter/src/test/java/org/apache/camel/observability/services/springboot/ObservabilityServicesOptInOverridesTest.java @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.camel.observability.services.springboot; + +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.core.env.ConfigurableEnvironment; + +import static org.junit.jupiter.api.Assertions.assertEquals; + +/** + * Verifies that the wider settings the starter no longer ships by default can be opted back into, which is what + * a deployment reached over the pod network has to do. + */ +@SpringBootTest(classes = ObservabilityServicesTestApplication.class, + properties = { + "management.server.address=0.0.0.0", + "management.endpoint.health.show-details=always", + "camel.health.exposure-level=full" }) +public class ObservabilityServicesOptInOverridesTest { + + @Autowired + private ConfigurableEnvironment environment; + + @Test + public void bindAddressCanBeWidened() { + assertEquals("0.0.0.0", environment.getProperty("management.server.address")); + } + + @Test + public void healthDetailsCanBeAlwaysShown() { + assertEquals("always", environment.getProperty("management.endpoint.health.show-details")); + } + + @Test + public void camelHealthExposureLevelCanBeSetToFull() { + assertEquals("full", environment.getProperty("camel.health.exposure-level")); + } +} diff --git a/docs/spring-boot/modules/ROOT/pages/starters/observability-services.adoc b/docs/spring-boot/modules/ROOT/pages/starters/observability-services.adoc index 3a383c14cca..93a054b48c2 100644 --- a/docs/spring-boot/modules/ROOT/pages/starters/observability-services.adoc +++ b/docs/spring-boot/modules/ROOT/pages/starters/observability-services.adoc @@ -3,6 +3,15 @@ = Observability Services :artifactid: camel-observability-services-starter +Spring Boot auto-configuration bundling the Camel observability services: metrics, tracing and health. + +The starter pulls in `camel-micrometer`, `micrometer-registry-prometheus`, `camel-opentelemetry2` and +`camel-management`, and contributes a set of defaults so that a Camel application exposes a Prometheus +scrape endpoint and Kubernetes-shaped liveness and readiness probes without any configuration. + +The defaults are registered as the lowest precedence property source, so anything you set in +`application.properties`, in an environment variable or on the command line overrides them. + == Maven coordinates [source,xml] @@ -13,3 +22,95 @@ ---- +== Usage + +Add the starter to the classpath and the endpoints below are available on a separate management listener, +bound to loopback on port `9876`: + +[width="100%",cols="3,5",options="header"] +|=== +| Endpoint | Purpose +| `http://127.0.0.1:9876/observe/health` | Aggregate health +| `http://127.0.0.1:9876/observe/health/live` | Kubernetes liveness probe +| `http://127.0.0.1:9876/observe/health/ready` | Kubernetes readiness probe +| `http://127.0.0.1:9876/observe/metrics` | Prometheus scrape endpoint +|=== + +=== Injected defaults + +This is the full set of properties the starter contributes. Every one of them can be overridden by your own +configuration: + +[width="100%",cols="4,2,4",options="header"] +|=== +| Property | Value | Notes +| `management.server.port` | `9876` | Management endpoints run on their own listener, separate from the application port. +| `management.server.address` | `127.0.0.1` | The listener binds to loopback. See <>. +| `management.endpoints.web.exposure.include` | `health,prometheus` | Only these two endpoints are exposed. +| `management.endpoints.web.base-path` | `/observe` | Replaces the `/actuator` base path. +| `management.endpoints.web.path-mapping.prometheus` | `metrics` | Serves the Prometheus endpoint at `/observe/metrics`. +| `camel.metrics.log-metrics-on-shutdown` | `true` | Dumps a metrics summary to the log when the application stops. +| `camel.metrics.log-metrics-on-shutdown-filters` | `app.info,camel.exchanges.*,process.cpu.usage,jvm.memory.max,jvm.memory.used` | Which metrics that summary contains. +| `camel.metrics.log-metrics-on-shutdown-format` | `prometheus` | Format of that summary. +| `camel.opentelemetry2.enabled` | `true` | Enables the OpenTelemetry tracing instrumentation. +| `management.endpoint.health.probes.enabled` | `true` | Enables the liveness and readiness health groups. +| `management.health.livenessState.enabled` | `true` | Registers the `livenessState` indicator. +| `management.health.readinessState.enabled` | `true` | Registers the `readinessState` indicator. +| `management.endpoint.health.show-details` | `when-authorized` | See <>. +| `management.endpoint.health.group.live.include` | `livenessState,camelLivenessState` | Contents of `/observe/health/live`. +| `management.endpoint.health.group.live.show-details` | `always` | See <>. +| `management.endpoint.health.group.ready.include` | `readinessState,camelReadinessState` | Contents of `/observe/health/ready`. +| `management.endpoint.health.group.ready.show-details` | `always` | See <>. +|=== + +=== Exposing the management listener + +The management listener binds to `127.0.0.1`, so out of the box the health and metrics endpoints are reachable +from the same host only. This matches the Spring Boot baseline, which ships no separate management listener at +all: opening one on every interface should be a decision you make, not one the starter makes for you. + +A Kubernetes deployment whose kubelet probes or Prometheus scrapers reach the pod over the network has to widen +the bind address: + +[source,properties] +---- +management.server.address = 0.0.0.0 +---- + +Pair that with a `NetworkPolicy`, or with authentication in front of the management port, so the endpoints are +only reachable from your probes and your scrapers. + +=== Health detail exposure + +`/observe/health` uses `when-authorized`, the setting that shows the individual health indicators to an +authenticated caller and the overall status alone to everybody else. Camel health checks report on the +resources a route talks to — broker connections, data sources, remote endpoints — and their detail can +identify those resources, so it is not something to hand to an unauthenticated caller. With no Spring Security +on the classpath there is no authenticated caller, and the endpoint behaves as `never`. + +To restore the previous behaviour of showing the details to anyone who asks: + +[source,properties] +---- +management.endpoint.health.show-details = always +---- + +The `live` and `ready` groups keep `always`. The kubelet reaches them unauthenticated, and it puts the response +body into the probe-failure event, so `kubectl describe pod` names the indicator that took the pod down. Both +groups contain availability-state indicators only — `livenessState`, `readinessState` and their Camel +counterparts report a status and carry no data — so there is nothing in those responses beyond the indicator +names and their state. + +=== Camel health exposure level + +`camel.health.exposure-level` is left at its Camel default of `default`, which filters health check metadata — +endpoint URIs, route and consumer identifiers — out of the health response while keeping the check names, +error messages and stack traces that tell you what failed. + +If you want the metadata as well, opt in: + +[source,properties] +---- +camel.health.exposure-level = full +---- +