Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
Spring Boot auto-configuration bundling the Camel observability services: metrics, tracing and health.

The starter pulls in `camel-micrometer`, `micrometer-registry-prometheus`, `camel-opentelemetry2` and
`camel-management`, and contributes a set of defaults so that a Camel application exposes a Prometheus
scrape endpoint and Kubernetes-shaped liveness and readiness probes without any configuration.

The defaults are registered as the lowest precedence property source, so anything you set in
`application.properties`, in an environment variable or on the command line overrides them.
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
Add the starter to the classpath and the endpoints below are available on a separate management listener,
bound to loopback on port `9876`:

[width="100%",cols="3,5",options="header"]
|===
| Endpoint | Purpose
| `http://127.0.0.1:9876/observe/health` | Aggregate health
| `http://127.0.0.1:9876/observe/health/live` | Kubernetes liveness probe
| `http://127.0.0.1:9876/observe/health/ready` | Kubernetes readiness probe
| `http://127.0.0.1:9876/observe/metrics` | Prometheus scrape endpoint
|===

=== Injected defaults

This is the full set of properties the starter contributes. Every one of them can be overridden by your own
configuration:

[width="100%",cols="4,2,4",options="header"]
|===
| Property | Value | Notes
| `management.server.port` | `9876` | Management endpoints run on their own listener, separate from the application port.
| `management.server.address` | `127.0.0.1` | The listener binds to loopback. See <<Exposing the management listener>>.
| `management.endpoints.web.exposure.include` | `health,prometheus` | Only these two endpoints are exposed.
| `management.endpoints.web.base-path` | `/observe` | Replaces the `/actuator` base path.
| `management.endpoints.web.path-mapping.prometheus` | `metrics` | Serves the Prometheus endpoint at `/observe/metrics`.
| `camel.metrics.log-metrics-on-shutdown` | `true` | Dumps a metrics summary to the log when the application stops.
| `camel.metrics.log-metrics-on-shutdown-filters` | `app.info,camel.exchanges.*,process.cpu.usage,jvm.memory.max,jvm.memory.used` | Which metrics that summary contains.
| `camel.metrics.log-metrics-on-shutdown-format` | `prometheus` | Format of that summary.
| `camel.opentelemetry2.enabled` | `true` | Enables the OpenTelemetry tracing instrumentation.
| `management.endpoint.health.probes.enabled` | `true` | Enables the liveness and readiness health groups.
| `management.health.livenessState.enabled` | `true` | Registers the `livenessState` indicator.
| `management.health.readinessState.enabled` | `true` | Registers the `readinessState` indicator.
| `management.endpoint.health.show-details` | `when-authorized` | See <<Health detail exposure>>.
| `management.endpoint.health.group.live.include` | `livenessState,camelLivenessState` | Contents of `/observe/health/live`.
| `management.endpoint.health.group.live.show-details` | `always` | See <<Health detail exposure>>.
| `management.endpoint.health.group.ready.include` | `readinessState,camelReadinessState` | Contents of `/observe/health/ready`.
| `management.endpoint.health.group.ready.show-details` | `always` | See <<Health detail exposure>>.
|===

=== Exposing the management listener

The management listener binds to `127.0.0.1`, so out of the box the health and metrics endpoints are reachable
from the same host only. This matches the Spring Boot baseline, which ships no separate management listener at
all: opening one on every interface should be a decision you make, not one the starter makes for you.

A Kubernetes deployment whose kubelet probes or Prometheus scrapers reach the pod over the network has to widen
the bind address:

[source,properties]
----
management.server.address = 0.0.0.0
----

Pair that with a `NetworkPolicy`, or with authentication in front of the management port, so the endpoints are
only reachable from your probes and your scrapers.

=== Health detail exposure

`/observe/health` uses `when-authorized`, the setting that shows the individual health indicators to an
authenticated caller and the overall status alone to everybody else. Camel health checks report on the
resources a route talks to — broker connections, data sources, remote endpoints — and their detail can
identify those resources, so it is not something to hand to an unauthenticated caller. With no Spring Security
on the classpath there is no authenticated caller, and the endpoint behaves as `never`.

To restore the previous behaviour of showing the details to anyone who asks:

[source,properties]
----
management.endpoint.health.show-details = always
----

The `live` and `ready` groups keep `always`. The kubelet reaches them unauthenticated, and it puts the response
body into the probe-failure event, so `kubectl describe pod` names the indicator that took the pod down. Both
groups contain availability-state indicators only — `livenessState`, `readinessState` and their Camel
counterparts report a status and carry no data — so there is nothing in those responses beyond the indicator
names and their state.

=== Camel health exposure level

`camel.health.exposure-level` is left at its Camel default of `default`, which filters health check metadata —
endpoint URIs, route and consumer identifiers — out of the health response while keeping the check names,
error messages and stack traces that tell you what failed.

If you want the metadata as well, opt in:

[source,properties]
----
camel.health.exposure-level = full
----
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,11 @@
* Contributes the opinionated observability defaults as the lowest precedence property source so that any
* user-provided configuration (application.properties, environment variables, system properties, ...) overrides them
* following the standard Spring Boot precedence rules.
* <p>
* The defaults stay within the Spring Boot baseline: the management listener binds to loopback, and the aggregate
* health endpoint only shows its details to an authorized caller. The {@code live} and {@code ready} probe groups
* keep {@code show-details=always} because they are consumed unauthenticated by the kubelet, and the indicators
* they contain report an availability state and nothing else.
*/
public class ObservabilityServicesEnvironmentPostProcessor implements EnvironmentPostProcessor, Ordered {

Expand All @@ -38,6 +43,8 @@ public class ObservabilityServicesEnvironmentPostProcessor implements Environmen
public void postProcessEnvironment(ConfigurableEnvironment environment, SpringApplication application) {
Map<String, Object> defaults = new LinkedHashMap<>();
defaults.put("management.server.port", "9876");
// bind the management listener to loopback; exposing it beyond the host is a conscious step
defaults.put("management.server.address", "127.0.0.1");
defaults.put("management.endpoints.web.exposure.include", "health,prometheus");
defaults.put("management.endpoints.web.base-path", "/observe");
defaults.put("management.endpoints.web.path-mapping.prometheus", "metrics");
Expand All @@ -48,15 +55,15 @@ public void postProcessEnvironment(ConfigurableEnvironment environment, SpringAp
// Opentelemetry
defaults.put("camel.opentelemetry2.enabled", "true");
// Health
defaults.put("camel.health.exposure-level", "full");
defaults.put("management.endpoint.health.probes.enabled", "true");
defaults.put("management.health.readinessState.enabled", "true");
defaults.put("management.health.livenessState.enabled", "true");
defaults.put("management.endpoint.health.show-details", "always");
// /observe/health/live remap
defaults.put("management.endpoint.health.show-details", "when-authorized");
// /observe/health/live remap; the group carries availability states only, and the kubelet reads it
// unauthenticated, so its details stay visible
defaults.put("management.endpoint.health.group.live.include", "livenessState,camelLivenessState");
defaults.put("management.endpoint.health.group.live.show-details", "always");
// /observe/health/ready remap
// /observe/health/ready remap; see the note on the live group above
defaults.put("management.endpoint.health.group.ready.include", "readinessState,camelReadinessState");
defaults.put("management.endpoint.health.group.ready.show-details", "always");

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,18 @@
*/
package org.apache.camel.observability.services.springboot;

import java.util.Arrays;
import java.util.LinkedHashSet;
import java.util.Set;

import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.core.env.ConfigurableEnvironment;
import org.springframework.core.env.EnumerablePropertySource;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;

@SpringBootTest(classes = ObservabilityServicesTestApplication.class)
Expand All @@ -42,19 +48,68 @@ public void defaultsAreApplied() {
environment.getProperty("camel.metrics.log-metrics-on-shutdown-filters"));
assertEquals("prometheus",
environment.getProperty("camel.metrics.log-metrics-on-shutdown-format"));
assertEquals("full", environment.getProperty("camel.health.exposure-level"));
assertEquals("true", environment.getProperty("management.endpoint.health.probes.enabled"));
assertEquals("true", environment.getProperty("management.health.readinessState.enabled"));
assertEquals("true", environment.getProperty("management.health.livenessState.enabled"));
assertEquals("always", environment.getProperty("management.endpoint.health.show-details"));
assertEquals("livenessState,camelLivenessState",
environment.getProperty("management.endpoint.health.group.live.include"));
assertEquals("always", environment.getProperty("management.endpoint.health.group.live.show-details"));
assertEquals("readinessState,camelReadinessState",
environment.getProperty("management.endpoint.health.group.ready.include"));
}

@Test
public void managementListenerBindsToLoopback() {
// the port alone would put a second listener on every interface
assertEquals("127.0.0.1", environment.getProperty("management.server.address"));
}

@Test
public void aggregateHealthDetailsRequireAuthorization() {
assertEquals("when-authorized", environment.getProperty("management.endpoint.health.show-details"));
}

@Test
public void probeGroupsKeepTheirDetails() {
// the kubelet reads these unauthenticated and surfaces the body in a probe-failure event; the groups
// hold availability-state indicators only, which report a status and no data
assertEquals("always", environment.getProperty("management.endpoint.health.group.live.show-details"));
assertEquals("always", environment.getProperty("management.endpoint.health.group.ready.show-details"));
}

@Test
public void camelHealthExposureLevelIsNotForced() {
// 'full' is an opt-in; left alone, Camel's own 'default' level applies and filters health check metadata
assertNull(environment.getProperty("camel.health.exposure-level"));
}

@Test
public void injectedPropertiesAreTheDocumentedSet() {
EnumerablePropertySource<?> source = (EnumerablePropertySource<?>) environment.getPropertySources()
.get(ObservabilityServicesEnvironmentPostProcessor.PROPERTY_SOURCE_NAME);

Set<String> expected = new LinkedHashSet<>(Arrays.asList(
"management.server.port",
"management.server.address",
"management.endpoints.web.exposure.include",
"management.endpoints.web.base-path",
"management.endpoints.web.path-mapping.prometheus",
"camel.metrics.log-metrics-on-shutdown",
"camel.metrics.log-metrics-on-shutdown-filters",
"camel.metrics.log-metrics-on-shutdown-format",
"camel.opentelemetry2.enabled",
"management.endpoint.health.probes.enabled",
"management.health.readinessState.enabled",
"management.health.livenessState.enabled",
"management.endpoint.health.show-details",
"management.endpoint.health.group.live.include",
"management.endpoint.health.group.live.show-details",
"management.endpoint.health.group.ready.include",
"management.endpoint.health.group.ready.show-details"));

// keep this in sync with the table in src/main/doc/usage.adoc
assertEquals(expected, new LinkedHashSet<>(Arrays.asList(source.getPropertyNames())));
}

@Test
public void userApplicationPropertiesOverrideDefaults() {
// these values are set in src/test/resources/application.properties and must win
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.camel.observability.services.springboot;

import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.core.env.ConfigurableEnvironment;

import static org.junit.jupiter.api.Assertions.assertEquals;

/**
* Verifies that the wider settings the starter no longer ships by default can be opted back into, which is what
* a deployment reached over the pod network has to do.
*/
@SpringBootTest(classes = ObservabilityServicesTestApplication.class,
properties = {
"management.server.address=0.0.0.0",
"management.endpoint.health.show-details=always",
"camel.health.exposure-level=full" })
public class ObservabilityServicesOptInOverridesTest {

@Autowired
private ConfigurableEnvironment environment;

@Test
public void bindAddressCanBeWidened() {
assertEquals("0.0.0.0", environment.getProperty("management.server.address"));
}

@Test
public void healthDetailsCanBeAlwaysShown() {
assertEquals("always", environment.getProperty("management.endpoint.health.show-details"));
}

@Test
public void camelHealthExposureLevelCanBeSetToFull() {
assertEquals("full", environment.getProperty("camel.health.exposure-level"));
}
}
Loading
Loading