Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ Choose the ingestion path below that best matches your existing architecture, yo

<LinkCard
title="Prometheus Integration"
href="../../infrastructure/prometheus-integration/"
href="../../infrastructure/integrations/prometheus-integration/"
description="Use this when your metrics already live in Prometheus and you want KloudMate to ingest them through an OpenTelemetry-based path."
/>

Expand All @@ -57,7 +57,7 @@ If you are unsure where to begin, use this table to map your immediate goal to t
| --- | --- |
| Fastest overall onboarding | [KloudMate Agent](../../kloudmate-agent/) |
| Reuse existing OTel pipelines | [OpenTelemetry Collector](../../opentelemetry/) |
| Import Prometheus metrics | [Prometheus Integration](../../infrastructure/prometheus-integration/) |
| Import Prometheus metrics | [Prometheus Integration](../../infrastructure/integrations/prometheus-integration/) |
| Onboard AWS with minimal setup | [AWS Integration](../../aws-integration/) |
| Onboard Azure services | [Azure Integration](../../azure-integration/) |

Expand All @@ -67,7 +67,7 @@ For teams starting from scratch, we recommend this phased approach:

1. **Infrastructure First:** Start with the [KloudMate Agent](../../kloudmate-agent/) to get baseline telemetry, host metrics, and zero-code visibility.
2. **Application Tracing:** Add [OpenTelemetry](../../opentelemetry/) instrumentation to your applications where you need richer distributed tracing or custom metrics.
3. **Cloud Integrations:** Layer in cloud-specific integrations for [AWS](../../aws-integration/), [Azure](../../azure-integration/), or [Prometheus](../../infrastructure/prometheus-integration/) to centralize your remaining data sources.
3. **Cloud Integrations:** Layer in cloud-specific integrations for [AWS](../../aws-integration/), [Azure](../../azure-integration/), or [Prometheus](../../infrastructure/integrations/prometheus-integration/) to centralize your remaining data sources.

***

Expand Down
6 changes: 4 additions & 2 deletions src/content/docs/docs/infrastructure/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,10 @@ This section covers common infrastructure ingestion paths, whether you are monit
- [Hosts & Containers](./hosts/)
- [Advanced Host Configuration](./server-metrics-to-kloudmate/)
- [Kubernetes Monitoring](./kubernetes/)
- [Prometheus Integration](./prometheus-integration/)
- [vCenter Integration](./vcenter-integration/)
- [Prometheus Integration](./integrations/prometheus-integration/)
- [vCenter Integration](./integrations/vcenter-integration/)
- [Kafka Integration](./integrations/kafka-integration/)
- [Keycloak Integration](./integrations/keycloak-integration/)
- [AWS Integration](../../aws-integration/)
- [Azure Integration](../../azure-integration/)

Expand Down
4 changes: 4 additions & 0 deletions src/content/docs/docs/infrastructure/integrations/_meta.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"label": "Integrations",
"order": 4
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
---
title: "Kafka Integration"
description: "Collect Kafka broker, topic, and consumer metrics in KloudMate through the KloudMate Agent."
sidebar:
order: 3
---
Use this integration when you want to monitor Kafka brokers, topics, and consumers in KloudMate through the OpenTelemetry `kafkametrics` receiver running in the KloudMate Agent.

## Prerequisites

1. Install and run the [KloudMate Agent](../../../kloudmate-agent/).
2. Ensure your Kafka brokers are reachable from the node where the Agent is running.
3. Have your Kafka broker addresses, protocol version, and authentication credentials ready.

## Step 1: Define the Kafka Metrics Receiver

Add the following `kafkametrics` receiver to your Agent configuration file (typically `/etc/km-agent/config.yaml`). Replace the broker addresses, credentials, and protocol version to match your environment.

```yaml
extensions:
health_check:
endpoint: 0.0.0.0:13133

receivers:
kafkametrics:
brokers:
- kafka.example.com:9095
- kafka.example.com:9096
- kafka.example.com:9097
protocol_version: 2.8.0
auth:
plain_text:
username: <your-username>
password: <your-password>
scrapers:
- brokers
- topics
- consumers
metrics:
kafka.broker.log_retention_period:
enabled: true
kafka.topic.replication_factor:
enabled: true
kafka.topic.log_retention_period:
enabled: true
kafka.topic.log_retention_size:
enabled: true
kafka.topic.min_insync_replicas:
enabled: true
```

The `scrapers` block controls which data is collected. `brokers`, `topics`, and `consumers` are the three available options. The metrics listed under the `metrics` block are not enabled by default and must be explicitly set to `enabled: true` to be collected.

## Step 2: Add to Pipeline

Add the `kafkametrics` receiver to the metrics pipeline in your Agent configuration.

```yaml
service:
extensions:
- health_check
telemetry:
metrics:
level: none
pipelines:
metrics/kafkametrics:
receivers:
- kafkametrics
processors:
- batch
exporters:
- otlphttp
```

## Step 3: Restart the Agent

```bash
systemctl restart kmagent
systemctl status kmagent
```

## Step 4: Validate in KloudMate

After the Agent restarts:

- Open [Explore](../../../visualize-data/explore/) in KloudMate.
- Query one of the scraped Kafka metrics, such as `kafka.brokers` or `kafka.topic.partitions`.
- Build dashboards or alarms once the data is visible.

## Standard Kafka Dashboards

KloudMate provides prebuilt Kafka dashboards through [dashboard templates](https://templates.kloudmate.com/kafka/index.html). These dashboards help you visualize broker availability, topic partitions, consumer lag, offsets, and replication health.

To import and start using these templates, follow the steps described in [Import from Templates](../../../visualize-data/dashboards/create-a-dashboard/).

## Default Kafka Metrics

Kafka Monitoring automatically collects commonly used Kafka metrics when enabled.

Example Metrics

| **Metric Name** | **Description** |
| --- | --- |
| kafka.brokers | Number of brokers in the cluster. |
| kafka.topic.partitions | Number of partitions in a topic. |
| kafka.partition.current_offset | Current offset of a topic partition. |
| kafka.partition.oldest_offset | Oldest available offset of a topic partition. |
| kafka.partition.replicas | Number of replicas configured for a topic partition. |
| kafka.partition.replicas_in_sync | Number of in-sync replicas for a topic partition. |
| kafka.consumer_group.members | Number of members in a consumer group. |
| kafka.consumer_group.lag | Current approximate lag of a consumer group for a topic partition. |
| kafka.consumer_group.lag_sum | Current approximate sum of consumer group lag across all topic partitions. |
| kafka.consumer_group.offset | Current consumer group offset for a topic partition. |
| kafka.consumer_group.offset_sum | Sum of consumer group offsets across topic partitions. |

For the complete metrics list, refer to the [metrics reference](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/kafkametricsreceiver/documentation.md).
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
---
title: "Keycloak Integration"
description: "Collect Keycloak metrics in KloudMate through the KloudMate Agent."
sidebar:
order: 4
---
Use this integration when you want to scrape Keycloak metrics from its Prometheus-compatible metrics endpoint and send them to KloudMate through the KloudMate Agent.

## Prerequisites

1. Install and run the [KloudMate Agent](../../../kloudmate-agent/).
2. Ensure your Keycloak service exposes the `/auth/metrics` endpoint. If this endpoint is not available, no metrics will be scraped.
3. Keycloak must be reachable from the node where the KloudMate Agent is running.

## Step 1: Define the Prometheus Receiver

Add the following Prometheus receiver to your Agent configuration file (typically `/etc/km-agent/config.yaml`). Replace `<keycloak-service-name>` and `<namespace>` with your actual Kubernetes service name and namespace.

```yaml
extensions:
health_check:
endpoint: 0.0.0.0:13133

receivers:
prometheus:
config:
scrape_configs:
- job_name: keycloak
metrics_path: /auth/metrics
scheme: http
static_configs:
- targets:
- <keycloak-service-name>.<namespace>.svc.cluster.local:8080
```

:::info
Validate indentation carefully when copying `scrape_configs` into the Agent configuration.
:::

## Step 2: Add to Pipeline

Add the Prometheus receiver to the metrics pipeline in your Agent configuration.

```yaml
service:
extensions:
- health_check
telemetry:
metrics:
level: none
pipelines:
metrics/keycloak:
receivers:
- prometheus
processors:
- batch
exporters:
- otlphttp
```

## Step 3: Restart the Agent

```bash
systemctl restart kmagent
systemctl status kmagent
```

## Step 4: Validate in KloudMate

After the Agent restarts:

- Open [Explore](../../../visualize-data/explore/) in KloudMate.
- Query one of the scraped Keycloak metrics.
- Build dashboards or alarms once data is visible.
Original file line number Diff line number Diff line change
@@ -1,14 +1,16 @@
---
title: "Prometheus Integration"
description: "Ingest Prometheus metrics into KloudMate through the KloudMate Agent."
sidebar:
order: 1
---
Use this integration when your workloads already expose Prometheus metrics and you want KloudMate to ingest them without changing application code.

KloudMate does not scrape Prometheus directly. Instead, you configure the KloudMate Agent with the Prometheus receiver and export the collected metrics to KloudMate.

## Prerequisites

1. Install and run the [KloudMate Agent](../../kloudmate-agent/).
1. Install and run the [KloudMate Agent](../../../kloudmate-agent/).
2. Ensure you can access the Prometheus configuration or know the scrape targets you want the Agent to use.

## Step 1: Define the Prometheus Receiver
Expand Down Expand Up @@ -65,11 +67,11 @@ systemctl status kmagent

After the Agent restarts:

- open [Explore](../../visualize-data/explore/)
- open [Explore](../../../visualize-data/explore/)
- query one of the scraped Prometheus metrics
- build dashboards or alarms once the data is visible

## Related Paths

- [KloudMate Agent](../../kloudmate-agent/)
- [Explore](../../visualize-data/explore/)
- [KloudMate Agent](../../../kloudmate-agent/)
- [Explore](../../../visualize-data/explore/)
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
---
title: "vCenter Integration"
description: "Collect VMware vCenter metrics in KloudMate through the OpenTelemetry vCenter receiver."
sidebar:
order: 2
---
Use this guide to collect VMware vCenter metrics in KloudMate with the OpenTelemetry `vcenter` receiver running via the KloudMate Agent.

Expand All @@ -9,7 +11,7 @@ Use this guide to collect VMware vCenter metrics in KloudMate with the OpenTelem
1. Run vCenter 7.0 or 8.x.
2. Create a read-only vSphere user with access to the vCenter server, clusters, and resources you want to monitor.
3. Identify the vCenter or ESXi endpoint with SDK access enabled.
4. Install the [KloudMate Agent](../../kloudmate-agent/) on a host that can reach vCenter.
4. Install the [KloudMate Agent](../../../kloudmate-agent/) on a host that can reach vCenter.

## Step 1: Configure the Receiver

Expand Down Expand Up @@ -57,7 +59,7 @@ For Windows, restart the `KloudMate Agent` service from `services.msc`.

After the Agent restarts:

- open [Explore](../../visualize-data/explore/)
- open [Explore](../../../visualize-data/explore/)
- query vCenter metrics such as `vcenter_host_cpu_utilization` or `vcenter_vm_memory_usage`
- build dashboards and alerts once the data is visible

Expand Down
4 changes: 4 additions & 0 deletions src/content/docs/guides/network-monitoring/_meta.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"label": "Network Monitoring",
"order": 3
}
Loading
Loading