Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
282 changes: 282 additions & 0 deletions app/_how-tos/konnect-platform/manage-ai-gateway-with-kongctl.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,282 @@
---
title: Manage {{site.ai_gateway}} with kongctl
description: >-
Create {{site.ai_gateway}} and route an OpenAI chat request through a local
data plane.
content_type: how_to
permalink: /kongctl/manage-ai-gateway/

breadcrumbs:
- /kongctl/

products:
- ai-gateway
- konnect

works_on:
- konnect

tools:
- kongctl

min_version:
ai-gateway: '2.0'
kongctl: '1.14'

tags:
- ai
- declarative-config
- openai

published: false

tldr:
q: How do I manage {{site.ai_gateway}} with kongctl?
a: |
Declare {{site.ai_gateway}}, a Model Provider, a Model, and a data-plane
certificate in YAML, apply the configuration with kongctl, and connect a
data plane.

automated_tests: false

prereqs:
skip_product: false
show_works_on: false
inline:
- title: "{{site.konnect_product_name}}"
content: |
You need a {{site.konnect_short_name}} account and kongctl 1.14 or
later authenticated with `kongctl login`.
icon_url: /assets/icons/gateway.svg
- title: Local tools
content: |
Install Docker and OpenSSL. You also need an
[OpenAI API key](https://platform.openai.com/api-keys).
icon_url: /assets/icons/ai.svg

related_resources:
- text: Learn about {{site.ai_gateway}}
url: /ai-gateway/
- text: Declarative configuration with kongctl
url: /kongctl/declarative/
- text: kongctl declarative resource reference
url: /kongctl/supported-resources/#ai-gateway

next_steps:
- text: Explore {{site.ai_gateway}}
url: /ai-gateway/
- text: Learn about kongctl sync
url: /kongctl/sync/
---

This tutorial creates an {{site.ai_gateway}}, an OpenAI Model Provider and
Model, and a data-plane certificate. You then run a local data plane and send
an OpenAI-compatible chat request through it.

## Create a working directory

Create a directory for the configuration and certificate:

```sh
mkdir kongctl-ai-gateway
```

Change into the directory:

```sh
cd kongctl-ai-gateway
```

Create a certificate directory:

```sh
mkdir certs
```

Generate a self-signed certificate and private key:

```sh
openssl req -new -x509 -nodes -newkey rsa:2048 -days 365 \
-subj "/CN=openai-llm-data-plane/C=US" \
-keyout certs/data-plane.key \
-out certs/data-plane.crt
```

Allow the data-plane container group to read the private key:

```sh
chgrp "$(id -g)" certs/data-plane.key
chmod 640 certs/data-plane.key
```

Keep `certs/data-plane.key` private and out of version control. kongctl sends
only the public certificate to {{site.konnect_short_name}}.

## Declare {{site.ai_gateway}}

Create `ai-gateway.yaml` with the following configuration:

```yaml
_defaults:
kongctl:
namespace: openai-llm-example

ai_gateways:
- ref: openai-llm
name: openai-llm
display_name: OpenAI LLM Gateway
deployment_type: hybrid
description: Routes OpenAI-compatible chat traffic to OpenAI
proxy_urls:
- host: localhost
port: 8000
protocol: http
labels:
example: openai-llm
data_plane_certificates:
- ref: openai-llm-data-plane
title: openai-llm-data-plane
description: Local Docker data plane
cert: !file ./certs/data-plane.crt
model_providers:
- ref: openai
name: openai
display_name: OpenAI
type: openai
config:
auth:
type: basic
headers:
- name: Authorization
value: !secret
parts:
- "Bearer "
- !env OPENAI_API_KEY
models:
- ref: my-gpt-4o
name: my-gpt-4o
display_name: My GPT-4o
type: model
formats:
- type: openai
config:
route:
paths:
- /v1
model:
body_param: model
values:
- my-gpt-4o
targets:
- name: gpt-4o
provider: openai
config:
type: openai
policies: []
capabilities:
- generate
```

Set your OpenAI API key:

```sh
export OPENAI_API_KEY='YOUR_OPENAI_API_KEY'
```

Preview the changes:

```sh
kongctl diff --mode apply -f ai-gateway.yaml
```

Apply the configuration:

```sh
kongctl apply -f ai-gateway.yaml
```

The `!secret` value is resolved only during execution. The resolved OpenAI
key isn't stored in the configuration or plan.

## Connect the data plane

Read the configuration endpoint from {{site.ai_gateway}}:

```sh
export AIGW_CONTROL_PLANE="$(kongctl get ai-gateway \
'OpenAI LLM Gateway' --output json --jq \
'.endpoints.configuration | sub("^https://"; "") | sub(":443$"; "")' \
--jq-raw-output)"
```

Read the telemetry endpoint:

```sh
export AIGW_TELEMETRY="$(kongctl get ai-gateway \
'OpenAI LLM Gateway' --output json --jq \
'.endpoints.telemetry | sub("^https://"; "") | sub(":443$"; "")' \
--jq-raw-output)"
```

Start a {{site.ai_gateway}} 2.0 data plane:

```sh
docker run --detach --rm --name openai-llm-data-plane \
--group-add "$(id -g)" \
--env KONG_ROLE=data_plane \
--env KONG_DATABASE=off \
--env KONG_VITALS=off \
--env KONG_CLUSTER_MTLS=pki \
--env "KONG_CLUSTER_CONTROL_PLANE=$AIGW_CONTROL_PLANE:443" \
--env "KONG_CLUSTER_SERVER_NAME=$AIGW_CONTROL_PLANE" \
--env "KONG_CLUSTER_TELEMETRY_ENDPOINT=$AIGW_TELEMETRY:443" \
--env "KONG_CLUSTER_TELEMETRY_SERVER_NAME=$AIGW_TELEMETRY" \
--env KONG_CLUSTER_CERT=/etc/kong/certs/data-plane.crt \
--env KONG_CLUSTER_CERT_KEY=/etc/kong/certs/data-plane.key \
--env KONG_LUA_SSL_TRUSTED_CERTIFICATE=system \
--env KONG_KONNECT_MODE=on \
--volume "$PWD/certs:/etc/kong/certs:ro" \
--publish 8000:8000 \
--publish 8443:8443 \
kong/kong-ai-gateway:2.0.2
```

Confirm that the data plane connects:

```sh
kongctl get ai-gateway nodes --gateway-name "OpenAI LLM Gateway"
```

## Send a chat request

Send an OpenAI-compatible request through the local proxy:

```sh
curl --no-progress-meter --fail-with-body \
--request POST http://localhost:8000/v1/chat/completions \
--header "Accept: application/json" \
--json '{
"model": "my-gpt-4o",
"messages": [
{"role": "user", "content": "Say this is a test!"}
]
}'
```

A successful response contains a chat completion from `gpt-4o`.

## Clean up

Stop and remove the local data plane:

```sh
docker stop openai-llm-data-plane
```

Delete {{site.ai_gateway}} and its managed child resources:

```sh
kongctl delete -f ai-gateway.yaml
```

Delete the local certificate files when you no longer need them.
15 changes: 11 additions & 4 deletions app/_landing_pages/kongctl.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ rows:
The tool provides both declarative and imperative style resource management capabilities along with
other developer friendly features.

The declarative configuration feature allows you to define API Platform infrastructure as code using a YAML based syntax and a state free reconciliation system. The tool supports a growing list of {{site.konnect_short_name}} resource types including Dev Portals, control planes, APIs, and more.
The declarative configuration feature lets you define API platform infrastructure as code using YAML and a stateless reconciliation system. kongctl manages APIs, Dev Portals, control planes, {{site.ai_gateway}}, {{site.event_gateway_short}}, organization, and {{site.konnect_catalog}} resources.

kongctl also ships installable AI agent skills that help coding agents generate, review, and operate kongctl configuration from a repository.

Expand Down Expand Up @@ -92,6 +92,8 @@ rows:
**AI Agent Skills**: Install bundled skills that help coding agents discover resource schemas, generate
declarative configuration, build extensions, and preview changes with kongctl workflows.

**Audit Logs**: Pull a finite audit-log history, follow new organization events, or receive Dev Portal events with a webhook listener.

**Flexible Authorization**: kongctl communicates with {{site.konnect_short_name}} via its APIs and supports authentication
through browser-based device flow or personal access tokens.

Expand Down Expand Up @@ -326,18 +328,21 @@ rows:
Yes, but with caution. Since kongctl's declarative engine is stateless and calculates changes by querying live {{site.konnect_short_name}} resources, concurrent processes operating on the same resources could conflict. Use namespace isolation to separate resources managed by different processes or teams.
- q: What resources can kongctl manage declaratively?
a: |
The list of {{site.konnect_short_name}} resources supported is growing. The current list includes (but not limited to):
kongctl manages these {{site.konnect_short_name}} resource families:
* **APIs**: Including versions, publications, implementations, and documents
* **Dev Portals**: Including pages, teams, snippets, customizations, authentication strategies, and custom domains
* **Control planes**: For {{site.base_gateway}} control plane management _and_ decK integration
* **{{site.ai_gateway}}**: Including Model Providers, Auth Strategies, Policies, Agents, Consumers, Models, MCP Servers, Config Stores, secrets, Vaults, and data-plane certificates
* **{{site.event_gateway_short}}**: Including clusters, listeners, policies, certificates, schema registries, and keys
* **{{site.konnect_catalog}} and organization**: Including Catalog services, teams, roles, and custom dashboards

Support for additional resource types is planned for future releases.
Run `kongctl explain` for the version-specific resource list and see the [declarative resource reference](/kongctl/supported-resources/).
- q: How does kongctl authentication work?
a: |
kongctl supports two authentication methods:

* **Device Flow (Recommended)**: Run `kongctl login` to authenticate via your browser. Tokens are stored locally and refreshed automatically.
* **Personal Access Token**: Use the `--pat` flag or `KONGCTL_DEFAULT_KONNECT_PAT` environment variable for automation scenarios like CI/CD pipelines.
* **Personal or system account access token**: Create PATs and sPATs with kongctl, then use the `--pat` flag or `KONGCTL_DEFAULT_KONNECT_PAT` environment variable for automation.

Configuration and device flow authentication credentials are stored in `$XDG_CONFIG_HOME/kongctl/` (typically `~/.config/kongctl/`).
- q: Can I use kongctl in CI/CD pipelines?
Expand Down Expand Up @@ -379,6 +384,8 @@ rows:
- text: kongctl releases and changelog
type: github
url: https://github.com/kong/kongctl/releases
- text: Hands-on kongctl learning guide
url: https://kong.github.io/kongctl/
- text: Announcement blog
type: blog
url: https://konghq.com/blog/product-releases/kongctl
1 change: 1 addition & 0 deletions app/_landing_pages/konnect.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,7 @@ rows:
blocks:
- type: unordered_list
items:
- "[kongctl](/kongctl/): Manage {{site.konnect_short_name}} resources with imperative commands or declarative, plan-based workflows"
- "[decK](/deck/): Manage {{site.base_gateway}} configuration in {{site.konnect_short_name}} through declarative state files"
- "[Terraform](/terraform/): Manage infrastructure as code and automated deployments to streamline setup and configuration of {{site.konnect_short_name}}, {{site.base_gateway}}, and {{site.mesh_product_name}}"
- "[KIC](/kubernetes-ingress-controller/): Manage ingress traffic and routing rules for your services"
Expand Down
2 changes: 2 additions & 0 deletions app/kongctl/adopt/ai-gateway.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ breadcrumbs:
related_resources:
- text: kongctl adopt commands
url: /kongctl/adopt/
- text: Declarative configuration with kongctl
url: /kongctl/declarative/
---

Apply the KONGCTL-namespace label to an existing {{site.konnect_short_name}} {{site.ai_gateway}} that is not currently managed by kongctl.
Expand Down
1 change: 0 additions & 1 deletion app/kongctl/adopt/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ content_type: reference
layout: reference

works_on:
- on-prem
- konnect

tools:
Expand Down
1 change: 0 additions & 1 deletion app/kongctl/adopt/auth-strategy.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ content_type: reference
layout: reference

works_on:
- on-prem
- konnect

tools:
Expand Down
1 change: 0 additions & 1 deletion app/kongctl/adopt/control-plane.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ content_type: reference
layout: reference

works_on:
- on-prem
- konnect

tools:
Expand Down
1 change: 0 additions & 1 deletion app/kongctl/adopt/konnect.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ layout: reference


works_on:
- on-prem
- konnect

tools:
Expand Down
1 change: 0 additions & 1 deletion app/kongctl/adopt/organization.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ layout: reference


works_on:
- on-prem
- konnect

tools:
Expand Down
1 change: 0 additions & 1 deletion app/kongctl/adopt/portal.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ content_type: reference
layout: reference

works_on:
- on-prem
- konnect

tools:
Expand Down
Loading
Loading