Cogstack Openstack Kubernetes K3S Module

Terraform Module for provisioning VMs on Openstack as a kubernetes cluster using K3s

Features

• Creates a multi node kuberentes cluster in Openstack with K3s
• VMs will initialise their setup using K3s install scripts
• Setup Networking in Openstack to allow communications between expected services
• Return a KUBECONFIG file for integration with kubernetes and helm providers in terraform

Prerequisites

• An Openstack Environment able to create Compute and Networking rules
• An accessible Ubuntu image in the openstack environment

Example

This example deploys a two node kubernetes cluster in Openstack with K3s.

module "openstack_kubernetes_cluster" {
  source  = "CogStack/kubernetes-k3s/openstack"
  version = "0.1.0"

  ubuntu_image_name = "ubuntu-22.04"
  network = {
    name = "public"
  }
  host_instances = [
    { name = "controller", is_controller = true, flavour = "m1.small" },
    { name = "worker", flavour = "m1.small" },
  ]
}

Requirements

Name	Version
openstack	~> 3.0.0

Providers

Name	Version
cloudinit	n/a
local	n/a
null	n/a
openstack	~> 3.0.0
random	n/a

Resources

Name	Type
local_file.private_key	resource
local_file.public_key	resource
null_resource.copy_kubeconfig	resource
null_resource.kubernetes_nodes_provisioner	resource
null_resource.kubernetes_server_provisioner	resource
openstack_compute_instance_v2.kubernetes_nodes	resource
openstack_compute_instance_v2.kubernetes_server	resource
openstack_compute_keypair_v2.compute_keypair	resource
openstack_networking_floatingip_associate_v2.kubernetes_nodes_fip	resource
openstack_networking_floatingip_associate_v2.kubernetes_server_fip	resource
openstack_networking_secgroup_rule_v2.cogstack_apps_port_rules	resource
openstack_networking_secgroup_v2.cogstack_apps_security_group	resource
random_id.server	resource
random_password.k3s_token	resource
cloudinit_config.init_docker	data source
cloudinit_config.init_docker_controller	data source
local_file.kubeconfig_file	data source
openstack_compute_flavor_v2.available_compute_flavors	data source
openstack_images_image_v2.ubuntu	data source
openstack_networking_network_v2.network	data source
openstack_networking_port_v2.nodes_port	data source
openstack_networking_port_v2.server_port	data source

Inputs

Name	Description	Type	Default	Required
host_instances	Hosts to Create: name = Human readable hostname for this host. is_controller = Must be true for exactly one host. This will run the k3s "server". All other nodes run the k3s "agent". flavour = The openstack_compute_flavor_v2 for the host volume_size = Size in GB for the disk volume for the node image_uuid = (Optional) The Openstack image you want to run, to override the default in ubuntu_image_name floating_ip = (Optional) Floating IP configuration. Set use_floating_ip to true and provide address to associate a floating IP with this host	list(object({ name = string, flavour = string, volume_size = optional(number, 20), is_controller = optional(bool, false), image_uuid = optional(string, null), floating_ip = optional(object({ use_floating_ip = optional(bool, false) # Using a boolean to make it a plan time value. address = optional(string) }), null) }))	n/a	yes
network	Network configuration. Either provide 'name' to lookup the network by name, or 'network_id' to use a network UUID directly.	object({ name = optional(string) network_id = optional(string) })	n/a	yes
ubuntu_image_name	Name of an available Machine Image running ubuntu in the openstack environment	string	n/a	yes
allowed_ingress_ips_cidr	CIDR Block that is allowed ingress to deployed services	string	"0.0.0.0/24"	no
generate_random_name_prefix	Whether to generate a random prefix for hostnames. If false, hostnames will use only the name from host_instances	bool	true	no
k3s_version	Optional k3s version to install (INSTALL_K3S_VERSION), for example v1.32.2+k3s1. When unset, the installer downloads from the stable channel.	string	null	no
output_file_directory	Optional path to write output files to. If directory doesnt exist it will be created	string	null	no
prefix	Optional custom prefix for resource names. If provided will override generate_random_name_prefix	string	null	no
ssh_key_pair	Paths to an SSH Public and Private Keypair. If provided these will be given SSH access on the created hosts. If not provided, a Keypair will be generated and accessible as a local file	object({ public_key_file = string private_key_file = string })	null	no

Outputs

Name	Description
compute_keypair	Absolute path to a public and private SSH key pair that is granted login on created VMs
created_controller_host	Created Controller Host: A map of { hostname: { data } }
created_hosts	Created Hosts: A map of { hostname: { data } }
created_security_group	Security group associated to the created hosts
kubeconfig_file	Path to the generated KUBECONFIG file used to connect to kubernetes
kubeconfig_raw	Kubeconfig for this cluster

Existing Network and Floating IPs

You can use this module with a custom OpenStack network by specifying the network variable with your own network name or network ID.

Using a custom OpenStack network with your own subnet allows you to improve security by keeping most nodes private. This ensures worker nodes and other internal resources are not directly exposed, reducing security risks. Only the controller node needs a floating IP to be accessible in order to use the k8s api server (eg for kubectl to work from outside the network). Using floating ips will also have the advantage of being stable, so you can destroy/create VMs without having to update anywhere the IP address is referenced by reassigning the floating ip.

To use this configuration, you will probably need to assign a floating IP to the controller node so that it is accessible. You can configure floating IP assignment per node using the floating_ip block within each entry in the host_instances variable.

Please see this example for using this module with a custom network and floating IPs.

module "openstack_kubernetes_cluster" {
  source  = "CogStack/kubernetes-k3s/openstack"
  version = "0.1.0"

  ubuntu_image_name = "ubuntu-22.04"
  network = {
    network_id = openstack_networking_network_v2.example_network.id
  }
  host_instances = [
    {
      name          = "controller"
      is_controller = true
      flavour       = "m1.small"
      floating_ip = {
        use_floating_ip = true
        address         = "203.0.113.10" # Address of an existing floating_ip in openstack
      }
    },
    {
      name    = "worker"
      flavour = "m1.small"
      # Optionally also assign a floating IP here, or leave blank to keep it internal to the network
    },
  ]
}

resource "openstack_networking_network_v2" "example_network" {
  name           = "dev-example-network"
  admin_state_up = "true"
}

resource "openstack_networking_subnet_v2" "example_subnet" {
  name       = "dev-example-subnet"
  network_id = openstack_networking_network_v2.example_network.id
  cidr       = "192.168.0.0/24"
  ip_version = 4
}

resource "openstack_networking_router_v2" "example_router" {
  name                = "test-router"
  admin_state_up      = true
  external_network_id = data.openstack_networking_network_v2.public.id
}

data "openstack_networking_network_v2" "public" {
  name = "public"
}

When using a non-default network, ensure the controller host has a floating IP so Terraform can access it for provisioning.

Contributing

The source for this module lives in CogStack/cogstack-platform at terraform-modules/openstack-kubernetes-k3s. Contribute by opening pull requests or forks against that repository, with your changes in that directory.

CogStack/terraform-openstack-kubernetes-k3s is a read-only mirror. When a tag is created in cogstack-platform as terraform-modules/openstack-kubernetes-k3s-v*.*.* all the files will be synced accross.