Talos cluster template

The talos cluster template will create a cluster using Talos Control Plane provider.

A container running a haproxy server is used for the cluster load balancer endpoint. The load balancer endpoint will be the IP address of the haproxy container.

WARNING: The load balancer container is a single point of failure for the control plane of the workload cluster, therefore should only be used for development or evaluation purposes. For production grade clusters, you should instead use a different option for the load balancer, e.g. OVN

WARNING: This cluster template is not currently tested in CI, therefore could be broken. Please raise a GitHub issue if that is the case.

Table Of Contents

• Requirements
  • 1. CAPN version
  • 2. Install Talos control plane and bootstrap providers on the cluster
  • 3. Import Talos nocloud server image into Incus
• Configuration
• Generate cluster
• Cluster Template

Requirements

1. CAPN version

Talos cluster templates require CAPN version v0.8.5 or newer.

2. Install Talos control plane and bootstrap providers on the cluster

clusterctl init -c talos -b talos

Tested Versions:

• cluster-api-control-plane-provider-talos v0.5.12
• cluster-api-bootstrap-provider-talos v0.6.11
• talos linux version v1.12.2

3. Import Talos nocloud server image into Incus

Start from https://factory.talos.dev/

• Select Cloud Server
• Select Talos Linux Version (v1.12.2)
• Select nocloud variant
• Select Machine architecture (amd64)
• Select Secure Boot (true or false)
• Select optional System Extensions (none required)
• Select Bootloader (auto)
• Select BIOS Only boot (NOTE: UEFI boot seems to not work correctly)

# 1. download nocloud-amd64.raw.xz, extract and convert to "rootfs.qcow2"
curl -L "https://factory.talos.dev/image/9ed5fecdacb36b5c5427b87d409f1065cfb2df69b0f71c58b868d9d466d8dab3/v1.12.2/nocloud-amd64.raw.xz" -o nocloud-amd64.raw.xz
unxz nocloud-amd64.raw.xz
qemu-img convert -f raw -O qcow2 nocloud-amd64.raw rootfs.qcow2

# 1. download nocloud-amd64-secureboot.raw.xz, extract and convert to "rootfs.qcow2"
curl -L "https://factory.talos.dev/image/376567988ad370138ad8b2698212367b8edcb69b5fd68c80be1f2ec7d603b4ba/v1.12.2/nocloud-amd64-secureboot.raw.xz" -o nocloud-amd64-secureboot.raw.xz
unxz nocloud-amd64-secureboot.raw.xz
qemu-img convert -f raw -O qcow2 nocloud-amd64-secureboot.raw rootfs.qcow2

# 2. generate image metadata tarball "metadata.tar.gz"
SERIAL="$(date '+%s')"
VERSION=v1.12.2
echo "
architecture: amd64
creation_date: $SERIAL
expiry_date: 0
properties:
  architecture: amd64
  description: talos $VERSION amd64 ($SERIAL)
  name: talos-$VERSION
  os: talos
  release: $VERSION
  serial: $SERIAL
  variant: default
templates: {}
" | tee metadata.yaml
tar cvzf metadata.tar.gz metadata.yaml

Finally, import the image into Incus:

# c. import image into Incus
incus image import metadata.tar.gz rootfs.qcow2 --alias talos-v1.12.2

Make a note of:

• The image alias talos-v1.12.2, as we will later refer to it when generating the cluster manifest.
• Whether you used the SecureBoot image variant (configuration below needs to match).

Configuration

# Cluster version and size
export KUBERNETES_VERSION=v1.34.0
export CONTROL_PLANE_MACHINE_COUNT=3
export WORKER_MACHINE_COUNT=2

# Name of secret with server credentials
export LXC_SECRET_NAME=lxc-secret

# Talos version (tested v1.12)
export TALOS_VERSION=v1.12

# Talos image (tested talos/nocloud/v1.12.2/amd64)
export TALOS_IMAGE_NAME=talos-v1.12.2           # alias of imported Talos nocloud image
export TALOS_IMAGE_SECURE_BOOT=false            # 'true' or 'false' based on nocloud image

# Load balancer configuration
export LXC_LOAD_BALANCER_TYPE=lxc               # must be 'lxc' or 'oci'
export LOAD_BALANCER_MACHINE_PROFILES=[default] # profiles for the haproxy container
export LOAD_BALANCER_MACHINE_FLAVOR=c1-m1       # instance type for the haproxy container

# Control plane machine configuration
export CONTROL_PLANE_MACHINE_FLAVOR=c2-m4       # flavor for control plane nodes
export CONTROL_PLANE_MACHINE_PROFILES=[default] # profiles for control plane nodes

# Worker machine configuration
export WORKER_MACHINE_FLAVOR=c2-m4              # flavor for worker nodes
export WORKER_MACHINE_PROFILES=[default]        # profiles for worker nodes

Generate cluster

clusterctl generate cluster example-cluster -i incus --flavor talos

Cluster Template

---
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: ${CLUSTER_NAME}
spec:
  clusterNetwork:
    pods:
      cidrBlocks: ${POD_CIDR:=[10.244.0.0/16]}
    services:
      cidrBlocks: ${SERVICE_CIDR:=[10.96.0.0/12]}
    serviceDomain: cluster.local
  controlPlaneRef:
    apiVersion: controlplane.cluster.x-k8s.io/v1alpha3
    kind: TalosControlPlane
    name: ${CLUSTER_NAME}-control-plane
  infrastructureRef:
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha2
    kind: LXCCluster
    name: ${CLUSTER_NAME}
---
apiVersion: controlplane.cluster.x-k8s.io/v1alpha3
kind: TalosControlPlane
metadata:
  name: ${CLUSTER_NAME}-control-plane
spec:
  replicas: ${CONTROL_PLANE_MACHINE_COUNT}
  version: ${KUBERNETES_VERSION}
  infrastructureTemplate:
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha2
    kind: LXCMachineTemplate
    name: ${CLUSTER_NAME}-control-plane
  controlPlaneConfig:
    controlplane:
      generateType: controlplane
      talosVersion: ${TALOS_VERSION}
      hostname:
        source: InfrastructureName
---
apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
  name: ${CLUSTER_NAME}-md-0
spec:
  clusterName: ${CLUSTER_NAME}
  replicas: ${WORKER_MACHINE_COUNT}
  selector:
    matchLabels:
  template:
    spec:
      version: ${KUBERNETES_VERSION}
      clusterName: ${CLUSTER_NAME}
      bootstrap:
        configRef:
          apiVersion: bootstrap.cluster.x-k8s.io/v1alpha3
          kind: TalosConfigTemplate
          name: ${CLUSTER_NAME}-md-0
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1alpha2
        kind: LXCMachineTemplate
        name: ${CLUSTER_NAME}-md-0
---
apiVersion: bootstrap.cluster.x-k8s.io/v1alpha3
kind: TalosConfigTemplate
metadata:
  name: ${CLUSTER_NAME}-md-0
spec:
  template:
    spec:
      generateType: worker
      talosVersion: ${TALOS_VERSION}
      hostname:
        source: InfrastructureName
---
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha2
kind: LXCCluster
metadata:
  name: ${CLUSTER_NAME}
spec:
  secretRef:
    name: ${LXC_SECRET_NAME}
  cloudProviderNodePatch: true
  loadBalancer:
    ${LXC_LOAD_BALANCER_TYPE:=lxc}:
      instanceSpec:
        flavor: ${LOAD_BALANCER_MACHINE_FLAVOR:=""}
        profiles: ${LOAD_BALANCER_MACHINE_PROFILES:=[default]}
      disableHealthzCheck: true
---
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha2
kind: LXCMachineTemplate
metadata:
  name: ${CLUSTER_NAME}-control-plane
spec:
  template:
    spec:
      instanceType: virtual-machine
      config:
        security.secureboot: "${TALOS_IMAGE_SECURE_BOOT:=false}"
      devices:
      - cloud-init,type=disk,source=cloud-init:config
      image:
        name: ${TALOS_IMAGE_NAME}
      flavor: ${CONTROL_PLANE_MACHINE_FLAVOR}
      profiles: ${CONTROL_PLANE_MACHINE_PROFILES:=[default]}
---
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha2
kind: LXCMachineTemplate
metadata:
  name: ${CLUSTER_NAME}-md-0
spec:
  template:
    spec:
      instanceType: virtual-machine
      config:
        security.secureboot: "${TALOS_IMAGE_SECURE_BOOT:=false}"
      devices:
      - cloud-init,type=disk,source=cloud-init:config
      image:
        name: ${TALOS_IMAGE_NAME}
      flavor: ${WORKER_MACHINE_FLAVOR}
      profiles: ${WORKER_MACHINE_PROFILES:=[default]}