Attach Custom Cloud Servers to UpCloud Managed Kubernetes

UpCloud Kubernetes Service (UKS) provides a managed control plane. You scale capacity through node groups via the UKS API. But you can also attach individual cloud servers directly, outside the node group mechanism.

I tested this end-to-end on a UKS 1.35 cluster.

Why Bother?

Tools like Karpenter need this capability. Karpenter provisions servers one at a time, picking the most cost-efficient plan per workload. Without the ability to attach arbitrary VMs, Karpenter can't work with UKS.

And also because it's fun to experiment.

How It Works

Create a VM with POST /server, attach it to the cluster's private network, and run kubeadm join through a cloud-init script. The VM registers with the control plane like any kubeadm-managed worker as soon as the VM boots up. Cilium deploys its DaemonSet to the node, and pods schedule within a minute or two.

The key requirement: the VM must live in the same region as the cluster. UpCloud UKS SDN private networks don't span regions. A VM in fi-hel1 can't join a private network in de-fra1.

The Test

I ran two tests on a UKS 1.35 cluster with Cilium CNI and two existing nodes.

Test 1: Using only a public network

I created a CLOUDNATIVE-1xCPU-4GB VM using the UpCloud K8s 1.35 template with only public and utility interfaces. The cloud-init script ran kubeadm join against the public API server endpoint.

This did register the node, which kubectl get nodes showed, but it stayed NotReady. The API server's internal endpoint (:6443) was unreachable from outside the UKS network, and Cilium couldn't reach the control plane to initialize.

Test 2: attaching the private UKS network

I created a second VM with a plain Debian 13 template. This time I attached it to the cluster's private network (10.0.0.0/24) using --network "type=private,network=<uks-network-uuid>".

The cloud-init script installed containerd, kubeadm, kubelet, and kubectl, fixed a CNI path for Debian, then ran kubeadm join against the internal API server endpoint (:6443). The internal endpoint was immediately reachable over the private network.

Within 90 seconds the node reached Ready, with Cilium deployed:

NAME               STATUS   ROLES    VERSION   AGE
custom-worker-01   Ready    <none>   v1.35.6   3m15s

I scheduled an nginx pod and verified connectivity.

Step-by-Step

1. Create a Bootstrap Token

apiVersion: v1
kind: Secret
metadata:
  name: bootstrap-token-custom01
  namespace: kube-system
type: bootstrap.kubernetes.io/token
stringData:
  token-id: custom1
  token-secret: abcdef0123456789
  usage-bootstrap-authentication: "true"
  usage-bootstrap-signing: "true"
  auth-extra-groups: system:bootstrappers:kubeadm:default-node-token

Apply it:

kubectl apply -f bootstrap-token.yaml

2. Collect Cluster Info

# API server endpoint
kubectl config view --raw -o jsonpath='{.clusters[0].cluster.server}'

# CA certificate hash
openssl s_client -connect <api-server-host>:6443 </dev/null 2>/dev/null \
  | openssl x509 -pubkey -noout | sha256sum | awk '{print "sha256:"$1}'

3. Save the Cloud-Init Script for use at VM creation time

#cloud-config

packages:
  - apt-transport-https
  - ca-certificates
  - curl
  - gnupg

package_update: true
package_upgrade: true

write_files:
  - path: /etc/modules-load.d/containerd.conf
    content: |
      overlay
      br_netfilter
  - path: /etc/sysctl.d/99-kubernetes-cri.conf
    content: |
      net.bridge.bridge-nf-call-iptables = 1
      net.bridge.bridge-nf-call-ip6tables = 1
      net.ipv4.ip_forward = 1

runcmd:
  - modprobe overlay
  - modprobe br_netfilter
  - sysctl --system

  - apt-get install -y containerd
  - mkdir -p /etc/containerd
  - containerd config default > /etc/containerd/config.toml
  - sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
  - systemctl restart containerd

  - mkdir -p /etc/apt/keyrings
  - curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.35/deb/Release.key \
    | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
  - echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.35/deb/ /' \
    > /etc/apt/sources.list.d/kubernetes.list
  - apt-get update
  - apt-get install -y kubelet kubeadm kubectl
  - apt-mark hold kubelet kubeadm kubectl

  - ln -sf /opt/cni/bin /usr/lib/cni

  - >
    PRIVATE_IP=$(ip -4 addr show eth3 | grep -oP '(?<=inet\s)\d+(\.\d+){3}' | head -1);
    kubeadm join <API_SERVER_ENDPOINT> \
      --token custom1.abcdef0123456789 \
      --discovery-token-ca-cert-hash sha256:<CA_HASH> \
      --node-name karpenter-$(hostname)

  - systemctl enable kubelet

Replace <API_SERVER_ENDPOINT> and <CA_HASH> with values from step 2. Use the internal endpoint (:6443) since the VM will be on the private network.

4. Find Your UKS Network UUID

upctl kubernetes show <cluster-name>

Look for Network UUID in the output. Or pull it from an existing node:

kubectl get node <existing-node> -o jsonpath='{.metadata.annotations}' | grep upcloud-vm-private-nw-uuid

5. Create the VM

upctl server create \
  --zone de-fra1 \
  --hostname custom-worker-01 \
  --title "custom-worker-01" \
  --plan "CLOUDNATIVE-1xCPU-4GB" \
  --os "Debian GNU/Linux 13 (Trixie)" \
  --ssh-keys ~/.ssh/id_ed25519.pub \
  --network "type=public" \
  --network "type=utility" \
  --network "type=private,network=<uks-network-uuid>" \
  --enable-metadata \
  --user-data "$(cat cloud-init.yaml)" \
  --wait

Your zone must match the cluster's zone.

6. Verify

kubectl get nodes -o wide

Notes

  1. Your VM must share the cluster's region. SDN private networks are region-scoped. No cross-region attachments.
  2. API server port changes by context. The internal endpoint serves on 6443. The public load balancer uses 7443. VMs on the private network should use 6443.
  3. The CA hash in cluster-info ConfigMap may not match. I extracted the actual cert hash from the API server: openssl s_client -connect <host>:6443 </dev/null 2>/dev/null | openssl x509 -pubkey -noout | sha256sum.
  4. Debian puts CNI plugins in the wrong place. Kubelet looks in /usr/lib/cni. Cilium installs to /opt/cni/bin. The cloud-init script above creates a symlink. The UpCloud K8s template handles this.
  5. Kubelet picks the wrong IP. Without --node-ip, kubelet may register the public IP. The script above extracts the private IP from eth3 (the SDN interface) and passes it to kubeadm join. Confirm your interface numbering by running ip addr on an existing UKS node.
Kubernetes UpCloud kubeadm cloud-init Networking Cilium