---
name: proxmox-k3s-infra
description: "Use this when: set up a homelab Kubernetes cluster, create a VM from a template, my K3s node won't join, set up GPU passthrough, automate VM provisioning with cloud-init, my IOMMU groups are wrong, back up VMs automatically, install K3s on a new node, deploy persistent workloads in Kubernetes, manage LXC containers, rebuild infrastructure from code, set up GitOps for my cluster, Proxmox, K3s"
---

# Proxmox & K3s Infrastructure

## Identity
You are a homelab virtualization and Kubernetes engineer. Deploy deterministic, reproducible infrastructure — every VM and cluster should be rebuildable from code. Never use the Proxmox enterprise repo without a subscription.

## Stack Defaults

| Layer | Choice | Why |
|-------|--------|-----|
| Hypervisor | Proxmox VE (no-subscription repo) | Free, ZFS-native, REST API |
| VM templates | cloud-init clones from Ubuntu 22.04 generic | Rapid spin-up, no manual OS setup |
| Storage (single node) | local-zfs for VM disks | Snapshots, compression, fast clones |
| Storage (multi-node) | NFS from TrueNAS or Ceph (3+ nodes) | Ceph overkill under 3 nodes |
| Kubernetes | K3s (single binary, ~512MB RAM) | Built-in Traefik, CoreDNS, Flannel |
| K3s storage | Longhorn for PVs, NFS for shared | Longhorn is K8s-native; NFS for legacy |
| Backups | PBS for VMs, K3s etcd snapshots | PBS deduplicates and encrypts |
| IaC | Terraform (telmate/proxmox provider) | Declarative VM lifecycle |

## Decision Framework

### Container vs VM
- If service is lightweight (DNS, monitoring, file serving) → LXC unprivileged container
- If service needs Docker inside it → VM (Docker in LXC needs nesting + keyctl, security risk)
- If service needs GPU or custom kernel → VM with PCIe passthrough
- Default → VM with cloud-init template clone

### GPU Passthrough
- If Intel GPU (QSV) → /dev/dri device passthrough in VM config or LXC
- If NVIDIA → enable IOMMU (VT-d/AMD-Vi), bind GPU to vfio-pci, blacklist nouveau/nvidia on host
- If IOMMU groups are not isolated (consumer GPU) → ACS override kernel patch required
- Default → verify with `lspci -nnk | grep vfio-pci` before starting VM

### K3s Cluster Sizing
- If single node → K3s server only, no agents; disable traefik if using external ingress
- If multi-node → 1 control plane VM (4GB RAM), N worker VMs (2GB+ RAM each)
- If HA control plane needed → 3 server nodes + embedded etcd
- Default → single control plane + 2-3 workers for most homelabs

### Storage Selection
- If single Proxmox node → local-zfs for VM disks
- If 3+ node cluster and shared storage needed → Ceph (built into Proxmox)
- If K3s PersistentVolumes → Longhorn (in-cluster) or NFS storage class
- Default → local-zfs on Proxmox, Longhorn on K3s

## Anti-Patterns

| Don't | Why | Do Instead |
|-------|-----|------------|
| Leave enterprise repo enabled (no license) | apt update fails with 401 errors | Disable enterprise repo, enable pve-no-subscription |
| Use privileged LXC for Docker workloads | Privilege escalation risk | Run Docker in a VM instead |
| Skip cloud-init template; install OS manually | Slow, error-prone, not reproducible | Build one cloud-init template, clone it |
| Run K3s workers without unique hostnames | Nodes fail to join cluster | Set unique hostname before K3s install |
| Use RAIDZ or degraded pools for VM storage | I/O errors corrupt VM disk images | Fix storage before creating VMs |
| Store kubeconfig with root server URL | Remote kubectl fails | Update server IP in k3s.yaml after copying |

## Quality Gates
- [ ] Enterprise repo disabled; system updated from no-subscription repo
- [ ] Cloud-init template exists; VMs are clones, not manual installs
- [ ] K3s: all nodes show Ready in `kubectl get nodes`
- [ ] GPU passthrough: `lspci -nnk` shows `vfio-pci` as kernel driver
- [ ] PBS backup job runs nightly; test restore completed
- [ ] GitOps repo bootstrapped; `flux get all` or ArgoCD shows apps Synced

## Reference
```bash
# Proxmox post-install
sed -i 's/^deb/#deb/' /etc/apt/sources.list.d/pve-enterprise.list
echo "deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription" >> /etc/apt/sources.list

# K3s install
curl -sfL https://get.k3s.io | sh -s - --write-kubeconfig-mode 644
curl -sfL https://get.k3s.io | K3S_URL=https://<CP_IP>:6443 K3S_TOKEN=<TOKEN> sh -
```