L0 — Foundation (host, network, storage)
The physical + virtualization base everything runs on.
KVM host dl385-2
| Item | Value |
|---|---|
| OS | Ubuntu 24.04, system libvirt |
| CPU / RAM | 512 vCPU (2× AMD EPYC) / 1.5 TiB |
| Access | ssh ze@dl385-2 (passwordless sudo) |
| Nested virt | kvm_amd nested=1 → cluster nodes get /dev/kvm (--cpu host-passthrough) for CNV |
| oc-mirror kit | ~/ocp420-airgap/ (oc / oc-mirror in tools/bin, not in non-login PATH) |
| VMs | ~52 running (see Inventory) |
⚠️ The 5
ibbl-replica-lab-v3-*VMs are a separate lab — do not touch without explicit go-ahead.
Networking
| Item | Value |
|---|---|
| Lab network | libvirt ocpnet / virbr-ocp, 192.168.130.0/24 |
| Domain | airgap.lab (all *.airgap.lab resolve on the isolated net) |
| Bastion 2nd NIC | 192.168.122.61 (libvirt default net, reachable from dl385-2) |
| DNS / DHCP | dnsmasq on bastion (/etc/dnsmasq.d/ocp420.conf) |
| NTP | chrony on bastion .10 (PCI-DSS 10.4) — every node/VM syncs here |
| Reach from laptop | sshuttle -r ze@dl385-2 192.168.130.0/24 192.168.122.0/24 |
Storage layout & the etcd lesson
- All VM disks (OS + data, ~102 qcow2) live under
/var/lib/libvirt/imageson one physical volume (/dev/sda3, HPE MR408i HDD-RAID, 13 TB). - etcd is latency-sensitive. With ~52 VMs contending one HDD-RAID volume, disk
r_awaithit 50–171 ms → etcd fsync/heartbeat misses → leader flapping → cluster-wide API timeouts (context deadline exceeded), which wedged controllers. - Rule: keep the master VMs’ disks (or at least
/var/lib/etcd) on fast/dedicated storage (NVMe/SSD), and keep VM density per physical disk sane. See Gotchas.
GitOps vs. out-of-band
All of L0 is host-level / manual (libvirt, DNS, NTP, disk). Nothing here is GitOps.