unaen.org

About this homelab

4 min read

About this homelab

unaen.org is a homelab run as if it were production. Every service is declared in code, every change goes through a merge request, and what was figured out once gets written down here so it stays figured out. It is also a deliberate testbed: patterns are tried here first, at small scale, before they are used on much larger networks. So the designs favour the general and repeatable over the clever one-off.

Compute

  • A three-node Proxmox VE cluster. Two Xeon nodes carry the workloads, and a low-power Atom board provides the third quorum vote. Ceph (Squid) backs the VM disks, so guests can move between nodes and restart elsewhere under HA.
  • Backups go to a Proxmox Backup Server.
  • A small AMD APU cluster (Ryzen nodes with integrated Radeon GPUs) for AI and HPC experiments. It serves local LLMs through llama.cpp over Vulkan behind an OpenAI-compatible endpoint, and tests how far shared-memory iGPUs can be pushed.

Network

  • A VyOS router with separate VLANs for home devices, servers, management and a public segment. The forward chains drop by default, and external access is a narrow pinhole per host and service, never a blanket allow. New firewall rules are proven first with count-only probes and applied with commit-confirm.
  • IPv6 everywhere. Prefix delegation from the ISP sits alongside static gateway addresses, so a lapsed lease cannot take a VLAN down.
  • An InfiniBand fabric (Mellanox SX6005, ConnectX-3) for storage traffic, including NFS over RDMA. SR-IOV hands virtual functions only to the guests that really need them.
  • Two uplinks: a wired ISP plus Starlink. A WireGuard mesh routed with Babel (FRR) keeps the off-site nodes reachable whichever uplink is alive.
  • Remote access over two independent overlays: a self-hosted NetBird control plane and Tailscale. Both are kept on purpose, so an upgrade or a network that blocks one never locks anyone out.

Storage

  • ZFS pools on a storage server feed Proxmox over iSCSI through a Salt-driven control plane, with a deletion-proof backstop behind it. The largest consumer is a digital library of about 100 TiB.
  • JuiceFS (Redis metadata and S3-compatible object storage) provides a mount-anywhere filesystem for workstations, compute nodes and models.

Identity, certificates and DNS

  • FreeIPA handles users, hosts, Kerberos and LDAP. Its Dogtag CA issues the internal TLS certificates, and Keycloak provides single sign-on (OIDC).
  • Public certificates come from Let’s Encrypt via lego, using HTTP-01 or DNS-01 through a delegated acme-dns. A separate, standalone CA issues the client certificates the automation uses for mutual TLS.
  • DNS:
    • The public zones are edited as files in git and served by a hidden PowerDNS primary. Four public secondaries pull them over TSIG-signed zone transfers.
    • Internal names come from FreeIPA.
    • At home, dnsdist routes each query to the right place, including straight to the primary for the homelab’s own domains.

The public edge

There is exactly one public web edge: a small off-site VM. It hosts nothing of its own. Public sites, this page included, live at home and are reverse-proxied and cached by the edge over the WireGuard mesh. If home is unreachable, the edge keeps serving the last copy.

How it is run

  • Salt describes every host. States are served from a self-hosted GitLab, and every change is a merge request with unit tests. The house rule is that a converged host reports zero changes on re-apply, so any change in the output is real drift.
  • Secrets never go in git. They are generated on the host that needs them, or delivered by reference, and the automation is careful never to echo them into logs or job output.
  • An AI agent works alongside the operator. It has its own identity, with tiered permissions, LDAP-backed authentication and mutual-TLS client certificates, and it reaches the fleet through Salt’s API via an MCP server (how it works and its security model: mcp-salt-overview). It reads and proposes much like a junior engineer:
    • it forks repositories and opens merge requests, and a human merges them;
    • every state apply is a preview first, and is confirmed only once the diff has been reviewed;
    • it also keeps this wiki up to date.

This wiki

The wiki is a git repository of Markdown pages, rendered with Quartz. Most pages are internal. A page is public only when its frontmatter says publish: true. Public pages are then built as a separate site, so the search index, graph and feeds of the public site can never mention a page that wasn’t published.

Graph View

Backlinks