clover's snow globe

This codebase contains declarative configuration to run my home server, a computer sitting in my closet used daily to power my activities. It first and foremost acts as a file store for my ongoing and archived projects for paper clover. Secondly, it contains a ton of apps for me and friends to use, from Jellyfin and Navidrome for media consumption to git hosting and render farms.

The system is built out of a NixOS base (just to install the system itself), using managed docker containers (managed via Nomad) and virtual machines to run all the real services. There is a lot of custom tooling to spawn interactive testing environments. For example, without even pushing any code, I can edit the configuration of a service or upgrade it, then spin up an isolated domain like shale-4c84f9da.staging.paperclover.net to verify changes. Then, pushing to production ensures that the new deployment is healthy before routing traffic to it -- It's kind of like "easy kubernetes."

deployment loop

python3 tools/deploy.py stage shale  # preview the working files, even without a commit message
python3 tools/deploy.py stage shale  # update the same URL and staging data after another edit
python3 tools/deploy.py publish      # upload the described, conflict-free main commit for GUI review
python3 tools/deploy.py prod         # upload and deploy main directly

Production always deploys a snapshot exported from main, never working files or a stage. The dashboard's deploy main page shows the uploaded commit and its message; deploying applies the full repository configuration and retains production data. A newer upload invalidates an older deployment confirmation. Sibling application build sources declared in build-source.json are bundled at upload time. Their frozen contents are part of the release digest.

main joins the infra-2 and home-infra histories. Its tree contains infra-2; the retired configuration remains available in the home-infra parent history.

installer

After publishing main, build the prepared USB image on an x86 Linux host:

nix build --extra-experimental-features 'nix-command flakes' path:/opt/studio/main#installer

The ISO is in result/iso/. It boots a live installer with this Mac's SSH key, ZFS and migration tools, the uploaded repository at /etc/infra-2, and a cached dashboard image. It does not install automatically. The physical installation uses #zenith after generating its hardware configuration; the existing data pool and service state follow the handoff.

On Zenith, the live installer uses the same wired addresses and gateway as the installed system. Once firmware boots the USB, connect from this Mac with ssh root@10.0.0.1. SSH starts automatically and accepts the admin key only; no monitor, local login, or DHCP address lookup is needed after USB boot.

filesystem layout

The computer mounts the ZFS root dataset under /srv, meaning "server," loosely following the linux convention. Within, it's a unique structure:

srv/
+--- clover/ [dataset]       user-data storage, what I mount as /Volumes/clover
     +--- Archive/<year>     personal archive. one folder per year, scrambled within.
     +--- Asset/             resources, sample packs, audio plugins, stock videos
          +--- Blender/      
          +--- Font/         (moved from zenith Documents/Font, will be stable index)
          +--- Samples/      
          +--- Texture/      
          +--- Video/      
     +--- Documents/
     +--- Project/           currently active project files
     +--- Media/ [dataset]   files from the world-wide-web. managed partially manually
          +--- jellyfin/
          +--- mirror/
          +--- music/        (used by navidrome)
          +--- music-intake/ (to be manually indexed)
          +--- seedbox/
          +--- vm/           (virtual machine images)
     +--- Published/         source of truth for `paperclover.net/file`
+--- prod/ [dataset]         production application data
     +--- shale/ [dataset]   (each service is its own dataset)
     +--- ...
+--- staging/                staging deployments use this space for temporary clones
     +--- postgres-9f06f74b/ [dataset]
+--- vm/ [dataset]           virtual machines
     +--- clover-sandbox/    [dataset]

Media is it's own dataset so it can be snapshotted independently of my personal data (less frequent, lower retention), and all my personal files are on the same dataset to allow fast move/copying between top level folders. Services use their own datasets to implement copy-on-write forks.

testing domains on a Mac

tools/mac-domains.py routes .studio.test and its nested subdomains through the rehearsal VM's HTTPS, DNS, and login services. Start the rehearsal SSH forwards first, then run:

sudo /usr/bin/python3 tools/mac-domains.py start /path/to/rehearsal-ca.crt

The relay binds loopback ports, drops administrator privileges, and passes TLS through to the VM. Local UDP DNS queries use the SSH tunnel's TCP DNS connection. Existing certificate trust is preserved. Mac DNS and hosts settings are backed up before editing; sudo /usr/bin/python3 tools/mac-domains.py stop restores them and stops the relay. Undo refuses to overwrite later edits. After a Mac restart, run stop and start again to restart the relay.

The dashboard package uses home-dashboard. Existing state paths, environment names, service IDs, and telemetry names keep their old names for compatibility.

dashboard boundary

NixOS runs studio-dashboard in a non-root Podman container with a read-only root, private network, resource limits, and explicit data mounts. The small studio-host service authenticates the dashboard UID on its Unix socket and performs bounded ZFS, VM, deployment, host-sampling, and identity operations. Host control sockets and management credentials stay outside the container.

The MCP tab manages separate observability, agent, and Shale catalogs through the existing Keycloak realm. Each connection has explicit service, machine, or repository grants; Shale credentials belong to the signed-in user. Agent Relay's existing outbound client protocol connects to the Rust server.

Build the image with nix build .#dashboard-image on Linux. Run python3 tools/dashboard-unit-test.py --output /tmp/dashboard-checks.json on the rehearsal VM to exercise the generated NixOS units, containment, IAM, and MCP connectors with disposable fixtures. --relay-agent-dir includes the existing Agent Relay client interoperability check; --browser-ready-file temporarily routes the public dashboard to the fixture for browser and SSO load checks.