Skip to content
KirthPublic

About

NixOS OpenStack Deployment Tooling

Resources

Stars

1 star

Watchers

0 watching

Forks

Repository files navigation

NOStack

NOStack builds OpenStack clouds on NixOS from one inventory file. You describe the cloud (nodes, roles, addresses, NIC MAC addresses, the API VIP and the three networks) in inventory.nix. Every node's NixOS configuration, the deploy order and all HA wiring (Galera, OVN Raft, RabbitMQ, keepalived, HAProxy) are derived from it.

Supported layout: 3 controllers and any number of compute nodes. NOStack builds its own OpenStack 2026.1 packages on nixos-26.05 (pkgs/openstack/).

  • Controllers run MariaDB/Galera, RabbitMQ, HAProxy/keepalived, Keystone, Placement, Glance, the Nova control plane, the Neutron server, and the OVN NB/SB databases with ovn-northd. They are also the network gateway chassis.
  • Computes run nova-compute, libvirt/QEMU, OVS and ovn-controller.

Layout

  • flake.nix: library outputs (lib.mkCloud, nixosModules.default, templates.default) plus the lab instance.
  • lib/inventory.nix: the inventory schema, validation and derived facts.
  • lib/mkCloud.nix: inventory to nixosConfigurations, Colmena hive and inventory (JSON for tools).
  • pkgs/openstack/: the OpenStack package set (see its README).
  • modules/openstack/: the NixOS modules. inventory.nix maps the inventory onto them; wsgi.nix runs the API services under uWSGI.
  • templates/default/: starting point for your own cloud.
  • examples/lab/: the Incus dev lab (inventory, guest profile and tooling).
  • docs/: topology and bootstrap notes. secrets/: the secret contract.

Your own cloud

nix flake init -t github:nostack/nostack   # placeholder until published
$EDITOR inventory.nix hardware.nix
nix eval .#inventory --json | jq .order

To add a compute node, add one entry under nodes and deploy it.

The Incus dev lab

examples/lab/inventory.nix describes four VMs on this machine: three controllers and one compute, on Incus bridges incusbr0 (management), incusbr1 (tunnel) and incusbr2 (provider; the host is its gateway). The repo root's nixosConfigurations, colmenaHive and inventory are this lab. Run the tooling from the repository root:

# create (or recreate) the lab from scratch
INCUS=incus examples/lab/rebuild.sh
rill examples/lab/init.ril --rebuild_images --reimport_images --direct_incus

# networks, image, flavor, a test VM and a floating IP reachable from the host
rill examples/lab/bootstrap.ril --direct_incus

init.ril --dry_run prints every Incus change without making it.

Everyday changes: push deploys

examples/lab/deploy.sh                          # all nodes, in inventory order
examples/lab/deploy.sh nostack-controller-2     # one node
examples/lab/deploy.sh --dry-run nostack-compute

This builds locally and pushes the system over SSH with Colmena; disk images aren't rebuilt. --dry-run runs Colmena's dry-activate goal, which lists the units that would restart without switching. After each node, deploy.sh runs nostack-converge on it and doesn't move on until it passes:

  • it retries any unit that failed earlier;
  • it fails the deploy, printing the failed units' logs, if anything is still failed;
  • on controllers, it waits until all three Galera members are Primary.

The Colmena hive and the images build the same system derivation, so push deploys are the everyday loop. Rebuild from scratch now and then, to prove a cold start still works.

Cluster bootstrap

The controllers decide among themselves which one starts the Galera cluster (nostack-galera-elect, see docs/bootstrap-order.md):

  • A node joins any controller that is already a Primary member.
  • If none is, the controllers bootstrap only once all three have reported their position, and the one with the most recent data starts the cluster.

Boot order doesn't matter, and no manual bootstrap step is needed. Inspect the election with nostack-galera-status on any controller.

NICs are renamed by MAC address to mgmt, tunnel and provider, so PCI order and hypervisor device ordering can't swap networks.

Service configuration

Every OpenStack config file is rendered from Nix. Each service has a freeform settings option holding its INI sections. NOStack sets its own values with mkDefault, so you can override any key, or add one, from a node's modules in the inventory:

modules = [{
  nostack.openstack.nova.settings.DEFAULT.cpu_allocation_ratio = 4.0;
  nostack.openstack.neutron.ml2Settings.ml2.extension_drivers = "port_security,qos";
}];

The available options are keystone.settings, glance.settings, placement.settings, nova.settings, neutron.settings and neutron.ml2Settings.

Secrets never enter the Nix store. The store holds templates with placeholders such as @nova_database_password@, and a nostack-config-<file> unit fills them in at boot from /run/secrets/openstack/ into /run/nostack/<service>/. The usual /etc/<service>/ paths are symlinks to those files.

Each service restarts when its rendered template changes. A deploy restarts only the services whose config changed; deploy.sh --dry-run lists them first. The nostack-*-bootstrap units only run database setup and migrations, once per cluster (docs/bootstrap-order.md).

Neutron uses ML2/OVN only.

Known gaps

See docs/known-issues.md for details and planned fixes. The main ones:

  • the noVNC console proxy can stop accepting connections;
  • services write to their local Galera node instead of a single writer;
  • secrets are fixed development values (docs/secrets.md), and there's no TLS.

Checks

nix flake check path:$PWD    # evaluates every lab node and the template

If the sandbox cannot write to the default Nix cache directory, use XDG_CACHE_HOME=/tmp/nix-cache nix flake check path:$PWD.

About

NixOS OpenStack Deployment Tooling

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages