Who am I

• Dionysis Grigoropoulos (dgrig or Erethon)
• https://erethon.com
• SRE by trade, hackerman at heart

• email: dgrig@erethon.com
• matrix: @dgrig:erethon.com
• github: @erethon

Building blocks for building NixOS

nix-shell -p bash
 this path will be fetched
  (0.06 MiB download, 0.31 MiB unpacked):
  /nix/store/<hash>-bash-interactive-5.2p37-dev
 copying path '/nix/store/<hash>-bash-interactive-5.2p37-dev'
  from 'https://cache.nixos.org'...

• Not magic, this information is public in our git repos, anyone can figure this out.

Binary caches

• Nix can reuse already built binary packages from another store

man nix.conf
 substituters
  A list of URLs of Nix stores to be used as substituters,
  separated by whitespace. A substituter is an additional
  store from which Lix can obtain store objects instead of
  building them.

$ cat /etc/nix/nix.conf
  substituters = https://cache.nixos.org/

cache.nixos.org - DNS

$ whois nixos.org | grep -i "registrar url"
  Registrar URL: http://www.gandi.net

$ dig +short NS nixos.org
  ns-177-a.gandi.net.
  ns-61-b.gandi.net.
  ns-161-c.gandi.net.

dig +short cache.nixos.org
  dualstack.v2.shared.global.fastly.net.

cache.nixos.org - DNS

• Managed in github:nixos/infra/dns using DNSControl

~ cat nixos/infra/dns/nixos.org.js
  D("nixos.org",
    REG_NONE,
    DnsProvider(DSP_GANDI),
    ...
    CNAME("cache", "dualstack.v2.shared.global.fastly.net."),
    A("pluto", "37.27.99.100"))

cache.nixos.org - CDN

• Acts as a global CDN allowing for faster pull times
• Fastly serves the landing page, the narinfo and nar.xz files of packages

• Also used in (artifacts|channels|releases|tarballs).nixos.org

• Managed in github:nixos/infra/terraform/cache.tf using Terraform/OpenTofu

cache.nixos.org - Storage

• Amazon Web Services S3 for storing the built artifacts

• ~705 TB as of 2025/08/07, grows at ~210GB/day
• @arianvp shared a cost breakdown, ~15k USD/m
• Managed in github:nixos/infra/terraform/cache.tf using Terraform/OpenTofu

cache.nixos.org - Architecture

cache.nixos.org - Trust?

• Do we need to trust Amazon, Fastly or Gandi?

$ man nix.conf
 At least one of the following conditions must be met for
 Lix to accept copying a store object from another Nix
 store (such as a substituter):

 - the store object has been signed using a key in the
 trusted keys list

$ cat /etc/nix/nix.conf
substituters = https://cache.nixos.org/
trusted-public-keys =
 cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=

• However, we bootstrap our trust over these companies

The tools that build NixOS

Hydra - The NixOS CI

• Perl based Continuous Integration system
  • Runs NixOS tests, builds packages, signs them and uploads them to the cache

• hydra.nixos.org
  • 1 coordinator node & 1 database node
  • 2 x86_64-linux, 2 aarch64-linux builder nodes
  • 7 darwin builder nodes (5 M1, 2 M2)

Hydra - Infrastructure

• Coordinator, database, Linux and M1 darwin build hosts are hosted in Hetzner
• M2 darwin hosts are hosted in OakHost

Hydra - Architecture

GitHub - Source of truth

• Hosting all of NixOS repositories, issues, pull requests

• Nixpkgs has ~8800 open issues, ~7800 open pull requests

• GitHub Actions
  • Periodic labeling of PRs
  • linting, nixpkgs-vet, ping maintainers, eval
  • Uses Cachix as a shared cache
  • 900 thousand minutes run time in July

GitHub - Architecture

OfBorg - PR time builds and NixOS tests

• Builds packages and runs tests on PRs (before merge!)

• 3 x86_64, 2 aarch64 darwin hosts
• 5 builders, 1 core, 4 eval hosts

• Darwin builders in MacStadium, linux builders in tetaneutral.net and the core host is on Hetzner Cloud

• No trust is placed on OfBorg, since its built artifacts aren't re-used anywhere

Conclusion

• We only trust the Hydra infrastructure and the nixpkgs source code

• It's easy to get involved in all of the above aspects of NixOS. Usually, fixing something is only a pull request away.

Thank you - Questions?

• email: dgrig@erethon.com
• matrix: @dgrig:erethon.com