|
xrpld
|
This guide explains how to use Nix to set up a reproducible development environment for xrpld. Using Nix eliminates the need to manually install utilities and ensures consistent tooling across different machines.
The Nix development shell is the recommended way to develop xrpld. It unifies the development environment for everyone and synchronizes updates: the same tooling and compiler versions are used both here and in CI. Any custom environment (Homebrew packages or anything else) will continue to work, but then it is up to you to keep it in sync with the environment used in CI.
Please follow the official installation instructions of nix package manager for your system.
From the root of the xrpld repository, enter the default development shell:
This will:
The first time you run this command, it will take a few minutes to download and build the environment. Subsequent runs will be much faster.
for the custom-vs-plain toolchain trade-off.
A compiler can be chosen by providing its name with the .# prefix, e.g. nix develop .#clang.
On Linux, .#gcc and .#clang provide the exact toolchain CI uses: the compiler (pinned in nix/packages.nix) rebuilt against the pinned custom glibc (see nix/linux.nix). Building that toolchain the first time is slow unless it is fetched from a Nix binary cache. If you don't need the custom glibc, the Linux-only .#gcc-plain and .#clang-plain give you the stock nixpkgs compilers of the same versions. On macOS there is no custom glibc, so .#gcc and .#clang are already the plain nixpkgs toolchain, and the -plain variants do not exist.
Use nix flake show to see all the available development shells.
Use nix develop .#no-compiler to use the compiler from your system.
nix develop opens bash by default. To use another shell, pass it with the -c flag — this works with any shell, e.g. zsh or fish:
Once inside the Nix development shell, follow the standard build instructions. The Nix shell provides all necessary tools (CMake, Ninja, Conan, etc.).
Coverage builds (-Dcoverage=ON) work in the gcc shell (and gcc-plain on Linux): each ships a gcov matching its compiler, since Nix's cc-wrapper does not expose one. The clang shells do not include llvm-cov, so use a gcc shell for coverage.
Builds of the Rust crates (-Drust=ON) also work out of the box: every shell provides the Rust toolchain pinned in rust-toolchain.toml (see Rust), plus the cargo-audit, cargo-llvm-cov and cargo-nextest plugins.
The shell runs conan/init.sh on entry, so Set Up Conan is already done for you. It installs into the shell's own Conan home: CONAN_HOME=~/.conan2-nix.
On Linux, the binaries on the xrplf remote are built in this same Nix environment — CI runs in Docker images that bundle the dev shell's toolchain (see nix/docker) — so .#gcc and .#clang can reuse them. The -plain shells do not match that toolchain's glibc, so binaries from the remote are not a reliable match there.
On macOS, CI also builds in this Nix environment, in Debug and Release (the macos-arm64-*-nix configurations — Debug because the profile defaults to it). The Nix build resolves to compiler=clang, so it gets its own package IDs, separate from the Apple Clang ones. The dependency upload publishes them on pushes to develop and on manual runs — its nightly run rebuilds everything from source but uploads nothing — so once a set has been published nix develop can reuse it instead of compiling every dependency locally. These configurations run outside the reduced pull-request matrix, so label a PR Full CI build when it touches flake.lock or nix/.
To compile everything from source, add --build '*' to the conan install command.
A Conan package ID records the compiler and its major version, but nothing about the nixpkgs revision the toolchain came from — and flake.lock moves far more often than the toolchain meaningfully changes, so folding it in would rebuild every dependency on every bump for nothing.
That is safe as long as no cached artifact resolves a /nix/store path at run time, because store paths change on every update and the old ones disappear with nix-collect-garbage. With the clang toolchain macOS CI and the dev shell use, they do not: it links against /usr/lib/libc++ and /usr/lib/libSystem, and store paths reach the .a files only through debug info, which nothing resolves at link or run time.
This is checked rather than assumed. bin/check-nix-store-refs.sh takes one file or directory and fails if a binary under it resolves a store path at run time. CI runs it over the build output and the Conan cache, and again in the upload job before anything is published. You can run it yourself:
It works on Linux too, but asserts something narrower there: the toolchain always writes the store into PT_INTERP and RUNPATH. That is fine for the pinned glibc, whose path does not move, but not for the GCC runtime, which moves with every GCC update. So conan/profiles/default links build-context packages, whose executables run during the build, with -static-libstdc++ -static-libgcc -Wl,--as-needed, and conan/profiles/sanitizers does not instrument them. CI checks that they load nothing from the store but glibc, from the graph conan install --format=json writes:
Only the binaries PatchNixBinary.cmake retargets to the system loader have to be fully clean, and those are what CI checks:
This is not hypothetical: xrpld used to be caught by it. The c-ares package tells the linker to pass -lresolv, and nixpkgs keeps libresolv out of the macOS SDK and ships it as an ordinary store dylib — so every Nix-built xrpld recorded a /nix/store/…-libresolv-93/lib/libresolv.9.dylib load command and stopped running once that path was collected. Nothing in the link uses a single symbol from it.
Both environments now put a stub on the linker search path (libresolvSystemStub in nix/darwin.nix): the same library with its install name set to /usr/lib/libresolv.9.dylib, which is exactly the load command the Apple Clang build records.
Package IDs did not change, so Conan keeps serving anything built before the stub landed. If a binary fails to start with Library not loaded: /nix/store/…, see that entry in the troubleshooting guide.
direnv or nix-direnv can automatically activate the Nix development shell when you enter the repository directory.
This is also the most robust way to use the environment from any shell (bash, zsh, fish, …): direnv stays in your current shell and loads the environment after your shell's startup files have run, so the Nix-provided tools take precedence over anything your shell configuration adds to $PATH.
The repository already ships an .envrc at its root that activates the Nix flake development shell, so you don't need to create one. To use it:
To update flake.lock to the latest revision use nix flake update command.
The tool versions in each Nix environment are recorded in nix/check-tools/ and verified by CI. If you change the environment (bump the CI image tag, update flake.lock, or edit the tool list in bin/check-tools.sh), CI fails until you regenerate and commit the affected snapshot — see nix/check-tools/README.md.
See Troubleshooting Nix problems for common issues, such as nix develop failing inside Git worktrees.