The open-source Nix ecosystem is witnessing an exciting development with nix-ninja, a tool that promises to revolutionize compilation workflows by leveraging Nix's experimental dynamic derivations feature. This innovation has sparked significant interest in the developer community, particularly among those working with complex build systems.
Dynamic Derivations: The Path to Stable Release
One of the most discussed aspects in the community is the timeline for stabilizing dynamic derivations in Nix. Currently, nix-ninja depends on experimental features from an unreleased version of Nix, which raises questions about when these features might become officially supported.
A core Nix developer involved with dynamic derivations has provided some clarity on the roadmap, explaining that dynamic derivations depend on content-addressing derivations, which has been in experimental-feature limbo for some time. However, with focused effort and scope management, stabilization could be achievable in the near future. The developer characterized the remaining work as not actually a huge amount despite being a series of yak shaves - a term referring to a chain of seemingly unrelated tasks that must be completed before addressing the main goal.
Performance Potential for Complex Builds
The performance implications of nix-ninja are particularly promising. The tool aims to enable truly incremental compilation, where changing a single C/C++ file anywhere in a build graph would require recompiling only that file and relinking just the affected executables or shared libraries.
While still in development, early testing shows that Nix's sandboxing overhead is negligible for incremental compilation tasks. However, some features are still missing, such as proper handling of targets that depend on generated source files (like Nix's bison parser), which is currently being tracked as an open issue.
A long term end goal is that if you don't change headers and only change is single C/C++, anywhere in the build graph for your entire system you should get a quick recompile of just that file + relink just the executables/shared libraries the output object file is built in.
Key Features of nix-ninja
- Parses ninja.build files and generates a derivation per compilation unit
- Stores build inputs & outputs in content-addressed derivations for granular incrementality
- Compatible CLI for ninja (can be used as a drop-in replacement)
- Supports running locally or inside a Nix derivation
Requirements
- Nix with experimental features enabled:
- nix-command
- dynamic-derivations
- ca-derivations
- recursive-nix
Development Milestones
- 0.1.0: First release focusing on correctness
- 0.2.0: Major performance features to make incremental builds productive
Strategic Focus on Ninja Build Files
The developers chose to target Ninja build files specifically, which has proven to be a strategic decision. By supporting Ninja as the build graph representation, nix-ninja can work with numerous popular build systems like CMake, meson, premake, and gn that output Ninja files.
This approach is particularly relevant for the Nix community since Nix itself uses meson and outputs Ninja files. The choice allows for an incremental, crowd-sourceable approach to improving build performance across the ecosystem, starting with large projects like LLVM and Chromium that would benefit most from incremental compilation.
As nix-ninja continues to develop toward its 0.1.0 milestone focused on correctness and the subsequent 0.2.0 release targeting performance features, the project represents a promising step toward making Nix builds more efficient without requiring massive rewrites of existing build systems. For developers working with complex codebases, this tool could significantly improve productivity by reducing compilation times and providing more granular incrementality through Nix's content-addressed derivations.
Reference: nix-ninja