How to install Nix in a Qubes OS AppVM

I wrote how to install Nix in a Qube

9 Likes

Nix template is more interesting.

1 Like

I didn’t find any Nix related template even in the community testing repository :frowning:

I installed Guix OS in a HVM, but in fact it’s more useful in an AppVM, see the unman’s guide:

Nix is a package system, not an OS, so you can adapt the solene’s guide to a dedicated Nix templateVM (by cloning a Debian or Fedora templateVM).

I didn’t use Guix foreign for a long time, but using it in an AppVM (non-guix) should be pretty similar to Nix in stand alone right?

Using Nix in a template seems tricky to me, because you may want to provide a pre-filled store in the template, but you may also want to have additional changes in an AppVM, I don’t think the persistent mode allow to merge a template directory + a persistent directory :thinking:

1 Like

This could be improved soon when the stacking nix store will be available, see Super Colliding Nix Stores - Announcements - NixOS Discourse and my comment Super Colliding Nix Stores - #6 by Solene - Announcements - NixOS Discourse for the use case

2 Likes

Hi. I did some quick trials, using disposables to try out nix packages for software that I had never ran before, and ran into problems running most programs with errors about MESA, or GLX, or some other type of OpenGL error.

Could you try some GL related nix packages out on your setup and tell me if you have the same problem? One example:

nix-env -iA nixpkgs.oh-my-git
(it’s supposedly software that teaches you about git, but since it wont run I cant confirm that)

The full procedure I used was:
(starting from a debian based disposable)

sudo apt update
sudo apt install nix nix-setup-systemd

sudo adduser user nix-users

sudo su -
su - user
###the above 2 lines reloads the user so that the active user is in the nix-users group.

nix-channel --add https://nixos.org/channels/nixpkgs-unstable
nix-channel --update
###(update takes around 1 min)

PATH=$PATH:~/.nix-profile/bin

nix-env -iA nixpkgs.oh-my-git

oh-my-git

Doing the above returned:

user@host:~$ oh-my-git 
Godot Engine v3.5.2.stable.custom_build - https://godotengine.org
ERROR: Condition "!fbc" is true. Returned: ERR_UNCONFIGURED
   at: initialize (platform/x11/context_gl_x11.cpp:167)
ERROR: Error initializing GLAD
   at: is_viable (drivers/gles3/rasterizer_gles3.cpp:141)

(zenity:7195): dbind-WARNING **: 18:35:02.387: AT-SPI: Error retrieving accessibility bus address: org.freedesktop.DBus.Error.ServiceUnknown: The name org.a11y.Bus was not provided by any .service files

(plus a popup that says “Your video card driver does not support any of the supported OpenGL versions.
Please update your drivers or if you have a very old or integrated GPU, upgrade it.
If you have updated your graphics drivers recently, try rebooting.
Alternatively, you can force software rendering by running Godot with the LIBGL_ALWAYS_SOFTWARE=1
environment variable set, but this will be very slow.”)

Then doing:

user@host:~$ export LIBGL_ALWAYS_SOFTWARE=1
user@host:~$ oh-my-git

produces the same error

Qubes OS doesn’t have graphical hardware acceleration, so programs trying to do 3D operations may fail. Sometimes it can work because of the software rendering done on the CPU with the llvmpipe driver, but it’s a very limited driver with regards to 3D features.

You should ask oh-my-git why it needs OpenGL

Maybe related to a Nix issue though? Getting libGL in a devshell produced from a flake - Help - NixOS Discourse

I believe that qubes uses mesa in the individual qubes to do software rendering.

apt list --installed | grep -i mesa
returns:

libegl-mesa0/oldstable,now 20.3.5-1 amd64 [installed,automatic]
libgl1-mesa-dri/oldstable,now 20.3.5-1 amd64 [installed,automatic]
libglapi-mesa/oldstable,now 20.3.5-1 amd64 [installed,automatic]
libglx-mesa0/oldstable,now 20.3.5-1 amd64 [installed,automatic]

I’ve never had a problem running debian opengl problems under qubes (except occasionally trying to run a opengl program that has a performance problem, but that’s my own fault :slight_smile:

for example running amoeba in that qube after doing sudo apt install amoeba works fine. (clarification: I didn’t check it past the first few seconds but at least the first few seconds of 3d motion worked fine). However running amoeba in the exact same qube after uninstalling the debian version, and running time nix-env -iA nixpkgs.amoeba, results in:

user@host:~$ amoeba 

(process:415590): Gtk-WARNING **: 17:07:40.062: Locale not supported by C library.
	Using the fallback 'C' locale.

(amoeba:415590): dbind-WARNING **: 17:07:40.077: AT-SPI: Error retrieving accessibility bus address: org.freedesktop.DBus.Error.ServiceUnknown: The name org.a11y.Bus was not provided by any .service files
Unhandled exception: No usable visuals! (Try 32bpp mode if you are not already)

My guess is that for running nix applications, nix has replaced some of the debian graphics libraries and graphics configurations with nix versions of the graphics libraries and nix graphics configurations, and whatever magic was done to get 3d software emulation working in qubes debian, also needs to be done in the nix configuration.
However before pushing that theory, I think it would make sense to make sure that I didn’t screw up the nix install.
If you didn’t already, would you try running a nix package that uses OpenGL on your setup to see if you can see any nix package that uses OpenGL works? Every single nix OpenGL package that I have tried has given a similar error message. Every OpenGL program I have tried from debian packages has worked fine?

(Note: This is my first attempt at using nix, meaning my understanding of the “nix way” of configuring things will be quite limited and my ability to screw up a nix install should be quite high :slight_smile:

I appreciate the sentiment, but unfortunately many programs now require OpenGL that you would not expect. I believe I even heard that one of the big desktops (KDE or gnome) is planning to “drop support for 2d graphics” (however, I can’t seem to find a reference to where I read that)

Maybe? I don’t know enough about nix yet to be able to tell :slight_smile:

Nix bundles all the libs and doesn’t use the libraries from the host system, so your guess is right. There are regular issues with Nix and OpenGL… :frowning: are you using 23.11 or nixpkgs-unstable?

I’m using unstable. Or more precicely, I typed the commands:

nix-channel --add https://nixos.org/channels/nixpkgs-unstable
nix-channel --update

In this link: Differences between Nix channels - Help - NixOS Discourse
I see the following:

Despite all the docs advising nixos-* for NixOS and nixpkgs-* for NaaPM, there is theoretically nothing that prevents you from using nixos-* for NaaPM and nixpkgs-* for NixOS. It is merely assumed that NaaPM users already have a stable system and want cutting-edge packages, whilst NixOS users need to rely on a more stable set. Beware, however, that something may break, because maintainers will assume this convention is followed.

nixpkgs-unstable is a statement of fact rather than a variant - there is no nixpkgs-stable (except maybe for Darwin?)

The set of packages in nixpkgs-unstable is not related in any way to nixos-unstable. nixos-unstable is only slighty less stable than nixos-stable (and is intended for rolling updates), nixpkgs-unstable is quite a bit looser.

so, acording to that, it’s fine to use nixos, not just nixpkgs. So I should use nixos-23.11 instead? I.E. you are proposing:

nix-channel --add https://nixos.org/channels/nixos-23.11
nix-channel --update

?
I can do that. I’ll do it in a brand new qube.

:frowning: That’s dissapointing.

Knowing that, a search shows:
https://nixos.wiki/wiki/Nixpkgs_with_OpenGL_on_non-NixOS

Which says that aparently OpenGL with nix package manager is a problem, but OpenGL with nixOS works fine.

Then points to this as a possible solution:

So the 2 potential solutions seem to be

  1. nixGL with nix, or
  2. full NixOS template for qubes.

As for a NixOS template for qubes:
Did you notice nix has a qubes package?: NixOS Search

It looks like someone started the packaging that would be required for a NixOS template on qubes, but then stopped.

There are also several people posting about it, but nobody actually doing it :slight_smile:

As for NixGL:
I’ll try it and get back to you.

Also, this (especially the 3rd comment) might be helpful if nixGL does not work out:

I remember using it now you linked it, it should work fine, but it’s annoying to have to use it :frowning:

Sound is broken, but OpenGL now works. Thanks for your help. Pointing me in the right direction saved me lots of time. Did you do anything special to get sound working?

Here’s the new, revised procedure to run amoeba via nix:

(start with a debian disposableVM)

sudo apt update
sudo apt install nix nix-setup-systemd

sudo adduser user nix-users

sudo su -
su - user
###the above 2 lines reloads the user so that the active user is in the nix-users group.

nix-channel --add https://nixos.org/channels/nixpkgs-unstable
nix-channel --update
###(update takes around 1 min)

####POSSIBLE OPENGL FIX?
nix-channel --add https://github.com/guibou/nixGL/archive/main.tar.gz nixgl
nix-channel --update
export NIX_PATH=/nix/var/nix/profiles/per-user/user/channels
nix-env -iA nixgl.auto.nixGLDefault   # or replace `nixGLDefault` with your desired wrapper

PATH=$PATH:~/.nix-profile/bin

nix-env -iA nixpkgs.amoeba
nixGL amoeba

(select “no sound” at the first menu)

try to install pulseaudio?

Found the issues. Aparently the audio problem was related to installing it the “debian way” (I.E.:
sudo apt install nix nix-setup-systemd
)

instead of the “nix way” (I.E.:
sh <(curl -L https://nixos.org/nix/install) --no-daemon
)

Anyway, on the nixGL side, I would recommend adding a comment about nixGL somewhere in your instructions.

Thanks for your help

I assume he was referencing nixOS for a template.

For anyone who does not already know, what he said about nix being a package system that can be installed in a cloned debian or fedora templateVM is correct. Then there is NixOS which is a entire distribution based on nix packages, which would require a full NixOS template, at least one project has started porting the qubes packages to nixpkgs, but did not finish, so to the best of my knowledge, a NixOS template is not available at this time.

Lastly, when searching for nix documentation, you often find NixOS documentation, which is not necessarily obvious whether if it also applies for nix packages or not, so a NixOS template would be easier to find documentation for.

@solene Thank you so much for this guide. I used it as part of testing of Use nix for reproducible build env by mmlb · Pull Request #1269 · linuxboot/heads · GitHub just wanted to let you know! Might have some improvements that I will suggest :slight_smile:

2 Likes

Thank you for your awesome guide.
It really helped me setting things up.

There is one addition/ suggestion I’d like to make.

QubesOS’ bind-dirs seem to attach to the sys / root partition.

That means in an AppVM based off a standard template e.g. debian-12-xfce sth like nix-shell -p htop will work, while sth like nix-shell -p eclipses.eclipse-java will not.

RN I’m having trouble finding the original NixOS’ discourse post that got me to that direction (will add it later when I find it), but the solution was to bind-mount to another location.

What I ended up with was appending this to /rw/config/rc.local:

sudo mkdir -p /nix /home/nix && sudo mount -o bind /home/nix /nix 

same applies to guix of course

sudo mkdir -p /gnu /home/gnu && sudo mount -o bind /home/gnu /gnu 

For my purposes, this aligns more neatly with other software, like virtualenv, pipx, ghcup, rustup, cargo, etc that all tend to install their stuff into /home somewhere.
It gives me “thin” templates and “fat” AppVMs.

1 Like