Persistent System Logs in Qubes OS using rsyslog and qrexec

Community Guides
1

This guide is made using AI.
The model used was qwen3.5:35B-A3B

This guide details how to set up an rsyslog service in Qubes OS using qrexec as the communication channel. This setup enables the centralization of persistent logs from multiple AppVMs onto a single target, facilitating better system debugging and forensics.

Create the rsyslog template

You can skip this step if you already have a template with rsyslog installed.

In dom0, install debian-13-minimal.

qvm-template install debian-13-minimal

In dom0, clone the template.

qvm-clone debian-13-minimal debian-rsyslog

In dom0, open a terminal with root privileges in the template.

qvm-run --user root debian-rsyslog xterm

In the template, make the following changes.

# Update and upgrade system packages
apt update && apt upgrade -y 

# Install Qubes OS agent packages
apt install qubes-core-agent-passwordless-root \
qubes-core-agent-networking \

# Install system utilities
apt install vim rsyslog

# Disable rsyslog
systemctl stop rsyslog
systemctl disable rsyslog
systemctl mask rsyslog 

# Configure locales
dpkg-reconfigure locales 

#Shutdown the template
reboot

Create the sys-rsyslog AppVM

This AppVM will be running the rsyslog service, and data persistence will be set up using dir-binds.

In dom0, create the AppVM

qvm-create --class AppVM --template debian-rsyslog --label red sys-rsyslog

In dom0, open the terminal in the AppVM

qvm-run sys-rsyslog xterm

Configure bind-dirs

sudo mkdir -p /rw/bind-dirs/var/log/qubeslogs
sudo mkdir -p /rw/config/qubes-bind-dirs.d
sudo touch /rw/config/qubes-bind-dirs.d/50_user.conf

Add the following content to /rw/config/qubes-bind-dirs.d/50_user.conf

# rsyslog needs to modified
binds+=('/etc/rsyslog.conf')

# qubeslogs will contain the persistant logs
binds+=('/var/log/qubeslogs')

Restart sys-rsyslog to activate the binds.

Add the following content to /rw/config/rc.local

# rsyslog used by qubeslogs 
systemctl unmask rsyslog.service
systemctl enable rsyslog.service
systemctl start rsyslog.service

Modify /etc/rsyslog.conf
Remove # from the TCP module and input lines.

# used by qubeslog
module(load=imtcp)
input(type="imtcp" port="514")

Add the following content

# create logfiles for each qube by hostname
$template QubeslogsHostFormat,"/var/log/qubeslogs/%hostname%.log"
*.* ?QubeslogsHostFormat

Restart sys-rsyslog to start rsyslog.

Configure rsyslog client-side

In this example I will be using debian-rsyslog, but this should be the template of the appVM where you want to run rsyslog.

In dom0, open a terminal with root privileges in the template.

qvm-run --user root debian-rsyslog xterm

Install rsyslog

apt install rsyslog

# disable rsyslog
systemctl stop rsyslog
systemctl disable rsyslog
systemctl mask rsyslog

Edit the file /etc/rsyslog.conf, and add the following content

# custom log template used by qubeslog
template(name="QubeslogFormat" type="list"){
	property(name="timegenerated")
	constant(value=" ")
	constant(value="hostname")
	constant(value=" ")
	property(name="syslogtag")
	property(name="msg" droplastlf="on")
	constant(value="\n")
}

# remote logging should only be active in the appVM
#*.* @@127.0.0.1:5140;QubeslogFormat

This is added to rsyslog.conf in the template because it makes it easier to reuse it in any appVM that uses the template.

Shut down the template.

In this example I will be using rsyslog-client as the appVM, but this should be the appVM where you want to run rsyslog.

In dom0, open a terminal in the appVM

qvm-run rsyslog-client xterm

Create the file /rw/config/qubes-bind-dirs.d/50_user.conf

sudo mkdir -p /rw/config/qubes-bind-dirs.d
sudo touch /rw/config/qubes-bind-dirs.d/50_user.conf

Add the following content to /rw/config/qubes-bind-dirs.d/50_user.conf

# rsyslog needs to modified
binds+=('/etc/rsyslog.conf')

Restart to active the binds.

Create the file /rw/config/qubeslog.socket with the following content

[Unit]
Description=qubeslog socket

[Socket]
ListenStream=127.0.0.1:5140
Accept=true

[Install]
WantedBy=socket.target

Create the file /rw/config/qubeslog@.service with the following content

[Unit]
Description=qubeslog service
StartLimitBurst=3
StartLimitIntervalSec=15
Requires=qubeslog.socket
After=qubeslog.socket

[Service]
Type=simple
ExecStart=qrexec-client-vm '' qubes.ConnectTCP+514
StandardInput=socket
StandardOutput=inherit
Restart=on-failure
RestartSec=5

Edit the file /rw/config/rc.local and add the following content

cp -r /rw/config/qubeslog* /lib/systemd/system
systemctl daemon-reload
systemctl start qubeslog.socket

systemctl unmask rsyslog.service
systemctl enable rsyslog.service
systemctl start rsyslog.service

Edit the file /etc/rsyslog.conf
Activate the remote logging

*.* @@127.0.0.1:5140;QubeslogFormat

Change the hostname to the name of the qube.

constant(value="hostname")

Shutdown the AppVM.

Create the qrexec policy

In dom0, create the file /etc/qubes/policy.d/30-user-networking.policy, and allow rsyslog-client to access sys-rsyslog

qubes.ConnectTCP +514 rsyslog-client @default allow target=sys-rsyslog

Change the names to match your system.

You should be able to start the AppVM (rsyslog-client) and it should automatically start sys-rsyslog and send logs using rsyslog.

2 Likes