What are the hard links used for? I though just for dedupe. The issue you linked also mentions that it should work without it?
Well rclone apparently does not support hard links yet Preserve hard links · Issue #6160 · rclone/rclone · GitHub.
Still as far as I understand it really should not matter that hard links are broken. Why shouldn’t they could you please explain?
@tee Is this the wyng-util-qubes v0.9 beta rel 20250820 version, running on Qubes 4.2? If so, you could try going back a couple versions (rel 20250731) which was geared to Qubes 4.2. The latest version is supposed to support both (device assignments differ between 4.2 and 4.3), but you may have encountered a bug.
“Breaking” or duplicating hard links will cause the archive copy to occupy more space than the original (its not an issue if you didn’t use deduplication). The archive copy will be fine otherwise and you can easily reclaim the space later with wyng arch-deduplicate. The Wyng user guide has some advice on creating duplicate archives and there is a script for duplicating archives that uses rsync (which does preserve hard links).
To clarify: Wyng doesn’t use hard links unless you tell it to deduplicate. Also, in order to create/use an archive Wyng has to see a POSIX type filesystem, so direct “cloud” access won’t work unless the type of cloud storage you have can be mounted as a filesystem. If the fs doesn’t support hard links, Wyng should work anyways unless you are doing wyng arch-deduplicate or wyng --dedup send. Wyng doesn’t use dedup by default and will still efficiently send incremental backups without dedup.
@tee If you are trying to recover the existing Qubes installation, not starting from a new install, then its possible the version value in /etc/os-release has gotten out of sync with the rest of the OS. That would cause wyng-util-qubes to access the wrong Qubes function for devices settings. You may have to restore or edit it to reflect the correct version.
Yes, it is Wyng-util-qubes v0.9 beta rel 20250820, it’s showing 4.3 alpha because the upgrade script did 1or 2 stages before the issues.
Yes, I was trying to with my existing Qubes installation. I am trying to learn Qubes, correct me if I am wrong I dont see anything atleast here. The backup is from 4.2.4 version.
$ cat /etc/os-release
NAME="Qubes OS"
VERSION="4.3 (R4.3)"
ID=qubes
VERSION_ID=4.3
PRETTY_NAME="Qubes OS 4.3-alpha (R4.3)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:ITL:qubes:4.3"
Thank you …
I don’t know if this can work under Qubes 4.3 alpha. It may be that version still has the 4.2 device interface, in which case you could try forcing wyng-util-qubes into using the old function. On line 323 changing oldqver = ctx.get_qubes_version()[:2] < (4,3) to this oldqver = True will do it.
Otherwise, you can try an updated version of the util in the testing branch here. It will skip the device assignment if an error occurs. This will leave sys-net with no devices and a “backup-restore-in-progress” tag that prevents it from starting; you can manually assign the net devices on the VM’s Devices tab and remove the tag this way in dom0: qvm-tags sys-net del backup-restore-in-progress
I also created an issue for this:
I’ve been trying to get a wyng backup going on 4.3rc3 for a number of weeks to no avail. I have set wyng up as per the guide at the start of this post and I am using what I believe to be the latest versions of wyng and wyng-util qubes. My attempted backup is to a SSD mounted to my personal VM so maybe that is the problem. Anyway I can successfully create an archive on the SSD; the minute I attempt a backup:
sudo wyng-util-qubes backup irc personal www
I get this python error immediately:
Wyng 0.8 rc3 release 20251007
ln: failed to create hard link ‘.hardlink’ => ‘archive.dat’: Operation not permitted
[0, 1]
Encrypted archive ‘qubes://personal/media/user/Samsung_T5/Backups/wyng/2025-11-05_08-55’
Last updated 2025-11-05 08:56:01.077570 (+10:00)
Preparing snapshots in ‘/dev/qubes_dom0/’…
Queuing full scan of ‘vm-irc-private’
Queuing full scan of ‘vm-personal-private’
Queuing full scan of ‘vm-www-private’
Traceback (most recent call last):
File “/usr/local/bin/wyng”, line 5924, in
raise e
File “/usr/local/bin/wyng”, line 5763, in
sid, ses_tags = monitor_send(storage, aset, vols or selected_vols, monitor_only=False,
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
use_sesid=sid, ses_tags=ses_tags, imports=other_vols,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
benchmark=options.dry_run, vname_sz=vname_sz)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/bin/wyng”, line 4250, in monitor_send
= storage.prep_snapshots(storage, aset, datavols, monitor_only, other_vols=imports)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/bin/wyng”, line 3360, in prepare_snapshots_lvm
l_vol = lvols[datavol] ; snap1, snap2 = l_vol.snap1, l_vol.snap2
~~~~~^^^^^^^^^
KeyError: ‘wyng-qubes-metadata’
Wyng::Failed return code 1
Maybe 4.3rc3 is still not supported here, or my system is not properly configured? Maybe @tasket you recognise the error?
@qub411 I just recreated the KeyError on R4.3. It appears to be minor, as re-running the backup completes without the error (I think Mirai said they got the same result).
The issue for this bug is: #59 - KeyError: ‘wyng-qubes-metadata’ when backing up to new archive - tasket/wyng-util-qubes - Codeberg.org
The hard link “operation not permitted” is just a notice that the backup storage doesn’t support hard links so deduplication can’t be used (which is probably not a big deal unless you have multiple cloned VMs / templates in the backup).
Super quick response there Chris, many thanks. I’ll give this another shot in the next few days.
@tasket Your fix indeed worked to solve that issue and I was able to run a three VM backup. I thought I would try to rerun a backup of one of the same VMs to the same archive and then got this error. Subsequent backups to the same archive also failed:
Traceback (most recent call last):
File “/usr/local/bin/wyng”, line 5944, in
raise e
File “/usr/local/bin/wyng”, line 5780, in
sid, ses_tags = monitor_send(storage, aset, vols or selected_vols, monitor_only=False,
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
use_sesid=sid, ses_tags=ses_tags, imports=other_vols,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
benchmark=options.dry_run, vname_sz=vname_sz)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/bin/wyng”, line 4241, in monitor_send
for vol in aset.vols.values(): autoprune(vol, apmode=“full”)
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File “/usr/local/bin/wyng”, line 4491, in autoprune
startdate = to_date(sessions[0]) ; nthresh = 3 if apmode == “on” else 0
~~~~~~~~^^^
IndexError: list index out of range
It resolved the issue for me on 4.2
@qub411 This is triggered by using ‘autoprune’ when there are too few sessions to prune. The fix08 branch has been updated with a fix, but you could also turn off autoprune to avoid the error.
BTW can I use wyng to restore my LVM backup to a new btrfs install? I would like to switch
@mirai Yes, its designed to do that when using the wyng-util-qubes program. It can also handle a system configured for both types of storage at the same time.
The latest fixed worked well and I am now able to create a full backup to a local SSD. The next part of the plan was to look at Wyng backups to my NAS, and here is where I’m getting another issue.
I am able to successfully create an archive using the following command (I have not changed my /etc/wyng/wyng.ini configs here):
sudo wyng arch-init --dest=qubes-ssh://personal:[ssh]/[path]
But when I do a backup with this:
sudo wyng-util-qubes backup --includes --dest=qubes-ssh://personal:[ssh]/[path]
I get this error:
Wyng-util-qubes v0.9 beta rel 20251105
Wyng 0.8 (fix08) release 20251107
Traceback (most recent call last):
File “/usr/local/bin/wyng”, line 5945, in
raise e
File “/usr/local/bin/wyng”, line 5732, in
aset = get_configs(options)
File “/usr/local/bin/wyng”, line 2560, in get_configs
aset = get_configs_remote(dest, cachedir, opts) ; os.utime(aset.path)
File “/usr/local/bin/wyng”, line 2579, in get_configs_remote
fetch_file_blobs(root_list, tmpdir, dest, skip0=True)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/bin/wyng”, line 3308, in fetch_file_blobs
if recvp.stdout.read(3) != magic: raise ValueError(“Bad magic.”)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: Bad magic.
Wyng::Failed return code 1
Any idea what is going on here?
@qub411 I can’t reproduce the error when accessing an Ubuntu ssh server. Probably what is happening is the remote helper process is terminating early; it may be that one of the commands (i.e. ‘python3’) doesn’t exist on the server. It needs a full python3 version 3.8 or later.
If python isn’t the issue, you can check the actual remote error by adding -w debug to the command line and then looking in /etc/wyng-debug/receive.log.
Since this is getting more technical, and this is a ‘noobs’ thread, you might want to create an issue in the tracker.
(BTW, I’ll be updating that section of code to be more descriptive. At least it should say the reason for not getting the correct ‘magic’ is that the helper terminated.)
Thanks, I will have a look at the points you’ve raised. I applaud your efforts Chris, and yes, I am a firm believer that no utility should crash out ungracefully like this, so a meaningfull error message and clean exit is always preferable.