Send files directly to external storage mounted in other qube

Maybe there is a way to mount NFS with specific options that will allow all operation for user on NFS mount.

It’s better to configure the SELinux properly.

I actually don’t seem to understand the way permissions are handled.

When configuring the NFS share on the server with mapall user and mapall group to user and setting the permissions on the server to user/group=full access I am able to mount the share from qubes and access everything. Except the qvm-copy throws an operation not permitted error after copying because it probably tries to chown as root.

When configuring the NFS share on the server with maproot user and maproot group to user I am not able to access the mounted share in qubes anymore. But if I try to access it with sudo cd QubesIncoming I am actually able to access it. So in that case it seems to be only accessible by the root user.

I have no idea how to configure it on the server side so that all access is mapped to user nevermind if this is a qubes-user or a qubes-root access.

1 Like

I’ve been having a similar issue since upgrading to 4.2–Whenever I mount an external drive, I have to also chown user /mnt/mountPoint in order to write (and sometimes to read too),

There are other bugs with qvm-copy and qvm-move (like inconsistent and fatal mishandling of some non-unicode characters in filenames) that have popped up too.

I give up. I spent so many hours now on trying to get that working but it won’t work. I am not able to copy from qube A to qube B which contains a nfs share mounted. Copying to qube B and then copying into the nfs share works without problems but the simplification to directly copy to the nfs share will not work for me. Sad bud true :frowning:

Edit: I finally tweaked the NFS share to allow user and root access at the same time. In a terminal in qube B I am able to cd into the share and to create files and I am also able to sudo chown a file. But still when copying from qube A to qube B, the file is actually copied and then an operation not permitted error is thrown :frowning:

What if you do it from dom0 (not super practical though)

  • qvm-run VM "cat file" | qvm-run DEST "cat > /mnt/file"

What if you do it from dom0 (not super practical though)

This does actually copy a file without giving an error, but the file is empty.
qvm-run NAME_OF_SOURCE_QUBE "cat /home/user/Downloads/test.txt"
does not show anything but does not throw an error either.

Oh sorry, I forgot you need to use --pass-io on each qvm-run otherwise there are no input/output…

That does actually work without problems, the file is directly copied from qube A to nfs share in qube B without any error.

Because sudo chown in qube B now (with changed NFS share configuration) does work my assumption that qvm-copy performs chown at the end is probably wrong. I don’t know what qvm-copy tries to do after the copying is finished and therefore I don’t know what actually causes the operation not permitted.

Here is the exact error message again, maybe you can see any other important information in it.
qfile-agent: Fatal error: File copy: "Operation not permitted; Last file: test.txt" (error type: Operation not permitted)

It’s your NFS share that is misconfigured, it works for me.

I mount my share in /home/user/remote, and the remote endpoint is chown/chgrp to user.

1 Like

I now moved the mount point from /mnt/remote to /home/user/remote but the error still occurs.

It’s your NFS share that is misconfigured, it works for me.

Are you using NFSv3 oder NFSv4 (as I do)?

Do you also have to disable SELinux in the remote qube?

And just to make sure: You are copying from a different qube A to qube B which links the folder of the incoming qube directly to the mounted NFS share?

I’m using NFS v4.

I don’t use SELinux, both the local and remote qube is running Debian 12, so It might be a SELinux issue.

I call this script from rc.local, it’s all I do to mount the share

#!/bin/sh

systemctl unmask rpcbind.service
/usr/bin/mount -t nfs NFS-PATH /home/user/remote

And it works if I symlink ~/QubesIncomming to ~/remote/QubesIncomming

I now created two more qubes running debian for testing, but the same error occurs.

Are you sure you are using NFSv4, I thought mount -t nfs4 would be required for that?
I added to rc.local

mkdir /home/remote
mount -t nfs4 -o proto=tcp,port=2049 NFS-PATH /home/remote

So I might try NFSv3 for testing, but this is actually not really an option because other systems are mounting different shares on that server with NFSv4.

I don’t understand what I am doing wrong…

Yes. I didn’t realize you had moved it under “input devices.” Thanks!

What you have there looks good actually. Very minimalist with little explanation, but now I see that this is what we’re after here. The general option (where the mounted offline storage ends up with a top level directory /[ORIGIN_QUBE]) is the easiest and that’s what’s described. There are of course a zillion variations on this! (Including the one where you link to ~QubesIncoming/[ORIGIN_QUBE]). We can rely on users to figure that out once we give them the initial idea.

For instance I’m going to have to download the 4.2 ISO at some point, and I’m going to have to get creative about how to accommodate that file. Mounting offline storage (in this particular case it will be a thumb drive) to ~/Downloads will do it, I think. And really that’s just the same tip, reworked slightly.

Yeah, well, the controversy is my fault; my original steps were defective. I hope there’s no more damage from that going forward.

1 Like

I don’t see how there’s actual damage, but as long as the top post here and the Quick QoL entry are both updated with tested instructions, everything should be fine.

I don’t use SELinux, both the local and remote qube is running Debian 12, so It might be a SELinux issue.

I finally found the problem in my configuration: I changed permissions on the server side (using TrueNAS) from NFSv4 ACL to POSIX and now the operation not permitted error is not thrown anymore. I don’t know the exact reason why NFSv4 ACL is a problem in the qvm-copy situation (I am accessing it with other Linux based systems without problems), but I can live with that.

There are still some speed issues: I can copy from qube A to qube B (local) with >300 MB/s. I can copy from qube B (local) to the NFS-share with approx 105 MB/s. But when trying to copy from qube A to qube B and directly into the linked NFS-share the speed drops to approx 35 MB/s. I checked the app “TaskManager” in source/destination qubes, but neither cpu nor ram are overloaded. Any ideas where to check next?

105 MB/s could be NFS being limited by 1gbps networking, not sure about the 35MB/s.

If the qube file copy does some type of data sync, like reading the data back to make sure it’s written correctly, that can be slow over NFS.

If the qube file copy does some type of data sync, like reading the data back to make sure it’s written correctly, that can be slow over NFS.

But it should not slow down that much, shouldn’t it?

I found the options Transfer files in parallel (set it to Always) and Verify file checksum on copy (was already set to Never) in Thunar File Manager but I guess they don’t have an impact on the speed.

Can qvm-copy be configured in some way?

The SELinux part of this issue is now being tracked on GitHub.

1 Like