Hi Everyone,
I’m installed 4.2.0-RC4 (I beliece dom0 has been updated to RC5) and ran into the following issue: when I try to copy files or folders between appVMs (click copy to other qube in file manager) that have foreign characters in their names (e.g., Arabic or Hebrew), the process fails without any warning.
To reproduce the problem, create a file named عربية or עברית and try to copy it to another appVM.
Curiously, when I tried it with some old files, it did fail with a warning about qfile-agent fatal error file copy invalid or incomplete multibyte or wide characters (maybe it was folders with names encoded differently somehow?).
I have been copying between fedora-38 template based appVM. Both appVMs allow to create files with foreign characters in their names.
This is a critical problem, since it can very easily lead to data loss.
I haven’t had a chance to test this in previous versions of Qubes.
Any thoughts about what might be wrong?
B
Maybe not exactly your case, but might be close… I found this brief mention of issue copying text with unicode content (I assume that was with characters that are beyond the ASCII range):
Thanks you. The clipboard seems to work fine (I used it to copy the strings to this message). It is the file copy that doesn’t work. It might be related somewhere under the hood though.
Unless I am missing something very specific to my setup, I would argue that one might want to hold back on releasing 4.2.0 with this problem since it can easily lead to data loss.
This has been reported and seems to be working as intended if your filename strings are not UTF-8. @marmarek wrote:
Indeed R4.2 introduces stricter filename validation, it needs to be (among other things) proper UTF-8 string. Maybe you use some different encoding for your file names?
However, if it’s not giving you any warning message then that may be a bug; I suggest commenting under the linked GitHub issue thread.
The filenames should be encoded as UTF-8 (which is a pretty reasonable encoding for a multi-language system).
What happens is you create new files with those same names (I mean typing the names, not copying them from your existing files)?
I would expect files newly created in whatever AppVM to be created with UTF-8 names, and to be copied successfully with qvm-copy.
If that’s the case, I think renaming your existing files to make sure their names are encoded in UTF-8 should be enough (though it may still be a significant amount of effort).
If the new files fail to be copied, however, whether because the names are not UTF-8 or otherwise, I’d dig further:
- what locale the AppVM is using
- try to figure out why it’s not UTF-8
- or try to figure out why those UTF-8 names break
qvm-copy
Thanks!
So, this does happen when I just type the names while creating new files. I’ll dig deeper to check the locales I’m using and write back.
In either case, there has to be an error message. It seems to exist for some incompatible file names, but it does not seem to cover all the cases. I don’t know how the validation works, but it would be good to have the same validation be used for error reporting.
It is reasonable to expect people to have legacy files or receive files from other sources. Ideally, these would be copied, but if their are not copied there should be an error message about it.
I created an issue: Error message sometimes missing when inter-qube file copy/move fails due to disallowed characters in filename · Issue #8738 · QubesOS/qubes-issues · GitHub
I think referenced the original issue, but the fact that an error message isn’t displayed seems to me to justify a separate issue.
I’m not entirely sure what locale I am using, but the command I used to switch the keyboard is just setxkbmap which I thought was pretty standard.
The crucial system components qvm-copy and qvm-move are not great, with major problems over the years. E.g. they failed to copy files without warnings due to access rights (chown/chmod), and not with R4.2 there is even worse, because of more strict copy policy that is intended for better user security by limiting what they can do.
Symlinks also won’t be copied properly anymore, user can never the sure about copying file tree, because some files will not be copied due to non-utf8 characters in the name (completely valid for filesystem, but not allowed by Qubes OS) and symlinks that can be somewhere inside the file tree.
The way to “workaround” this ‘improvement’ will be to tar files before copying to preserve user data. But it destroys all the purpose of this R4.2 changes and makes user experience worse (tar each time user copies file tree).
I personally think, that not being able to copy symlinks properly and files with non-utf-8 chars in the name anymore is a major design mistake of R4.2. The user data should be preserved at all costs.
I agree. There is a critical data integrity issue here.
Is there any post discussing the security concerns?
You may participate in the discussion about this user data loss with @marmarek and @Demi here: Error message sometimes missing when inter-qube file copy/move fails due to disallowed characters in filename · Issue #8738 · QubesOS/qubes-issues · GitHub
UPD: Or, sorry, have not notice that it is your issue ticket.
I do not think that files with invalid utf-8 names or symlinks are that dangerous that copy tool should skip/loose user’s data because of it.
Inability to copy symlinks from R4.2 I consider ridiculous. Have of the GNU/Linux is based on symlinks and scripts.
I’m facing this problem. I cannot copy from VMs tens of thousands of files because hundreds have French and Spanish characters. I read through this thread and I can’t find a solution. Anyone can help? I’m using Debian 12 Minimal VMs on Qubes 4.2.
A dirty fix I could think of is to create an archive of these files, copy it to the other qube and de-archive it. That’s not ideal but it will work 
@oijawyuh
qvm-copy and qvm-move are BROKEN in R4.2 by design. They do not copy files and symlinks, but instead copy only some files with an undesired and filtering instead. Filtering that nobody was even hoping to have, as I understand.
You should tar your files and untar in the target qube. Yes, it is less secure and convenient than it was in R4.1. But this is what users have to do in R4.2.
I consider it to be a huge major regression of R4.2, not a feature. At least until this is a default and unchangeable behavior.
Dangerous regression that obviously can and does lead to the user data loss.
You can share your opinion about this decision of @marmarek to affect qvm-copy/qvm-move in this way here: