Duplicate files on /home and /rw folders?

tanky0u · January 26, 2023, 2:45pm

I am seeing that my /home and /rw folders in my qube filesystem are identical, and thus the data in my /home folder is replicated on the /rw folder, making the storage size requirement for my qube to be doubled. I mean, if I am storing 50 GB of video files in my qube in the /home/user/Videos folder, that 50 GB gets duplicated on the /rw folder, thus making the disk space requirement of my qube to a 100 GB.

Why does /rw duplicate the files in the /home folder? It seems awfully redundant, unless there is a good reason for it (which is foreign to me for the time being).

Now, I need to give my qube 200 GB of private disk space, if I want to store 100 GB of size video files in it. Why?

rustybird · January 26, 2023, 3:06pm

/home is bind mounted from /rw/home, so it doesn’t duplicate any data:

github.com

QubesOS/qubes-core-agent-linux/blob/v4.1.38/filesystem/fstab#L6


      
          # Accessible filesystems, by reference, are maintained under '/dev/disk'
          # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
          #
          /dev/mapper/dmroot /                       ext4 defaults,discard,noatime        1 1
          /dev/xvdb		/rw			auto	noauto,defaults,discard,nosuid,nodev	1 2
          /rw/home        /home       none    noauto,bind,defaults,nosuid,nodev 0 0
          /rw/usrlocal        /usr/local       none    noauto,bind,defaults 0 0
          /dev/xvdc1      swap                    swap    defaults        0 0
          tmpfs                   /dev/shm                tmpfs   defaults,size=1G        0 0
          devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
          sysfs                   /sys                    sysfs   defaults        0 0
          proc                    /proc                   proc    defaults        0 0
          /dev/xvdi	/mnt/removable	auto noauto,user,rw 0 0

https://man7.org/linux/man-pages/man8/mount.8.html

tanky0u · January 26, 2023, 3:08pm

How so?

$ sudo du -sh /      
97G     /
zsh: exit 1     sudo du -sh /
user@host:~ $ du -sh /home
46G     /home

As you can see, the root filesystem is the same size as the /home filesystem size. Also, I am visually confirming that my ~/Videos/ directory contents are exactly identical to the ones in /rw/user/home/Videos/ directory.

So, it seems like it is duplicating the data?

rustybird · January 26, 2023, 3:14pm

In other words, /home is just another “view” of /rw/home - kind of like a symlink or hardlink. Search for “Bind mount” in the mount(8) manpage.

du is not aware of this, and sums up both views as if they were completely separate.

tanky0u · January 26, 2023, 3:17pm

Alright. But that doesn’t change the fact that it seems like my qube is out of space? See, that qube has 100GB of “Private Storage” space setup in the Qube Manager GUI. And now that its filled to 47 GB, my yt-dlp video downloads are giving “Post Processing” errors, which, a quick github search seems to suggest that it is related to running out of disk space.

So, either I am interpreting my yt-dlp video download woes wrongfully (and putting the consistent errors at video downloads, wrongfully, to my 100 GB private storage space qube running out of storage space due to “duplication” of data between /home and /rw), or there is something else going on.

rustybird · January 26, 2023, 3:19pm

A reliable way to check the disk space of the private volume is df -h /rw (note that it’s df, not du)

tanky0u · January 26, 2023, 3:21pm

This returns:

 $ sudo df -h /rw      
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvdb        98G   48G   51G  49% /rw

I guess you are right. And I am wrongfully blaming a data duplication, instead of seeking the error of the cause of my yt-dlp errors elsewhere.