Ephemeral DVMs in fully ephemeral thin pools (ephemeral encryption, zram-disk, tmpfs). Removing DVM logs and metadata

Forensic survey on metadata and records from the dvm created in the zram-pool in the dom0

Looking through all the files in the dom0 in Qubes, after installing zram-pool and creating a dvm named anon-whonix-zram, filenames are created with the name anon-whonix-zram and the name anon-whonix-zram appears inside, written in several files…

Below is the list of directories where those records are:
In the case the user of this qube is user, then /home/user

/etc/libvirt/libxl/
/etc/lvm/archive/
/etc/lvm/backup/
/var/log/libvirt/libxl/
/var/log/qubes/
/var/log/xen/console/
/var/lib/qubes/appvms/
/var/lib/qubes/backup/
/home/user/.local/share/applications/
/home/user/.local/share/qubes-appmenus/anon-whonix-tmpfs/
/home/user/.local/share/qubes-appmenus/anon-whonix-tmpfs/apps/
/home/user/.local/share/desktop-directories/
/home/user/.local/state/wireplumber/
/run/udev/links/ #ignore runtime, volatile
/run/udev/data/ #ignore runtime, volatile
/tmp/ #by default it stays in tmpfs, ignore
/dev/zram_vg/ #this is the zram itself, ignore

Many of these files are records, netvm, when the netvm changes, when it’s turned on, when it’s turned off, and these are metadata that can be used to correlate online activity!
Example: a certain user logs into a social network and makes posts for time x1 to x2, on day xB!
Files in the dom0 may have metadata that matches that time and day, hour when they accessed, closed, and day!
If this repeats 10 or 100 times, it may be possible to infer that you are the user of that social network because it’s too much of a coincidence!
This metadata is eliminated when we use the dom0 100% in RAM and all of this is destroyed!
When the dom0 is not in RAM, apparently all of it stays there, but maybe there’s more…
To use zram-pool, it’s necessary to run the directories with the files where this metadata lives 100% anonymously in RAM using overlayfs or tmpfs!
This needs to be handled!

You can have more details
using on dom0 this script…
create the zram-pool, create a dvm or use a dvm that is in the zram-pool, turn it on, navigate a bit, turn it off.
Let’s say the name of this test dvm is dvm-test
then run the script inside dom0
sudo ./qubes-forensic-hunter.sh
put the dvm name, (dvm-test)
report mode: concise or detailed

After the report, it’s necessary to find out what each file does and whether they are accessed, modified, and record metadata such as start of use, end of use, time, day, etc… when the dvm-test and other dvm’s in the zram-pool are being used!
I didn’t have time to investigate each directory and file and exactly what each one does and other details…
After that, it is necessary to create overlayfs or tmpfs setup for all directories that contain the critical metadata we are discussing here!

qubes-forensic-hunter.sh (forensic script to trace appvm/dvm records in dom0

run inside dom0

#!/bin/bash
# ==============================================================================
#  dom0-appvm-forensic-scan.sh  v3
#  Qubes OS dom0 — AppVM Forensic Scanner
#
#  FILENAME SEARCH  → entire filesystem (including /proc /sys /boot /dev)
#                     read-only, only the path string is checked, nothing opened
#
#  CONTENT SEARCH   → safe text/config/log files only (no binaries, no disk
#                     images, no media) — NO size limit, every line is checked
#
#  TWO REPORT MODES:
#    [1] CONCISE  — filenames containing the AppVM name (path + metadata)
#                   + files where the name appears in content (path + count)
#    [2] DETAILED — same as concise, plus every matching line with line number
#
#  Report saved as: <appvm-name>-rastros.txt
# ==============================================================================

set -uo pipefail

# ──────────────────────────────────────────────────────────────────────────────
# ROOTS FOR CONTENT SCAN (safe — no /proc /sys /boot /dev)
# ──────────────────────────────────────────────────────────────────────────────
CONTENT_ROOTS=(
    /etc
    /var
    /home
    /root
    /run
    /tmp
    /usr
    /opt
    /srv
)

# ──────────────────────────────────────────────────────────────────────────────
# ROOTS FOR FILENAME SCAN (entire filesystem — only path string is read)
# ──────────────────────────────────────────────────────────────────────────────
FILENAME_ROOTS=(
    /etc
    /var
    /home
    /root
    /run
    /tmp
    /usr
    /opt
    /srv
    /boot
    /proc
    /sys
    /dev
    /mnt
    /media
)

# ──────────────────────────────────────────────────────────────────────────────
# EXTENSIONS SAFE FOR CONTENT SCAN (no size limit applied to these)
# ──────────────────────────────────────────────────────────────────────────────
SAFE_TEXT_EXTENSIONS=(
    conf cfg ini xml json yaml yml toml
    env profile rc
    sh bash zsh fish py rb pl php js ts lua
    log out err
    txt md rst html htm css
    pem crt key csr
    service unit socket timer target mount
    rules policy te
    sql ks preseed xsl xslt
    desktop
    bak orig dpkg-old dpkg-dist rpmsave rpmnew
    qube qubes policy2
)

# ──────────────────────────────────────────────────────────────────────────────
# EXTENSIONS TO SKIP FOR CONTENT SCAN (binary / media / archives / disk images)
# ──────────────────────────────────────────────────────────────────────────────
SKIP_CONTENT_EXTENSIONS=(
    jpg jpeg png gif bmp ico tiff webp svg
    mp3 mp4 avi mkv mov wav flac ogg opus webm
    zip tar gz bz2 xz zst 7z rar
    pdf doc docx xls xlsx ppt pptx odt ods odp
    db sqlite sqlite3 img iso qcow2 vmdk vhd vhdx raw cow
    rpm deb snap appimage
    ttf otf woff woff2
    so dll exe bin pyc pyo o a class jar elf
    swp swo swap
)

# ──────────────────────────────────────────────────────────────────────────────
# HELPERS
# ──────────────────────────────────────────────────────────────────────────────
CYAN='\033[0;36m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'
RED='\033[0;31m'; BOLD='\033[1m'; RESET='\033[0m'

log()    { echo -e "${CYAN}[INFO]${RESET}  $*"; }
ok()     { echo -e "${GREEN}[ OK ]${RESET}  $*"; }
warn()   { echo -e "${YELLOW}[WARN]${RESET}  $*"; }
divider(){ printf '%0.s─' {1..80}; echo; }

in_array() {
    local needle="$1"; shift
    for item in "$@"; do [[ "${item}" == "${needle}" ]] && return 0; done
    return 1
}

get_ext() {
    local base
    base=$(basename "$1" | tr '[:upper:]' '[:lower:]')
    if [[ "${base}" == .* ]] && [[ "${base}" != *.*.* ]]; then
        echo "${base#.}"
    else
        echo "${base##*.}"
    fi
}

# Returns 0 = safe to scan content, 1 = skip
is_content_safe() {
    local fpath="$1"
    local ext
    ext=$(get_ext "${fpath}")

    in_array "${ext}" "${SKIP_CONTENT_EXTENSIONS[@]}" && return 1
    in_array "${ext}" "${SAFE_TEXT_EXTENSIONS[@]}"    && return 0

    # Unknown extension: ask file(1) for MIME type
    local mime
    mime=$(file -b --mime-type "${fpath}" 2>/dev/null || echo "application/octet-stream")
    case "${mime}" in
        text/*|application/json|application/xml|application/x-sh|\
        application/javascript|inode/x-empty)
            return 0 ;;
        *)
            return 1 ;;
    esac
}

# ──────────────────────────────────────────────────────────────────────────────
# INPUT
# ──────────────────────────────────────────────────────────────────────────────
echo
echo -e "${BOLD}════════════════════════════════════════════════════════════════════${RESET}"
echo -e "${BOLD}    Qubes OS dom0 — AppVM Forensic Scanner  v3                     ${RESET}"
echo -e "${BOLD}════════════════════════════════════════════════════════════════════${RESET}"
echo

[[ $EUID -ne 0 ]] && warn "Not root — some paths may be unreadable. Recommend: sudo $0" && echo

read -rp "$(echo -e "${BOLD}AppVM name to search for:${RESET} ")" APPVM_NAME
[[ -z "${APPVM_NAME}" ]] && echo -e "${RED}Name cannot be empty.${RESET}" && exit 1

echo
echo -e "  ${BOLD}Report mode:${RESET}"
echo -e "  ${CYAN}[1]${RESET} CONCISE  — filenames containing the AppVM name + files where it"
echo -e "             appears in content (path + occurrence count only)"
echo -e "  ${CYAN}[2]${RESET} DETAILED — same as above, plus every matching line with line number"
echo
read -rp "$(echo -e "${BOLD}Choose mode [1/2]:${RESET} ")" MODE_CHOICE

case "${MODE_CHOICE}" in
    1) REPORT_MODE="concise"  ;;
    2) REPORT_MODE="detailed" ;;
    *) echo -e "${YELLOW}Invalid choice — defaulting to DETAILED.${RESET}"
       REPORT_MODE="detailed" ;;
esac

REPORT_FILE="${APPVM_NAME}-rastros.txt"
TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')
HOST_VAL=$(hostname)

echo
log "AppVM target  : ${APPVM_NAME}"
log "Report mode   : ${REPORT_MODE}"
log "Report file   : ${REPORT_FILE}"
log "Started       : ${TIMESTAMP}"
echo

# Counters
FNAME_HIT=0
CONTENT_HIT_FILES=0
CONTENT_HIT_LINES=0
FILES_SCANNED=0
FILES_SKIPPED=0

# ──────────────────────────────────────────────────────────────────────────────
# REPORT HEADER
# ──────────────────────────────────────────────────────────────────────────────
{
cat <<HEADER
================================================================================
  QUBES OS dom0 — FORENSIC SCAN REPORT
================================================================================
  AppVM Target  : ${APPVM_NAME}
  Report mode   : $(echo "${REPORT_MODE}" | tr '[:lower:]' '[:upper:]')
  Host          : ${HOST_VAL}
  Scan started  : ${TIMESTAMP}
  Run by        : $(whoami)
================================================================================

HEADER
} > "${REPORT_FILE}"

# ──────────────────────────────────────────────────────────────────────────────
# SECTION 1 — FILENAME MATCHES
# Covers entire filesystem. Only the path string is examined — no file is
# opened, read, or modified. Safe for /proc /sys /boot /dev.
# ──────────────────────────────────────────────────────────────────────────────
log "SECTION 1 — Filename scan across entire filesystem..."

{
cat <<'S1'
================================================================================
  SECTION 1 — FILES / DIRECTORIES WHOSE NAME CONTAINS THE APPVM NAME
================================================================================
  Covers : /etc /var /home /root /run /tmp /usr /opt /srv
           /boot /proc /sys /dev /mnt /media
  Method : path string check only — no file is opened or modified
--------------------------------------------------------------------------------

S1
} >> "${REPORT_FILE}"

for ROOT in "${FILENAME_ROOTS[@]}"; do
    [[ -e "${ROOT}" ]] || continue

    # Limit depth on virtual filesystems to avoid hangs
    DEPTH_ARG=()
    [[ "${ROOT}" == /proc || "${ROOT}" == /sys || "${ROOT}" == /dev ]] \
        && DEPTH_ARG=(-maxdepth 4)

    while IFS= read -r -d '' FPATH; do
        FTYPE=$(file -b --mime-type "${FPATH}" 2>/dev/null || echo "unknown")
        FSIZE=$(stat -c '%s'         "${FPATH}" 2>/dev/null || echo "?")
        FMOD=$(stat -c '%y'          "${FPATH}" 2>/dev/null | cut -d'.' -f1 || echo "?")
        OWNER=$(stat -c '%U:%G'      "${FPATH}" 2>/dev/null || echo "?")

        {
        printf "  PATH     : %s\n" "${FPATH}"
        printf "  TYPE     : %s\n" "${FTYPE}"
        printf "  SIZE     : %s bytes\n" "${FSIZE}"
        printf "  MODIFIED : %s\n" "${FMOD}"
        printf "  OWNER    : %s\n" "${OWNER}"
        echo   "  ──────────────────────────────────────────────────────────────"
        } >> "${REPORT_FILE}"

        (( FNAME_HIT++ )) || true
    done < <(find "${ROOT}" "${DEPTH_ARG[@]}" -name "*${APPVM_NAME}*" -print0 2>/dev/null)
done

[[ ${FNAME_HIT} -eq 0 ]] && echo "  [NO FILENAME MATCHES FOUND]" >> "${REPORT_FILE}"

{
echo ""
printf "  Total filename matches: %d\n" "${FNAME_HIT}"
echo ""
} >> "${REPORT_FILE}"

ok "Section 1 done — ${FNAME_HIT} filename match(es)."

# ──────────────────────────────────────────────────────────────────────────────
# SECTION 2 — CONTENT MATCHES
# Scans every text/config/log file in safe roots.
# No size limit — every line of every qualifying file is checked.
# ──────────────────────────────────────────────────────────────────────────────
log "SECTION 2 — Content scan (no size limit, line by line)..."

if [[ "${REPORT_MODE}" == "concise" ]]; then
    FMT_NOTE="  FORMAT: FILE PATH  |  OCCURRENCE COUNT"
else
    FMT_NOTE="  FORMAT: file header block + each matching line with its line number"
fi

{
cat <<S2
================================================================================
  SECTION 2 — FILE CONTENT MATCHES
================================================================================
  Scope : /etc /var /home /root /run /tmp /usr /opt /srv
  Files : all text, config, log, script, xml, json, yaml, conf, sh, py ...
  Skips : binaries, media, archives, disk images, compiled objects
  Size  : NO limit — every line of every qualifying file is checked
${FMT_NOTE}
--------------------------------------------------------------------------------

S2
} >> "${REPORT_FILE}"

TMP_CONTENT=$(mktemp /tmp/dom0-content-XXXXXX.tmp)

for ROOT in "${CONTENT_ROOTS[@]}"; do
    [[ -d "${ROOT}" ]] || continue

    while IFS= read -r -d '' FPATH; do
        [[ -f "${FPATH}" ]] || continue

        (( FILES_SCANNED++ )) || true

        if ! is_content_safe "${FPATH}"; then
            (( FILES_SKIPPED++ )) || true
            continue
        fi

        # -i case-insensitive, -n line numbers, skip binary files automatically
        GREP_OUT=$(grep -in "${APPVM_NAME}" "${FPATH}" 2>/dev/null) || continue
        [[ -n "${GREP_OUT}" ]] || continue

        LINE_COUNT=$(echo "${GREP_OUT}" | wc -l)
        FTYPE=$(file -b --mime-type "${FPATH}" 2>/dev/null || echo "unknown")
        FMOD=$(stat -c '%y' "${FPATH}" 2>/dev/null | cut -d'.' -f1 || echo "?")

        if [[ "${REPORT_MODE}" == "concise" ]]; then
            {
            printf "  %-72s | %d occurrence(s)\n" "${FPATH}" "${LINE_COUNT}"
            } >> "${TMP_CONTENT}"
        else
            {
            echo "════════════════════════════════════════════════════════════════════════════════"
            printf "  FILE NAME     : %s\n"         "$(basename "${FPATH}")"
            printf "  FULL PATH     : %s\n"         "${FPATH}"
            printf "  TYPE          : %s\n"         "${FTYPE}"
            printf "  LAST MODIFIED : %s\n"         "${FMOD}"
            printf "  OCCURRENCES   : %d line(s)\n" "${LINE_COUNT}"
            echo   "  MATCHING LINES:"
            while IFS= read -r MATCH; do
                LNUM="${MATCH%%:*}"
                LTEXT="${MATCH#*:}"
                printf "    LINE %-7s: %s\n" "${LNUM}" "${LTEXT}"
            done <<< "${GREP_OUT}"
            echo ""
            } >> "${TMP_CONTENT}"
        fi

        (( CONTENT_HIT_FILES++ )) || true
        (( CONTENT_HIT_LINES += LINE_COUNT )) || true

    done < <(find "${ROOT}" -type f -print0 2>/dev/null)
done

if [[ ${CONTENT_HIT_FILES} -eq 0 ]]; then
    echo "  [NO CONTENT MATCHES FOUND]" >> "${REPORT_FILE}"
else
    cat "${TMP_CONTENT}" >> "${REPORT_FILE}"
fi
rm -f "${TMP_CONTENT}"

{
echo ""
printf "  Files examined for content : %d\n" "${FILES_SCANNED}"
printf "  Files skipped (binary etc) : %d\n" "${FILES_SKIPPED}"
printf "  Files with matches         : %d\n" "${CONTENT_HIT_FILES}"
printf "  Total matching lines       : %d\n" "${CONTENT_HIT_LINES}"
echo ""
} >> "${REPORT_FILE}"

ok "Section 2 done — ${CONTENT_HIT_FILES} file(s), ${CONTENT_HIT_LINES} matching line(s)."

# ──────────────────────────────────────────────────────────────────────────────
# SECTION 3 — QUBES / XEN / LIBVIRT SPECIFIC RECORDS
# ──────────────────────────────────────────────────────────────────────────────
log "SECTION 3 — Qubes-specific records..."

{
cat <<'S3'
================================================================================
  SECTION 3 — QUBES-SPECIFIC RECORDS
================================================================================
S3
} >> "${REPORT_FILE}"

# qvm-ls
{
echo ""
echo "  ── qvm-ls ──────────────────────────────────────────────────────────────────"
} >> "${REPORT_FILE}"
if command -v qvm-ls &>/dev/null; then
    OUT=$(qvm-ls 2>/dev/null | grep -i "${APPVM_NAME}" || true)
    [[ -n "${OUT}" ]] \
        && printf "%s\n" "${OUT}" >> "${REPORT_FILE}" \
        || echo "  [not found in qvm-ls]" >> "${REPORT_FILE}"
else
    echo "  [qvm-ls not available in this environment]" >> "${REPORT_FILE}"
fi

# qvm-prefs
{
echo ""
echo "  ── qvm-prefs ───────────────────────────────────────────────────────────────"
} >> "${REPORT_FILE}"
if command -v qvm-prefs &>/dev/null; then
    qvm-prefs "${APPVM_NAME}" 2>/dev/null >> "${REPORT_FILE}" \
        || echo "  [could not retrieve prefs — VM may not exist or be inaccessible]" >> "${REPORT_FILE}"
else
    echo "  [qvm-prefs not available]" >> "${REPORT_FILE}"
fi

# qvm-firewall
{
echo ""
echo "  ── qvm-firewall ────────────────────────────────────────────────────────────"
} >> "${REPORT_FILE}"
if command -v qvm-firewall &>/dev/null; then
    qvm-firewall "${APPVM_NAME}" list 2>/dev/null >> "${REPORT_FILE}" \
        || echo "  [no firewall rules found or VM inaccessible]" >> "${REPORT_FILE}"
else
    echo "  [qvm-firewall not available]" >> "${REPORT_FILE}"
fi

# journald
{
echo ""
echo "  ── journald ────────────────────────────────────────────────────────────────"
} >> "${REPORT_FILE}"
if command -v journalctl &>/dev/null; then
    JOUT=$(journalctl --no-pager -g "${APPVM_NAME}" 2>/dev/null || true)
    if [[ -n "${JOUT}" ]]; then
        JCOUNT=$(echo "${JOUT}" | wc -l)
        if [[ "${REPORT_MODE}" == "concise" ]]; then
            printf "  %d journal line(s) matched.\n" "${JCOUNT}" >> "${REPORT_FILE}"
            printf "  To view: journalctl --no-pager -g '%s'\n" "${APPVM_NAME}" >> "${REPORT_FILE}"
        else
            echo "${JOUT}" >> "${REPORT_FILE}"
        fi
    else
        echo "  [no journal entries found]" >> "${REPORT_FILE}"
    fi
else
    echo "  [journalctl not available]" >> "${REPORT_FILE}"
fi

echo "" >> "${REPORT_FILE}"
ok "Section 3 done."

# ──────────────────────────────────────────────────────────────────────────────
# SECTION 4 — SUMMARY
# ──────────────────────────────────────────────────────────────────────────────
END_TIME=$(date '+%Y-%m-%d %H:%M:%S')

{
cat <<SUMM
================================================================================
  SECTION 4 — SUMMARY
================================================================================
  AppVM searched              : ${APPVM_NAME}
  Report mode                 : $(echo "${REPORT_MODE}" | tr '[:lower:]' '[:upper:]')
  Started                     : ${TIMESTAMP}
  Finished                    : ${END_TIME}

  [FILENAME SCAN]
  Matches found               : ${FNAME_HIT}
  Searched in                 : /etc /var /home /root /run /tmp /usr /opt /srv
                                /boot /proc /sys /dev /mnt /media

  [CONTENT SCAN]
  Files examined              : ${FILES_SCANNED}
  Files skipped (binary etc)  : ${FILES_SKIPPED}
  Files with matches          : ${CONTENT_HIT_FILES}
  Total matching lines        : ${CONTENT_HIT_LINES}
  Searched in                 : /etc /var /home /root /run /tmp /usr /opt /srv

================================================================================
  END OF REPORT
  Saved to: ${REPORT_FILE}
================================================================================
SUMM
} >> "${REPORT_FILE}"

# ──────────────────────────────────────────────────────────────────────────────
# TERMINAL FINAL MESSAGE
# ──────────────────────────────────────────────────────────────────────────────
echo
divider
echo -e "${BOLD}${GREEN}  Scan complete!${RESET}"
echo -e "  Report file  : ${BOLD}${REPORT_FILE}${RESET}"
echo -e "  Mode         : ${REPORT_MODE}"
echo -e "  Filename hits: ${FNAME_HIT}"
echo -e "  Content hits : ${CONTENT_HIT_FILES} file(s) — ${CONTENT_HIT_LINES} line(s)"
divider
echo

This handling must be done, because if this metadata exists, in case of a successful physical attack, correlation could deanonymize many users. With this handling, using the dom0 100% in RAM or in overlayfs with the appvms inside the varlibpool (of the dom0) continues to be the best strategy since this metadata will not exist after reboot and anti-cold-boot by dracut! The problem is: reboot, configure, use, and then go back to normal every time—this destroys usability for the user and wastes a lot of time. Hard to use for everyone, and that’s a problem!

1 Like