Guide: How to judge if a hardware device or operating system is secure

(I originally posted this on GrapheneOS forum, but I am posting it here too, since I think it might be interesting to QubesOS users as well.)

It is often I see questions being raised on this and other forums like “Is this device I heard of secure?” or “Can this operating system I heard of be trusted?”. Unfortunately, it isn’t easy to judge whether a device or operating system is secure or can be trusted to protect you, and to make things harder, there are a lot of misinformation out there and even outright fraudulent advertisements. A lot of companies want to earn your money, even if they cannot live up to their promises.

So I wrote this guide, in the hope that is will be useful for some people. Here are the things to look for in a device and in an operating system, to see if it lives up to the current community standards with regard to security and privacy. I realize it might still be hard to figure out if a device or operating system really fulfills these things I have listed, but at least I hope it will point you in the direction of what to look for, and give the understanding of why it is so important.

And if a device manufacturer or operating system provider is doing a serious job at trying to reach these security goals listed, they can most likely also be trusted with actually protecting your security and privacy. The ones that aren’t trusted wouldn’t go to such extends in making a secure system, but would just rely on their fraudulent or misleading advertisement.

Also, as far as I know, there is no device and operating system combination that fulfills all these goals yet, but some are really close. This is more to help you compare, and to detect the ones that doesn’t even seem to take security seriously at all, regardless of their reputation.

I won’t claim this list is complete in any sense, and I acknowledge that each point in these lists might be differently important depending on your specific threat model.

How to judge if a device manufacturer takes security seriously:

  • Firmware updates are provided frequently and promptly, preferably at least once a month.

  • Firmware updates are provided for all hardware components, but at least for the boot firmware (UEFI for computers), 3G/4G/5G modem (baseband), Wifi and Bluetooth hardware, wired network, the TPM or secure element, and the CPU and GPU respectively, as those are security critical and usually have many serious security vulnerabilities found each year, each. Beware of device manufacturers that claims any hardware component is firmware-free, no hardware works without firmware, and if they claim that, they won’t be providing any firmware updates for that hardware component.

  • Firmware updates are provided for at least as many years as you intend to use the device.

  • Firmware updates are properly signed, and installation is refused unless the signature matches the enrolled key, and the signed version number is higher than the currently installed version.

  • That the device offers a secure implementation of verified boot or similar, that ensures the boot firmware, all other loaded firmware components, as well as the loaded operating system itself are properly signed with the enrolled key, and is of at least the version that was booted last time, so rollback attacks are not possible. Replacing the enrolled key must destroy all your data, so an evil-maid cannot make use of your disk encryption password to gain access to your data, even if they tricked you into entering it in their compromised operating system.

  • That the device offers a secure factory reset functionality, or secure way to format the device and reinstall the verified operating system, that can be invoked without loading any user data, and where all user data gets deleted, so the next boot of the operating system is guaranteed to not load any non-verified data at all. Without this, it would not be possible to clean a compromised system.

  • That the device offers a secure way to erase all your data such that it can never be recovered through any means, for example by securely deleting and regenerating a transparent disk encryption key, and that there is a way for you as a user to invoke that functionality. This functionality is usually either built-in in the disk controller itself or a secure element, and might be exposed as part of a secure factory reset, but sometimes require executing special commands to the disk controller or secure element instead. Shredding data by overwriting is not possible on modern flash based storage devices, so beware if this is advertised. Likewise, issuing TRIM commands to the flash based storage device will not securely delete the data, and is never enough. Only cryptographic destruction is reliable today.

  • That the device offers RAM memory encryption with a per-boot key, to prevent cold-boot attacks, where data can be extracted if the RAM memory modules are removed while the device is running, and placed in a special extraction device, or where the device is booted up again immediately after having been shut down. RAM memory modules usually take up to a few minutes after powerloss until they have fully lost all their data.

  • That the device offers a way to leave individual radios completely powered off (3G/4G/5G baseband, Wifi and Bluetooth), even if the rest of the device is running. This is to allow turning off long-distance radios when not used, to reduce risk of getting hacked, and to avoid getting your movements tracked unnecessarily.

  • That the device offers whatever hardware support is needed for the operating system to be able to harden operating system components, drivers and apps that are written in non-memory-safe languages such as C and C++. Typically this might be proper support for running virtual machines, including with IOMMU support, and advanced hardening functionalities such as memory tagging.

  • That the device manufacturer admits even up-to-date firmware might have security issues, and that all hardware components, especially radios (3G/4G/5G, Wifi, Bluetooth), are properly isolated from all other hardware components on bus level, as well as from the CPU and RAM memory using IOMMU or similar, so that they cannot access any other hardware component in any way at all, or access user app data or saved files in any way at all, without the operating system permitting it. Beware of subsystems that are specifically exempted from such isolation, for example to enable remote control or management of your device. They are an attractive target for attacks, as if compromised, they give undetected full and persistent access to all your device usage and data.

How to judge if an operating system takes security seriously:

  • Updates for the operating system, all hardware drivers, as well as all installed apps are provided frequently and promptly, preferably at least once a month. Be aware of operating systems that only provide updates for the most popular drivers and apps.

  • Updates for the operating system, all hardware drivers, as well as all installed apps are full version updates, not just backports of fixes for known security vulnerabilities with assigned CVEs. Many security issues that gets fixed and quality improvements to the code that is made never get any CVE assigned to them.

  • Updates for the operating system, all hardware drivers, as well as all installed apps are provided for at least as many years as you intend to use the device with this specific operating system.

  • Updates for the operating system, all hardware drivers, as well as all installed apps are signed, and installation is refused unless the signature matches the enrolled key, and the signed version number is higher than the currently installed version.

  • That the operating system including drivers and bundled apps are distributed as a monolithic signed image, and each app also distributed as a monolithic signed image, so that verified boot can protect far into the booted operating system.

  • That the operating system provider admits even an up-to-date app might be compromised, and makes sure that no app can access application data for any other app by any means at all, ie, that if the web browser is compromised it cannot access the data for your end-to-end encrypted chat app, nor other way around.

  • That the operating system provider admits even an up-to-date app might be compromised, and makes sure that no app can access privacy sensitive hardware such as the microphone, webcam, sensors or GPS without you explicitly granting that access for that specific app and occasion. It is okay that you can give an app access to a specific hardware component persistently, as long as you can choose not to.

  • That the operating system provider admits even an up-to-date app might be compromised, and makes sure that no app can read, write, delete or even detect the presence of any of your saved files without you explicitly granting that specific access for that specific file, app and occasion. Usually this is done by the app being able to open a system provided file picker for open respectively save operations, and then only being granted access to that file. It is okay that you can give an app access to all files in a folder, or all files of a certain file type, as long as you can choose not to.

  • That the operating system provider admits even an up-to-date app might be compromised, and makes sure that no app by any means can bypass the network assigned to it, whether that is a specific physical network, specific VPN or even no network access at all. No app must be able to detect your real IP address or any other externally exposed network identifier by any means, nor send out data, unless you permit it. Not even by talking to another app. Be aware of VPN leaks, many operating systems to not provide perfectly leak-proofed VPN implementations.

  • That the operating system provider admits even an up-to-date app might be compromised, and makes sure that no app can spoof being another more privacy sensitive app. One way is to make the icons, colors and app names in the title bars or app switcher unspoofable.

  • That the operating system provider admits even an up-to-date app might be compromised, and makes sure that no app can access the clipboard without user permission, nor perform screenshots or screen recording without user permission, nor being able to render over other applications or full screen itself without user permission, nor catch or record input events intended for another app without user permission, nor generate input events to simulate a user interacting with the device without user permission.

  • That the operating system provider admits even an up-to-date driver might be compromised, and makes sure drivers, at least for networking (wired, 3G/4G/5G, Wifi, Bluetooth) and external devices (USB), aren’t running with more privileges than regular user-installed apps, except for what is minimally required for them to function. In particular, no driver must be able to access the user’s app data or saved files by any means at all, nor interact with other unrelated drivers.

  • That the operating system provider admits users easily do mistakes that might harm their privacy a lot, and prevents this by making it hard for the user to copy-paste text or share files to apps where it was not intended. For example by requiring extra confirmation the first time files from a certain folder is shared with a certain app, or text is copied from a certain app to another certain app, and allowing the user to select between “allow only this time” or “allow everytime”.

  • That the operating system provider admits non-memory-safe programming languages like C and C++ are not suitable for any parts of the operating system, drivers nor apps, and that as much code as possible is written in memory safe languages, that more code is actively being ported to memory safe languages, and that the code that is not yet ported is hardened as much as possible.

  • That the operating system provider admits the importance of root of trust, and provides a trusted way for users to find and install apps, such that the user can be certain it is a legitimate release of that app. Typically this is done by bundling an app store with the operating system, that has all the apps you might want, whether maintained by the operating system or a third-party they trust.

  • That the operating system itself does not leak or spy on your activity at all, intentionally or through negligence. One may think an operating system that takes security very seriously wouldn’t do that, but unfortunately many do. Such spying is usually done in relation to app scanning or anti-virus scanning, screening functionalities of various kinds, cloud storage services, automatic metadata fetching or AI based tagging for media files, telemetry and automatic crash reporting. Beware of operating systems where such functionalities are enabled by default, or easy to enable by accident, such as by merely requiring flipping a switch. Ideally, an operating systems should not include such functionality bundled by default at all.

  • That the operating system provides built-in full disk encryption for all your data and metadata, including all your app data and saved files, including for files you save to external storage devices for backup keeping. The disk encryption should only be unlockable by knowing your password, by making the encryption key itself actually being derived from the password. Otherwise someone disassembling your device or using advanced forensics tools may be able to recover the encryption key. Beware of operating systems only offering disk encryption solutions that relies on a TPM releasing the encryption key, or where a backup of the encryption key is stored in an online account.

  • That the operating system does not provide any way to access app data, cached data such as thumbnails or search indices, keyboard data such as typing history, deleted data, or raw storage devices, not even to the legitimate user, and that the internal storage medium is unusable if attached to another device instead or accessed through another operating system, for example by encryption key being partially derived from a token enrolled in a secure element, which is released to the CPU through a secure and pre-authenticated channel, and is destroyed if the verified boot keys are replaced or OS version downgraded. This is so files, chats, web history and other data the user has already deleted cannot be easily recovered even if the user is forced to surrender the disk encryption password. Beware, issuing a TRIM operation to the underlying storage medium is not enough to prevent file recovery.

  • That only data that is intended for storage is written to storage devices, and that runtime data and data that should be deleted at application closure is kept in RAM memory as far as possible, and written to storage encrypted with a per-boot key otherwise. Files the user is working on should only be written to storage if the user chooses to do so, and should be sharable to other apps without touching storage. The operating system as well as apps should leave as little traces as possible on the storage device of their usage. Even if making data extraction hard is good, the less data there even is to extract, the better.

  • That the operating system only gives a limited number of attempts to unlock the screen with fingerprint, face, PIN or short password, before rebooting or requiring your disk encryption password again. This is because the disk encryption password is much stronger, and many failed attempts at unlocking the screen may indicate attempt at unauthorized access.

  • That the operating system shuts down or reboots with RAM memory wiping if left locked too long, as it is likely the device is no longer in the user’s control, but has been taken. This is to prevent future data extraction or screen unlock attempts.

  • The screen should automatically lock within a few minutes of inactivity. It is easy to forget to or fail to lock the screen, especially if distracted. Changing security features and installing apps should not be possible without authenticating again, even if the screen is unlocked, to prevent compromise.

  • That the operating system has a means to detect unauthorized access even when unlocked, and locks the screen, such as by detecting sudden strong movements that might indicate the device was snatched, or detecting device being disconnected from external power that might indicate the device is being stolen.