Some general information about starting a vm

General Discussion
1

Hi all,

I’m searching for some general information, what happens on the host and on the vm sides when

a) starting sys-net
b) starting sys-firewall
c) starting an app-vm

What are the points, the different domains communicate to others and what are the scripts and config files, one can look into, to understand some details?

Maybe there exist some draws already, which describe this?

Thanks in advance
Timewulf

1 Like
2

Really no one with an idea or tip for me?

Given that the underlaying Xen code and structure works as it should (We do so or we wouldn’t use Qubes-OS!), the security of the whole system is based on what happens here.

So if anyone wants to contribute without risking to smash the security of the system, he will need this knowledge. I can’t believe, that there aren’t any organisational charts to help for an overview, as it doesn’t make sense, that all who want to help, have to reinvent the wheel.

I’m willing to read scripts, configs and code to understand this all, but to save time I’ll need help where to start.

3

What about starting by the official docs?

4

Thanks, yes, I found them already, but those are really general.

They describe the fundamentals, what is important, too. But what I’m searching for is something, what describes the implementation, the data flow between the different VMs during those phases.

What scripts manage these phases?

It is important to know, what mechanisms are started, at what point during startup and in which order this is done. Without this knowledge there’s a great risk to break something.

5

You can usually find three layers to most tools:

  • Official docs
  • Developer docs
  • Code

Example:

6

Thanks again for your answers.

It’s taking the right direction, but my question goes deeper:

For example: The moment sys-firewall gets started, there has to be some synchronizing between this and sys-net for setting routes for example. Something similar must happen when other VMs start and get connected to sys-firewall.

What are the envolved scripts? - Is all communication started from a script in dom0, what starts sys-firewall and tells sys-net what has to be done or is there a script started in sys-firewall on startup, what communicates with sys-net?

I want to understand this workflow. The general conception is mostly clear.

7

If an AppVM’s (e.g. my-app-vm) NetVM is sys-firewall and if sys-firewall’s NetVM is sys-net, when all three are shutdown, the following command will start all three of them:

# dom0

qvm-start my-app-vm

I’m afraid I don’t have useful direct pointers, but I’d start reading through the qvm-start code to see what happens from there. Surely everything that has to happen (identifying VMs, starting them, configuring networks, etc.) does happen from there. :slightly_smiling_face:

8

You can use systemd-analyze to see what happens when an app qube start.
It could have other things (e.g. ~/.config/autostart/, /rw/config/ etc …),
but systemd-analyze is a good starting point.

[user@some_qube ~]$ systemd-analyze critical-chain

There are some qubes services defined here:

[user@some_qube ~]$ /usr/lib/systemd/system/qubes-*.service

You can also look in the entire /usr/lib/systemd/ directory.
e.g.

[user@some_qube ~]$ /usr/lib/systemd/user-preset/74-qubes-vm.preset
[user@some_qube ~]$ /usr/lib/systemd/system-preset/75-qubes-vm.preset
etc ...

There are some scripts here:

[user@some_qube ~]$ /usr/lib/qubes/
[user@some_qube ~]$ /etc/qubes/autostart/
[user@some_qube ~]$ /etc/qubes/post-install.d/
etc ...

To dig more you can use find:

[user@some_qube ~]$ sudo find / -iname "*qube*"

This command should give you a lot of qubes related folders/files.

And as gonzalo-bulnes mentioned, looking at the source code should also give you answers.

9

I looked into the sources belonging to qvm-start in dom0, but that’s not really what I’m looking for.
There’s “/usr/bin/qvm-start” what imports “/usr/lib/python3.11/site-packages/qubesadmin/tools/qvm_start.py”.
As I understand both are responsible for preparing the boot devices of the VMs to start and for the starting of the VMs. But I can’t see anything responsible for the network devices and related things.

The systemd-approach seems to show promise.

I’ll detail what I’ve found, later.

10

I was speaking about all source code, not speciffically about qvm-start (my bad, I should have been more specific).
Yeah I know, there are a lot of source code.

About app qube load/start, there could be some events that trigger action.
https://dev.qubes-os.org/projects/core-admin/en/latest/qubes-events.html
https://github.com/QubesOS/qubes-core-admin/tree/main/qubes/ext

e.g. event 'domain-start' or 'domain-load' or 'domain-init'
https://github.com/QubesOS/qubes-core-admin/blob/main/qubes/ext/audio.py#L82
https://github.com/QubesOS/qubes-core-admin/blob/main/qubes/ext/services.py#L138

There are maybe more events related to start/load. i didn’t look everythings. :slight_smile:

good hunting ^^