api: cache system info #651

marmarek · 2025-02-04T01:43:43Z

collecting the information used for qrexec policy evaluation can be
expensive, but it doesn't change that often. cache it inside qubesd daemon,
and invalidate the cache based on events related to any of the included
information.

fixes QubesOS/qubes-issues#9362

Collecting the information used for qrexec policy evaluation can be expensive, but it doesn't change that often. Cache it inside qubesd daemon, and invalidate the cache based on events related to any of the included information. Fixes QubesOS/qubes-issues#9362

codecov · 2025-02-04T23:06:48Z

Codecov Report

Attention: Patch coverage is 81.63265% with 9 lines in your changes missing coverage. Please review.

Project coverage is 69.69%. Comparing base (c1cdf38) to head (953620a).

Files with missing lines	Patch %	Lines
qubes/api/internal.py	80.85%	9 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #651      +/-   ##
==========================================
- Coverage   69.71%   69.69%   -0.02%     
==========================================
  Files          58       58              
  Lines       12499    12511      +12     
==========================================
+ Hits         8714     8720       +6     
- Misses       3785     3791       +6

Flag	Coverage Δ
unittests	`69.69% <81.63%> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

qubesos-bot · 2025-02-06T07:50:39Z

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025020921-4.3&flavor=pull-requests

Test run included the following:

Do not close FDs 0, 1, or 2 qubes-core-qrexec#186 (QubesOS/qubes-core-qrexec@24ef8b1)
SystemD dropins for minimal sys-net and sys-usb qubes-core-agent-linux#540 (QubesOS/qubes-core-agent-linux@451ca9e)
Add filter for qubes stored on inaccessible storage pools qubes-manager#399 (QubesOS/qubes-manager@7dcb8b7)
tests: add simple storage performance tests #649 (664c5dd)
api: cache system info #651 (030f5f3)
Add Arch packaging qubes-core-admin-client#332 (QubesOS/qubes-core-admin-client@e22c531)

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025020404-4.3&flavor=update

system_tests_pvgrub_salt_storage
- TC_41_HVMGrub_debian-12-xfce: test_000_standalone_vm (error)
  qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
- TC_41_HVMGrub_debian-12-xfce: test_001_standalone_vm_dracut (error)
  qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
- TC_41_HVMGrub_debian-12-xfce: test_010_template_based_vm (error)
  qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
- TC_41_HVMGrub_debian-12-xfce: test_011_template_based_vm_dracut (error)
  qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
- TC_41_HVMGrub_fedora-41-xfce: test_000_standalone_vm (error)
  qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
- TC_41_HVMGrub_fedora-41-xfce: test_010_template_based_vm (error)
  qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
system_tests_extra
- TC_00_QVCTest_whonix-gateway-17: test_010_screenshare (failure)
  ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 0 == 0
- TC_00_QVCTest_whonix-workstation-17: test_010_screenshare (failure)
  ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 0 == 0
system_tests_kde_gui_interactive
- gui_keyboard_layout: wait_serial (wait serial expected)
  # wait_serial expected: "echo -e '[Layout]\nLayoutList=us,de' | sud...
- gui_keyboard_layout: Failed (test died)
  # Test died: command 'test "$(cd ~user;ls e1*)" = "$(qvm-run -p wor...
system_tests_suspend@hw1
- suspend: unnamed test (unknown)
- suspend: Failed (test died)
  # Test died: no candidate needle with tag(s) 'SUSPEND-FAILED' match...
system_tests_whonix@hw7
- whonixcheck: fail (unknown)
  Whonixcheck for sys-whonix failed...
- whonixcheck: unnamed test (unknown)
system_tests_guivm_gui_interactive
- simple_gui_apps: unnamed test (unknown)
- simple_gui_apps: Failed (test died)
  # Test died: no candidate needle with tag(s) 'vm-settings-app-xterm...
- simple_gui_apps: unnamed test (unknown)
- simple_gui_apps: wait_serial (wait serial expected)
  # wait_serial expected: qr/2E8vz-\d+-/...

Failed tests

18 failures

system_tests_pvgrub_salt_storage
- TC_41_HVMGrub_debian-12-xfce: test_000_standalone_vm (error)
  qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
- TC_41_HVMGrub_debian-12-xfce: test_001_standalone_vm_dracut (error)
  qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
- TC_41_HVMGrub_debian-12-xfce: test_010_template_based_vm (error)
  qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
- TC_41_HVMGrub_debian-12-xfce: test_011_template_based_vm_dracut (error)
  qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
- TC_41_HVMGrub_fedora-41-xfce: test_000_standalone_vm (error)
  qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
- TC_41_HVMGrub_fedora-41-xfce: test_010_template_based_vm (error)
  qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
system_tests_extra
- TC_00_QVCTest_whonix-gateway-17: test_010_screenshare (failure)
  ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 0 == 0
- TC_00_QVCTest_whonix-workstation-17: test_010_screenshare (failure)
  ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 0 == 0
system_tests_kde_gui_interactive
- gui_keyboard_layout: wait_serial (wait serial expected)
  # wait_serial expected: "echo -e '[Layout]\nLayoutList=us,de' | sud...
- gui_keyboard_layout: Failed (test died)
  # Test died: command 'test "$(cd ~user;ls e1*)" = "$(qvm-run -p wor...
system_tests_suspend@hw1
- suspend: unnamed test (unknown)
- suspend: Failed (test died)
  # Test died: no candidate needle with tag(s) 'SUSPEND-FAILED' match...
system_tests_whonix@hw7
- whonixcheck: fail (unknown)
  Whonixcheck for sys-whonix failed...
- whonixcheck: unnamed test (unknown)
system_tests_guivm_gui_interactive
- simple_gui_apps: unnamed test (unknown)
- simple_gui_apps: Failed (test died)
  # Test died: no candidate needle with tag(s) 'vm-settings-app-xterm...
- simple_gui_apps: unnamed test (unknown)
- simple_gui_apps: wait_serial (wait serial expected)
  # wait_serial expected: qr/2E8vz-\d+-/...

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/127852#dependencies

31 fixed

system_tests_suspend
- mount_and_boot_options: unnamed test (unknown)
- mount_and_boot_options: Failed (test died)
  # Test died: no candidate needle with tag(s) 'x11' matched...
system_tests_qrexec
- TC_00_Qrexec_debian-12-xfce: test_070_qrexec_vm_simultaneous_write (failure)
  exited with non-zero code 125; stderr: b"dd: error writing 'standa...
system_tests_dispvm
- TC_20_DispVM_fedora-41-xfce: test_100_open_in_dispvm (failure)
  AssertionError: './open-file test.txt' failed with ./open-file test...
system_tests_devices
- TC_00_List_whonix-gateway-17: test_000_list_loop (error)
  subprocess.CalledProcessError: Command 'set -e;truncate -s 128M /tm...
- TC_00_List_whonix-gateway-17: test_001_list_loop_mounted (error)
  subprocess.CalledProcessError: Command 'set -e;truncate -s 128M /tm...
- TC_00_List_whonix-gateway-17: test_010_list_dm (error)
  subprocess.CalledProcessError: Command 'set -e;truncate -s 128M /tm...
- TC_00_List_whonix-gateway-17: test_011_list_dm_mounted (error)
  subprocess.CalledProcessError: Command 'set -e;truncate -s 128M /tm...
- TC_00_List_whonix-gateway-17: test_012_list_dm_delayed (error)
  subprocess.CalledProcessError: Command 'set -e;truncate -s 128M /tm...
- TC_00_List_whonix-gateway-17: test_013_list_dm_removed (error)
  subprocess.CalledProcessError: Command 'set -e;truncate -s 128M /tm...
- TC_00_List_whonix-gateway-17: test_020_list_loop_partition (error)
  subprocess.CalledProcessError: Command 'set -e;truncate -s 128M /tm...
- TC_00_List_whonix-gateway-17: test_021_list_loop_partition_mounted (error)
  subprocess.CalledProcessError: Command 'set -e;truncate -s 128M /tm...
- TC_00_List_whonix-workstation-17: test_000_list_loop (error)
  subprocess.CalledProcessError: Command 'set -e;truncate -s 128M /tm...
- TC_00_List_whonix-workstation-17: test_001_list_loop_mounted (error)
  subprocess.CalledProcessError: Command 'set -e;truncate -s 128M /tm...
- TC_00_List_whonix-workstation-17: test_010_list_dm (error)
  subprocess.CalledProcessError: Command 'set -e;truncate -s 128M /tm...
- TC_00_List_whonix-workstation-17: test_011_list_dm_mounted (error)
  subprocess.CalledProcessError: Command 'set -e;truncate -s 128M /tm...
- TC_00_List_whonix-workstation-17: test_012_list_dm_delayed (error)
  subprocess.CalledProcessError: Command 'set -e;truncate -s 128M /tm...
- TC_00_List_whonix-workstation-17: test_013_list_dm_removed (error)
  subprocess.CalledProcessError: Command 'set -e;truncate -s 128M /tm...
- TC_00_List_whonix-workstation-17: test_020_list_loop_partition (error)
  subprocess.CalledProcessError: Command 'set -e;truncate -s 128M /tm...
- TC_00_List_whonix-workstation-17: test_021_list_loop_partition_mounted (error)
  subprocess.CalledProcessError: Command 'set -e;truncate -s 128M /tm...
- TC_10_Attach_whonix-gateway-17: test_000_attach_reattach (error)
  subprocess.CalledProcessError: Command 'set -e;truncate -s 128M /tm...
- TC_10_Attach_whonix-workstation-17: test_000_attach_reattach (error)
  subprocess.CalledProcessError: Command 'set -e;truncate -s 128M /tm...
system_tests_kde_gui_interactive
- clipboard_and_web: unnamed test (unknown)
- clipboard_and_web: Failed (test died)
  # Test died: no candidate needle with tag(s) 'menu-vm-firefox' matc...
system_tests_audio
- TC_20_AudioVM_PipeWire_whonix-workstation-17: test_250_audio_playback_audiovm_pipewire (failure)
  AssertionError: pacat for test-inst-vm1 (xid 63) running(False) in ...
- TC_20_AudioVM_PipeWire_whonix-workstation-17: test_251_audio_playback_audiovm_pipewire_late_start (failure)
  AssertionError: too short audio, expected 10s, got 9.42956916099773...
system_tests_qrexec_perf@hw1
- TC_00_QrexecPerf_debian-12-xfce: test_110_simple_data_duplex (failure)
  AssertionError: '/usr/lib/qubes/tests/qrexec_perf.py --vm1=test-ins...
system_tests_storage_perf@hw1
- integ: storage_perf (error)
  ModuleNotFoundError: No module named 'qubes.tests.integ.storage_perf'
system_tests_basic_vm_qrexec_gui_ext4
- switch_pool: Failed (test died)
  # Test died: command 'printf "label: gpt\n,,L" | sfdisk /dev/sdb' f...
system_tests_backup
- TC_10_BackupVM_whonix-gateway-17: test_110_send_to_vm_no_space (error)
  subprocess.CalledProcessError: Command 'mknod /dev/loop0 b 7 0;trun...
- TC_10_BackupVM_whonix-workstation-17: test_110_send_to_vm_no_space (error)
  subprocess.CalledProcessError: Command 'mknod /dev/loop0 b 7 0;trun...

Unstable tests

## Performance Tests ### Performance degradation: No issues ### Remaining performance tests: No remaining performance tests

Especially get fork-server logs from .xsession-errors.

Fedora 41 use yet another desktop app id. QubesOS/qubes-issues#9244

tests: add sbin to PATH so mkfs.ext4 works under Whonix

ece1603

marmarek force-pushed the system-info-cache branch from 6f13b22 to 085ef16 Compare February 4, 2025 19:18

api: cache system info

0f3d7fa

Collecting the information used for qrexec policy evaluation can be expensive, but it doesn't change that often. Cache it inside qubesd daemon, and invalidate the cache based on events related to any of the included information. Fixes QubesOS/qubes-issues#9362

marmarek force-pushed the system-info-cache branch from 085ef16 to 0f3d7fa Compare February 4, 2025 20:47

ci: use default docker image

11e7b33

marmarek added the openqa-pending label Feb 5, 2025

tests: fix qrexec_perf tests to actually run on all templates

953620a

marmarek added 2 commits February 10, 2025 03:32

tests: collect more info on qrexec test failure

a5351b8

Especially get fork-server logs from .xsession-errors.

tests: support the new(er) Thunderbird name on F41+

30bee56

Fedora 41 use yet another desktop app id. QubesOS/qubes-issues#9244

marmarek force-pushed the system-info-cache branch from 030f5f3 to 30bee56 Compare February 10, 2025 02:32

marmarek merged commit 30bee56 into QubesOS:main Feb 10, 2025
2 of 3 checks passed

marmarek deleted the system-info-cache branch February 18, 2025 11:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

api: cache system info #651

api: cache system info #651

marmarek commented Feb 4, 2025

codecov bot commented Feb 4, 2025 •

edited

Loading

qubesos-bot commented Feb 6, 2025 •

edited

Loading

api: cache system info #651

api: cache system info #651

Conversation

marmarek commented Feb 4, 2025

codecov bot commented Feb 4, 2025 • edited Loading

Codecov Report

qubesos-bot commented Feb 6, 2025 • edited Loading

OpenQA test summary

New failures, excluding unstable

Failed tests

Fixed failures

Unstable tests

codecov bot commented Feb 4, 2025 •

edited

Loading

qubesos-bot commented Feb 6, 2025 •

edited

Loading