Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server: fix pod retrieval during volume attach #10324

Draft
wants to merge 2 commits into
base: 4.19
Choose a base branch
from

Conversation

shwstppr
Copy link
Contributor

@shwstppr shwstppr commented Feb 4, 2025

Description

Fixes issue highlighted in #9315 (comment)

A volume in Allocated state can be attached to a stopped VM even when the actual volume is not created in the primary store.
test_12_start_vm_multiple_volumes_allocated in test/integration/smoke/test_vm_life_cycle.py does the same.
After changes in #10267, a scenario was failing as a suitable pool was not found for an allocated volume when attached to a VM which has never been started

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • build/CI
  • test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

Verified test_vm_life_cycle.py with changes,

[root@pr9315-t7075-kvm-rocky8-marvin marvin]# nosetests --with-xunit --xunit-file=results.xml --with-marvin --marvin-config=./pr9315-t7075-kvm-rocky8-advanced-cfg -s -a tags=advanced --hypervisor=KVM tests/smoke/test_vm_life_cycle.py 
/usr/local/lib/python3.6/site-packages/paramiko/transport.py:32: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography. The next release of cryptography will remove support for Python 3.6.
  from cryptography.hazmat.backends import default_backend

==== Marvin Init Started ====

=== Marvin Parse Config Successful ===

=== Marvin Setting TestData Successful===

==== Log Folder Path: /marvin/MarvinLogs/Feb_05_2025_08_29_31_XD4RCY All logs will be available here ====

=== Marvin Init Logging Successful===

==== Marvin Init Successful ====
=== TestName: test_advZoneVirtualRouter | Status : SUCCESS ===

=== TestName: test_deploy_vm | Status : SUCCESS ===

=== TestName: test_deploy_vm_multiple | Status : SUCCESS ===

=== TestName: test_01_migrate_VM_and_root_volume | Status : SUCCESS ===

=== TestName: test_02_migrate_VM_with_two_data_disks | Status : SUCCESS ===

Securing Host pr9315-t7075-kvm-rocky8-kvm1
Host 9ba61cf4-a8a2-4ad7-afe7-f421a58ce2dd now showing as Up
Securing Host pr9315-t7075-kvm-rocky8-kvm2
Host 82c7aff5-ead2-4576-ba98-bfc4cdd81f8b now showing as Up
====Trying SSH Connection: Host:10.1.35.116 User:root                                   Port:22 RetryCnt:60===
===SSH to Host 10.1.35.116 port : 22 SUCCESSFUL===
{Cmd: grep -a listen_tls=1 /etc/libvirt/libvirtd.conf | tail -1 via Host: 10.1.35.116} {returns: ['listen_tls=1']}
Securing Host pr9315-t7075-kvm-rocky8-kvm1
Host 9ba61cf4-a8a2-4ad7-afe7-f421a58ce2dd now showing as Up
Securing Host pr9315-t7075-kvm-rocky8-kvm2
Host 82c7aff5-ead2-4576-ba98-bfc4cdd81f8b now showing as Up
=== TestName: test_01_secure_vm_migration | Status : SUCCESS ===

Securing Host pr9315-t7075-kvm-rocky8-kvm1
Host 9ba61cf4-a8a2-4ad7-afe7-f421a58ce2dd now showing as Up
Securing Host pr9315-t7075-kvm-rocky8-kvm2
Host 82c7aff5-ead2-4576-ba98-bfc4cdd81f8b now showing as Up
====Trying SSH Connection: Host:10.1.35.116 User:root                                   Port:22 RetryCnt:60===
===SSH to Host 10.1.35.116 port : 22 SUCCESSFUL===
{Cmd: rm -f /etc/cloudstack/agent/cloud* &&                       service cloudstack-agent stop ;                       service libvirtd stop ;                       service libvirt-bin stop ;                       sed -i 's/listen_tls.*/listen_tls=0/g' /etc/libvirt/libvirtd.conf &&                       sed -i 's/listen_tcp.*/listen_tcp=1/g' /etc/libvirt/libvirtd.conf &&                       sed -i '/.*_file=.*/d' /etc/libvirt/libvirtd.conf &&                       sed -i 's/vnc_tls.*/vnc_tls=0/g' /etc/libvirt/qemu.conf &&                       service libvirtd start ;                       service libvirt-bin start ;                       sleep 30 ;                       service cloudstack-agent start via Host: 10.1.35.116} {returns: ['Redirecting to /bin/systemctl stop cloudstack-agent.service', 'Redirecting to /bin/systemctl stop libvirtd.service', 'Redirecting to /bin/systemctl stop libvirt-bin.service', 'Failed to stop libvirt-bin.service: Unit libvirt-bin.service not loaded.', 'Redirecting to /bin/systemctl start libvirtd.service', 'Redirecting to /bin/systemctl start libvirt-bin.service', 'Failed to start libvirt-bin.service: Unit libvirt-bin.service not found.', 'Redirecting to /bin/systemctl start cloudstack-agent.service']}
Unsecuring Host: pr9315-t7075-kvm-rocky8-kvm1
Host 9ba61cf4-a8a2-4ad7-afe7-f421a58ce2dd now showing as Up
====Trying SSH Connection: Host:10.1.34.196 User:root                                   Port:22 RetryCnt:60===
===SSH to Host 10.1.34.196 port : 22 SUCCESSFUL===
{Cmd: rm -f /etc/cloudstack/agent/cloud* &&                       service cloudstack-agent stop ;                       service libvirtd stop ;                       service libvirt-bin stop ;                       sed -i 's/listen_tls.*/listen_tls=0/g' /etc/libvirt/libvirtd.conf &&                       sed -i 's/listen_tcp.*/listen_tcp=1/g' /etc/libvirt/libvirtd.conf &&                       sed -i '/.*_file=.*/d' /etc/libvirt/libvirtd.conf &&                       sed -i 's/vnc_tls.*/vnc_tls=0/g' /etc/libvirt/qemu.conf &&                       service libvirtd start ;                       service libvirt-bin start ;                       sleep 30 ;                       service cloudstack-agent start via Host: 10.1.34.196} {returns: ['Redirecting to /bin/systemctl stop cloudstack-agent.service', 'Redirecting to /bin/systemctl stop libvirtd.service', 'Redirecting to /bin/systemctl stop libvirt-bin.service', 'Failed to stop libvirt-bin.service: Unit libvirt-bin.service not loaded.', 'Redirecting to /bin/systemctl start libvirtd.service', 'Redirecting to /bin/systemctl start libvirt-bin.service', 'Failed to start libvirt-bin.service: Unit libvirt-bin.service not found.', 'Redirecting to /bin/systemctl start cloudstack-agent.service']}
Unsecuring Host: pr9315-t7075-kvm-rocky8-kvm2
Host 82c7aff5-ead2-4576-ba98-bfc4cdd81f8b now showing as Up
====Trying SSH Connection: Host:10.1.35.116 User:root                                   Port:22 RetryCnt:60===
===SSH to Host 10.1.35.116 port : 22 SUCCESSFUL===
{Cmd: grep -a listen_tcp=1 /etc/libvirt/libvirtd.conf | tail -1 via Host: 10.1.35.116} {returns: ['listen_tcp=1']}
Securing Host pr9315-t7075-kvm-rocky8-kvm1
Host 9ba61cf4-a8a2-4ad7-afe7-f421a58ce2dd now showing as Up
Securing Host pr9315-t7075-kvm-rocky8-kvm2
Host 82c7aff5-ead2-4576-ba98-bfc4cdd81f8b now showing as Up
=== TestName: test_02_unsecure_vm_migration | Status : SUCCESS ===

Securing Host pr9315-t7075-kvm-rocky8-kvm1
Host 9ba61cf4-a8a2-4ad7-afe7-f421a58ce2dd now showing as Up
Securing Host pr9315-t7075-kvm-rocky8-kvm2
Host 82c7aff5-ead2-4576-ba98-bfc4cdd81f8b now showing as Up
====Trying SSH Connection: Host:10.1.35.116 User:root                                   Port:22 RetryCnt:60===
===SSH to Host 10.1.35.116 port : 22 SUCCESSFUL===
{Cmd: rm -f /etc/cloudstack/agent/cloud* &&                       service cloudstack-agent stop ;                       service libvirtd stop ;                       service libvirt-bin stop ;                       sed -i 's/listen_tls.*/listen_tls=0/g' /etc/libvirt/libvirtd.conf &&                       sed -i 's/listen_tcp.*/listen_tcp=1/g' /etc/libvirt/libvirtd.conf &&                       sed -i '/.*_file=.*/d' /etc/libvirt/libvirtd.conf &&                       sed -i 's/vnc_tls.*/vnc_tls=0/g' /etc/libvirt/qemu.conf &&                       service libvirtd start ;                       service libvirt-bin start ;                       sleep 30 ;                       service cloudstack-agent start via Host: 10.1.35.116} {returns: ['Redirecting to /bin/systemctl stop cloudstack-agent.service', 'Redirecting to /bin/systemctl stop libvirtd.service', 'Redirecting to /bin/systemctl stop libvirt-bin.service', 'Failed to stop libvirt-bin.service: Unit libvirt-bin.service not loaded.', 'Redirecting to /bin/systemctl start libvirtd.service', 'Redirecting to /bin/systemctl start libvirt-bin.service', 'Failed to start libvirt-bin.service: Unit libvirt-bin.service not found.', 'Redirecting to /bin/systemctl start cloudstack-agent.service']}
Unsecuring Host: pr9315-t7075-kvm-rocky8-kvm1
Host 9ba61cf4-a8a2-4ad7-afe7-f421a58ce2dd now showing as Up
====Trying SSH Connection: Host:10.1.34.196 User:root                                   Port:22 RetryCnt:60===
===SSH to Host 10.1.34.196 port : 22 SUCCESSFUL===
{Cmd: grep -a listen_tls=1 /etc/libvirt/libvirtd.conf | tail -1 via Host: 10.1.34.196} {returns: ['listen_tls=1']}
Securing Host pr9315-t7075-kvm-rocky8-kvm1
Host 9ba61cf4-a8a2-4ad7-afe7-f421a58ce2dd now showing as Up
Securing Host pr9315-t7075-kvm-rocky8-kvm2
Host 82c7aff5-ead2-4576-ba98-bfc4cdd81f8b now showing as Up
=== TestName: test_03_secured_to_nonsecured_vm_migration | Status : SUCCESS ===

Securing Host pr9315-t7075-kvm-rocky8-kvm1
Host 9ba61cf4-a8a2-4ad7-afe7-f421a58ce2dd now showing as Up
Securing Host pr9315-t7075-kvm-rocky8-kvm2
Host 82c7aff5-ead2-4576-ba98-bfc4cdd81f8b now showing as Up
====Trying SSH Connection: Host:10.1.35.116 User:root                                   Port:22 RetryCnt:60===
===SSH to Host 10.1.35.116 port : 22 SUCCESSFUL===
{Cmd: rm -f /etc/cloudstack/agent/cloud* &&                       service cloudstack-agent stop ;                       service libvirtd stop ;                       service libvirt-bin stop ;                       sed -i 's/listen_tls.*/listen_tls=0/g' /etc/libvirt/libvirtd.conf &&                       sed -i 's/listen_tcp.*/listen_tcp=1/g' /etc/libvirt/libvirtd.conf &&                       sed -i '/.*_file=.*/d' /etc/libvirt/libvirtd.conf &&                       sed -i 's/vnc_tls.*/vnc_tls=0/g' /etc/libvirt/qemu.conf &&                       service libvirtd start ;                       service libvirt-bin start ;                       sleep 30 ;                       service cloudstack-agent start via Host: 10.1.35.116} {returns: ['Redirecting to /bin/systemctl stop cloudstack-agent.service', 'Redirecting to /bin/systemctl stop libvirtd.service', 'Redirecting to /bin/systemctl stop libvirt-bin.service', 'Failed to stop libvirt-bin.service: Unit libvirt-bin.service not loaded.', 'Redirecting to /bin/systemctl start libvirtd.service', 'Redirecting to /bin/systemctl start libvirt-bin.service', 'Failed to start libvirt-bin.service: Unit libvirt-bin.service not found.', 'Redirecting to /bin/systemctl start cloudstack-agent.service']}
Unsecuring Host: pr9315-t7075-kvm-rocky8-kvm1
Host 9ba61cf4-a8a2-4ad7-afe7-f421a58ce2dd now showing as Up
====Trying SSH Connection: Host:10.1.35.116 User:root                                   Port:22 RetryCnt:60===
===SSH to Host 10.1.35.116 port : 22 SUCCESSFUL===
{Cmd: grep -a listen_tcp=1 /etc/libvirt/libvirtd.conf | tail -1 via Host: 10.1.35.116} {returns: ['listen_tcp=1']}
Securing Host pr9315-t7075-kvm-rocky8-kvm1
Host 9ba61cf4-a8a2-4ad7-afe7-f421a58ce2dd now showing as Up
Securing Host pr9315-t7075-kvm-rocky8-kvm2
Host 82c7aff5-ead2-4576-ba98-bfc4cdd81f8b now showing as Up
=== TestName: test_04_nonsecured_to_secured_vm_migration | Status : SUCCESS ===

=== TestName: test_01_stop_vm | Status : SUCCESS ===

=== TestName: test_01_stop_vm_forced | Status : SUCCESS ===

=== TestName: test_02_start_vm | Status : SUCCESS ===

=== TestName: test_03_reboot_vm | Status : SUCCESS ===

=== TestName: test_04_reboot_vm_forced | Status : SUCCESS ===

=== TestName: test_06_destroy_vm | Status : SUCCESS ===

=== TestName: test_07_restore_vm | Status : SUCCESS ===

=== TestName: test_08_migrate_vm | Status : SUCCESS ===

=== TestName: test_09_expunge_vm | Status : SUCCESS ===

====Trying SSH Connection: Host:10.1.53.86 User:root                                   Port:22 RetryCnt:20===
===SSH to Host 10.1.53.86 port : 22 SUCCESSFUL===
{Cmd: mkdir -p /mnt/tmp via Host: 10.1.53.86} {returns: []}
{Cmd: mount -rt iso9660 /dev/vdc /mnt/tmp via Host: 10.1.53.86} {returns: ['mount: special device /dev/vdc does not exist']}
{Cmd: mount -rt iso9660 /dev/vdb /mnt/tmp via Host: 10.1.53.86} {returns: ['mount: special device /dev/vdb does not exist']}
{Cmd: mount -rt iso9660 /dev/hdb /mnt/tmp via Host: 10.1.53.86} {returns: ['mount: special device /dev/hdb does not exist']}
{Cmd: mount -rt iso9660 /dev/hdc /mnt/tmp via Host: 10.1.53.86} {returns: []}
{Cmd: mount |grep /dev/hdc|head -1 via Host: 10.1.53.86} {returns: ['/dev/hdc on /mnt/tmp type iso9660 (ro)']}
{Cmd: du /dev/hdc | tail -1 via Host: 10.1.53.86} {returns: ['0\t/dev/hdc']}
{Cmd: umount /mnt/tmp via Host: 10.1.53.86} {returns: []}
{Cmd: mount |grep /dev/hdc|head -1 via Host: 10.1.53.86} {returns: []}
=== TestName: test_10_attachAndDetach_iso | Status : SUCCESS ===

=== TestName: test_11_destroy_vm_and_volumes | Status : SUCCESS ===

=== TestName: test_12_start_vm_multiple_volumes_allocated | Status : SUCCESS ===

=== Final results are now copied to: /marvin//MarvinLogs/test_vm_life_cycle_HAM8FF ===
[root@pr9315-t7075-kvm-rocky8-marvin marvin]# cat /marvin//MarvinLogs/test_vm_life_cycle_HAM8FF/results.txt 
Test advanced zone virtual router ... === TestName: test_advZoneVirtualRouter | Status : SUCCESS ===
ok
Test Deploy Virtual Machine ... === TestName: test_deploy_vm | Status : SUCCESS ===
ok
Test Multiple Deploy Virtual Machine ... === TestName: test_deploy_vm_multiple | Status : SUCCESS ===
ok
Test VM will be migrated with it's root volume ... === TestName: test_01_migrate_VM_and_root_volume | Status : SUCCESS ===
ok
Test VM will be migrated with it's root volume ... === TestName: test_02_migrate_VM_with_two_data_disks | Status : SUCCESS ===
ok
Test VM will be migrated with it's root volume ... SKIP: VM Migration with Volumes is not supported on other than VMware
Test VM will be migrated with it's root volume ... SKIP: VM Migration with Volumes is not supported on other than VMware
Test VM will be migrated with it's root volume ... SKIP: VM Migration with Volumes is not supported on other than VMware
Test VM will be migrated with it's root volume ... SKIP: VM Migration with Volumes is not supported on other than VMware
Test secure VM migration ... === TestName: test_01_secure_vm_migration | Status : SUCCESS ===
ok
Test Non-secured VM Migration ... === TestName: test_02_unsecure_vm_migration | Status : SUCCESS ===
ok
Test destroy Virtual Machine ... === TestName: test_03_secured_to_nonsecured_vm_migration | Status : SUCCESS ===
ok
Test Non-secured VM Migration ... === TestName: test_04_nonsecured_to_secured_vm_migration | Status : SUCCESS ===
ok
Test the following for all found ovf templates: ... SKIP: Skipping test: Reason -  hypervisorNotSupported
Test Stop Virtual Machine ... === TestName: test_01_stop_vm | Status : SUCCESS ===
ok
Test Force Stop Virtual Machine ... === TestName: test_01_stop_vm_forced | Status : SUCCESS ===
ok
Test Start Virtual Machine ... === TestName: test_02_start_vm | Status : SUCCESS ===
ok
Test Reboot Virtual Machine ... === TestName: test_03_reboot_vm | Status : SUCCESS ===
ok
Test Force Reboot Virtual Machine ... === TestName: test_04_reboot_vm_forced | Status : SUCCESS ===
ok
Test destroy Virtual Machine ... === TestName: test_06_destroy_vm | Status : SUCCESS ===
ok
Test recover Virtual Machine ... === TestName: test_07_restore_vm | Status : SUCCESS ===
ok
Test migrate VM ... === TestName: test_08_migrate_vm | Status : SUCCESS ===
ok
Test destroy(expunge) Virtual Machine ... === TestName: test_09_expunge_vm | Status : SUCCESS ===
ok
Test for attach and detach ISO to virtual machine ... === TestName: test_10_attachAndDetach_iso | Status : SUCCESS ===
ok
Test destroy Virtual Machine and it's volumes ... === TestName: test_11_destroy_vm_and_volumes | Status : SUCCESS ===
ok
Test attaching multiple datadisks and start VM ... === TestName: test_12_start_vm_multiple_volumes_allocated | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 26 tests in 1770.170s

OK (SKIP=5)

How did you try to break this feature and the system with this change?

Fixes issue highlighted in apache#9315 (comment)

Signed-off-by: Abhishek Kumar <[email protected]>
@shwstppr
Copy link
Contributor Author

shwstppr commented Feb 4, 2025

@blueorangutan package

@blueorangutan
Copy link

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Copy link

codecov bot commented Feb 4, 2025

Codecov Report

Attention: Patch coverage is 87.50000% with 1 line in your changes missing coverage. Please review.

Project coverage is 15.16%. Comparing base (97be6f2) to head (daa379e).
Report is 12 commits behind head on 4.19.

Files with missing lines Patch % Lines
...n/java/com/cloud/storage/VolumeApiServiceImpl.java 87.50% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##              4.19   #10324       +/-   ##
============================================
+ Coverage     4.29%   15.16%   +10.86%     
- Complexity       0    11314    +11314     
============================================
  Files          366     5409     +5043     
  Lines        29602   474474   +444872     
  Branches      5193    57877    +52684     
============================================
+ Hits          1272    71944    +70672     
- Misses       28185   394487   +366302     
- Partials       145     8043     +7898     
Flag Coverage Δ
uitests 4.29% <ø> (-0.01%) ⬇️
unittests 15.88% <87.50%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@DaanHoogland DaanHoogland added this to the 4.19.2 milestone Feb 4, 2025
Copy link
Contributor

@DaanHoogland DaanHoogland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clgtm

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12327

Copy link
Member

@weizhouapache weizhouapache left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code lgtm

@shwstppr
thanks for the quick fix

@DaanHoogland
Copy link
Contributor

@blueorangutan test

@blueorangutan
Copy link

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan
Copy link

[SF] Trillian test result (tid-12306)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 48183 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10324-t12306-kvm-ol8.zip
Smoke tests completed. 132 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_02_unsecure_vm_migration Error 499.08 test_vm_life_cycle.py
test_12_start_vm_multiple_volumes_allocated Error 17.37 test_vm_life_cycle.py

@shwstppr
Copy link
Contributor Author

shwstppr commented Feb 5, 2025

@blueorangutan package

@blueorangutan
Copy link

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Copy link
Contributor

@DaanHoogland DaanHoogland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code looks good, let's test ;)

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12339

@DaanHoogland
Copy link
Contributor

@shwstppr , is this one ready or awaiting further improvements pending the revert of #10267 ?

@shwstppr
Copy link
Contributor Author

shwstppr commented Feb 5, 2025

@DaanHoogland this is ready now. If we revert #10267 then I'll add commits from that here

@shwstppr
Copy link
Contributor Author

shwstppr commented Feb 6, 2025

@blueorangutan test

@blueorangutan
Copy link

@shwstppr a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants