Blackhole Galaxy reset for tt-burnin #16
Description
When running tt-burnin without the --no-reset flag on a blackhole galaxy, reset does not execute as expected. On a blackhole galaxy, we should use ipmitool to reset the UBB, which is implemented in tt-smi using the -glx_reset flag.
(my-env) joshwang@bh-glx6u-11:~$ tt-burnin
Detected Chips: 32
Detecting ARC: \
Detecting DRAM: \
[] [14/14] ETH: \
┏━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Pci Dev ID ┃ Board Type ┃ Device Series ┃ Board Number ┃ Coordinates ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ 0 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 1 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 2 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 3 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 4 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 5 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 6 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 7 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 8 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 9 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 10 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 11 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 12 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 13 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 14 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 15 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 16 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 17 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 18 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 19 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 20 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 21 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 22 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 23 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 24 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 25 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 26 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 27 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 28 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 29 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 30 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
│ 31 │ Blackhole │ N/A │ 47131831011 │ (0, 0, 0, 0) │
└────────────┴────────────┴───────────────┴──────────────┴──────────────┘
Resetting devices on host...
Warning: Secondary bus reset not completed for device at PCI index 0. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 1. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 2. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 3. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 4. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 5. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 6. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 7. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 8. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 9. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 10. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 11. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 12. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 13. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 14. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 15. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 16. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 17. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 18. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 19. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 20. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 21. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 22. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 23. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 24. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 25. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 26. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 27. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 28. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 29. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 30. Continuing with reset.
Warning: Secondary bus reset not completed for device at PCI index 31. Continuing with reset.
Waiting for 12.8 seconds for potential hotplug removal.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 0.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 1.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 2.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 3.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 4.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 5.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 6.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 7.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 8.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 9.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 10.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 11.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 12.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 13.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 14.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 15.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 16.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 17.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 18.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 19.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 20.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 21.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 22.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 23.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 24.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 25.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 26.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 27.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 28.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 29.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 30.
Waiting for devices to reappear on pci bus...
Reset successfully completed for device at PCI index 31.
^C^C
Metadata
Metadata
Assignees
Labels
No labels