amd-snp.md

Miscellaneous Notes on AMD SEV-SNP

#VC exception handling in Linux

• Based on Linux 6.8
• #VC can happen in both user and kernel space
• user-space entry point
• kernel-space entry point

Detail

• Main handler is vc_handle_exitcode()
• cpuid
  • The kernel contains code for both SEV(-ES) and SNP
    • SNP VM can have cpuid page
  • SNP
    • The main part is vc_hdnale_cpuid_snp()
    • The handler tries to get cpuid from the cpuid page
    • Then only call VMGEXIT for fixup if needed
• rdmsr / wrmsr
  • VMGEIXT SVM_EXIT_MSR
• pio
  • VMGEXIT SVM_EXIT_IOIO
• hypercall (vmmcall)
  • VMGEIXT SVM_EXIT_VMMCALL

Enable AVX 512 in the guest

QEMU 8.1.5 requires to use not -cpu host but -cpu EPYC-v4 to boot SNP guests, but this CPU model misses several CPUIDs. Especially, AMD512-related CPUIDs are not enabled, resulting the guest application won't use AVX512. This affects the performance of HPC and AI/ML applications. We can manually enable these CPUIDs so that the guest can use AVX512.

Check available AVX instructions reported by CPUID

• EPYC 9334 (on the host)

% cat /proc/cpuinfo | grep flags | head -n1 | sed -e 's/ /\n/g' | sort | uniq | grep avx
avx
avx2
avx512_bf16
avx512_bitalg
avx512bw
avx512cd
avx512dq
avx512f
avx512ifma
avx512vbmi
avx512_vbmi2
avx512vl
avx512_vnni
avx512_vpopcntdq

• QEMU -cpu EPYC-v4 model

# cat /proc/cpuinfo | grep flags | head -n1 | sed -e 's/ /\n/g' | sort | uniq | grep avx
avx
avx2

We can enable AVX512 in the guest by manually adding features in the command line.

-cpu EPYC-v4,host-phys-bits=true,+avx512f,+avx512dq,+avx512cd,+avx512bw,+avx512vl,+avx512ifma,+avx512vbmi,+avx512vbmi2,+avx512vnni,+avx512bitalg
# +avx512vpopcntdq is not supported in QEMU 8.15. Future versions support it.

Now we get

# cat /proc/cpuinfo | grep flags | head -n1 | sed -e 's/ /\n/g' | sort | uniq | grep avx
avx
avx2
avx512_bitalg
avx512bw
avx512cd
avx512dq
avx512f
avx512ifma
avx512vbmi
avx512_vbmi2
avx512vl
avx512_vnni

Comment

• Are there any other missing important CPUIDs?