Shell not triggering

[YummYume]

Hello, thank you for making this!

As a new year resolution, I decided to set up kdump properly to try and resolve these rare but annoying system freezes I get. I stumbled upon this on the Arch wiki and decided to give it a go. After following the setup, I got the service up and running but after triggering a crash manually (su and then sync; echo c > /proc/sysrq-trigger), the system froze as expected but the emergency shell never launched. I waited for about 10 minutes before forcing a shutdown. I'm not sure how I can troubleshoot this issue, is there something I missed in the configuration ?

This is what my /etc/conf.d/simple-kdump.conf file looks like :

# Kernel and initramfs for the kexec environment.
# Recommended to use the linux or linux-lts kernel with 'default' preset.
KERNEL=/boot/vmlinuz-linux
INITRAMFS=/boot/amd-ucode.img

# No crashkernel= option for the kexec kernel cmdline.
# Just regular boot options, the extra needed ones will be added by
# simple-kdump-setup service automatically.
BOOT_OPTIONS=root=/dev/os/root rw loglevel=5 console=ttyS0

I didn't really change anything except for the INITRAMFS one. And I do have a crashkernel=768M kernel parameter. The service seems to be running fine :

Starting Setup kexec environment for simple-kdump...
Finished Setup kexec environment for simple-kdump.

[adam900710]

Sorry, your INITRAMFS= parameter is wrong. In your case you should go with the default one (/boot/initramfs-linux.img), not the microcode.

Furthermore, your BOOT_OPTIONS must match your environment.
You can check your existing boot options through /proc/cmdline.

I guess you're using GRUB2, thus are not that familiar with the kernel/initramfs/cmdline options.
(Although I hate GRUB2, it's really doing all the detection and cmdline generation pretty well for most users)

Please provide your /proc/cmdline output so that I can help you figuring out the needed options.

In the long run, I will add extra BOOT_OPTIONS auto-detection (which is not that hard using /proc/cmdline)

[YummYume]

@adam900710 Thank you for the fast answer. The main issue is that I do not have a /boot/initramfs-linux.img, which is why I went with the only .img I could find. Is there something I can do to add it or is it perhaps not located in /boot ?

The content of /proc/cmdline :

cryptdevice=PARTUUID=29dee41f-12a1-4e85-8ca5-211bb6ced0f9:root root=/dev/mapper/root zswap.enabled=0 rw rootfstype=ext4 crashkernel=768M

[adam900710]

Since you're using encrypted devices, I'm pretty sure you must have an initramfs.

Are you using Unified kernel images? In that case it's not going to work because the UKI images contains all of kernel, initramfs and boot options.

Would it be possible switch to regular separate kernel + initramfs options?

In that case, you may want to check this wiki instead:
https://wiki.archlinux.org/title/Mkinitcpio

But I'm also very new to UKI images. What's your bootloader? Or are you directly booting the UKI payload from UEFI?

And since you're using encrypted rootfs, it may not be a good idea to let kdump to save the image into rootfs, as

• You need to manually enter the passcode
  Meanwhile in the kexec environment the console may not be properly initialized.

• The current decryption process may need way memory than the reserved one

Thus for other server distros, they go the heavy lifting by making the initramfs fully independent and then just copy the vmcore to another server.

So unfortunately your case will be a little more complex to go through and may not even be suitable for the project.
And even with the vmcore saved, you will not be able to extract dmesg, without a proper vmlinux with all the debug info.
(Default Archlinux one has all of them striped)

Thus I recommend to go with netconsole instead to catch the dying message, and I'm pretty sure it will be some amdgpu related bugs.

[YummYume]

I did not think that far, but yes, I am using unified kernel images. I've been able to sign my .efi file for secure boot very easily that way (although the files get unsigned after a system crash and I have to add them back, how ironic). I'm also using systemd-boot to load the .efi files. I did use mkinitcpio as you linked to add the crashkernel parameter. I thought avoiding GRUB altogether would make things easier overall, but it seems to also bring its lot of trouble.

As a side note, I do have the vmlinux file built and ready to use just in case. Ultimately, I just want to see the cause of the kernel panic or whatever is causing the crash, because trying various fixes randomly and wait for another crash to happen days later has just not been very fun.

I did not know about netconsole, however isn't it usually used for servers and not physical machines ? It would mean my laptop would need to be constantly connected to a network, and especially the moment it crashes ? But other than that it seems to be what I'm looking for, and I do have a remote server I could use to send the logs to.

[shayaknyc]

Furthermore, your BOOT_OPTIONS must match your environment.
You can check your existing boot options through /proc/cmdline.

I have the same entry as @YummYume for my BOOT_OPTIONS in my /etc/conf.d/simple-kdump.conf file. This is the output of my /proc/cmdline:

BOOT_IMAGE=/@/boot/vmlinuz-linux-zen root=UUID=8dcc0c9c-8fe9-4fc6-b189-
a8db2cfa3823 rw rootflags=subvol=@ rd.driver.pre=vfio-pci intel_iommu=o
n i915.enable_guc=3 i915.max_vfs=7 quiet resume=UUID=a14a443b-9c6b-47d0
-b49a-d9686d767b7c loglevel=3 sysrq_always_enabled=1 split_lock_detect=
off nvidia-drm.modeset=1 pci=disable_acs_redir=pci:12D8:2308 crashkerne
l=512M ibt=off

You'll forgive me, but I'm not sure how I'm meant to adapt this to the BOOT_OPTION in the conf file.

Edit: I took a gamble and modified my /etc/conf.d/simple-kdump.conf file as follows, adapting the boot options from /proc/cmdline:

# Kernel and initramfs for the kexec environment.
# Recommended to use the linux or linux-lts kernel with 'default' preset.
KERNEL=/boot/vmlinuz-linux
INITRAMFS=/boot/initramfs-linux.img

# No crashkernel= option for the kexec kernel cmdline.
# Just regular boot options, the extra needed ones will be added by
# simple-kdump-setup service automatically.
#BOOT_OPTIONS=root=/dev/os/root rw loglevel=5 console=ttyS0
BOOT_OPTIONS=root=UUID=8dcc0c9c-8fe9-4fc6-b189-a8db2cfa3823 rw rootflags=subvol=@ rd.driver.pre=vfio-pci intel_iommu=on i915.enable_guc=3 i915.max_vfs=7 quiet resume=UUID=a14a443b-9c6b-47d0-b49a-d9686d767b7c loglevel=5 console=ttyS0 sysrq_always_enabled=1 split_lock_detect=off nvidia-drm.modeset=1 pci=disable_acs_redir=pci:12D8:2308 ibt=off

After saving this change, and running sudo systemctl restart simple-kdump-setup, I got this output:

● simple-kdump-setup.service - Setup kexec environment for simple-kdump
     Loaded: loaded (/usr/lib/systemd/system/simple-kdump-setup.service; enabled; preset: disabled)
     Active: active (exited) since Thu 2025-04-24 23:21:59 EDT; 7min ago
 Invocation: a9062e7b17144162808371737b207ae5
    Process: 1489530 ExecStart=/usr/bin/kexec -p $KERNEL --initrd $INITRAMFS --append ${BOOT_OPTIONS} nr_cpus=1 reset_devices systemd.unit=emergency-kdump.target (c>
   Main PID: 1489530 (code=exited, status=0/SUCCESS)
   Mem peak: 30M
        CPU: 164ms

Apr 24 23:21:59 shaya-desktop-intel-garuda systemd[1]: Starting Setup kexec environment for simple-kdump...
Apr 24 23:21:59 shaya-desktop-intel-garuda systemd[1]: Finished Setup kexec environment for simple-kdump.

Previously, it indicated that it was Running, and now it appears it's not. Am I on the right track?

Edit2: rebooted and caused a kernel crash, and nothing happened. no reboot, no nuthin. Not sure what I'm doing wrong.... Would graciously accept any guidance.