Cannot launch Ubuntu 22.04 multipass on my Ubuntu 24.04 machine

[sjg20]

Describe the bug
I am trying to launch Ubuntu 22.04 multipass on my Ubuntu 24.04 machine. But I must be doing something horribly wrong as it just hangs.

To Reproduce
How, and what happened?

1. multipass start jammy -vvvv

Expected behavior
I expect it to start quickly and give me a shell

Logs
log.txt

Additional info

• OS: Ubuntu 24.04

• CPU architecture or model: x86 (amd64)

• multipass version
  multipass 1.15.0
  multipassd 1.15.0

• multipass info

• Name: jammy
  State: Unknown
  Snapshots: 0
  IPv4: --
  Release: --
  Image hash: 610e1d37fe06 (Ubuntu 22.04 LTS)
  CPU(s): --
  Load: --
  Disk usage: --
  Memory usage: --
  Mounts:

• multipass get local.driver
  qemu

Additional context
Add any other context about the problem here.

I tried to watch the network, but it seems there is no bootp server running. My local network has one on 192.168.4.1 but I'm not sure if multipass is trying to use that or its own?

sudo tcpdump -i mpqemubr0 -n udp port 67 and port 68
libibverbs: Warning: couldn't open config directory '/etc/libibverbs.d'.
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on mpqemubr0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
14:01:02.405819 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 52:54:00:dd:a6:5b, length 277
14:01:06.181737 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 52:54:00:dd:a6:5b, length 277
14:01:13.543157 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 52:54:00:dd:a6:5b, length 277
14:01:28.836497 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 52:54:00:dd:a6:5b, length 277
14:02:01.535738 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 52:54:00:dd:a6:5b, length 277

[georgeliao]

@sjg20
Sorry that you have encountered this.
I have two wondering.

multipass start jammy -vvvv

This is the command for starting the jammy instance. This implies that have launched an instance jammy and stopped it. Is that the case?

By scanning your log.txt, I noticed

Jan 27 09:20:02 ellesmere dnsmasq[1190049]: failed to create listening socket for 10.165.74.1: Address already in use
Jan 27 09:20:02 ellesmere dnsmasq[1190049]: FAILED to start up
Jan 27 09:20:02 ellesmere multipassd[1189715]: 
                                               dnsmasq: failed to create listening socket for 10.165.74.1: Address already in use
Jan 27 09:20:02 ellesmere multipassd[1189715]: dnsmasq died: Process returned exit code: 2. Ensure nothing is using port 53.

which points to dnsmasq process, do you happen to have other dnsmasq processes which was not closed properly?

[sjg20]

Thanks for the response.

1. Yes, I started the jammy instance, with 'multipass launch -n jammy 22.04' and it hung. Then I used 'multipass stop jammy' to stop it, then ''multipass start jammy' to try again.

2. Yes, with libvirt:

ellesmere:~$ ps ax |grep dnsmasq
   3441 ?        S      0:00 /snap/multipass/14086/usr/sbin/dnsmasq --keep-in-foreground --strict-order --bind-interfaces --pid-file --domain=multipass --local=/multipass/ --except-interface=lo --interface=mpqemubr0 --listen-address=10.206.166.1 --dhcp-no-override --dhcp-ignore-clid --dhcp-authoritative --dhcp-leasefile=/var/snap/multipass/common/data/multipassd/network/dnsmasq.leases --dhcp-hostsfile=/var/snap/multipass/common/data/multipassd/network/dnsmasq.hosts --dhcp-range 10.206.166.2,10.206.166.254,infinite --conf-file=/var/snap/multipass/common/data/multipassd/network/dnsmasq-ujpdIp.conf
   6110 ?        S      0:00 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
   6112 ?        S      0:00 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
2787470 pts/6    S+     0:00 grep dnsmasq

which is:

$ sudo cat /var/lib/libvirt/dnsmasq/default.conf
##WARNING:  THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
##OVERWRITTEN AND LOST.  Changes to this configuration should be made using:
##    virsh net-edit default
## or other application using the libvirt API.
##
## dnsmasq conf file created by libvirt
strict-order
user=libvirt-dnsmasq
pid-file=/run/libvirt/network/default.pid
except-interface=lo
bind-dynamic
interface=virbr0
dhcp-range=192.168.122.2,192.168.122.254,255.255.255.0
dhcp-no-override
dhcp-authoritative
dhcp-lease-max=253
dhcp-hostsfile=/var/lib/libvirt/dnsmasq/default.hostsfile
addn-hosts=/var/lib/libvirt/dnsmasq/default.addnhosts

Perhaps the 'bind-dynamic' is a problem?

[georgeliao]

@sjg20
I do no think libvirt is the issue. Many users have libvirt with bind-dynamic, and none have encountered this problem. You can explore two possible directions to troubleshoot.:

1. It could be some staled multipass processes that have not shutdown properly. Try removing and reinstalling multipass, or manually terminating the multipass and multipass dnsmasq process. Then, restart everything with sudo snap start multipass.multipassd.
2. If the issue persists, you have to watch the which another process is using the port by using command like sudo lsof -i :53, maybe systemd-resolved is occupying that. If so, temporarily stop the conflicting services and try again.

[sjg20]

After 'snap remove multipass':

 lsof -i :53
COMMAND    PID            USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
systemd-r 2080 systemd-resolve   14u  IPv4  12260      0t0  UDP _localdnsstub:domain 
systemd-r 2080 systemd-resolve   15u  IPv4  12261      0t0  TCP _localdnsstub:domain (LISTEN)
systemd-r 2080 systemd-resolve   16u  IPv4  12262      0t0  UDP _localdnsproxy:domain 
systemd-r 2080 systemd-resolve   17u  IPv4  12263      0t0  TCP _localdnsproxy:domain (LISTEN)
unbound   3244         unbound    3u  IPv6  23153      0t0  UDP ip6-localhost:domain 
unbound   3244         unbound    4u  IPv6  23154      0t0  TCP ip6-localhost:domain (LISTEN)
unbound   3244         unbound    5u  IPv4  23155      0t0  UDP localhost:domain 
unbound   3244         unbound    6u  IPv4  23156      0t0  TCP localhost:domain (LISTEN)
dnsmasq   6674 libvirt-dnsmasq    5u  IPv4  50293      0t0  UDP ellesmere:domain 
dnsmasq   6674 libvirt-dnsmasq    6u  IPv4  50294      0t0  TCP ellesmere:domain (LISTEN)

Then I updated my 24.04 host with 'apt dist-upgrade' and rebooted!

Then install it:

root@ellesmere:/home/sglass# snap install multipass
multipass 1.15.0 from Canonical✓ installed
root@ellesmere:/home/sglass# lsof -i :53
COMMAND     PID            USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
systemd-r  2080 systemd-resolve   14u  IPv4  12260      0t0  UDP _localdnsstub:domain 
systemd-r  2080 systemd-resolve   15u  IPv4  12261      0t0  TCP _localdnsstub:domain (LISTEN)
systemd-r  2080 systemd-resolve   16u  IPv4  12262      0t0  UDP _localdnsproxy:domain 
systemd-r  2080 systemd-resolve   17u  IPv4  12263      0t0  TCP _localdnsproxy:domain (LISTEN)
unbound    3244         unbound    3u  IPv6  23153      0t0  UDP ip6-localhost:domain 
unbound    3244         unbound    4u  IPv6  23154      0t0  TCP ip6-localhost:domain (LISTEN)
unbound    3244         unbound    5u  IPv4  23155      0t0  UDP localhost:domain 
unbound    3244         unbound    6u  IPv4  23156      0t0  TCP localhost:domain (LISTEN)
dnsmasq    6674 libvirt-dnsmasq    5u  IPv4  50293      0t0  UDP ellesmere:domain 
dnsmasq    6674 libvirt-dnsmasq    6u  IPv4  50294      0t0  TCP ellesmere:domain (LISTEN)
dnsmasq   31797          nobody    6u  IPv4 197982      0t0  UDP ellesmere:domain 
dnsmasq   31797          nobody    7u  IPv4 197983      0t0  TCP ellesmere:domain (LISTEN)

$ multipass launch 22.04 -vvv

and it works!

I'm not sure what was wrong.

Thank you for your help!

[sjg20]

Hmm but it seems to be broken on my 24.04 laptop too. I filed an issue in case it helps:

#3917

$ multipass list
list failed: cannot connect to the multipass socket
$ snap remove multipass 

took about 5 minutes

If it is helpful I will keep this laptop in its current state in case you want to login and take a look.

I'm going to remove and install multipass on the laptop....

Hmmm , that fixed it (snap remove multipass, snap install multipass), without any updates or a reboot.

[sjg20]

I am seeing the same thing again today. I just can't get it to be reliable:

sudo snap restart multipass
2025-02-05T10:05:32-07:00 INFO Waiting for "snap.multipass.multipassd.service" to stop.
Restarted.

Then 'multipass start xienial'

and it hangs. I notice that the 'qemu' process starts and runs for a while, then quits. Is there a log file that could help?

[ricab]

Hi @sjg20, what do you mean by "hangs" exactly? Does the daemon reply back if you do multipass version in a different terminal window? What does multipass list say at that point?

The multipass start command waits for the Multipass daemon to let it know what is going on, giving up and existing after a certain timeout (5 min by default, I believe). A successful start transition can take a while (YMMV, but usually around 1 min). If the qemu process quits, the command should also exit shortly after. Can you confirm this?

Looking at your original logs, it looks like the instance was unable to obtain an IP. Without that, Multipass will not declare the start-up complete and will consider the instance to be in an "unknown" status. That dnsmasq process that @georgeliao mentioned is the one that serves DHCP to instances. I see also error binding DHCP socket to device mpqemubr0 sprinkled in your logs... If dnsmasq is unable to fulfil its function for some reason, that will prevent instances from getting an IP and completing the start transition.

However, the qemu-system (not to be confused with qemu-img) process quitting on its own is a different matter entirely. I don't know what could be causing it. Could there be something in the instance deciding to shut it down? Could it be starving for resources and the OS giving up entirely? Could the image be corrupted and causing QEMU to crash? Multipass unfortunately does not capture boot logs today, but you could try to run the same command that it runs and observe what happens.

Please have a look at this troubleshooting that we put together for such cases and let us know what you find.

[sjg20]

OK, here you go.

$ multipass shell xienial
start failed: The following errors occurred:
xienial: timed out waiting for response
ellesmere:~$ 

$ multipass version
multipass   1.15.0
multipassd  1.15.0

I cam happily shell into jammy as it is already running.

QEMU appears to stop after about 15-20 seconds, but the multipass command keeps running for another ~5mins.

The QEMU cmdline is:

[2025-02-05T14:03:34.517] [debug] [qemu-system-x86_64] [1889570] started: qemu-system-x86_64 -bios OVMF.fd --enable-kvm -cpu host -nic tap,ifname=tap-7ce8d9d6c6d,script=no,downscript=no,model=virtio-net-pci,mac=52:54:00:ac:20:4f -device virtio-scsi-pci,id=scsi0 -drive file=/var/snap/multipass/common/data/multipassd/vault/instances/xienial/ubuntu-16.04-server-cloudimg-amd64-uefi1.img,if=none,format=qcow2,discard=unmap,id=hda -device scsi-hd,drive=hda,bus=scsi0.0 -smp 1 -m 1024M -qmp stdio -chardev null,id=char0 -serial chardev:char0 -nographic -cdrom /var/snap/multipass/common/data/multipassd/vault/instances/xienial/cloud-init-config.iso

So when I try that:

Failed to get "write" lock
Is another process using the image [/var/snap/multipass/common/data/multipassd/vault/instances/xienial/ubuntu-16.04-server-cloudimg-amd64-uefi1.img]?

So then I see:

ellesmere:~$ ps ax |grep qemu
   3485 ?        S      0:00 /snap/multipass/14086/usr/sbin/dnsmasq --keep-in-foreground --strict-order --bind-interfaces --pid-file --domain=multipass --local=/multipass/ --except-interface=lo --interface=mpqemubr0 --listen-address=10.128.75.1 --dhcp-no-override --dhcp-ignore-clid --dhcp-authoritative --dhcp-leasefile=/var/snap/multipass/common/data/multipassd/network/dnsmasq.leases --dhcp-hostsfile=/var/snap/multipass/common/data/multipassd/network/dnsmasq.hosts --dhcp-range 10.128.75.2,10.128.75.254,infinite --conf-file=/var/snap/multipass/common/data/multipassd/network/dnsmasq-QzedpH.conf
1889570 ?        Sl     0:20 /snap/multipass/14086/usr/bin/qemu-system-x86_64 -bios OVMF.fd --enable-kvm -cpu host -nic tap,ifname=tap-7ce8d9d6c6d,script=no,downscript=no,model=virtio-net-pci,mac=52:54:00:ac:20:4f -device virtio-scsi-pci,id=scsi0 -drive file=/var/snap/multipass/common/data/multipassd/vault/instances/xienial/ubuntu-16.04-server-cloudimg-amd64-uefi1.img,if=none,format=qcow2,discard=unmap,id=hda -device scsi-hd,drive=hda,bus=scsi0.0 -smp 1 -m 1024M -qmp stdio -chardev null,id=char0 -serial chardev:char0 -nographic -cdrom /var/snap/multipass/common/data/multipassd/vault/instances/xienial/cloud-init-config.iso
1897044 pts/17   S+     0:00 grep qemu

So it is running, just not connecting.

[sjg20]

Yes, I went through the troubleshooting before filing an issue. I'll have another look.

[sjg20]

When I kill -9 the running qemu and start it myself, I see it boot, with a warning:

Then it continues to the xienial prompt. I'm not sure how to login though.

At this point:

$ ps ax |grep qemu
   3485 ?        S      0:00 /snap/multipass/14086/usr/sbin/dnsmasq --keep-in-foreground --strict-order --bind-interfaces --pid-file --domain=multipass --local=/multipass/ --except-interface=lo --interface=mpqemubr0 --listen-address=10.128.75.1 --dhcp-no-override --dhcp-ignore-clid --dhcp-authoritative --dhcp-leasefile=/var/snap/multipass/common/data/multipassd/network/dnsmasq.leases --dhcp-hostsfile=/var/snap/multipass/common/data/multipassd/network/dnsmasq.hosts --dhcp-range 10.128.75.2,10.128.75.254,infinite --conf-file=/var/snap/multipass/common/data/multipassd/network/dnsmasq-QzedpH.conf
1899966 pts/19   Sl+    0:18 qemu-system-x86_64 -bios OVMF.fd --enable-kvm -cpu host -nic tap,ifname=tap-7ce8d9d6c6d,script=no,downscript=no,model=virtio-net-pci,mac=52:54:00:ac:20:4f -device virtio-scsi-pci,id=scsi0 -drive file=/var/snap/multipass/common/data/multipassd/vault/instances/xienial/ubuntu-16.04-server-cloudimg-amd64-uefi1.img,if=none,format=qcow2,discard=unmap,id=hda -device scsi-hd,drive=hda,bus=scsi0.0 -smp 1 -m 1024M -qmp stdio -chardev null,id=char0 -cdrom /var/snap/multipass/common/data/multipassd/vault/instances/xienial/cloud-init-config.iso

[ricab]

Ah OK, I think I know what is going on. Xenial is unsupported:

$ multipass find xenial
No images or blueprints found.
$ multipass find --show-unsupported xenial
Image                       Aliases           Version          Description
xenial                                        20211001         Ubuntu 16.04 LTS
daily:xenial                                  20211001         Ubuntu 16.04 LTS

And a decision was made a while ago to scope such images out of what Multipass is supposed to support. See #3443 and #1068 for reference.

Your title says 22.04 though. Do you still have issues with Jammy instances?