RamaLama can't discover RX 5700 XT

[Split7fire]

Describe the bug

Installing RamaLama through pip (as official documentation said). And expect it to recognize my GPU (RX 5700 XT).

Yes, it is not officially supported in ROCm but according to user @RealVishy it works in Aurora-dx.

Crossref: ollama/ollama#2503

What did you expect to happen?

I expect RamaLama to recognize my GPU (RX 5700 XT).

Output of bootc status

Staged ostree
         Commit: 3bef95cfdead2f95d3ec2fa651aa0292bc6c9d9986cd092328929c0f7738edf1

● Booted ostree
           Commit: e0ec67f8b42d669fe2716b1c700cece261bef1cba385560e7b92e6f87bd5ebda

  Rollback ostree
           Commit: 37abc3a1eb69f5ecdc1d0db2a339d4d41641b481d9ee2b457aa8e5ce4d5d7fdf

Output of groups

vlad wheel docker lxd incus-admin libvirt

Extra information or context

No response

[castrojo]

What image are you on? The version we're shipping is on the -dx image, and mine pulled the rocm container for my 5700XT and that worked out of the box.

[Split7fire]

Bluefin-dx stable image.

[castrojo]

Can you try with the one on the image? /usr/bin/ramalama

[Split7fire]

@castrojo I tried:

❯ /usr/bin/ramalama run llama3.2
Loading model                                                                                                                                                               
~ took 5s 

My system info:

# System Details Report
---

## Report details
- **Date generated:**                              2025-02-11 20:41:29

## Hardware Information:
- **Hardware Model:**                              HUANANZHI X79-ZD3
- **Memory:**                                      64.0 GiB
- **Processor:**                                   Intel® Xeon® E5-1650 0 × 12
- **Graphics:**                                    AMD Radeon™ RX 5700 XT
- **Disk Capacity:**                               1.0 TB

## Software Information:
- **Firmware Version:**                            4.6.5
- **OS Name:**                                     Bluefin (Version: 41.20250209.1 / FROM Fedora Silverblue 41)
- **OS Build:**                                    dd783ec
- **OS Type:**                                     64-bit
- **GNOME Version:**                               47
- **Windowing System:**                            Wayland
- **Kernel Version:**                              Linux 6.12.9-200.fc41.x86_64

[Split7fire]

Also, I couldn't imagine that ramalama is installed. Where I can see a list of installed non-standard apps?

[castrojo]

https://github.com/ublue-os/bluefin/blob/main/packages.json

DX section

[RealVishy]

I tried:

❯ /usr/bin/ramalama run llama3.2
Loading model                                                                                                                                                               
~ took 5s 

Hi, can you run it with the --debug flag?
Like in this issue containers/ramalama#638

Additionally, try removing the podman image it pulled and try again, maybe there was an issue when the container was pulled.

Also, I recommend making an issue in the upstream repo, they might know how to debug it better.

[Split7fire]

I tried:

❯ /usr/bin/ramalama run llama3.2
Loading model                                                                                                                                                               
~ took 5s 

Hi, can you run it with the --debug flag? Like in this issue containers/ramalama#638

Additionally, try removing the podman image it pulled and try again, maybe there was an issue when the container was pulled.

Also, I recommend making an issue in the upstream repo, they might know how to debug it better.

Sure:

❯ /usr/bin/ramalama --debug run llama3.2
run_cmd:  podman inspect quay.io/ramalama/rocm:0.5
Working directory: None
Ignore stderr: False
Ignore all: True
exec_cmd:  podman run --rm -i --label RAMALAMA --security-opt=label=disable --name ramalama_KvKhPCcWPu --pull=newer -t --device /dev/dri --device /dev/kfd -e HIP_VISIBLE_DEVICES=0 --mount=type=bind,src=/var/home/vlad/.local/share/ramalama/models/ollama/llama3.2:latest,destination=/mnt/models/model.file,ro quay.io/ramalama/rocm:latest llama-run -c 2048 --temp 0.8 -v /mnt/models/model.file
Loading modelggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no                                                                                                                     
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
  Device 0: AMD Radeon RX 5700 XT, gfx1010:xnack- (0x1010), VMM: no, Wave Size: 32

~ took 4s 

-e HIP_VISIBLE_DEVICES=0 seems odd

[Split7fire]

Also, I recommend making an issue in the upstream repo, they might know how to debug it better.

I'm using 0.5.2 from Bluefin. Their most probable advice will be to update to 0.5.5 at least.

[castrojo]

Yeah we're working on getting updated versions into Bluefin asap.