Low loading speed on linux

[andutei]

Hi all. Sorry if this is the wrong forum to post this. I am getting low loading speed on ubuntu, about 2.4GB/s with one core at 100% usage during loading. I have two Lexar NQ790 2TB SSDs configured as software RAID0. I am expecting >7GB/s (they are connected to PCIe 3.0 instead of 4.0 at the moment). I've tried with and without mmap, build from source and releases, same result.
fio testing gives same speed, but if I use --direct=1 I get the expected result, >7GB/s with very low CPU usage:
sudo fio --name=read --rw=read --bs=128M --size=8G --filename=/dev/md0 --numjobs=1 --direct=1
I've tried to profile dd without iflag=direct to see what hogs the CPU:
sudo perf record -g -- dd if=/dev/md0 of=/dev/null bs=128M count=128
_copy_to_iter: 22%, clear_page_erms: 9%, xa_load: 5%
I've tried fio and dd on the devices themselves /dev/nvme0n1 and /dev/nvme1n1. I get the same 2.4GB/s (as md0) without O_DIRECT and 3.5GB/s with. fio with more threads increases the speed without direct. So reads without O_DIRECT are CPU bound.
Elevator is set to [none] for both nvme, nr_requests is 1023. ubuntu is freshly installed and up to date.
The core that goes to 100% boosts to 3.7GHz.
Also during the model loading with llama.cpp I see 4.8GB/s in htop, so 2.4GB/s actual speed, htop just adds all block devices speed, doesn't account for RAID.
My system is: single socket epyc 77V13 zen3 2.45 GHz 64-core, 512GB DDR4-2400 in 8 channel mode (8 x 64GB RDIMMs). I've tested the memory bandwidth with:
sysbench memory --memory-block-size=1G --memory-total-size=256G --memory-oper=write --threads=64 run
I get 150-160GB/s.
Inference is running fine, I get about 4-5 tok/s with R1 Q4 at the beginning with low context. So I don't think anything is wrong with the hardware.
Any pointers much appreciated.