Skip to content

Latest commit

 

History

History
71 lines (53 loc) · 4.21 KB

README.md

File metadata and controls

71 lines (53 loc) · 4.21 KB

InstructLab toolbox container for AMD ROCm GPUs

The ROCm container file is designed for AMD GPUs with RDNA3 architecture (gfx1100). The container can be build for RDNA2 (gfx1030) and older GPUs, too. Please refer to AMD's system requirements for a list of officially supported cards. ROCm is known to work on more consumer GPUs.

The container file creates a toolbox container for toolbox(1) command line tool. A toolbox containers has seamless access to the entire system including user's home directory, networking, hardware, SSH agent, and more.

The container has all Python dependencies installed in a virtual env. The virtual env is already activated when you enter the container.

Quick start

  1. git clone the instructlab and taxonomy project into a common folder in your home directory (e.g. ~/path/to/instructlab)
  2. add your account to render and video group: sudo usermod -a -G render,video $LOGNAME
  3. install build dependency for this container: sudo dnf install toolbox podman make rocminfo
  4. build the container: make rocm or make rocm-gfx1100
  5. create a toolbox make rocm-toolbox
  6. enter toolbox toolbox enter instructlab. The container has your home directory mounted.

To update InstructLab CLI to latest version: pip install -e ~/path/to/instructlab/instructlab

ilab generate and ilab chat use the GPU automatically. ilab train needs more powerful and recent GPU and therefore does not use GPU by default. To train on a GPU, run ilab train --device cuda.

Building for other GPU architectures

Use the amdgpu-arch or rocminfo tool to get the short name

dnf install clang-tools-extra rocminfo
amdgpu-arch
rocminfo | grep gfx

Map the name to a LLVM GPU target and an override GFX version. PyTorch 2.2.1+rocm5.7 provides a limited set of rocBLAS Kernels. Fedora 40's ROCm packages have more Kernels. For now we are limited to what PyTorch binaries provide until Fedora ships python-torch with ROCm support.

Name XNACK/USM Version PyTorch Fedora
gfx900 9.0.0 ✅ ✅
gfx906 xnack- 9.0.6 ✅ ✅
gfx908 xnack- 9.0.8 ✅ ✅
gfx90a xnack- 9.0.10 ✅ ✅
gfx90a xnack+ 9.0.10 ✅ ✅
gfx940 ❌ ✅
gfx941 ❌ ✅
gfx942 ❌ ✅
gfx1010 ❌ ✅
gfx1012 ❌ ✅
gfx1030 10.3.0 ✅ ✅
gfx1100 11.0.0 ✅ ✅
gfx1101 ❌ ✅
gfx1102 ❌ ✅

If your card is not listed or unsupported, try the closest smaller value, e.g. for gfx1031 use target gfx1030 and override 10.3.0. See ROCm/ROCR-Runtime isa.cpp and LLVM User Guide for AMDGPU for more information.

Marketing Name Name Arch Target GFX version Memory Chat Train
AMD Radeon RX 7900 XT gfx1100 RDNA3 gfx1100 11.0.0 20 GiB ✅ ✅
AMD Radeon RX 7900 XTX RDNA3 24 GiB ✅ ✅
AMD Radeon RX 6700 gfx1031 RDNA2 gfx1030 10.3.0 10 GiB ✅ ❌

Build the container with additional build arguments:

make rocm-gfx1100 BUILD_ARGS=

Known issues

AMD Instinct MI210 with ISA amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- is not supported by Fedora build rocblas-6.0.2-3. As of late April 2024, Fedora has gfx90a:xnack+ and gfx90a:xnack- but lacks gfx90a:sramecc+:xnack-.