Skip to content

Previous Ollama Build Steps

Jump to bottom
likelovewant edited this page Feb 10, 2025 · 1 revision

Welcome to the ollama-for-amd wiki!

This wiki aims to extend support for AMD GPUs that Ollama Official doesn't currently cover due to limitations in official ROCm on Windows. The goal is to remove these GPU limitations and include support for more AMD graphics card models. Remember, this help is provided voluntarily by community.

Important

"This repository is essentially a fork of ollama, primarily modified the gpu lists to work with more AMD GPUs. It doesn't introduce any new features beyond that."

(For those who prefer another language, right-click and use the auto-translate feature in your browser or device. Most platforms have this built-in!.)

Need Help?

issues? Please share your server logs in the issue section .also check out the official ollama issue tracker: (ollama/issues) for more in-depth assistance or get help from similiar issue.If you suspect the issue originates from the Ollama program itself, please hold off on further troubleshooting for now. ollama teams are working on updates to address this problem. Keep an eye out for a new release!"

Tip

Compilation Not Required? Try the Demo!

If you're not comfortable compiling or build the software yourself, no problem! You can skip the development steps and jump straight to using the Demo Release Version.

To use the demo, simply replace the rocblas files in the Rocblas Support Section with those provided in the demo release.

Development Guide

（"This guide outlines how the repository builds the program. You don't need to follow these steps unless you're interested in customizing the build process. One benefit of following these steps is that the compiled program may take up less space, as it will only be optimized for your specific GPU architecture instead of a wide range of architectures.").

Follow the guide on development.md for Windows development setup.

  • Important( only for version before v0.4.0): Make sure you have Strawberry Perl installed. If you encounter a build error like "ninja: build stopped: subcommand failed", try moving the Strawberry Perl environment path before the MinGW-w64 path.（ Thanks ByronLeeeee for this tips)

Downloading & Editing

  1. Clone the Ollama Repository: 
    git clone https://github.com/ollama/ollama.git

or simpley clone this repo.

  1. For version before v0.4.0 Edit the gen_windows.ps1 File:

For version v0.4.0 and after Edit the Makefile.rocm File:

  • Locate this file: ollama/make/Makefile.rocm

  • Add your GPU model to the existing list at line 8. Here's an example of the official supported list:

HIP_ARCHS_COMMON := gfx900 gfx940 gfx941 gfx942 gfx1010 gfx1012 gfx1030 gfx1100 gfx1101 gfx1102

"gfx803"(rocm5),"gfx900:xnack-"(rocm5), "gfx902" (rocm5),"gfx90c:xnack-"(rocm5,only available at self-build),//"gfx906:xnack-","gfx1010:xnack-", "gfx1011", "gfx1012:xnack-", "gfx1031", "gfx1032", "gfx1034", "gfx1035", "gfx1036","gfx1103", "gfx1150"

Tip

To optimize builds and reduce file size, focus on including only your target architectures in support lists. Removing unnecessary entries can significantly improve build speed and final file size. For instance, if you only support gfx1032, keeping it as the sole entry HIP_ARCHS_COMMON := gfx1032 will result in a lean build focused solely on that architecture.

ROCm Support

  • Ensure you have a compatible Hip SDK installed. Install HIP SDK 6.1.2/HIP SDK 5.7.1 from hip-sdk.

  • Important( this is optional due to the update of upstream ,simply skip this steps unless your build won't works for your old arches ( eg, gfx803 ,gfx90c,...): Currently, HIP SDK 5.7 is out of supported. can be enabled by change the amdhip64_6.dll to amdhip64.dll in line 38 amd_hip_windows.go and line 27,change the C:\\Program Files\\AMD\\ROCm\\6.1\\bin" to C:\\Program Files\\AMD\\ROCm\\5.7\\bin" in line 26 /discover/amd_windows.go

  • Enable HSA_OVERRIDE_GFX_VERSION edits，optional for some undetected device in self-build, eg, gfx90c or gfx90c:xnack-: ,

Official AMD ROCm Supported GPUs: You can find a complete list of officially supported GPUs here: rocm.docs.amd.com

For supported GPUs, please use the official ROCm version from AMD also the Ollama Official. You don't need to build anything unless you want to learn how. (Examples:gfx900 gfx940 gfx941 gfx942 gfx1010 gfx1012 gfx1030 gfx1100 gfx1101 gfx1102)

If your GPU is not on the list:

  • Try pre-built ROCm libraries:
    • Some pre-built rocblas libraries are available for specific GPUs at:

ROCmLibs for HIP SDK 6.1.2

ROCmLibs for HIP SDK 5.7

  • Choose the appropriate library based on your GPU model (e.g., gfx902, gfx1032).

Building ROCm libraries:

Installing and Replacing Libraries:

  1. Install HIP SDK: This will create a C:\Program Files\AMD\ROCm\6.1\bin folder.
  2. Place rocblas.dll in the C:\Program Files\AMD\ROCm\6.1\bin folder. Replace any existing rocblas.dll.
  3. Replace library within rocblas\library: This ensures your code uses the correct library files.
  4. Add bin in the PATH: eg , C:\Program Files\AMD\ROCm\6.1\bin\ to Path( systeme envrimental , PATH , most people forgot this)

(change the path to 5.7 if you want to work on HIP 5.7 )

Community Support:

  • If you encounter issues or need assistance in rocblas and library, please leave a message on the project's issues page for community support.

Building the ollama.exe:**

make -j 9

Throughout the examples below -j 9 is suggested for 9 parallel jobs to speed up the build. You can adjust the job count based on your CPU Core count to reduce build times.

After build.

you can start the server in your Ollama directory:

./ollama serve

Finally, in a separate shell, run a model(eg):

./ollama run llama3

Building the Installer:**

  • Install Inno Setup: To build an installer for Ollama, you'll need to install Inno Setup: https://jrsoftware.org/isinfo.php.
  • Run PowerShell Script: In the top directory of the Ollama repository, run this PowerShell script: 
    powershell -ExecutionPolicy Bypass -File .\scripts\build_windows.ps1
  • The Installer: After the build is complete, you'll find the OllamaSetup.exe installer in the dist folder. This will work exactly like the official release.

Demo Release Version:

  • Test Drive Ollama: If you want to test before building from source, download and install a demo release from ollama-for-amd/releases.
  • Replace Files: As instructed in the ROCm supprot, replace files in your Ollama program folder with the rocblas.dll and rocblas/library folder from the demo release that matches your GPU architecture.(eg. the file in （`C:\Users\usrname\AppData\Local\Programs\Ollama\lib\ollama)

This repository serves as an example, and won't be regularly updated.

Try pre-built ROCm libraries:

  • Some pre-built rocblas libraries are available for specific GPUs at:

ROCmLibs for HIP SDK 6.1.2

ROCmLibs for HIP SDK 5.7

  • Choose the appropriate library based on your GPU model (e.g., gfx902, gfx1032).

Multi gpu Notes: if your has multi gpu and the arches not on the support lists . please consider build your rocmlibs and olloma as the guide in development-guide .

Note

gfx1103 (AMD 780M) is natively supported by this repository, so no file replacements are needed.

gfx803 hip memory error fixed by this rocm 5.7 libsrocm.gfx803.optic.test.version.7z for rocm5.7 in rocm 5.7 ,received feedback for gfx900 series ( gfx803;gfx900:xnack,gfx90c:xnack-...) can not detacted in Hip SDK 6.1.2 and forward . please consider build ollama with rocm5.7 yourself if there is no ollama-windows-amd64-rocm5.7.7z package available at release page .the ROCmlibs for 5.7 available at HIP SDK 5.7

Note

"The HSA_OVERRIDE_GFX_VERSION approach no longer works.Optional by force enable it in self-build gfx90c:xnack-, gfx1010:xnack-, gfx1012：xnack- and Similar Architectures:

  • Start by trying the pre-built rocblas and library files for your GPU. *If you encounter issues, try setting the environment variable HSA_OVERRIDE_GFX_VERSION=9.0.12（for gfx90c），HSA_OVERRIDE_GFX_VERSION=10.1.2（ gfx1012） . Refer to the Windows documentation for guidance on setting environment variables: https://www.computerhope.com/issues/ch000549.htm.

  • Running Your Models: After setting up ROCm and Ollama, run your models in a separate shell using the command: ./ollama run [model name], replacing "[model name]" with the actual name of your model.

and start the server in C:\Users\usrname\AppData\Local\Programs\Ollama\:

./ollama serve

Finally, in a separate shell, run a model(eg):

./ollama run llama3

  • Update Carefully: If you are using a demo release, DO NOT click the "Update" button if Ollama prompts you. Download updates manually from this release page: ollama-for-amd/releases

  • or update via Ollama-For-AMD-Installer created by ByronLeeeee, a simply click will do the update and replace libs process .

Trouble shootings on Windows:

error 0xc0000005

  1. make sure get the latest drivers.and replace the files match your gpu.
  2. some proven solutions as shows in 8#,14#,18#,by mannuly move the all the .dll files in Ollama\lib\ollama and rocblas folder into Ollama\lib\ollama\runners\rocm_v6.1.

No GPU detected

evel=INFO source=gpu.go:386 msg="no compatible GPUs were discovered

install HIPSDK , add hipinfo in path , ie, rocm/bin in the path,refer to hipinfo#, that's usually happens in ROCm5.7 package or Release.

Hide Nvida GPU detected

CUDA_VISIBLE_DEVICES=-1,0,1,2... can be get info by run nvidia-smi -L

Hide AMD GPU detected

HIP_VISIBLE_DEVICES=-1,0,1,2... can be get info by run hipinfo

Multiple AMD GPU OVERRIDE Have mulitiple OVERRIDE_GFX_VERSION can add extra tag 0,1 behidn version

HSA_OVERRIDE_GFX_VERSION_0=10.3.0 HSA_OVERRIDE_GFX_VERSION_1=11.0.0

Troubleshooting AMD GPU Support in Linux:

For users who prefer a self-build approach or use ollama official release on unsupported arches, setting environment variables like HSA_OVERRIDE_GFX_VERSION=9.0.0 or 9.0.6, 10.1.0, 10.3.0, 11.0.0 can enable GPU support on Linux systems. Configuring Environment Variables

Setting on Linux For Linux, set variables using systemd: 1, Edit your systemd service:

sudo systemctl edit ollama.service

2, Add the following line under the [Service]

Environment="HSA_OVERRIDE_GFX_VERSION=10.3.0"

Have mulitiple GPUs ,can add extra tag 0,1 behidn version

HSA_OVERRIDE_GFX_VERSION_0=10.3.0 HSA_OVERRIDE_GFX_VERSION_1=11.0.0

3, Reload the service and restart:

$ sudo systemctl daemon-reload
$ sudo systemctl restart ollama.service

However, some users may encounter issues such as:

  • "amdgpu too old gfx%d%x%x"
  • "unsupported Radeon iGPU detected skipping"
  • "unable to verify rocm library: %w", err)

These errors due to incompatibility between the installed AMDGPU drivers and rocm supports issue. To potentially resolve these issues, consider modifying the discover/amd_linux.go file within your ollama directory if the HSA_OVERRIDE_GFX_VERSIONnot works for you.

Caution: Editing this file can have unintended consequences. It's recommended to back up the original file before making any changes.

Specifically, focus on lines 295 to 320 and either delete or edit the relevant content. The exact modifications required will depend on your specific hardware and driver version. Refer to online forums and documentation for guidance on tailoring these edits for your situation.

Clone this wiki locally