Skip to content

Commit

Permalink
Make shortfin server heading for more visible doc links,
Browse files Browse the repository at this point in the history
Add iree-base-runtime, iree-base-compiler and iree-turbine to nightly install instructions
  • Loading branch information
stbaione committed Dec 18, 2024
1 parent a32a2eb commit bfe1058
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 0 deletions.
5 changes: 5 additions & 0 deletions docs/shortfin/llm/user/e2e_llama8b_mi300x.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,11 @@ To install nightly packages:
```bash
pip install shark-ai[apps] sharktank \
--pre --find-links https://github.com/nod-ai/shark-ai/releases/expanded_assets/dev-wheels
pip install -f https://iree.dev/pip-release-links.html --pre --upgrade \
iree-base-compiler \
iree-base-runtime \
iree-turbine \
"numpy<2.0"
```

See also the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,9 @@ For this tutorial, you will need to meet the following prerequisites:
- You can check out [pyenv](https://github.com/pyenv/pyenv)
as a good tool to be able to manage multiple versions of python
on the same system.

### Shortfin LLM Server

- A running `shortfin` LLM server. Directions on launching the llm server on one system can be found [here](https://github.com/nod-ai/shark-ai/blob/main/docs/shortfin/llm/user/e2e_llama8b_mi300x.md) and for launching
on a kubernetes cluster, please look [here](https://github.com/nod-ai/shark-ai/blob/main/docs/shortfin/llm/user/e2e_llama8b_k8s.md)
- We will use the shortfin server as the `backend` to generate completions
Expand Down

0 comments on commit bfe1058

Please sign in to comment.