Skip to content

Commit

Permalink
Remove pip install sentencepiece,
Browse files Browse the repository at this point in the history
Update `stable` and `nightly` installation instructs,
Only set lower-bound for `gguf`
  • Loading branch information
stbaione committed Dec 12, 2024
1 parent d0af5c0 commit 011f273
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 22 deletions.
31 changes: 10 additions & 21 deletions docs/shortfin/llm/user/e2e_llama8b_mi300x.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,38 +22,27 @@ python -m venv --prompt shark-ai .venv
source .venv/bin/activate
```

### Install `shortfin`
## Install stable shark-ai packages

You can install either the `latest stable` version of shortfin by installing
`shark-ai` or the `nightly` version directly:

#### Stable
<!-- TODO: Add `sharktank` to `shark-ai` meta package -->

```bash
pip install shark-ai[apps]
pip install shark-ai[apps] sharktank
```

#### Nightly

```bash
pip install shortfin[apps] --pre -f https://github.com/nod-ai/shark-ai/releases/expanded_assets/dev-wheels
```
### Nightly packages

<!-- TODO: Remove when sharktank added to `shark-ai[apps]` -->
### Install `sharktank`
To install nightly packages:

Install the `nightly` version of sharktank:
<!-- TODO: Add `sharktank` to `shark-ai` meta package -->

```bash
pip install sharktank --pre -f https://github.com/nod-ai/shark-ai/releases/expanded_assets/dev-wheels
pip install shark-ai[apps] sharktank \
--pre --find-links https://github.com/nod-ai/shark-ai/releases/expanded_assets/dev-wheels
```

<!-- TODO: Remove once `sentencepiece` added to nightly `sharktank` -->
### Install `sentencepiece`

```bash
pip install sentencepiece
```
See also the
[instructions here](https://github.com/nod-ai/shark-ai/blob/main/docs/nightly_releases.md).

### Define a directory for export files

Expand Down
2 changes: 1 addition & 1 deletion sharktank/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
iree-turbine

# Runtime deps.
gguf==0.11.0
gguf>=0.11.0
numpy<2.0

# Model deps.
Expand Down

0 comments on commit 011f273

Please sign in to comment.