Skip to content

Commit

Permalink
doc: Update llamaedge app docs
Browse files Browse the repository at this point in the history
Signed-off-by: Zhang Tianyang <[email protected]>
  • Loading branch information
Burning1020 committed Aug 19, 2024
1 parent 9ecee15 commit 569e10c
Showing 1 changed file with 12 additions and 1 deletion.
13 changes: 12 additions & 1 deletion docs/wasm/How-to-run-Llama-3-8B-with-Kubernetes.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ which introducing how to create an OpenAI-compatible API service for Llama-3-8B.
+ Install WasmEdge and plugins:
`curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install.sh | bash -s -- -v 0.13.5 --plugins wasi_logging wasi_nn-ggml`


### 1. Build docker image

We already have an example docker image on dockerhub: `docker.io/kuasario/llama-api-server:v1`.
Expand Down Expand Up @@ -50,6 +49,18 @@ CMD ["llama-api-server.wasm", "--prompt-template", "llama-3-chat", "--ctx-size",
```
Build it with `docker build -t docker.io/kuasario/llama-api-server:v1 .`

Otherwise, you can use this DOCKERFILE if you are debugging while developing:
```dockerfile
FROM ubuntu:22.04
RUN apt-get update && apt-get install -y curl
RUN curl -LO https://huggingface.co/second-state/Llama-3-8B-Instruct-GGUF/resolve/main/Meta-Llama-3-8B-Instruct-Q5_K_M.gguf
RUN curl -LO https://github.com/second-state/chatbot-ui/releases/latest/download/chatbot-ui.tar.gz; tar xzf chatbot-ui.tar.gz; rm chatbot-ui.tar.gz
COPY *.wasm .
```

This would download all prerequisites to docker image cache, allowing you pay attention only on development.
`docker build -f Dockerfile -t docker.io/kuasario/llama-api-server:v2 $path_to_app`

### 2. Build and run Kuasar Wasm Sandboxer

```bash
Expand Down

0 comments on commit 569e10c

Please sign in to comment.