Skip to content

Commit

Permalink
add ET Runner instructions (#691)
Browse files Browse the repository at this point in the history
Co-authored-by: Michael Gschwind <[email protected]>
  • Loading branch information
2 people authored and malfet committed Jul 17, 2024
1 parent b597373 commit 6dec867
Showing 1 changed file with 16 additions and 0 deletions.
16 changes: 16 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,22 @@ export TORCHCHAT_ROOT=${PWD}
./scripts/install_et.sh
```

### Test it out using our Executorch runner
Build the runner
```bash
scripts/build_native.sh et
```

**Get a PTE file if you don't have one already**
```
python3 torchchat.py export llama3 --quantize config/data/mobile.json --output-pte-path llama3.pte
```

Execute using the runner
```bash
cmake-out/et_run llama3.pte -z ~/.torchchat/model-cache/meta-llama/Meta-Llama-3-8B-Instruct/tokenizer.model -i "Once upon a time"
```

### Export for mobile
The following example uses the Llama3 8B Instruct model.

Expand Down

0 comments on commit 6dec867

Please sign in to comment.