We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 02b2179 commit 57fd6feCopy full SHA for 57fd6fe
README.md
@@ -198,8 +198,6 @@ export TORCHCHAT_ROOT=${PWD}
198
### Export for mobile
199
The following example uses the Llama3 8B Instruct model.
200
201
-[comment default]: echo '{"embedding": {"bitwidth": 4, "groupsize" : 32}, "linear:a8w4dq": {"groupsize" : 32}}' >./config/data/mobile.json
202
-
203
```
204
# Export
205
python3 torchchat.py export llama3 --quantize config/data/mobile.json --output-pte-path llama3.pte
0 commit comments