You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
1 - Fix an extraneous skip end that is out of order with a skip begin.
2 - fix some typos
PS: This might cause some README tests to fail, as they have not been run in a long time.
<summary>Additional Model Inventory Management Commands</summary>
172
172
173
173
### Where
174
-
This subcommand shows location of a particular model.
174
+
This subcommand shows the location of a particular model.
175
175
```bash
176
176
python3 torchchat.py where llama3.1
177
177
```
@@ -216,7 +216,6 @@ This mode generates text based on an input prompt.
216
216
python3 torchchat.py generate llama3.1 --prompt "write me a story about a boy and his bear"
217
217
```
218
218
219
-
[skip default]: end
220
219
221
220
### Server
222
221
This mode exposes a REST API for interacting with a model.
@@ -286,14 +285,16 @@ First, follow the steps in the Server section above to start a local server. The
286
285
streamlit run torchchat/usages/browser.py
287
286
```
288
287
288
+
[skip default]: end
289
+
289
290
Use the "Max Response Tokens" slider to limit the maximum number of tokens generated by the model for each response. Click the "Reset Chat" button to remove the message history and start a fresh chat.
290
291
291
292
292
293
## Desktop/Server Execution
293
294
294
295
### AOTI (AOT Inductor)
295
296
[AOTI](https://pytorch.org/blog/pytorch2-2/) compiles models before execution for faster inference. The process creates a [DSO](https://en.wikipedia.org/wiki/Shared_library) model (represented by a file with extension `.so`)
296
-
that is then loaded for inference. This can be done with both Python and C++ enviroments.
297
+
that is then loaded for inference. This can be done with both Python and C++ environments.
297
298
298
299
The following example exports and executes the Llama3.1 8B Instruct
299
300
model. The first command compiles and performs the actual export.
0 commit comments