Skip to content

Commit 6be901c

Browse files
danbevtybalex
authored andcommitted
infill : add download instructions for model (ggml-org#6626)
* infill : add download instructions for model This commit adds instructions on how to download a CodeLlama model using the `hf.sh` script. This will download the model and place it in the `models` directory which is the same model use later by the infill example. Signed-off-by: Daniel Bevenius <[email protected]> * squash! infill : add download instructions for model Clarify the reason for using CodeLlama. Signed-off-by: Daniel Bevenius <[email protected]> --------- Signed-off-by: Daniel Bevenius <[email protected]>
1 parent 2d8f549 commit 6be901c

File tree

1 file changed

+5
-0
lines changed

1 file changed

+5
-0
lines changed

examples/infill/README.md

+5
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,11 @@ The `infill` program offers a seamless way to interact with LLaMA models, allowi
3636

3737
### Example
3838

39+
Download a model that supports infill, for example CodeLlama:
40+
```console
41+
scripts/hf.sh --repo TheBloke/CodeLlama-13B-GGUF --file codellama-13b.Q5_K_S.gguf --outdir models
42+
```
43+
3944
```bash
4045
./infill -t 10 -ngl 0 -m models/codellama-13b.Q5_K_S.gguf -c 4096 --temp 0.7 --repeat_penalty 1.1 -n 20 --in-prefix "def helloworld():\n print(\"hell" --in-suffix "\n print(\"goodbye world\")\n "
4146
```

0 commit comments

Comments
 (0)