Skip to content

Files

Latest commit

ggerganovggerganov
llama : refactor llama_context, llama_kv_cache, llm_build_context (gg…
Mar 13, 2025
e0dbec0 · Mar 13, 2025

History

History
This branch is 80 commits behind ggml-org/llama.cpp:master.

embedding

README.md

llama.cpp/example/embedding

This example demonstrates generate high-dimensional embedding vector of a given text with llama.cpp.

Quick Start

To get started right away, run the following command, making sure to use the correct path for the model you have:

Unix-based systems (Linux, macOS, etc.):

./llama-embedding -m ./path/to/model --pooling mean --log-disable -p "Hello World!" 2>/dev/null

Windows:

llama-embedding.exe -m ./path/to/model --pooling mean --log-disable -p "Hello World!" 2>$null

The above command will output space-separated float values.

extra parameters

--embd-normalize i n t e g e r

i n t e g e r description formula
− 1 none
0 max absolute int16 32760 ∗ x i max | x i |
1 taxicab x i ∑ | x i |
2 euclidean (default) x i ∑ x i 2
> 2 p-norm x i ∑ | x i | p p

--embd-output-format ′ s t r i n g ′

′ s t r i n g ′ description
'' same as before (default)
'array' single embeddings [ [ x 1 , . . . , x n ] ]
multiple embeddings [ [ x 1 , . . . , x n ] , [ x 1 , . . . , x n ] , . . . , [ x 1 , . . . , x n ] ]
'json' openai style
'json+' add cosine similarity matrix

--embd-separator " s t r i n g "

" s t r i n g "
"\n" (default)
"<#embSep#>" for exemple
"<#sep#>" other exemple

examples

Unix-based systems (Linux, macOS, etc.):

./llama-embedding -p 'Castle<#sep#>Stronghold<#sep#>Dog<#sep#>Cat' --pooling mean --embd-separator '<#sep#>' --embd-normalize 2  --embd-output-format '' -m './path/to/model.gguf' --n-gpu-layers 99 --log-disable 2>/dev/null

Windows:

llama-embedding.exe -p 'Castle<#sep#>Stronghold<#sep#>Dog<#sep#>Cat' --pooling mean --embd-separator '<#sep#>' --embd-normalize 2  --embd-output-format '' -m './path/to/model.gguf' --n-gpu-layers 99 --log-disable 2>/dev/null