How to prefix the data generation

[lfoppiano]

I've got a question about generation using a prefix, meaning providing the first words to the LLM. Is it possible?

So for example, the prompt in plain text is:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are an expert on .... blablabla.<|eot_id|><|start_header_id|>user<|end_header_id|>
Usser prompt balblabla

Question: {question}<|eot_id|><|start_header_id|>assistant<|end_header_id|>Explanation: 

So we prefix the assistant: "Explanation: " to be the next token to provide a first token

I'm using llama-server via curl / openai python library

[foldl]

Have a check on 'user_suffix'?

[lfoppiano]

Sorry, I could not find it, is it documented somewhere?

[foldl]

Sorry. It is --in-suffix.

I have implemented this feature in chatllm.cpp. I have tried to make a PR to llama.cpp on this, but found --in-suffix which seemed to be something alike. Correct me if I am wrong.

[lfoppiano]

I cannot find the option in llama-server, are you sure is there? Can you please link it in the documentation?

[foldl]

https://github.com/ggerganov/llama.cpp/blob/4ddd199f6f6b980e0a7ed9f9b44efeae2fbdf5c4/examples/main/README.md?plain=1#L115