How to create your own LoRA adapter?

[maddes8cht]

I like the possibility of using LORAs. But how can I create my own LoRAs for use in Llamacpp?
With which software?
It is easy to find appropriate instructions for StableDiffusion, but for Llamacpp or LLMs it is much harder.
Is there somewhere suitable resources for beginners in the matter?

It would be nice if an ecosystem of LoRAs for LLMs would emerge, as has happened with StableDiffusion. I do not see that yet. (Do I miss something?)

[slaren]

You can use https://github.com/oobabooga/text-generation-webui
Check https://github.com/oobabooga/text-generation-webui/blob/main/docs/Training-LoRAs.md for more details.

[x4080]

I tried using 4bit quantification training and produce example lora, then how to use it with llama cpp ?
output files are

adapter_config.json
adapter_model.bin

thanks

[KerfuffleV2]

Looks like LoRAs still use GGML format.

[Green-Sky]

oh hmm, #2632 allows you to produce GGUF loras. I think we need to update the convert script or something.

[KerfuffleV2]

You'd need to update both the convert script and the LoRA loader in llama.cpp. I don't think there's really a user-visible benefit at the moment but it would be good from a code cleanup perspective. The existing LoRA loader special cases loading GGML and doesn't even check that the file magic is valid.

[Green-Sky]

and the LoRA loader in llama.cpp. I don't think there's really a user-visible benefit at the moment but it would be good from a code cleanup perspective. The existing LoRA loader special cases loading GGML and doesn't even check that the file magic is valid.

checkout #2632 :), it produces and loads GGUF loras

[x4080]

@Green-Sky Cool!

[earonesty]

one day someone will come along and add a lora training command to llama cpp....

[Green-Sky]

see #2632