How to train this model

[CodingGreatEmperor]

Due diligence

• I have done my due diligence in trying to find the answer myself.

Topic

The paper

Question

For mini model, moshi model, and tokenizer. I just wanna customize a little bit of model params of them

[yukiarimo]

+1

[bruno-hays]

https://github.com/kyutai-labs/moshi/blob/main/FAQ.md

We will release some training / fine-tuning code, but we do not have any timeline yet. Please be patient.

We don't have any other information or timeline at the moment, AFAIK

[Airoura]

It's too hard to implement llama version of moshi from scratch, need to work together.

https://github.com/Airoura/LlamaMoshi

[2010b9]

Hi! 🙂

I have some questions (maybe they don't make total sense – sorry if that's the case! – but I'm still learning about this):

• Will the fine-tuning / training code make it possible for the model to "speak" in other languages? And to perform tool calling? If so, do you have any idea how many training examples would be needed? And for how long and in which hardware should I train the model?

These are two requirements I have for my use case. Maybe there's more that I don't know about, but these seem the two most important ones – speak in another language and perform tool calling.

Thanks in advance 🙂

[davidbrowne17]

https://github.com/yangdongchao/RSTnet this repo provides code to finetune Moshi, I have used it (with some modifications) to finetune moshi.

[yukiarimo]

Would love to have a Colab of that