Will a model's own parameter values overwrite the package's default?

[yuanf8]

Hi, may I check if a model's own parameter values will overwrite the package's default?

For example, Llama-2 has a default max_length of 4096, but LLaMA-Factory's max_length defaults to 512. If I don't supply max_length when fine-tuning Llama-2, which value will max_length take? 4096 or 512?

[hiyouga]

the model's own parameter cannot overwrite the default one in the package. In fine-tuning, we use the cutoff_len parameter to control the max input length, you need to supply a new value if the default one is not wanted

[yuanf8]

Thanks for the reply!

Just to clarify, cutoff_len is only for training/fine-tuning and max_length (or max_new_tokens) is only for inference/prediction?

[hiyouga]

exactly