keep_tokens?

[caniyabanci76]

can anyone tell me what keep_tokens does exactly?

[DKnight54]

The number of tokens in your captions to not drop or shuffle. Does nothing if you don't activate shuffle_caption or caption_tag_dropout_rate.
Assuming you activate either one of the options, keep_tokens will indicate the number of tags (from the front) that will be skipped when shuffling tags or dropping tags. Useful if you have an activation keyword. Only useful if you are doing tag style training or have multiple captions. If only one caption, not very useful.

Examples:
Let's say you are training a dataset on dogs with the activation keyword 'sks_canine' and have caption files with the following format,
sks_canine, dog, dog_atrib1, dog_atrib2, dog_atrib3, dog_atrib4, dog_atrib5

You might want to set keep_tokens to 2 so that the activation keyword and the class identifier('dog') do not get shuffled or dropped during training.

As to why you might want to activate shuffling and dropping of keywords, it's supposed to train flexibility into the model/LoRA.