Support for Janus-Pro series of model

[Ankur-singh]

Any thoughts on supporting Janus Series VLMs?

Official Git Repo: https://github.com/deepseek-ai/Janus

[felipemello1]

Hey @Ankur-singh , thanks for bringing it up! Janus look nice, but It is a bit hard for us to onboard new models at the moment. We have some members on PTO and there is some work we have to prioritize first. I see that you have been helping on a lot of issues lately. In case you are curious, I listed some of these below, in no order of importance:

• Instantiating tokenizers using tokenizer config, instead of creating a new torchtune tokenizer per model -- this will make onboarding models easier
• Fast unit testing with increased coverage (permutations of quantization, performance flags, distributed, lora, etc) -- this will make onboarding models easier
• Cleaner / moduralized recipes
• Fast inference
• Efficient RL // parallel faster inference (instead of train -> inference -> train)

[felipemello1]

Added a bit more detail to the response :)