I need help to quantize video-llava model to quant size 4 or 8

[dnabanita7]

I am trying to convert video-llava according to instructions at #2948 but I am having issues because of convert.py file. Since, this is llava + vicuna should I add anything extra in the convert.py file or let it remain the same?

[Abusheik008]

you can use bitsandbytes to convert the model, i assume it is an easiest way load the model from transformer