Model inferencing from the saved files generated by train.py

[rahulanilal]

I have prepared a custom annotated dataset to fine tune DONUT, specifically for Invoice Data Extraction. The train.py file works just fine, and provides good results on the validation dataset which is part of the training script. But I am not able to get the fine-tuned model back from the saved files.

The errors I have encountered include 'SwinTransformer' object has no attribute 'pos_drop', 'The checkpoint you are trying to load has model type donut but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.', and some key mismatch issues. I have also tried cloning the entire repo and running the train and test commands as provided by them.

I have also seen that it is officially posted on the GitHub that there are issues in configuring the testing environments for DONUT, have they been resolved yet? Does my problem come under that as well?

PFA the files that are generated by the train.py

[david0ql]

did you solved it?

[fmr-itam]

Hope you found the solution already, but if not:

About the issue you mentioned with SwinTransformer, it is solved in some other closed/opened issues. It has to be with the dependencies versions. In my case, this one got solved using:

!pip install transformers==4.38.2
!pip install timm==0.6.13

Hope it helps a bit!

[david0ql]

don't use donut, it's to hard to keep all in mind and also very specific versions. i found there is a better alternative to be using unsloth