Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more transformer test models #42

Merged
merged 1 commit into from
Jul 12, 2024
Merged

Conversation

kevinwuTT
Copy link
Contributor

Added the following test models:

  • bloom
  • falcon
  • gpt2
  • llama
  • yolos

Updated README to show the current status of the models

Model Run Success Torch Ops (Before) Torch Ops (Remain) To/From_Device Ops Original Run Time (s) Compiled Run Time(s) Accuracy
Mnist (Eval) Yes 14 5 36 0.0113227 2.2285212059505284 0.9871560472189823
Mnist (Train) Yes 14 7 42 0.0120502 0.3592461119405925 0.7461481019967987
ResNet18 Yes 70 42 132 1.82999 8.746782089932822 0.9999111054719786
Bloom No N/A N/A N/A 5.5785 N/A N/A
YOLOS No N/A N/A N/A 0.27139 N/A N/A
Llama No 3 2 6 38.4944 N/A N/A
BERT Yes 1393 539 3862 61.9702 36.7952290129615 0.9864120722326671
Falcon No 3 2 6 34.9667 N/A N/A
GPT-2 No N/A N/A N/A 1.07637 N/A N/A

@kevinwuTT kevinwuTT requested a review from ayerofieiev-tt July 12, 2024 22:01
@ayerofieiev-tt ayerofieiev-tt merged commit c30b897 into main Jul 12, 2024
1 check passed
@ayerofieiev-tt ayerofieiev-tt deleted the kw/more_transformer_models branch July 12, 2024 22:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants