New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

推理时间问题 #36

Open

128veg80 opened this issue Feb 12, 2025 · 1 comment

128veg80 commented Feb 12, 2025

我在v100显卡对CogAgent-9B-20241220进行本地部署，单步推理的时间在8s左右。请问这个时长是否正常，亦或是我推理方式有问题？

leeaction commented Feb 12, 2025

啊？你在V100上有这么快的速度吗？我在v100上单步操作会用到40s左右你是量化过吗？

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment