Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add DeepSeek R1 Distill 8B #1488

Merged
merged 4 commits into from
Feb 3, 2025
Merged

Add DeepSeek R1 Distill 8B #1488

merged 4 commits into from
Feb 3, 2025

Conversation

Jack-Khuu
Copy link
Contributor

@Jack-Khuu Jack-Khuu commented Feb 1, 2025

Adding Configs for https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B

Examples:

python torchchat.py generate deepseek-r1:8b --compile --prompt "If x+7=9, solve for x"

python3 torchchat.py export deepseek-r1:8b --output-aoti-package-path r1.pt2
python3 torchchat.py generate deepseek-r1:8b--aoti-package-path r1.pt2 --prompt "If x+7=9, solve for x"

@Jack-Khuu Jack-Khuu requested a review from Gasoonjia February 1, 2025 00:08
Copy link

pytorch-bot bot commented Feb 1, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/1488

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 3 Pending

As of commit ea4f895 with merge base 162a38b (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label Feb 1, 2025
@Jack-Khuu Jack-Khuu added enhancement New feature or request triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module labels Feb 1, 2025
@Jack-Khuu Jack-Khuu requested a review from metascroy February 1, 2025 00:16
@Jack-Khuu Jack-Khuu requested a review from byjlw February 1, 2025 00:29
@mikekgfb
Copy link
Contributor

mikekgfb commented Feb 1, 2025

Nice!

@orionr
Copy link
Contributor

orionr commented Feb 3, 2025

Excellent! Do you want to also update the README.md at https://github.com/pytorch/torchchat?tab=readme-ov-file#models so people know you can run DeepSeek R1 8B?

@Jack-Khuu Jack-Khuu merged commit d607ecc into main Feb 3, 2025
69 checks passed
vmpuri pushed a commit that referenced this pull request Feb 4, 2025
* Add DeepSeek R1 Distill 8B

* Update aliases to match Ollama

* Update README
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Meta Open Source bot. enhancement New feature or request triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants