Skip to content

Commit a390bcf

Browse files
committed
fixes
1 parent aa989b0 commit a390bcf

File tree

4 files changed

+214
-204
lines changed

4 files changed

+214
-204
lines changed

notebooks/deepseek-r1/README.md

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,17 @@
1-
# LLM reasoning with Deepseek-R1 distilled models
1+
# LLM reasoning with DeepSeek-R1 distilled models
22

33
[DeepSeek-R1](https://github.com/deepseek-ai/DeepSeek-R1/blob/main/DeepSeek_R1.pdf) is an open-source reasoning model developed by DeepSeek to address tasks requiring logical inference, mathematical problem-solving, and real-time decision-making. With DeepSeek-R1, you can follow its logic, making it easier to understand and, if necessary, challenge its output. This capability gives reasoning models an edge in fields where outcomes need to be explainable, like research or complex decision-making.
44

5-
Distillation in AI creates smaller, more efficient models from larger ones, preserving much of their reasoning power while reducing computational demands. DeepSeek applied this technique to create a suite of distilled models from R1, using Qwen and Llama architectures. That allows us to try Deepseek-R1 capability locally on usual laptops.
5+
Distillation in AI creates smaller, more efficient models from larger ones, preserving much of their reasoning power while reducing computational demands. DeepSeek applied this technique to create a suite of distilled models from R1, using Qwen and Llama architectures. That allows us to try DeepSeek-R1 capability locally on usual laptops.
66

7-
In this tutorial, we consider how to run Deepseek-R1 distilled models using OpenVINO.
7+
In this tutorial, we consider how to run DeepSeek-R1 distilled models using OpenVINO.
8+
9+
The tutorial supports different models, you can select one from the provided options to compare the quality of LLM solutions:
10+
11+
* **DeepSeek-R1-Distill-Llama-8B** is a distilled model based on [Llama-3.1-8B](https://huggingface.co/meta-llama/Llama-3.1-8B), that prioritizes high performance and advanced reasoning capabilities, particularly excelling in tasks requiring mathematical and factual precision. Check [model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B) for more info.
12+
* **DeepSeek-R1-Distill-Qwen-1.5B** is the smallest Deekseek-R1 distilled modell based on [Qwen2.5-Math-1.5B](https://huggingface.co/Qwen/Qwen2.5-Math-1.5B). Despite its compact size, the model demonstrates strong capabilities in solving basic mathematical tasks, at the same time its programming capabilities are limited. Check [model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B) for more info.
13+
* **DeepSeek-R1-Distill-Qwen-7B** is a distilled model based on [Qwen-2.5-Math-7B](https://huggingface.co/Qwen/Qwen2.5-Math-7B). The model demonstrates a good balance between mathematical and factual reasoning and can be less suited for complex coding tasks. Check [model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B) for more info.
14+
* **DeepSeek-R1-Distil-Qwen-14B** is a distilled model based on [Qwen2.5-14B](https://huggingface.co/Qwen/Qwen2.5-14B) that has great competence in factual reasoning and solving complex mathematical tasks. Check [model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-15B) for more info.
815

916
## Notebook Contents
1017

@@ -15,6 +22,8 @@ The tutorial consists of the following steps:
1522
- Compress model weights to INT4 or INT8 precision using [NNCF](https://github.com/openvinotoolkit/nncf)
1623
- Create an inference pipeline
1724
- Run interactive demo
25+
26+
![](https://github.com/user-attachments/assets/9062bdc4-0338-4555-a863-87b5a71236e9)
1827

1928
## Installation Instructions
2029
This is a self-contained example that relies solely on its own code.</br>

0 commit comments

Comments
 (0)