You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/mddocs/Quickstart/install_windows_gpu.md
+42-27Lines changed: 42 additions & 27 deletions
Original file line number
Diff line number
Diff line change
@@ -123,21 +123,15 @@ To monitor your GPU's performance and status (e.g. memory consumption, utilizati
123
123
124
124
## A Quick Example
125
125
126
-
Now let's play with a real LLM. We'll be using the [Qwen-1.8B-Chat](https://huggingface.co/Qwen/Qwen-1_8B-Chat) model, a 1.8 billion parameter LLM for this demonstration. Follow the steps below to setup and run the model, and observe how it responds to a prompt "What is AI?".
126
+
Now let's play with a real LLM. We'll be using the [Qwen2-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2-1.5B-Instruct) model, a 1.8 billion parameter LLM for this demonstration. Follow the steps below to setup and run the model, and observe how it responds to a prompt "What is AI?".
127
127
128
128
- Step 1: Follow [Runtime Configurations Section](#step-1-runtime-configurations) above to prepare your runtime environment.
129
129
130
-
- Step 2: Install additional package required for Qwen-1.8B-Chat to conduct:
- Step 3: Create code file. IPEX-LLM supports loading model from Hugging Face or ModelScope. Please choose according to your requirements.
130
+
- Step 2: Create code file. IPEX-LLM supports loading model from Hugging Face or ModelScope. Please choose according to your requirements.
137
131
138
132
- For **loading model from Hugging Face**:
139
133
140
-
Create a new file named `demo.py` and insert the code snippet below to run [Qwen-1.8B-Chat](https://huggingface.co/Qwen/Qwen-1_8B-Chat) model with IPEX-LLM optimizations.
134
+
Create a new file named `demo.py` and insert the code snippet below to run [Qwen2-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2-1.5B-Instruct) model with IPEX-LLM optimizations.
141
135
142
136
```python
143
137
# Copy/Paste the contents to a new file demo.py
@@ -147,24 +141,34 @@ Now let's play with a real LLM. We'll be using the [Qwen-1.8B-Chat](https://hugg
@@ -195,10 +199,9 @@ Now let's play with a real LLM. We'll be using the [Qwen-1.8B-Chat](https://hugg
195
199
pip install modelscope==1.11.0
196
200
```
197
201
198
-
Create a new file named `demo.py`and insert the code snippet below to run [Qwen-1.8B-Chat](https://www.modelscope.cn/models/qwen/Qwen-1_8B-Chat/summary) model withIPEX-LLM optimizations.
202
+
Create a new file named `demo.py`and insert the code snippet below to run [Qwen2-1.5B-Instruct](https://www.modelscope.cn/models/qwen/Qwen2-1.5B-Instruct/summary) model withIPEX-LLM optimizations.
199
203
200
204
```python
201
-
202
205
# Copy/Paste the contents to a new file demo.py
203
206
import torch
204
207
from ipex_llm.transformers import AutoModelForCausalLM
@@ -207,11 +210,11 @@ Now let's play with a real LLM. We'll be using the [Qwen-1.8B-Chat](https://hugg
@@ -257,7 +269,7 @@ Now let's play with a real LLM. We'll be using the [Qwen-1.8B-Chat](https://hugg
257
269
> When running LLMs on Intel iGPUs with limited memory size, we recommend setting `cpu_embedding=True`in the `from_pretrained` function.
258
270
> This will allow the memory-intensive embedding layer to utilize the CPU instead of GPU.
259
271
260
-
- Step 4. Run `demo.py` within the activated Python environment using the following command:
272
+
- Step 3. Run `demo.py` within the activated Python environment using the following command:
261
273
262
274
```cmd
263
275
python demo.py
@@ -267,9 +279,12 @@ Now let's play with a real LLM. We'll be using the [Qwen-1.8B-Chat](https://hugg
267
279
268
280
Example output on a system equipped with an Intel Core Ultra 5 125H CPU and Intel Arc Graphics iGPU:
269
281
```
270
-
user: What is AI?
271
-
272
-
assistant: AI stands for Artificial Intelligence, which refers to the development of computer systems that can perform tasks that typically require human intelligence, such as visual perception, speech recognition,
282
+
<|im_start|>system
283
+
You are a helpful assistant.<|im_end|>
284
+
<|im_start|>user
285
+
What is AI?<|im_end|>
286
+
<|im_start|>assistant
287
+
Artificial Intelligence (AI) refers to the simulation of human intelligence in machines that are programmed to think and act like humans. It involves the development of algorithms,
0 commit comments