Skip to content

Commit 5dc121e

Browse files
authored
Add guide for running bigdl-example using ipex-llm libs (#28)
* add guide * update
1 parent a1048ca commit 5dc121e

File tree

1 file changed

+33
-0
lines changed

1 file changed

+33
-0
lines changed

docs/bigdl-llm-upgrade.md

+33
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# bigdl-llm upgrade guide
2+
## Upgrade bigdl-llm to ipex-llm
3+
Need to uninstall bigdl-llm and install ipex-llm first.
4+
```bash
5+
pip uninstall -y bigdl-llm
6+
pip install --pre --upgrade ipex-llm[all] # for cpu
7+
pip install --pre --upgrade ipex-llm[xpu] -f https://developer.intel.com/ipex-whl-stable-xpu # for xpu
8+
```
9+
## Run bigdl-llm example code
10+
Choose a way to modify the example code to use.
11+
12+
### 1.Run bigdl-llm example code with ipex-llm in compatible mode
13+
Minimal change: Use `BIGDL_COMPATIBLE_MODE` to run, need to add `import ipex_llm` at the
14+
beginning of code with bigdl-llm examples.
15+
```python
16+
# need to add before import bigdl.llm
17+
import ipex_llm
18+
from bigdl.llm.transformers import AutoModelForCausalLM
19+
model = AutoModelForCausalLM.from_pretrained(model_path,
20+
load_in_4bit=True,
21+
trust_remote_code=True)
22+
```
23+
24+
### 2.Upgrade bigdl-llm example code to ipex-llm
25+
Need to replace all `bigdl.llm` with `ipex_llm`.
26+
```python
27+
#from bigdl.llm.transformers import AutoModelForCausalLM
28+
from ipex_llm.transformers import AutoModelForCausalLM
29+
model = AutoModelForCausalLM.from_pretrained(model_path,
30+
load_in_4bit=True,
31+
trust_remote_code=True)
32+
```
33+

0 commit comments

Comments
 (0)