File tree 1 file changed +33
-0
lines changed
1 file changed +33
-0
lines changed Original file line number Diff line number Diff line change
1
+ # bigdl-llm upgrade guide
2
+ ## Upgrade bigdl-llm to ipex-llm
3
+ Need to uninstall bigdl-llm and install ipex-llm first.
4
+ ``` bash
5
+ pip uninstall -y bigdl-llm
6
+ pip install --pre --upgrade ipex-llm[all] # for cpu
7
+ pip install --pre --upgrade ipex-llm[xpu] -f https://developer.intel.com/ipex-whl-stable-xpu # for xpu
8
+ ```
9
+ ## Run bigdl-llm example code
10
+ Choose a way to modify the example code to use.
11
+
12
+ ### 1.Run bigdl-llm example code with ipex-llm in compatible mode
13
+ Minimal change: Use ` BIGDL_COMPATIBLE_MODE ` to run, need to add ` import ipex_llm ` at the
14
+ beginning of code with bigdl-llm examples.
15
+ ``` python
16
+ # need to add before import bigdl.llm
17
+ import ipex_llm
18
+ from bigdl.llm.transformers import AutoModelForCausalLM
19
+ model = AutoModelForCausalLM.from_pretrained(model_path,
20
+ load_in_4bit = True ,
21
+ trust_remote_code = True )
22
+ ```
23
+
24
+ ### 2.Upgrade bigdl-llm example code to ipex-llm
25
+ Need to replace all ` bigdl.llm ` with ` ipex_llm ` .
26
+ ``` python
27
+ # from bigdl.llm.transformers import AutoModelForCausalLM
28
+ from ipex_llm.transformers import AutoModelForCausalLM
29
+ model = AutoModelForCausalLM.from_pretrained(model_path,
30
+ load_in_4bit = True ,
31
+ trust_remote_code = True )
32
+ ```
33
+
You can’t perform that action at this time.
0 commit comments