Update_docker by heyang (#29)

Author: hzjane

docker/llm/README.md

@@ -2,13 +2,13 @@
 
 [Alpaca Lora](https://github.com/tloen/alpaca-lora/tree/main) uses [low-rank adaption](https://arxiv.org/pdf/2106.09685.pdf) to speed up the finetuning process of base model [Llama2-7b](https://huggingface.co/meta-llama/Llama-2-7b), and tries to reproduce the standard Alpaca, a general finetuned LLM. This is on top of Hugging Face transformers with Pytorch backend, which natively requires a number of expensive GPU resources and takes significant time.
 
-By constract, BigDL here provides a CPU optimization to accelerate the lora finetuning of Llama2-7b, in the power of mixed-precision and distributed training. Detailedly, [Intel OneCCL](https://www.intel.com/content/www/us/en/developer/tools/oneapi/oneccl.html), an available Hugging Face backend, is able to speed up the Pytorch computation with BF16 datatype on CPUs, as well as parallel processing on Kubernetes enabled by [Intel MPI](https://www.intel.com/content/www/us/en/developer/tools/oneapi/mpi-library.html). 
+By constract, IPEX-LLM here provides a CPU optimization to accelerate the lora finetuning of Llama2-7b, in the power of mixed-precision and distributed training. Detailedly, [Intel OneCCL](https://www.intel.com/content/www/us/en/developer/tools/oneapi/oneccl.html), an available Hugging Face backend, is able to speed up the Pytorch computation with BF16 datatype on CPUs, as well as parallel processing on Kubernetes enabled by [Intel MPI](https://www.intel.com/content/www/us/en/developer/tools/oneapi/mpi-library.html). 
 
 The architecture is illustrated in the following:
 
 ![image](https://llm-assets.readthedocs.io/en/latest/_images/llm-finetune-lora-cpu-k8s.png)
 
-As above, BigDL implements its MPI training with [Kubeflow MPI operator](https://github.com/kubeflow/mpi-operator/tree/master), which encapsulates the deployment as MPIJob CRD, and assists users to handle the construction of a MPI worker cluster on Kubernetes, such as public key distribution, SSH connection, and log collection. 
+As above, IPEX-LLM implements its MPI training with [Kubeflow MPI operator](https://github.com/kubeflow/mpi-operator/tree/master), which encapsulates the deployment as MPIJob CRD, and assists users to handle the construction of a MPI worker cluster on Kubernetes, such as public key distribution, SSH connection, and log collection. 
 
 Now, let's go to deploy a Lora finetuning to create a LLM from Llama2-7b.
 
@@ -20,7 +20,7 @@ Follow [here](https://github.com/kubeflow/mpi-operator/tree/master#installation)
 
 ### 2. Download Image, Base Model and Finetuning Data
 
-Follow [here](https://github.com/intel-analytics/BigDL/tree/main/docker/llm/finetune/lora/docker#prepare-bigdl-image-for-lora-finetuning) to prepare BigDL Lora Finetuning image in your cluster.
+Follow [here](https://github.com/intel-analytics/IPEX-LLM/tree/main/docker/llm/finetune/lora/docker#prepare-ipex-llm-image-for-lora-finetuning) to prepare IPEX-LLM Lora Finetuning image in your cluster.
 
 As finetuning is from a base model, first download [Llama2-7b model from the public download site of Hugging Face](https://huggingface.co/meta-llama/Llama-2-7b). Then, download [cleaned alpaca data](https://raw.githubusercontent.com/tloen/alpaca-lora/main/alpaca_data_cleaned_archive.json), which contains all kinds of general knowledge and has already been cleaned. Next, move the downloaded files to a shared directory on your NFS server.
 
@@ -34,21 +34,21 @@ After preparing parameters in `./kubernetes/values.yaml`, submit the job as befl
 
 ```bash
 cd ./kubernetes
-helm install bigdl-lora-finetuning .
+helm install ipex-llm-lora-finetuning .
 ```
 
 ### 4. Check Deployment
 ```bash
-kubectl get all -n bigdl-lora-finetuning # you will see launcher and worker pods running
+kubectl get all -n ipex-llm-lora-finetuning # you will see launcher and worker pods running
 ```
 
 ### 5. Check Finetuning Process
 
 After deploying successfully, you can find a launcher pod, and then go inside this pod and check the logs collected from all workers.
 
 ```bash
-kubectl get all -n bigdl-lora-finetuning # you will see a launcher pod
-kubectl exec -it <launcher_pod_name> bash -n bigdl-ppml-finetuning # enter launcher pod
+kubectl get all -n ipex-llm-lora-finetuning # you will see a launcher pod
+kubectl exec -it <launcher_pod_name> bash -n ipex-llm-lora-finetuning # enter launcher pod
 cat launcher.log # display logs collected from other workers
 ```
 

docker/llm/finetune/lora/cpu/README.md

@@ -12,13 +12,13 @@ FROM mpioperator/intel as builder
 ARG http_proxy
 ARG https_proxy
 ENV PIP_NO_CACHE_DIR=false
-COPY ./requirements.txt /bigdl/requirements.txt
+COPY ./requirements.txt /ipex_llm/requirements.txt
 
 # add public key
 COPY --from=key-getter /root/intel-oneapi-archive-keyring.gpg /usr/share/keyrings/intel-oneapi-archive-keyring.gpg
 RUN echo "deb [signed-by=/usr/share/keyrings/intel-oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main " > /etc/apt/sources.list.d/oneAPI.list
 
-RUN mkdir /bigdl/data && mkdir /bigdl/model && \
+RUN mkdir /ipex_llm/data && mkdir /ipex_llm/model && \
 # install pytorch 2.0.1
     apt-get update && \
     apt-get install -y python3-pip python3.9-dev python3-wheel git software-properties-common && \
@@ -29,12 +29,12 @@ RUN mkdir /bigdl/data && mkdir /bigdl/model && \
     pip install intel_extension_for_pytorch==2.0.100 && \
     pip install oneccl_bind_pt -f https://developer.intel.com/ipex-whl-stable && \
 # install transformers etc.
-    cd /bigdl && \
+    cd /ipex_llm && \
     git clone https://github.com/huggingface/transformers.git && \
     cd transformers && \
     git reset --hard 057e1d74733f52817dc05b673a340b4e3ebea08c && \
     pip install . && \
-    pip install -r /bigdl/requirements.txt && \
+    pip install -r /ipex_llm/requirements.txt && \
 # install python
     add-apt-repository ppa:deadsnakes/ppa -y && \
     apt-get install -y python3.9 && \
@@ -56,9 +56,9 @@ RUN mkdir /bigdl/data && mkdir /bigdl/model && \
     echo "    UserKnownHostsFile /dev/null" >> /etc/ssh/ssh_config && \
     sed -i 's/#\(StrictModes \).*/\1no/g' /etc/ssh/sshd_config
 
-COPY ./bigdl-lora-finetuing-entrypoint.sh /bigdl/bigdl-lora-finetuing-entrypoint.sh
-COPY ./lora_finetune.py /bigdl/lora_finetune.py
+COPY ./ipex-llm-lora-finetuing-entrypoint.sh /ipex_llm/ipex-llm-lora-finetuing-entrypoint.sh
+COPY ./lora_finetune.py /ipex_llm/lora_finetune.py
 
-RUN chown -R mpiuser /bigdl
+RUN chown -R mpiuser /ipex_llm
 USER mpiuser
 ENTRYPOINT ["/bin/bash"]

docker/llm/finetune/lora/cpu/docker/Dockerfile

@@ -1,11 +1,11 @@
 ## Fine-tune LLM with One CPU
 
-### 1. Prepare BigDL image for Lora Finetuning
+### 1. Prepare IPEX LLM image for Lora Finetuning
 
 You can download directly from Dockerhub like:
 
 ```bash
-docker pull intelanalytics/bigdl-llm-finetune-lora-cpu:2.5.0-SNAPSHOT
+docker pull intelanalytics/ipex-llm-finetune-lora-cpu:2.5.0-SNAPSHOT
 ```
 
 Or build the image from source:
@@ -17,7 +17,7 @@ export HTTPS_PROXY=your_https_proxy
 docker build \
   --build-arg http_proxy=${HTTP_PROXY} \
   --build-arg https_proxy=${HTTPS_PROXY} \
-  -t intelanalytics/bigdl-llm-finetune-lora-cpu:2.5.0-SNAPSHOT \
+  -t intelanalytics/ipex-llm-finetune-lora-cpu:2.5.0-SNAPSHOT \
   -f ./Dockerfile .
 ```
 
@@ -27,13 +27,13 @@ Here, we try to finetune [Llama2-7b](https://huggingface.co/meta-llama/Llama-2-7
 
 ```
 docker run -itd \
- --name=bigdl-llm-fintune-lora-cpu \
+ --name=ipex-llm-fintune-lora-cpu \
  --cpuset-cpus="your_expected_range_of_cpu_numbers" \
  -e STANDALONE_DOCKER=TRUE \
  -e WORKER_COUNT_DOCKER=your_worker_count \
- -v your_downloaded_base_model_path:/bigdl/model \
- -v your_downloaded_data_path:/bigdl/data/alpaca_data_cleaned_archive.json \
- intelanalytics/bigdl-llm-finetune-lora-cpu:2.5.0-SNAPSHOT \
+ -v your_downloaded_base_model_path:/ipex_llm/model \
+ -v your_downloaded_data_path:/ipex_llm/data/alpaca_data_cleaned_archive.json \
+ intelanalytics/ipex-llm-finetune-lora-cpu:2.5.0-SNAPSHOT \
  bash
 ```
 
@@ -44,21 +44,21 @@ You can adjust the configuration according to your own environment. After our te
 Enter the running container:
 
 ```
-docker exec -it bigdl-llm-fintune-lora-cpu bash
+docker exec -it ipex-llm-fintune-lora-cpu bash
 ```
 
 Then, run the script to start finetuning:
 
 ```
-bash /bigdl/bigdl-lora-finetuing-entrypoint.sh
+bash /ipex_llm/ipex-llm-lora-finetuing-entrypoint.sh
 ```
 
 After minutes, it is expected to get results like:
 
 ```
 Training Alpaca-LoRA model with params:
-base_model: /bigdl/model/
-data_path: /bigdl/data/alpaca_data_cleaned_archive.json
+base_model: /ipex_llm/model/
+data_path: /ipex_llm/data/alpaca_data_cleaned_archive.json
 output_dir: /home/mpiuser/finetuned_model
 batch_size: 128
 micro_batch_size: 8

docker/llm/finetune/lora/cpu/docker/README.md

@@ -15,9 +15,9 @@ then
      -genv KMP_AFFINITY="granularity=fine,none" \
      -genv KMP_BLOCKTIME=1 \
      -genv TF_ENABLE_ONEDNN_OPTS=1 \
-     python /bigdl/lora_finetune.py \
-       --base_model '/bigdl/model/'  \
-       --data_path "/bigdl/data/alpaca_data_cleaned_archive.json" \
+     python /ipex_llm/lora_finetune.py \
+       --base_model '/ipex_llm/model/'  \
+       --data_path "/ipex_llm/data/alpaca_data_cleaned_archive.json" \
        --output_dir "/home/mpiuser/finetuned_model" \
        --micro_batch_size 8 \
        --bf16 
@@ -29,7 +29,7 @@ else
   if [ "$WORKER_ROLE" = "launcher" ]
   then
     sed "s/:1/ /g" /etc/mpi/hostfile > /home/mpiuser/hostfile
-    export DATA_PATH="/bigdl/data/$DATA_SUB_PATH"
+    export DATA_PATH="/ipex_llm/data/$DATA_SUB_PATH"
     sleep 10
     mpirun \
       -n $WORLD_SIZE \
@@ -40,8 +40,8 @@ else
       -genv KMP_AFFINITY="granularity=fine,none" \
       -genv KMP_BLOCKTIME=1 \
       -genv TF_ENABLE_ONEDNN_OPTS=1 \
-      python /bigdl/lora_finetune.py \
-        --base_model '/bigdl/model/'  \
+      python /ipex_llm/lora_finetune.py \
+        --base_model '/ipex_llm/model/'  \
         --data_path "$DATA_PATH" \
         --output_dir "/home/mpiuser/finetuned_model" \
         --micro_batch_size $MICRO_BATCH_SIZE \

docker/llm/finetune/lora/cpu/docker/bigdl-lora-finetuing-entrypoint.sh docker/llm/finetune/lora/cpu/docker/ipex-llm-lora-finetuing-entrypoint.sh

@@ -1,6 +1,6 @@
 apiVersion: v2
 name: trusted-fintune-service
-description: A Helm chart for BigDL PPML Trusted BigData Service on Kubernetes
+description: A Helm chart for IPEX-LLM Finetuning Service on Kubernetes
 type: application
 version: 1.1.27
 appVersion: "1.16.0"

docker/llm/finetune/lora/cpu/kubernetes/Chart.yaml

@@ -1,8 +1,8 @@
 apiVersion: kubeflow.org/v2beta1
 kind: MPIJob
 metadata:
-  name: bigdl-lora-finetuning-job
-  namespace: bigdl-lora-finetuning
+  name: ipex-llm-lora-finetuning-job
+  namespace: ipex-llm-lora-finetuning
 spec:
   slotsPerWorker: 1
   runPolicy:
@@ -20,10 +20,10 @@ spec:
                claimName: nfs-pvc
            containers:
            - image: {{ .Values.imageName }}
-             name: bigdl-ppml-finetuning-launcher
+             name: ipex-llm-lora-finetuning-launcher
              securityContext:
               runAsUser: 1000
-             command: ['sh' , '-c', 'bash /bigdl/bigdl-lora-finetuing-entrypoint.sh']
+             command: ['sh' , '-c', 'bash /ipex_llm/ipex-llm-lora-finetuing-entrypoint.sh']
              env:
              - name: WORKER_ROLE
                value: "launcher"
@@ -34,7 +34,7 @@ spec:
              - name: MASTER_PORT
                value: "42679"
              - name: MASTER_ADDR
-               value: "bigdl-lora-finetuning-job-worker-0.bigdl-lora-finetuning-job-worker"
+               value: "ipex-llm-lora-finetuning-job-worker-0.ipex-llm-lora-finetuning-job-worker"
              - name: DATA_SUB_PATH
                value: "{{ .Values.dataSubPath }}"
              - name: OMP_NUM_THREADS
@@ -46,20 +46,20 @@ spec:
              volumeMounts:
              - name: nfs-storage
                subPath: {{ .Values.modelSubPath }}
-               mountPath: /bigdl/model
+               mountPath: /ipex_llm/model
              - name: nfs-storage
                subPath: {{ .Values.dataSubPath }}
-               mountPath: "/bigdl/data/{{ .Values.dataSubPath }}"
+               mountPath: "/ipex_llm/data/{{ .Values.dataSubPath }}"
     Worker:
       replicas: {{ .Values.trainerNum }}
       template:
         spec:
           containers:
           - image: {{ .Values.imageName }}
-            name: bigdl-ppml-finetuning-worker
+            name: ipex-llm-lora-finetuning-worker
             securityContext:
               runAsUser: 1000
-            command: ['sh' , '-c', 'bash /bigdl/bigdl-lora-finetuing-entrypoint.sh']
+            command: ['sh' , '-c', 'bash /ipex_llm/ipex-llm-lora-finetuing-entrypoint.sh']
             env:
             - name: WORKER_ROLE
               value: "trainer"
@@ -70,18 +70,18 @@ spec:
             - name: MASTER_PORT
               value: "42679"
             - name: MASTER_ADDR
-              value: "bigdl-lora-finetuning-job-worker-0.bigdl-lora-finetuning-job-worker"
+              value: "ipex-llm-lora-finetuning-job-worker-0.ipex-llm-lora-finetuning-job-worker"
             - name: LOCAL_POD_NAME
               valueFrom:
                 fieldRef:
                   fieldPath: metadata.name
             volumeMounts:
             - name: nfs-storage
               subPath: {{ .Values.modelSubPath }}
-              mountPath: /bigdl/model
+              mountPath: /ipex_llm/model
             - name: nfs-storage
               subPath: {{ .Values.dataSubPath }}
-              mountPath: "/bigdl/data/{{ .Values.dataSubPath }}"
+              mountPath: "/ipex_llm/data/{{ .Values.dataSubPath }}"
             resources:
               requests:
                 cpu: {{ .Values.cpuPerPod }}

docker/llm/finetune/lora/cpu/kubernetes/templates/bigdl-lora-finetuning-job.yaml docker/llm/finetune/lora/cpu/kubernetes/templates/ipex-llm-lora-finetuning-job.yaml

@@ -1,4 +1,4 @@
 apiVersion: v1
 kind: Namespace
 metadata:
-  name: bigdl-qlora-finetuning
+  name: ipex-llm-lora-finetuning

docker/llm/finetune/qlora/cpu/kubernetes/templates/bigdl-qlora-finetuning-namespace.yaml docker/llm/finetune/lora/cpu/kubernetes/templates/ipex-llm-lora-finetuning-namespace.yaml

@@ -1,8 +1,8 @@
 apiVersion: v1
 kind: PersistentVolume
 metadata:
-  name: nfs-pv-bigdl-lora-finetuning
-  namespace: bigdl-lora-finetuning
+  name: nfs-pv-ipex-llm-lora-finetuning
+  namespace: ipex-llm-lora-finetuning
 spec:
   capacity:
     storage: 15Gi

docker/llm/finetune/lora/cpu/kubernetes/templates/nfs-pv.yaml

@@ -2,7 +2,7 @@ kind: PersistentVolumeClaim
 apiVersion: v1
 metadata:
   name: nfs-pvc
-  namespace: bigdl-lora-finetuning
+  namespace: ipex-llm-lora-finetuning
 spec:
   accessModes:
   - ReadWriteOnce

docker/llm/finetune/lora/cpu/kubernetes/templates/nfs-pvc.yaml

@@ -1,4 +1,4 @@
-imageName: intelanalytics/bigdl-llm-finetune-lora-cpu:2.5.0-SNAPSHOT
+imageName: intelanalytics/ipex-llm-finetune-lora-cpu:2.5.0-SNAPSHOT
 trainerNum: 8
 microBatchSize: 8
 nfsServerIp: your_nfs_server_ip

docker/llm/finetune/lora/cpu/kubernetes/values.yaml

@@ -18,7 +18,7 @@ ENV TRANSFORMERS_COMMIT_ID=95fe0f5
 COPY --from=key-getter /root/intel-oneapi-archive-keyring.gpg /usr/share/keyrings/intel-oneapi-archive-keyring.gpg
 RUN echo "deb [signed-by=/usr/share/keyrings/intel-oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main " > /etc/apt/sources.list.d/oneAPI.list
 
-RUN mkdir -p /bigdl/data && mkdir -p /bigdl/model && \
+RUN mkdir -p /ipex_llm/data && mkdir -p /ipex_llm/model && \
     # install pytorch 2.1.0
     apt-get update && \
     apt-get install -y --no-install-recommends python3-pip python3.9-dev python3-wheel python3.9-distutils git software-properties-common && \
@@ -27,8 +27,8 @@ RUN mkdir -p /bigdl/data && mkdir -p /bigdl/model && \
     pip3 install --upgrade pip && \
     export PIP_DEFAULT_TIMEOUT=100 && \
     pip install --upgrade torch==2.1.0 && \
-    # install CPU bigdl-llm
-    pip3 install --pre --upgrade bigdl-llm[all] && \
+    # install CPU ipex-llm
+    pip3 install --pre --upgrade ipex-llm[all] && \
     # install ipex and oneccl
     pip install https://intel-extension-for-pytorch.s3.amazonaws.com/ipex_stable/cpu/intel_extension_for_pytorch-2.1.0%2Bcpu-cp39-cp39-linux_x86_64.whl && \
     pip install oneccl_bind_pt -f https://developer.intel.com/ipex-whl-stable && \
@@ -41,16 +41,16 @@ RUN mkdir -p /bigdl/data && mkdir -p /bigdl/model && \
     apt-get update && apt-get install -y curl wget gpg gpg-agent software-properties-common libunwind8-dev && \
     # get qlora example code
     ln -s /usr/bin/python3 /usr/bin/python && \
-    cd /bigdl && \
-    git clone https://github.com/intel-analytics/BigDL.git && \
-    mv BigDL/python/llm/example/CPU/QLoRA-FineTuning/* . && \
+    cd /ipex_llm && \
+    git clone https://github.com/intel-analytics/IPEX-LLM.git && \
+    mv IPEX-LLM/python/llm/example/CPU/QLoRA-FineTuning/* . && \
     mkdir -p /GPU/LLM-Finetuning && \
-    mv BigDL/python/llm/example/GPU/LLM-Finetuning/common /GPU/LLM-Finetuning/common && \
-    rm -r BigDL && \
-    chown -R mpiuser /bigdl
+    mv IPEX-LLM/python/llm/example/GPU/LLM-Finetuning/common /GPU/LLM-Finetuning/common && \
+    rm -r IPEX-LLM && \
+    chown -R mpiuser /ipex_llm
 
 # for standalone
-COPY ./start-qlora-finetuning-on-cpu.sh /bigdl/start-qlora-finetuning-on-cpu.sh
+COPY ./start-qlora-finetuning-on-cpu.sh /ipex_llm/start-qlora-finetuning-on-cpu.sh
 
 USER mpiuser
 

docker/llm/finetune/qlora/cpu/docker/Dockerfile

@@ -19,7 +19,7 @@ ENV TRANSFORMERS_COMMIT_ID=95fe0f5
 COPY --from=key-getter /root/intel-oneapi-archive-keyring.gpg /usr/share/keyrings/intel-oneapi-archive-keyring.gpg
 RUN echo "deb [signed-by=/usr/share/keyrings/intel-oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main " > /etc/apt/sources.list.d/oneAPI.list
 
-RUN mkdir -p /bigdl/data && mkdir -p /bigdl/model && \
+RUN mkdir -p /ipex_llm/data && mkdir -p /ipex_llm/model && \
     apt-get update && \
     apt install -y --no-install-recommends openssh-server openssh-client libcap2-bin gnupg2 ca-certificates \ 
     python3-pip python3.9-dev python3-wheel python3.9-distutils git software-properties-common && \
@@ -40,8 +40,8 @@ RUN mkdir -p /bigdl/data && mkdir -p /bigdl/model && \
     pip3 install --upgrade pip && \
     export PIP_DEFAULT_TIMEOUT=100 && \
     pip install --upgrade torch==2.1.0 --index-url https://download.pytorch.org/whl/cpu && \
-    # install CPU bigdl-llm
-    pip3 install --pre --upgrade bigdl-llm[all] && \
+    # install CPU ipex-llm
+    pip3 install --pre --upgrade ipex-llm[all] && \
     # install ipex and oneccl
     pip install intel_extension_for_pytorch==2.0.100 && \
     pip install oneccl_bind_pt -f https://developer.intel.com/ipex-whl-stable && \
@@ -59,14 +59,14 @@ RUN mkdir -p /bigdl/data && mkdir -p /bigdl/model && \
     rm -rf /var/lib/apt/lists/* && \
     # get qlora example code
     ln -s /usr/bin/python3 /usr/bin/python && \
-    cd /bigdl && \
-    git clone https://github.com/intel-analytics/BigDL.git && \
-    mv BigDL/python/llm/example/CPU/QLoRA-FineTuning/* . && \
-    rm -r BigDL && \
-    chown -R mpiuser /bigdl
+    cd /ipex_llm && \
+    git clone https://github.com/intel-analytics/IPEX-LLM.git && \
+    mv IPEX-LLM/python/llm/example/CPU/QLoRA-FineTuning/* . && \
+    rm -r IPEX-LLM && \
+    chown -R mpiuser /ipex_llm
 
 # for k8s
-COPY ./bigdl-qlora-finetuing-entrypoint.sh /bigdl/bigdl-qlora-finetuing-entrypoint.sh
+COPY ./ipex-llm-qlora-finetuing-entrypoint.sh /ipex_llm/ipex-llm-qlora-finetuing-entrypoint.sh
 
 USER mpiuser