Skip to content

Commit 015a4c8

Browse files
authored
Add CPU and GPU Frequency Locking Instructions to Documentation (#12947)
1 parent cb3c4b2 commit 015a4c8

File tree

3 files changed

+60
-0
lines changed

3 files changed

+60
-0
lines changed

docker/llm/serving/xpu/docker/README.md

+20
Original file line numberDiff line numberDiff line change
@@ -179,6 +179,26 @@ The following example files are available in `/llm/` within the container:
179179

180180
## 4. Benchmarking
181181

182+
> [!TIP]
183+
> Before running benchmarks, it's recommended to lock CPU and GPU frequencies to ensure more stable, reliable, and better performance data.
184+
>
185+
> **Lock CPU Frequency:**
186+
> Use the following command to set the minimum CPU frequency (adjust based on your CPU model):
187+
>
188+
> ```bash
189+
> sudo cpupower frequency-set -d 3.8GHz
190+
> ```
191+
>
192+
> **Lock GPU Frequencies:**
193+
> Use these commands to lock GPU frequencies to 2400MHz:
194+
>
195+
> ```bash
196+
> sudo xpu-smi config -d 0 -t 0 --frequencyrange 2400,2400
197+
> sudo xpu-smi config -d 1 -t 0 --frequencyrange 2400,2400
198+
> sudo xpu-smi config -d 2 -t 0 --frequencyrange 2400,2400
199+
> sudo xpu-smi config -d 3 -t 0 --frequencyrange 2400,2400
200+
> ```
201+
182202
### 4.1 Online Benchmark through API Server
183203
184204
To benchmark the API server and estimate TPS (transactions per second), follow these steps:

docs/mddocs/DockerGuides/vllm_docker_quickstart.md

+20
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,26 @@ root@arda-arc12:/# sycl-ls
5959

6060
## Running vLLM serving with IPEX-LLM on Intel GPU in Docker
6161

62+
> [!TIP]
63+
> Before running benchmarks, it's recommended to lock CPU and GPU frequencies to ensure more stable, reliable, and better performance data.
64+
>
65+
> **Lock CPU Frequency:**
66+
> Use the following command to set the minimum CPU frequency (adjust based on your CPU model):
67+
>
68+
> ```bash
69+
> sudo cpupower frequency-set -d 3.8GHz
70+
> ```
71+
>
72+
> **Lock GPU Frequencies:**
73+
> Use these commands to lock GPU frequencies to 2400MHz:
74+
>
75+
> ```bash
76+
> sudo xpu-smi config -d 0 -t 0 --frequencyrange 2400,2400
77+
> sudo xpu-smi config -d 1 -t 0 --frequencyrange 2400,2400
78+
> sudo xpu-smi config -d 2 -t 0 --frequencyrange 2400,2400
79+
> sudo xpu-smi config -d 3 -t 0 --frequencyrange 2400,2400
80+
> ```
81+
6282
We have included multiple vLLM-related files in `/llm/`:
6383
6484
1. `vllm_offline_inference.py`: Used for vLLM offline inference example,

docs/mddocs/Quickstart/vLLM_quickstart.md

+20
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,26 @@ Currently, IPEX-LLM integrated vLLM only supports the following models:
2121

2222
## Quick Start
2323

24+
> [!TIP]
25+
> Before running benchmarks, it's recommended to lock CPU and GPU frequencies to ensure more stable, reliable, and better performance data.
26+
>
27+
> **Lock CPU Frequency:**
28+
> Use the following command to set the minimum CPU frequency (adjust based on your CPU model):
29+
>
30+
> ```bash
31+
> sudo cpupower frequency-set -d 3.8GHz
32+
> ```
33+
>
34+
> **Lock GPU Frequencies:**
35+
> Use these commands to lock GPU frequencies to 2400MHz:
36+
>
37+
> ```bash
38+
> sudo xpu-smi config -d 0 -t 0 --frequencyrange 2400,2400
39+
> sudo xpu-smi config -d 1 -t 0 --frequencyrange 2400,2400
40+
> sudo xpu-smi config -d 2 -t 0 --frequencyrange 2400,2400
41+
> sudo xpu-smi config -d 3 -t 0 --frequencyrange 2400,2400
42+
> ```
43+
2444
This quickstart guide walks you through installing and running `vLLM` with `ipex-llm`.
2545
2646
### 1. Install IPEX-LLM for vLLM

0 commit comments

Comments
 (0)