Skip to content

Commit efa471a

Browse files
lua1235lua1235
authored
Derive LDS properties for AMDGPUs from empirical measurements (#2919)
Add LDS bank count and phase group properties for some AMD GPU architectures. These were derived from latency measurements: an LDS bank conflict is observed between two threads accessing the same LDS bank only when those threads belong to the same access phase group. Phase groups were collected for the `ds_read_32`, `ds_read_64`, and `ds_read_128` instructions.
1 parent 1b6dc7c commit efa471a

15 files changed

Lines changed: 3283 additions & 0 deletions

‎docs/amdgpu_kernel_optimization_guide.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -337,6 +337,10 @@ For `ds_read_b128`, the access happens in four phases of 16 threads each:
337337
> to avoid bank conflicts when accessing LDS in a > column-wise fashion, as with MFMA instructions.
338338
> Instead, prefer XOR-based swizzling as described [here](https://rocm.blogs.amd.com/software-tools-optimization/lds-bank-conflict/README.html)
339339
340+
For empirically discovered LDS latency tables and phase-groups on MI300X, MI350X, RX 9070
341+
XT, and W7900, see the
342+
[empirical LDS measurements](./empirical-lds/lds_summary.md).
343+
340344
### Global Memory
341345
342346
To achieve peak kernel performance on MI300, it's crucial to access the global

‎docs/empirical-lds/cdna3info.md‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# CDNA3 MI300X LDS Measurements
2+
3+
Measurements were collected on MI300X (`gfx942`), which has a wavefront size of
4+
64 threads. The LDS bank count was measured to be 32. With 4-byte banks, the
5+
bank mapping repeats every 128 bytes, so an access stride of 128 bytes causes
6+
the maximum possible number of bank conflicts.
7+
8+
Each table starts at the natural byte stride for that access width: 4 bytes for
9+
`ds_read_b32`, 8 bytes for `ds_read_b64`, and 16 bytes for `ds_read_b128`. The
10+
values were collected from `profile_bank_count.py` with 3 runs per stride
11+
using the shared 64-read `lds_phase_mask` timing block. Bank-conflict counts
12+
are collected with rocprof, normalized by the 4096 launched workgroups, and
13+
cover the full 64-read block. Latencies are average active-thread timer values
14+
from `clock64()`, reported in cycles, and were collected using the macro
15+
payload with 64 `ds_read` instructions.
16+
17+
## ds_read_b32 Latencies
18+
19+
| stride (bytes) | bank conflicts per block | avg thread latency (cycles) |
20+
| -------------- | ------------------------ | --------------------------- |
21+
| 4 | 0 | 500.756 |
22+
| 8 | 128 | 570.156 |
23+
| 16 | 384 | 956.776 |
24+
| 32 | 896 | 1833.150 |
25+
| 64 | 1920 | 3511.675 |
26+
| 128 | 3968 | 6810.440 |
27+
| 256 | 3968 | 6781.011 |
28+
| 512 | 3968 | 6798.651 |
29+
30+
When performing a `ds_read_b32`, lanes access LDS in 2 phases: T0-T31, then
31+
T32-T63. The fully conflicted case has 31 conflicts in each phase, or 62
32+
conflicts per `ds_read`. Across the 64-read timing block, that saturates at
33+
3968 conflicts per block.
34+
35+
## ds_read_b64 Latencies
36+
37+
| stride (bytes) | bank conflicts per block | avg thread latency (cycles) |
38+
| -------------- | ------------------------ | --------------------------- |
39+
| 8 | 0 | 852.258 |
40+
| 16 | 256 | 991.498 |
41+
| 32 | 768 | 1855.603 |
42+
| 64 | 1792 | 3625.288 |
43+
| 128 | 3840 | 7157.675 |
44+
| 256 | 3840 | 7177.591 |
45+
| 512 | 3840 | 7095.679 |
46+
47+
For `ds_read_b64`, the access happens in four phases of 16 lanes each:
48+
T0-T15, T16-T31, T32-T47, then T48-T63. The fully conflicted case has 15
49+
conflicts per phase, or 60 conflicts per `ds_read`. Across the 64-read timing
50+
block, that saturates at 3840 conflicts per block.
51+
52+
## ds_read_b128 Latencies
53+
54+
| stride (bytes) | bank conflicts per block | avg thread latency (cycles) |
55+
| -------------- | ------------------------ | --------------------------- |
56+
| 16 | 0 | 1384.460 |
57+
| 32 | 512 | 1941.316 |
58+
| 64 | 1536 | 3803.940 |
59+
| 128 | 3584 | 7614.521 |
60+
| 256 | 3584 | 7545.271 |
61+
| 512 | 3584 | 7573.651 |
62+
63+
For `ds_read_b128`, the access happens in eight phases of 8 lanes each.
64+
65+
1. T0-T3 and T20-T23
66+
2. T32-T35 and T52-T55
67+
3. T4-T7 and T16-T19
68+
4. T36-T39 and T48-T51
69+
5. T8-T11 and T28-T31
70+
6. T40-T43 and T60-T63
71+
7. T12-T15 and T24-T27
72+
8. T44-T47 and T56-T59
73+
74+
The fully conflicted case has 7 conflicts per phase, or 56 conflicts per
75+
`ds_read`. Across the 64-read timing block, that saturates at 3584 conflicts
76+
per block. The baseline latency is higher than the 32- and 64-bit reads, but
77+
the same pattern is visible.
78+
79+
## Summary
80+
81+
On MI300X, these measurements support the expected 32-bank, 4-byte bank model:
82+
the conflict pattern repeats every 128 bytes, and the saturated conflict count
83+
depends on the number of LDS phases required by the access width.
84+
85+
| access width | phase count | lanes per phase | saturated conflicts per ds_read | saturated conflicts per 64-read block |
86+
| ------------ | ----------- | --------------- | ------------------------------- | ------------------------------------- |
87+
| 32-bit | 2 | 32 | 62 | 3968 |
88+
| 64-bit | 4 | 16 | 60 | 3840 |
89+
| 128-bit | 8 | 8 | 56 | 3584 |
90+
91+
All three access widths classify the LDS bank count as 32 using average thread
92+
latency. PMC counters are displayed when available, but benchmark timer latency
93+
is the default classification signal because counters can be unavailable or
94+
incoherent on some ROCm/GPU combinations.

‎docs/empirical-lds/cdna4info.md‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
# CDNA4 MI350X LDS Measurements
2+
3+
Measurements were collected on MI350X (`gfx950`), which has a wavefront size of
4+
64 threads. The LDS bank count was measured to be 64. With 4-byte banks, the
5+
bank mapping repeats every 256 bytes, so an access stride of 256 bytes causes
6+
the maximum possible number of bank conflicts.
7+
8+
The captured bank-count sweep in this note covers `ds_read_b64`. No usable
9+
normalized rocprof bank-conflict counter values were recorded for this capture,
10+
so the table uses benchmark timer values. Latencies are average active-thread
11+
timer values from `clock64()`, reported in cycles, and were collected using the
12+
macro payload with 64 `ds_read` instructions. Phase groups are discovered by
13+
comparing pairwise latencies of threads mapping to the same bank. Threads in the
14+
same access phase experience bank conflicts.
15+
16+
## ds_read_b32 Phase Group
17+
18+
For `ds_read_b32`, the access happens in a single phase: T0-T63.
19+
20+
## ds_read_b64 Latencies
21+
22+
| stride (bytes) | avg thread latency (cycles) |
23+
| -------------- | --------------------------- |
24+
| 8 | 583.353 |
25+
| 16 | 819.207 |
26+
| 32 | 1490.275 |
27+
| 64 | 2758.733 |
28+
| 128 | 5089.500 |
29+
| 256 | 10024.566 |
30+
| 512 | 10014.062 |
31+
32+
For `ds_read_b64`, the access happens in two phases of 32 lanes each: T0-T31,
33+
then T32-T63. The high-latency bucket starts at the 256-byte stride and remains
34+
saturated for the 512-byte stride.
35+
36+
## ds_read_b128 Phase Groups
37+
38+
For `ds_read_b128`, the access happens in four phases of 16 lanes each.
39+
40+
1. T0-T3, T12-T15, T20-T23, T24-T27
41+
2. T32-T35, T44-T47, T52-T55, T56-T59
42+
3. T4-T7, T8-T11, T16-T19, T28-T31
43+
4. T36-T39, T40-T43, T48-T51, T60-T63
44+
45+
## Summary
46+
47+
On MI350X, the recorded `ds_read_b64` sweep supports a 64-bank, 4-byte bank
48+
model: the latency pattern saturates at 256-byte stride, and larger strides stay
49+
in the same high-latency bucket.
50+
51+
| access width | phase count | lanes per phase | most likely bank count |
52+
| ------------ | ----------- | --------------- | ---------------------- |
53+
| 32-bit | 1 | 64 | 64 |
54+
| 64-bit | 2 | 32 | 64 |
55+
| 128-bit | 4 | 16 | 64 |
56+
57+
The profiler classified the recorded sweep using average thread latency. The
58+
CDNA4 phase groups above follow the 64-bank MI350 LDS behavior: a 32-bit read
59+
can cover the whole 64-lane wave in one phase, while 64- and 128-bit reads split
60+
the wave into two and four phases respectively.

‎docs/empirical-lds/harness/.gitignore‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Python generated files
2+
__pycache__/
3+
*.py[cod]
4+
*$py.class
5+
6+
# Build outputs
7+
build/
8+
9+
# Profiling/tool outputs
10+
rocprof/
11+
rocprofatt/
12+
.rocprofv3/
13+
14+
# Local environments
15+
venv/

0 commit comments

Comments
 (0)