Skip to content

Commit 83632c0

Browse files
authored
[lld][BP] Order .Tgm symbols for startup (#126328)
The Global Function Merger (https://discourse.llvm.org/t/rfc-global-function-merging/82608) pass optimistically creates merged instances of functions and suffixes their names with `.Tgm`. Then in the linker, ICF will (hopefully) fold these `.Tgm` functions. For example, a function `foo` might become a thunk `foo` that calls a merged function `foo.Tgm`. Since IRPGO runs before the global merger, we will only have a profile for `foo`. We want to correlate this profile to both `foo` and `foo.Tgm` so they can both be ordered to improve startup time. I built a large binary and found that it increased the number of functions ordered for startup, as expected. ``` Functions for startup: 12049 -> 12697 Functions for compression: 34733 -> 34707 ``` The reason why we don't see a larger improvement is because there are some cases where the code was accidentally working: `getRootSymbol("foo.llvm.5555.Tgm")` already returns `foo`.
1 parent b5aa1c4 commit 83632c0

File tree

2 files changed

+32
-10
lines changed

2 files changed

+32
-10
lines changed

lld/include/lld/Common/BPSectionOrdererBase.inc

+7-4
Original file line numberDiff line numberDiff line change
@@ -147,11 +147,14 @@ static SmallVector<std::pair<unsigned, UtilityNodes>> getUnsForCompression(
147147
return sectionUns;
148148
}
149149

150-
/// Symbols can be appended with "(.__uniq.xxxx)?.llvm.yyyy" where "xxxx" and
151-
/// "yyyy" are numbers that could change between builds. We need to use the
152-
/// root symbol name before this suffix so these symbols can be matched with
153-
/// profiles which may have different suffixes.
150+
/// Symbols can be appended with "(.__uniq.xxxx)?(.llvm.yyyy)?(.Tgm)?" where
151+
/// "xxxx" and "yyyy" are numbers that could change between builds, and .Tgm is
152+
/// the global merge functions suffix
153+
/// (see GlobalMergeFunc::MergingInstanceSuffix). We need to use the root symbol
154+
/// name before this suffix so these symbols can be matched with profiles which
155+
/// may have different suffixes.
154156
inline StringRef getRootSymbol(StringRef name) {
157+
name.consume_back(".Tgm");
155158
auto [P0, S0] = name.rsplit(".llvm.");
156159
auto [P1, S1] = P0.rsplit(".__uniq.");
157160
return P1;

lld/test/MachO/bp-section-orderer.s

+25-6
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,12 @@
55
# RUN: llvm-profdata merge %t/a.proftext -o %t/a.profdata
66

77
# RUN: %no-fatal-warnings-lld -arch arm64 -lSystem -e _main -o %t/a.out %t/a.o --irpgo-profile-sort=%t/a.profdata --verbose-bp-section-orderer 2>&1 | FileCheck %s --check-prefix=STARTUP
8-
# RUN: %no-fatal-warnings-lld -arch arm64 -lSystem -e _main -o %t/a.out %t/a.o --irpgo-profile-sort=%t/a.profdata --verbose-bp-section-orderer --icf=all --compression-sort=none 2>&1 | FileCheck %s --check-prefix=STARTUP
8+
# RUN: %no-fatal-warnings-lld -arch arm64 -lSystem -e _main -o %t/a.out %t/a.o --irpgo-profile-sort=%t/a.profdata --verbose-bp-section-orderer --icf=all --compression-sort=none 2>&1 | FileCheck %s --check-prefix=STARTUP-ICF
99

1010
# RUN: %lld -arch arm64 -lSystem -e _main -o %t/a.out %t/a.o --irpgo-profile %t/a.profdata --bp-startup-sort=function --verbose-bp-section-orderer 2>&1 | FileCheck %s --check-prefix=STARTUP
11-
# RUN: %lld -arch arm64 -lSystem -e _main -o %t/a.out %t/a.o --irpgo-profile=%t/a.profdata --bp-startup-sort=function --verbose-bp-section-orderer --icf=all --bp-compression-sort=none 2>&1 | FileCheck %s --check-prefix=STARTUP
12-
# STARTUP: Ordered 3 sections using balanced partitioning
11+
# RUN: %lld -arch arm64 -lSystem -e _main -o %t/a.out %t/a.o --irpgo-profile=%t/a.profdata --bp-startup-sort=function --verbose-bp-section-orderer --icf=all --bp-compression-sort=none 2>&1 | FileCheck %s --check-prefix=STARTUP-ICF
12+
# STARTUP: Ordered 5 sections using balanced partitioning
13+
# STARTUP-ICF: Ordered 4 sections using balanced partitioning
1314

1415
# Check that orderfiles take precedence over BP
1516
# RUN: %no-fatal-warnings-lld -arch arm64 -lSystem -e _main -o - %t/a.o -order_file %t/a.orderfile --irpgo-profile-sort=%t/a.profdata | llvm-nm --numeric-sort --format=just-symbols - | FileCheck %s --check-prefix=ORDERFILE
@@ -26,6 +27,8 @@
2627
# ORDERFILE-DAG: _main
2728
# ORDERFILE-DAG: _B
2829
# ORDERFILE-DAG: l_C
30+
# ORDERFILE-DAG: merged1.Tgm
31+
# ORDERFILE-DAG: merged2.Tgm
2932

3033
# Data
3134
# ORDERFILE: s3
@@ -80,10 +83,10 @@ F:
8083
add w0, w0, #3
8184
bl l_C.__uniq.111111111111111111111111111111111111111.llvm.2222222222222222222
8285
ret
83-
merged1:
86+
merged1.Tgm:
8487
add w0, w0, #101
8588
ret
86-
merged2:
89+
merged2.Tgm:
8790
add w0, w0, #101
8891
ret
8992

@@ -114,7 +117,7 @@ r4:
114117
1
115118
# Weight
116119
1
117-
A, B, C.__uniq.555555555555555555555555555555555555555.llvm.6666666666666666666
120+
A, B, C.__uniq.555555555555555555555555555555555555555.llvm.6666666666666666666, merged1, merged2
118121

119122
A
120123
# Func Hash:
@@ -148,6 +151,22 @@ D
148151
# Counter Values:
149152
1
150153

154+
merged1
155+
# Func Hash:
156+
5555
157+
# Num Counters:
158+
1
159+
# Counter Values:
160+
1
161+
162+
merged2
163+
# Func Hash:
164+
6666
165+
# Num Counters:
166+
1
167+
# Counter Values:
168+
1
169+
151170
#--- a.orderfile
152171
A
153172
F

0 commit comments

Comments
 (0)