Skip to content

Commit eff2557

Browse files
lemirelemire
committed
Saving...
1 parent 112963c commit eff2557

File tree

4 files changed

+156
-10
lines changed

4 files changed

+156
-10
lines changed

‎README.md

+19-6
Original file line numberDiff line numberDiff line change
@@ -15,10 +15,11 @@ We do not need report the timings (ns) for precision reasons. Only the number of
1515

1616
## Systems
1717

18-
The rome machine refers to an AMD Rome server (AMD EPYC 7262 8-Core Processor) running Ubuntu 11.3.
18+
The rome machine refers to an AMD Rome server (AMD EPYC 7262 8-Core Processor) running Ubuntu with GCC 11.3. The m2 machine refers to a MacBook Air 2022 with LLVM 14.
1919

2020
## Python scripts
2121

22+
2223
We require a recent Python 3 interpreter. The Python scripts require pandas, numpy, sklearn, seaborn and matplotlib.
2324
You may be able to install these dependencies with `pip` or `pip3` as follows:
2425

@@ -28,15 +29,27 @@ pip install pandas matplotlib scikit-learn numpy seaborn
2829

2930
(Replace `pip` with `pip3` if needed.)
3031

31-
Example:
32+
33+
### Overall data (Google benchmark)
34+
35+
36+
The overall data comes from various datasets and is produced by the google-benchmark executable `benchdata` from the main `ada-url/ada` repository.
37+
38+
You can process the overall data and generate the tables and figures.
3239

3340
```
34-
python scripts/regress.py big_url_set/ada_url_aggregator_rome.txt
41+
python scripts/overall.py
3542
```
3643
(replace python by `python3` if needed)
3744

3845

39-
You can process the over data.
46+
### jsurl
47+
48+
The jsurl data refers to https://github.com/ada-url/js_url_benchmark
49+
50+
51+
4052
```
41-
python scripts/overall.py
42-
```
53+
python scripts/jsurl.py
54+
```
55+
(replace python by `python3` if needed)

‎jsurl/m2.txt

+99
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
Now using node v20.1.0 (npm v9.6.4)
2+
cpu: unknown
3+
runtime: deno 1.32.5 (aarch64-apple-darwin)
4+
5+
benchmark time (avg) (min … max) p75 p99 p995
6+
------------------------------------------------------------------- -----------------------------
7+
fixtures/linux_files.txt 141.17 ms/iter (137.75 ms … 153.24 ms) 138.77 ms 153.24 ms 153.24 ms
8+
fixtures/kasztp.txt 35.54 ms/iter (35.32 ms … 35.74 ms) 35.62 ms 35.74 ms 35.74 ms
9+
fixtures/userbait.txt 13.06 ms/iter (12.98 ms … 13.33 ms) 13.05 ms 13.33 ms 13.33 ms
10+
fixtures/top100.txt 102 ms/iter (101.6 ms … 103.83 ms) 102 ms 103.83 ms 103.83 ms
11+
fixtures/wikipedia_100k.txt 68.22 ms/iter (67.84 ms … 68.86 ms) 68.3 ms 68.86 ms 68.86 ms
12+
Average URL size: 61 bytes
13+
Ratio of bad URLs: 0.02%
14+
15+
cpu: Apple M2
16+
runtime: bun 0.5.9 (arm64-darwin)
17+
18+
benchmark time (avg) (min … max) p75 p99 p995
19+
------------------------------------------------------------------- -----------------------------
20+
fixtures/linux_files.txt 54.51 ms/iter (52.32 ms … 60.04 ms) 53.74 ms 60.04 ms 60.04 ms
21+
fixtures/kasztp.txt 23.52 ms/iter (23.06 ms … 30.56 ms) 23.34 ms 30.56 ms 30.56 ms
22+
fixtures/userbait.txt 7.02 ms/iter (6.87 ms … 13.49 ms) 6.96 ms 13.49 ms 13.49 ms
23+
fixtures/top100.txt 66.9 ms/iter (66.03 ms … 72.72 ms) 66.37 ms 72.72 ms 72.72 ms
24+
fixtures/wikipedia_100k.txt 34.78 ms/iter (33.7 ms … 40.65 ms) 34.24 ms 40.65 ms 40.65 ms
25+
Average URL size: 61 bytes
26+
Ratio of bad URLs: 0.03%
27+
28+
Now using node v16.20.0 (npm v8.19.4)
29+
cpu: Apple M2
30+
runtime: node v16.20.0 (arm64-darwin)
31+
32+
benchmark time (avg) (min … max) p75 p99 p995
33+
------------------------------------------------------------------- -----------------------------
34+
fixtures/linux_files.txt 171.61 ms/iter (170.47 ms … 172.97 ms) 172.01 ms 172.97 ms 172.97 ms
35+
fixtures/kasztp.txt 56.18 ms/iter (55.69 ms … 56.74 ms) 56.37 ms 56.74 ms 56.74 ms
36+
fixtures/userbait.txt 16.76 ms/iter (16.44 ms … 16.96 ms) 16.81 ms 16.96 ms 16.96 ms
37+
fixtures/top100.txt 147.05 ms/iter (146.59 ms … 147.54 ms) 147.22 ms 147.54 ms 147.54 ms
38+
fixtures/wikipedia_100k.txt 105.13 ms/iter (104.77 ms … 105.48 ms) 105.36 ms 105.48 ms 105.48 ms
39+
Average URL size: 62 bytes
40+
Ratio of bad URLs: 0.02%
41+
42+
Now using node v17.9.1 (npm v8.11.0)
43+
cpu: Apple M2
44+
runtime: node v17.9.1 (arm64-darwin)
45+
46+
benchmark time (avg) (min … max) p75 p99 p995
47+
------------------------------------------------------------------- -----------------------------
48+
fixtures/linux_files.txt 168.84 ms/iter (168.48 ms … 169.35 ms) 168.92 ms 169.35 ms 169.35 ms
49+
fixtures/kasztp.txt 47.57 ms/iter (47.3 ms … 47.77 ms) 47.65 ms 47.77 ms 47.77 ms
50+
fixtures/userbait.txt 14.42 ms/iter (14.31 ms … 14.7 ms) 14.44 ms 14.7 ms 14.7 ms
51+
fixtures/top100.txt 131.56 ms/iter (130.5 ms … 135.62 ms) 131.9 ms 135.62 ms 135.62 ms
52+
fixtures/wikipedia_100k.txt 89.13 ms/iter (87.89 ms … 90.31 ms) 89.1 ms 90.31 ms 90.31 ms
53+
Average URL size: 61 bytes
54+
Ratio of bad URLs: 0.02%
55+
56+
Now using node v18.15.0 (npm v9.5.0)
57+
cpu: Apple M2
58+
runtime: node v18.15.0 (arm64-darwin)
59+
60+
benchmark time (avg) (min … max) p75 p99 p995
61+
------------------------------------------------------------------- -----------------------------
62+
fixtures/linux_files.txt 159.45 ms/iter (158.64 ms … 161.12 ms) 159.75 ms 161.12 ms 161.12 ms
63+
fixtures/kasztp.txt 46.55 ms/iter (45.97 ms … 47.71 ms) 46.67 ms 47.71 ms 47.71 ms
64+
fixtures/userbait.txt 14.06 ms/iter (13.91 ms … 14.23 ms) 14.11 ms 14.23 ms 14.23 ms
65+
fixtures/top100.txt 127.6 ms/iter (126.83 ms … 128.7 ms) 127.61 ms 128.7 ms 128.7 ms
66+
fixtures/wikipedia_100k.txt 86.21 ms/iter (85.76 ms … 87.97 ms) 86.24 ms 87.97 ms 87.97 ms
67+
Average URL size: 61 bytes
68+
Ratio of bad URLs: 0.02%
69+
70+
Now using node v18.16.0 (npm v9.5.1)
71+
cpu: Apple M2
72+
runtime: node v18.16.0 (arm64-darwin)
73+
74+
benchmark time (avg) (min … max) p75 p99 p995
75+
------------------------------------------------------------------- -----------------------------
76+
fixtures/linux_files.txt 108.57 ms/iter (108.36 ms … 108.9 ms) 108.6 ms 108.9 ms 108.9 ms
77+
fixtures/kasztp.txt 33.13 ms/iter (32.98 ms … 33.29 ms) 33.2 ms 33.29 ms 33.29 ms
78+
fixtures/userbait.txt 9.77 ms/iter (9.68 ms … 9.92 ms) 9.82 ms 9.92 ms 9.92 ms
79+
fixtures/top100.txt 86.38 ms/iter (85.55 ms … 87.73 ms) 86.42 ms 87.73 ms 87.73 ms
80+
fixtures/wikipedia_100k.txt 61.11 ms/iter (60.67 ms … 61.83 ms) 61.16 ms 61.83 ms 61.83 ms
81+
Average URL size: 61 bytes
82+
Ratio of bad URLs: 0.02%
83+
84+
Now using node v20.1.0 (npm v9.6.4)
85+
cpu: Apple M2
86+
runtime: node v20.1.0 (arm64-darwin)
87+
88+
benchmark time (avg) (min … max) p75 p99 p995
89+
------------------------------------------------------------------- -----------------------------
90+
fixtures/linux_files.txt 49.84 ms/iter (49.68 ms … 50.53 ms) 49.84 ms 50.53 ms 50.53 ms
91+
fixtures/kasztp.txt 14.27 ms/iter (14.15 ms … 14.38 ms) 14.3 ms 14.38 ms 14.38 ms
92+
fixtures/userbait.txt 4.02 ms/iter (3.9 ms … 4.38 ms) 4.07 ms 4.34 ms 4.38 ms
93+
fixtures/top100.txt 34.44 ms/iter (34.3 ms … 34.53 ms) 34.48 ms 34.53 ms 34.53 ms
94+
fixtures/wikipedia_100k.txt 26.7 ms/iter (26.44 ms … 26.9 ms) 26.82 ms 26.9 ms 26.9 ms
95+
Average URL size: 61 bytes
96+
Ratio of bad URLs: 0.03%
97+
98+
Restoring node version to v20.1.0
99+
Now using node v20.1.0 (npm v9.6.4)

‎jsurl/rome.txt

+38
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,41 @@
1+
runtime: deno 1.32.5
2+
3+
benchmark time (avg) (min … max) p75 p99 p995
4+
------------------------------------------------------------------- -----------------------------
5+
fixtures/linux_files.txt 178.48 ms/iter (177.19 ms … 181.27 ms) 178.93 ms 181.27 ms 181.27 ms
6+
fixtures/kasztp.txt 56.35 ms/iter (55.84 ms … 56.75 ms) 56.52 ms 56.75 ms 56.75 ms
7+
fixtures/userbait.txt 20.11 ms/iter (20.01 ms … 20.49 ms) 20.15 ms 20.49 ms 20.49 ms
8+
fixtures/top100.txt 151.64 ms/iter (150.42 ms … 152.82 ms) 152.08 ms 152.82 ms 152.82 ms
9+
fixtures/wikipedia_100k.txt 105.28 ms/iter (104.48 ms … 106.03 ms) 105.63 ms 106.03 ms 106.03 ms
10+
Average URL size: 62 bytes
11+
Ratio of bad URLs: 0.02%
12+
13+
cpu: AMD EPYC 7262 8-Core Processor
14+
runtime: bun 0.5.9 (x64-linux)
15+
16+
benchmark time (avg) (min … max) p75 p99 p995
17+
------------------------------------------------------------------- -----------------------------
18+
fixtures/linux_files.txt 94.01 ms/iter (89.26 ms … 105.37 ms) 93.6 ms 105.37 ms 105.37 ms
19+
fixtures/kasztp.txt 44.51 ms/iter (42.65 ms … 54.83 ms) 44.15 ms 54.83 ms 54.83 ms
20+
fixtures/userbait.txt 13.55 ms/iter (12.94 ms … 27.63 ms) 13.34 ms 27.63 ms 27.63 ms
21+
fixtures/top100.txt 126.37 ms/iter (124.24 ms … 137.77 ms) 126.28 ms 137.77 ms 137.77 ms
22+
fixtures/wikipedia_100k.txt 72.79 ms/iter (64.04 ms … 91.41 ms) 76.46 ms 91.41 ms 91.41 ms
23+
Average URL size: 61 bytes
24+
Ratio of bad URLs: 0.02%
25+
26+
Now using node v16.20.0 (npm v8.19.4)
27+
cpu: AMD EPYC 7262 8-Core Processor
28+
runtime: node v16.20.0 (x64-linux)
29+
30+
benchmark time (avg) (min … max) p75 p99 p995
31+
------------------------------------------------------------------- -----------------------------
32+
fixtures/linux_files.txt 632.24 ms/iter (629.75 ms … 635.23 ms) 633.13 ms 635.23 ms 635.23 ms
33+
fixtures/kasztp.txt 193.7 ms/iter (192.67 ms … 194.41 ms) 194.13 ms 194.41 ms 194.41 ms
34+
fixtures/userbait.txt 50.67 ms/iter (50.18 ms … 51.52 ms) 50.78 ms 51.52 ms 51.52 ms
35+
fixtures/top100.txt 479.62 ms/iter (474.31 ms … 481.64 ms) 480.56 ms 481.64 ms 481.64 ms
36+
fixtures/wikipedia_100k.txt 393.44 ms/iter (391.11 ms … 396.82 ms) 393.91 ms 396.82 ms 396.82 ms
37+
Average URL size: 62 bytes
38+
Ratio of bad URLs: 0.02%
139

240
Now using node v17.9.1 (npm v8.11.0)
341
cpu: AMD EPYC 7262 8-Core Processor

‎scripts/overall.py

-4
Original file line numberDiff line numberDiff line change
@@ -72,8 +72,6 @@ def round(x):
7272
print("{name} & {insperurl} & {cyclesperurl} & {inspercycle} \\\\".format(name = benchmark_name[1],insperurl= round(obj["instructions/url"]), cyclesperurl= round(obj["cycles/url"]), inspercycle = round(obj["instructions/cycle"])))
7373
print("\\bottomrule")
7474

75-
76-
7775
matplotlib.rcParams['font.family'] = 'serif'
7876
with PdfPages(maindirname+'.pdf') as pdf:
7977
x = np.arange(len(datasets)) # the label locations
@@ -85,10 +83,8 @@ def round(x):
8583
for attribute, measurement in speeds.items():
8684
offset = width * multiplier
8785
rects = ax.bar(x + offset, measurement, width, label=attribute)
88-
#ax.bar_label(rects, padding=3)
8986
multiplier += 1
9087

91-
# Add some text for labels, title and custom x-axis tick labels, etc.
9288
ax.set_ylabel('Millions of URLs per second')
9389
ax.set_xticks(x + width, datasets)
9490
ax.legend(loc='upper center', frameon=True, ncols=5, facecolor='white', framealpha=0.5, edgecolor='none', bbox_to_anchor=(0.5, 1.05))

0 commit comments

Comments
 (0)