Skip to content

Commit ed69959

Browse files
committed
update benchmarks
1 parent 3546fcf commit ed69959

File tree

4 files changed

+76
-48
lines changed

4 files changed

+76
-48
lines changed

README.md

+55-46
Original file line numberDiff line numberDiff line change
@@ -4,71 +4,80 @@ The tests below were performed to improve the quality of Sisk. All tests are rep
44

55
Both applications were compiled in RELEASE and return a plain "Hello, world!". The K6 version used in these tests was v0.46.0.
66

7-
Sisk could handle 44.029 requests per second while ASP.NET got 51.766, using their Kestrel socket implementation. Sisk does uses HttpListener. Sisk is around 15%-20% slower than ASP.NET/Kestrel. We need to improve it.
7+
Sisk could handle 55.462 requests per second while ASP.NET got 63.728, using their Kestrel socket implementation. Sisk does uses HttpListener.
88

99
## Benchmark.NET
1010

1111
```
12-
BenchmarkDotNet v0.13.7, Windows 11 (10.0.22631.2861)
12+
BenchmarkDotNet v0.13.7, Windows 10 (10.0.19044.4291/21H2/November2021Update)
1313
Intel Core i3-10105F CPU 3.70GHz, 1 CPU, 8 logical and 4 physical cores
14-
.NET SDK 8.0.100
15-
[Host] : .NET 8.0.0 (8.0.23.53103), X64 RyuJIT AVX2
16-
DefaultJob : .NET 8.0.0 (8.0.23.53103), X64 RyuJIT AVX2
17-
```
14+
.NET SDK 9.0.100-preview.3.24204.13
15+
[Host] : .NET 8.0.4 (8.0.424.16909), X64 RyuJIT AVX2
16+
DefaultJob : .NET 8.0.4 (8.0.424.16909), X64 RyuJIT AVX2
17+
1818
19-
| Framework | Mean | StdDev | StdErr | Op/s |
20-
|-------------- |----------:|---------:|---------:|---------:|
21-
| AspNetMinimal | 96.64 us | 0.380 us | 0.095 us | 10,348.1 |
22-
| SiskDotNet | 110.36 us | 0.383 us | 0.099 us | 9,061.4 |
19+
| Mean | StdDev | StdErr | Op/s | Framework |
20+
|---------:|---------:|---------:|---------:|-------------- |
21+
| 52.71 us | 1.754 us | 0.285 us | 18,970.8 | AspNetMinimal |
22+
| 63.14 us | 0.187 us | 0.050 us | 15,838.4 | SiskDotNet |
23+
24+
```
2325

2426
## K6
2527

2628
Sisk:
2729

2830
```
29-
scenarios: (100.00%) 1 scenario, 500 max VUs, 1m0s max duration (incl. graceful stop):
30-
* default: 500 looping VUs for 30s (gracefulStop: 30s)
31-
32-
33-
data_received..................: 246 MB 8.2 MB/s
34-
data_sent......................: 106 MB 3.5 MB/s
35-
http_req_blocked...............: avg=7.84µs min=0s med=0s max=70.35ms p(90)=0s p(95)=0s
36-
http_req_connecting............: avg=3.88µs min=0s med=0s max=22.57ms p(90)=0s p(95)=0s
37-
http_req_duration..............: avg=11.18ms min=0s med=10.53ms max=111.62ms p(90)=15.34ms p(95)=18.64ms
38-
{ expected_response:true }...: avg=11.18ms min=0s med=10.53ms max=111.62ms p(90)=15.34ms p(95)=18.64ms
39-
http_req_failed................: 0.00% ✓ 0 ✗ 1321362
40-
http_req_receiving.............: avg=41.89µs min=0s med=0s max=93.87ms p(90)=0s p(95)=0s
41-
http_req_sending...............: avg=15.95µs min=0s med=0s max=88.68ms p(90)=0s p(95)=0s
42-
http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
43-
http_req_waiting...............: avg=11.12ms min=0s med=10.52ms max=81.43ms p(90)=15.27ms p(95)=18.51ms
44-
http_reqs......................: 1321362 44029.579878/s
45-
iteration_duration.............: avg=11.31ms min=0s med=10.58ms max=111.62ms p(90)=15.55ms p(95)=19.48ms
46-
iterations.....................: 1321362 44029.579878/s
31+
execution: local
32+
script: k6-sisk.js
33+
output: -
34+
35+
scenarios: (100.00%) 1 scenario, 500 max VUs, 1m30s max duration (incl. graceful stop):
36+
* default: 500 looping VUs for 1m0s (gracefulStop: 30s)
37+
38+
39+
data_received..................: 630 MB 11 MB/s
40+
data_sent......................: 271 MB 4.5 MB/s
41+
http_req_blocked...............: avg=1.76µs min=0s med=0s max=26.63ms p(90)=0s p(95)=0s
42+
http_req_connecting............: avg=109ns min=0s med=0s max=4.98ms p(90)=0s p(95)=0s
43+
http_req_duration..............: avg=8.79ms min=0s med=8.1ms max=237ms p(90)=11.96ms p(95)=13.89ms
44+
{ expected_response:true }...: avg=8.79ms min=0s med=8.1ms max=237ms p(90)=11.96ms p(95)=13.89ms
45+
http_req_failed................: 0.00% ✓ 0 ✗ 3388041
46+
http_req_receiving.............: avg=264.8µs min=0s med=0s max=50.39ms p(90)=0s p(95)=0s
47+
http_req_sending...............: avg=11.59µs min=0s med=0s max=65.94ms p(90)=0s p(95)=0s
48+
http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
49+
http_req_waiting...............: avg=8.51ms min=0s med=8.06ms max=234.01ms p(90)=11.96ms p(95)=13.78ms
50+
http_reqs......................: 3388041 56462.140897/s
51+
iteration_duration.............: avg=8.84ms min=0s med=8.14ms max=244.61ms p(90)=11.96ms p(95)=13.95ms
52+
iterations.....................: 3388041 56462.140897/s
4753
vus............................: 500 min=500 max=500
4854
vus_max........................: 500 min=500 max=500
4955
```
5056

5157
ASP.NET Minimal:
5258

5359
```
54-
scenarios: (100.00%) 1 scenario, 500 max VUs, 1m0s max duration (incl. graceful stop):
55-
* default: 500 looping VUs for 30s (gracefulStop: 30s)
56-
57-
58-
data_received..................: 255 MB 8.5 MB/s
59-
data_sent......................: 124 MB 4.1 MB/s
60-
http_req_blocked...............: avg=105.19µs min=0s med=0s max=537.38ms p(90)=0s p(95)=0s
61-
http_req_connecting............: avg=97.47µs min=0s med=0s max=530.34ms p(90)=0s p(95)=0s
62-
http_req_duration..............: avg=9.03ms min=0s med=8.52ms max=306.34ms p(90)=11.24ms p(95)=14.81ms
63-
{ expected_response:true }...: avg=9.03ms min=0s med=8.52ms max=306.34ms p(90)=11.24ms p(95)=14.81ms
64-
http_req_failed................: 0.00% ✓ 0 ✗ 1553207
65-
http_req_receiving.............: avg=48.53µs min=0s med=0s max=296.69ms p(90)=0s p(95)=0s
66-
http_req_sending...............: avg=20.31µs min=0s med=0s max=86.06ms p(90)=0s p(95)=0s
67-
http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
68-
http_req_waiting...............: avg=8.96ms min=0s med=8.52ms max=196.61ms p(90)=11.14ms p(95)=14.49ms
69-
http_reqs......................: 1553207 51766.383908/s
70-
iteration_duration.............: avg=9.56ms min=0s med=8.52ms max=566.56ms p(90)=11.68ms p(95)=16.93ms
71-
iterations.....................: 1553207 51766.383908/s
60+
execution: local
61+
script: k6-asp.js
62+
output: -
63+
64+
scenarios: (100.00%) 1 scenario, 500 max VUs, 1m30s max duration (incl. graceful stop):
65+
* default: 500 looping VUs for 1m0s (gracefulStop: 30s)
66+
67+
data_received..................: 627 MB 10 MB/s
68+
data_sent......................: 306 MB 5.1 MB/s
69+
http_req_blocked...............: avg=2.44µs min=0s med=0s max=110.6ms p(90)=0s p(95)=0s
70+
http_req_connecting............: avg=109ns min=0s med=0s max=9.62ms p(90)=0s p(95)=0s
71+
http_req_duration..............: avg=7.61ms min=0s med=6.98ms max=269.14ms p(90)=9.25ms p(95)=11.12ms
72+
{ expected_response:true }...: avg=7.61ms min=0s med=6.98ms max=269.14ms p(90)=9.25ms p(95)=11.13ms
73+
http_req_failed................: 0.03% ✓ 1367 ✗ 3822574
74+
http_req_receiving.............: avg=316.12µs min=0s med=0s max=150.44ms p(90)=0s p(95)=0s
75+
http_req_sending...............: avg=16.03µs min=0s med=0s max=135.54ms p(90)=0s p(95)=0s
76+
http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
77+
http_req_waiting...............: avg=7.28ms min=0s med=6.98ms max=269.14ms p(90)=9.15ms p(95)=10.97ms
78+
http_reqs......................: 3823941 63728.188762/s
79+
iteration_duration.............: avg=7.8ms min=0s med=6.98ms max=276.77ms p(90)=9.65ms p(95)=11.7ms
80+
iterations.....................: 3823941 63728.188762/s
7281
vus............................: 500 min=500 max=500
7382
vus_max........................: 500 min=500 max=500
7483
```

sisk-benchmark/Program.cs

+19
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
using BenchmarkDotNet.Loggers;
77
using BenchmarkDotNet.Running;
88
using Microsoft.Extensions.Logging;
9+
using System.Diagnostics;
910

1011
namespace sisk_benchmark;
1112

@@ -16,6 +17,24 @@ private static void Main()
1617
BenchmarkRunner.Run<Test1>(new Config());
1718
Console.ReadKey();
1819
}
20+
21+
private static void RunRace(string label, Func<Task> action)
22+
{
23+
double max = TimeSpan.FromSeconds(10).TotalMilliseconds;
24+
int count = 0;
25+
Stopwatch sw = new Stopwatch();
26+
sw.Start();
27+
var raceTask = Task.Run(async () =>
28+
{
29+
while (sw.ElapsedMilliseconds < max)
30+
{
31+
await action();
32+
count++;
33+
}
34+
});
35+
raceTask.Wait();
36+
Console.WriteLine($"Race \"{label}\" result in 10 seconds: {count} iterations");
37+
}
1938
}
2039

2140
class Config : ManualConfig

sisk/Program.cs

+1-1
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ static void Main(string[] args)
77
{
88
var app = HttpServer.CreateBuilder(host => host.UseListeningPort(5222));
99

10-
app.Router.SetRoute(RouteMethod.Get, "/", req => new HttpResponse().WithContent("Hello, world!"));
10+
app.Router.SetRoute(RouteMethod.Get, "/", req => new HttpResponse("Hello, world!"));
1111

1212
app.Start();
1313
}

sisk/sisk.csproj

+1-1
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
</PropertyGroup>
1010

1111
<ItemGroup>
12-
<ProjectReference Include="..\..\core\src\Sisk.Core.csproj" />
12+
<ProjectReference Include="..\..\core\src\Sisk.Core.csproj" />
1313
</ItemGroup>
1414

1515
</Project>

0 commit comments

Comments
 (0)