Skip to content

Commit eebca95

Browse files
CypherPotatoCypherPotato
committed
update benchmarks
1 parent ed69959 commit eebca95

File tree

8 files changed

+81
-96
lines changed

8 files changed

+81
-96
lines changed

‎.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
## Get latest from https://github.com/github/gitignore/blob/main/VisualStudio.gitignore
55

66
# User-specific files
7+
*.sln
78
*.rsuser
89
*.suo
910
*.user

‎README.md

+49-40
Original file line numberDiff line numberDiff line change
@@ -4,23 +4,32 @@ The tests below were performed to improve the quality of Sisk. All tests are rep
44

55
Both applications were compiled in RELEASE and return a plain "Hello, world!". The K6 version used in these tests was v0.46.0.
66

7-
Sisk could handle 55.462 requests per second while ASP.NET got 63.728, using their Kestrel socket implementation. Sisk does uses HttpListener.
7+
Sisk could handle 80.850,89 requests per second while ASP.NET got 87.962,17, using their Kestrel socket implementation. An difference of ~8,79%. Sisk does uses HttpListener.
88

99
## Benchmark.NET
1010

1111
```
12-
BenchmarkDotNet v0.13.7, Windows 10 (10.0.19044.4291/21H2/November2021Update)
13-
Intel Core i3-10105F CPU 3.70GHz, 1 CPU, 8 logical and 4 physical cores
14-
.NET SDK 9.0.100-preview.3.24204.13
15-
[Host] : .NET 8.0.4 (8.0.424.16909), X64 RyuJIT AVX2
16-
DefaultJob : .NET 8.0.4 (8.0.424.16909), X64 RyuJIT AVX2
12+
// * Summary *
1713
14+
BenchmarkDotNet v0.13.12, Windows 10 (10.0.19044.4529/21H2/November2021Update)
15+
11th Gen Intel Core i5-11400F 2.60GHz, 1 CPU, 12 logical and 6 physical cores
16+
.NET SDK 9.0.100-preview.4.24267.66
17+
[Host] : .NET 8.0.5 (8.0.524.21615), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI [AttachedDebugger]
18+
DefaultJob : .NET 8.0.5 (8.0.524.21615), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
1819
19-
| Mean | StdDev | StdErr | Op/s | Framework |
20-
|---------:|---------:|---------:|---------:|-------------- |
21-
| 52.71 us | 1.754 us | 0.285 us | 18,970.8 | AspNetMinimal |
22-
| 63.14 us | 0.187 us | 0.050 us | 15,838.4 | SiskDotNet |
2320
21+
| Mean | StdDev | StdErr | Op/s | Framework |
22+
|---------:|---------:|---------:|---------:|-------------- |
23+
| 59.49 us | 1.355 us | 0.303 us | 16,809.0 | AspNetMinimal |
24+
| 68.19 us | 1.195 us | 0.309 us | 14,664.2 | SiskDotNet |
25+
26+
// * Legends *
27+
Mean : Arithmetic mean of all measurements
28+
StdDev : Standard deviation of all measurements
29+
StdErr : Standard error of all measurements
30+
Op/s : Operation per second
31+
Framework : Custom 'Framework' tag column
32+
1 us : 1 Microsecond (0.000001 sec)
2433
```
2534

2635
## K6
@@ -29,27 +38,27 @@ Sisk:
2938

3039
```
3140
execution: local
32-
script: k6-sisk.js
41+
script: sisk-test.k6.js
3342
output: -
3443
3544
scenarios: (100.00%) 1 scenario, 500 max VUs, 1m30s max duration (incl. graceful stop):
3645
* default: 500 looping VUs for 1m0s (gracefulStop: 30s)
3746
3847
39-
data_received..................: 630 MB 11 MB/s
40-
data_sent......................: 271 MB 4.5 MB/s
41-
http_req_blocked...............: avg=1.76µs min=0s med=0s max=26.63ms p(90)=0s p(95)=0s
42-
http_req_connecting............: avg=109ns min=0s med=0s max=4.98ms p(90)=0s p(95)=0s
43-
http_req_duration..............: avg=8.79ms min=0s med=8.1ms max=237ms p(90)=11.96ms p(95)=13.89ms
44-
{ expected_response:true }...: avg=8.79ms min=0s med=8.1ms max=237ms p(90)=11.96ms p(95)=13.89ms
45-
http_req_failed................: 0.00% ✓ 0 ✗ 3388041
46-
http_req_receiving.............: avg=264.8µs min=0s med=0s max=50.39ms p(90)=0s p(95)=0s
47-
http_req_sending...............: avg=11.59µs min=0s med=0s max=65.94ms p(90)=0s p(95)=0s
48-
http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
49-
http_req_waiting...............: avg=8.51ms min=0s med=8.06ms max=234.01ms p(90)=11.96ms p(95)=13.78ms
50-
http_reqs......................: 3388041 56462.140897/s
51-
iteration_duration.............: avg=8.84ms min=0s med=8.14ms max=244.61ms p(90)=11.96ms p(95)=13.95ms
52-
iterations.....................: 3388041 56462.140897/s
48+
data_received..................: 898 MB 15 MB/s
49+
data_sent......................: 388 MB 6.5 MB/s
50+
http_req_blocked...............: avg=2.05µs min=0s med=0s max=35.84ms p(90)=0s p(95)=0s
51+
http_req_connecting............: avg=376ns min=0s med=0s max=35.84ms p(90)=0s p(95)=0s
52+
http_req_duration..............: avg=6.12ms min=0s med=5.99ms max=60.38ms p(90)=8.35ms p(95)=9.87ms
53+
{ expected_response:true }...: avg=6.12ms min=0s med=5.99ms max=60.38ms p(90)=8.35ms p(95)=9.87ms
54+
http_req_failed................: 0.00% ✓ 0 ✗ 4851382
55+
http_req_receiving.............: avg=159.91µs min=0s med=0s max=38.88ms p(90)=0s p(95)=0s
56+
http_req_sending...............: avg=12.32µs min=0s med=0s max=40.99ms p(90)=0s p(95)=0s
57+
http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
58+
http_req_waiting...............: avg=5.95ms min=0s med=5.99ms max=37.02ms p(90)=8.22ms p(95)=9.6ms
59+
http_reqs......................: 4851382 80850.894543/s
60+
iteration_duration.............: avg=6.17ms min=0s med=5.99ms max=60.38ms p(90)=8.51ms p(95)=9.99ms
61+
iterations.....................: 4851382 80850.894543/s
5362
vus............................: 500 min=500 max=500
5463
vus_max........................: 500 min=500 max=500
5564
```
@@ -58,26 +67,26 @@ ASP.NET Minimal:
5867

5968
```
6069
execution: local
61-
script: k6-asp.js
70+
script: asp-test.k6.js
6271
output: -
6372
6473
scenarios: (100.00%) 1 scenario, 500 max VUs, 1m30s max duration (incl. graceful stop):
6574
* default: 500 looping VUs for 1m0s (gracefulStop: 30s)
6675
67-
data_received..................: 627 MB 10 MB/s
68-
data_sent......................: 306 MB 5.1 MB/s
69-
http_req_blocked...............: avg=2.44µs min=0s med=0s max=110.6ms p(90)=0s p(95)=0s
70-
http_req_connecting............: avg=109ns min=0s med=0s max=9.62ms p(90)=0s p(95)=0s
71-
http_req_duration..............: avg=7.61ms min=0s med=6.98ms max=269.14ms p(90)=9.25ms p(95)=11.12ms
72-
{ expected_response:true }...: avg=7.61ms min=0s med=6.98ms max=269.14ms p(90)=9.25ms p(95)=11.13ms
73-
http_req_failed................: 0.03% ✓ 1367 ✗ 3822574
74-
http_req_receiving.............: avg=316.12µs min=0s med=0s max=150.44ms p(90)=0s p(95)=0s
75-
http_req_sending...............: avg=16.03µs min=0s med=0s max=135.54ms p(90)=0s p(95)=0s
76-
http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
77-
http_req_waiting...............: avg=7.28ms min=0s med=6.98ms max=269.14ms p(90)=9.15ms p(95)=10.97ms
78-
http_reqs......................: 3823941 63728.188762/s
79-
iteration_duration.............: avg=7.8ms min=0s med=6.98ms max=276.77ms p(90)=9.65ms p(95)=11.7ms
80-
iterations.....................: 3823941 63728.188762/s
76+
data_received..................: 866 MB 14 MB/s
77+
data_sent......................: 422 MB 7.0 MB/s
78+
http_req_blocked...............: avg=2.87µs min=0s med=0s max=66ms p(90)=0s p(95)=0s
79+
http_req_connecting............: avg=132ns min=0s med=0s max=16.15ms p(90)=0s p(95)=0s
80+
http_req_duration..............: avg=5.45ms min=0s med=5ms max=98.68ms p(90)=6.99ms p(95)=8ms
81+
{ expected_response:true }...: avg=5.45ms min=0s med=5ms max=98.68ms p(90)=6.99ms p(95)=8ms
82+
http_req_failed................: 0.00% ✓ 182 ✗ 5277770
83+
http_req_receiving.............: avg=258.83µs min=0s med=0s max=88.2ms p(90)=0s p(95)=0s
84+
http_req_sending...............: avg=17.84µs min=0s med=0s max=87ms p(90)=0s p(95)=0s
85+
http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
86+
http_req_waiting...............: avg=5.17ms min=0s med=5ms max=98.18ms p(90)=6.99ms p(95)=7.99ms
87+
http_reqs......................: 5277952 87962.173721/s
88+
iteration_duration.............: avg=5.62ms min=0s med=5ms max=549.06ms p(90)=7ms p(95)=8.19ms
89+
iterations.....................: 5277952 87962.173721/s
8190
vus............................: 500 min=500 max=500
8291
vus_max........................: 500 min=500 max=500
8392
```

‎asp/asp.csproj

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
<Project Sdk="Microsoft.NET.Sdk.Web">
1+
<Project Sdk="Microsoft.NET.Sdk.Web">
22

33
<PropertyGroup>
44
<TargetFramework>net8.0</TargetFramework>

‎k6-scripts/asp-test.k6.js

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
import http from 'k6/http';
2+
import { sleep } from 'k6';
3+
export const options = {
4+
vus: 500,
5+
duration: '60s',
6+
};
7+
export default function () {
8+
http.get('http://localhost:5000/');
9+
}

‎k6-scripts/sisk-test.k6.js

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
import http from 'k6/http';
2+
import { sleep } from 'k6';
3+
export const options = {
4+
vus: 500,
5+
duration: '60s',
6+
};
7+
export default function () {
8+
http.get('http://localhost:5222/');
9+
}

‎sisk-benchmark.sln

-43
This file was deleted.

‎sisk-benchmark/sisk-benchmark.csproj

+11-11
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,15 @@
1-
<Project Sdk="Microsoft.NET.Sdk">
1+
<Project Sdk="Microsoft.NET.Sdk">
22

3-
<PropertyGroup>
4-
<OutputType>Exe</OutputType>
5-
<TargetFramework>net8.0</TargetFramework>
6-
<RootNamespace>sisk_benchmark</RootNamespace>
7-
<ImplicitUsings>enable</ImplicitUsings>
8-
<Nullable>enable</Nullable>
9-
</PropertyGroup>
3+
<PropertyGroup>
4+
<OutputType>Exe</OutputType>
5+
<TargetFramework>net8.0</TargetFramework>
6+
<RootNamespace>sisk_benchmark</RootNamespace>
7+
<ImplicitUsings>enable</ImplicitUsings>
8+
<Nullable>enable</Nullable>
9+
</PropertyGroup>
1010

11-
<ItemGroup>
12-
<PackageReference Include="BenchmarkDotNet" Version="0.13.7" />
13-
</ItemGroup>
11+
<ItemGroup>
12+
<PackageReference Include="BenchmarkDotNet" Version="0.13.12" />
13+
</ItemGroup>
1414

1515
</Project>

‎sisk/Program.cs

+1-1
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ class Program
55
{
66
static void Main(string[] args)
77
{
8-
var app = HttpServer.CreateBuilder(host => host.UseListeningPort(5222));
8+
var app = HttpServer.CreateBuilder(host => host.UseListeningPort(5222)).Build();
99

1010
app.Router.SetRoute(RouteMethod.Get, "/", req => new HttpResponse("Hello, world!"));
1111

0 commit comments

Comments
 (0)