Skip to content

Commit 6877f55

Browse files
fcostaoliveirafcostaoliveira
committed
Add 8 comprehensive expiration benchmark tests with pipeline 10
- Test 1: SET EX 3 (50M keys) - Short TTL expiration performance - Test 2: SET EX 20 (150M keys) - Medium TTL expiration performance - Test 3: SET EX 10 with precondition (50M+50M keys) - Mixed TTL scenarios - Test 4: SET→SET→DEL EX 120 (50M keys) - Command throughput without expiration - Test 5: SET→SET→DEL EX 36000 (10M keys) - Long TTL command throughput - Test 6: SET EX random range 20sec-30days (50M keys) - Wide TTL distribution - Test 7: SET EX 10 years (50M keys) - Very long TTL performance - Test 8: Multi-client AWS test (20M+80M keys) - Multi-client expiration All tests use realistic pipeline 10 configuration and cover various expiration scenarios for ebuckets performance testing.
1 parent 4a005bf commit 6877f55

12 files changed

+319
-25
lines changed

‎pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[tool.poetry]
22
name = "redis-benchmarks-specification"
3-
version = "0.1.270"
3+
version = "0.1.272"
44
description = "The Redis benchmarks specification describes the cross-language/tools requirements and expectations to foster performance and observability standards around redis related technologies. Members from both industry and academia, including organizations and individuals are encouraged to contribute."
55
authors = ["filipecosta90 <[email protected]>","Redis Performance Group <[email protected]>"]
66
readme = "Readme.md"

‎redis_benchmarks_specification/setups/builders/gcc:10.5.0-amd64-debian-bullseye-redisearch.yml

Lines changed: 0 additions & 24 deletions
This file was deleted.

‎redis_benchmarks_specification/test-suites/memtier_benchmark-10Mkeys-string-set-update-del-ex-36000-pipeline-10.yml

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
version: 0.4
2+
name: memtier_benchmark-10Mkeys-string-set-update-del-ex-36000-pipeline-10
3+
description: Multi-step test - SET EX 36000 (10 hours), then update same keys with SET EX 36000, then DEL. Tests command throughput with long TTL without active expiration.
4+
dbconfig:
5+
configuration-parameters:
6+
save: '""'
7+
check:
8+
keyspacelen: 0
9+
resources:
10+
requests:
11+
memory: 5g
12+
tested-commands:
13+
- set
14+
- del
15+
tested-groups:
16+
- string
17+
- generic
18+
redis-topologies:
19+
- oss-standalone
20+
build-variants:
21+
- gcc:8.5.0-amd64-debian-buster-default
22+
- dockerhub
23+
clientconfig:
24+
run_image: redislabs/memtier_benchmark:edge
25+
tool: memtier_benchmark
26+
arguments: '--command "SET __key__ BBB EX 36000" --command "SET __key__ BBB EX 36000" --command "DEL __key__" -c 1 -t 1 --command-key-pattern=P --pipeline 10 --hide-histogram --key-maximum 10000000 -n allkeys'
27+
resources:
28+
requests:
29+
cpus: '2'
30+
memory: 2g
31+
32+
priority: 1

‎redis_benchmarks_specification/test-suites/memtier_benchmark-150Mkeys-string-set-ex-20-pipeline-10.yml

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
version: 0.4
2+
name: memtier_benchmark-150Mkeys-string-set-ex-20-pipeline-10
3+
description: Runs memtier_benchmark, for a keyspace of 150M keys with SET EX 20 commands and pipeline 10. Tests medium TTL expiration performance.
4+
dbconfig:
5+
configuration-parameters:
6+
save: '""'
7+
check:
8+
keyspacelen: 0
9+
resources:
10+
requests:
11+
memory: 20g
12+
tested-commands:
13+
- set
14+
tested-groups:
15+
- string
16+
redis-topologies:
17+
- oss-standalone
18+
build-variants:
19+
- gcc:8.5.0-amd64-debian-buster-default
20+
- dockerhub
21+
clientconfig:
22+
run_image: redislabs/memtier_benchmark:edge
23+
tool: memtier_benchmark
24+
arguments: '--command "SET __key__ BBB EX 20" -c 1 -t 1 --command-key-pattern=P --pipeline 10 --hide-histogram --key-maximum 150000000 -n allkeys'
25+
resources:
26+
requests:
27+
cpus: '2'
28+
memory: 2g
29+
30+
priority: 1

‎redis_benchmarks_specification/test-suites/memtier_benchmark-1Mkeys-generic-scan-cursor-count-5000-pipeline-10.yml

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
version: 0.4
2+
name: memtier_benchmark-1Mkeys-generic-scan-cursor-count-5000-pipeline-10
3+
description: Runs memtier_benchmark, for a keyspace length of 1M keys focusing on SCAN performance with CURSOR and large COUNT.
4+
dbconfig:
5+
configuration-parameters:
6+
save: '""'
7+
check:
8+
keyspacelen: 1000000
9+
preload_tool:
10+
run_image: redislabs/memtier_benchmark:edge
11+
tool: memtier_benchmark
12+
arguments: '"--pipeline" "100" "--data-size" "100" "--command" "HSET __key__ field __data__" "--command-key-pattern" "P" "-c" "50" "-t" "2" "--hide-histogram" "--key-minimum" "1"'
13+
resources:
14+
requests:
15+
memory: 1g
16+
tested-commands:
17+
- scan
18+
tested-groups:
19+
- generic
20+
redis-topologies:
21+
- oss-standalone
22+
build-variants:
23+
- gcc:8.5.0-amd64-debian-buster-default
24+
- dockerhub
25+
clientconfig:
26+
run_image: redislabs/memtier_benchmark:edge
27+
tool: memtier_benchmark
28+
arguments: --command "SCAN __key__ COUNT 5000" --key-prefix "" --distinct-client-seed --key-maximum 1000000 --pipeline 10 --command-key-pattern="R" -c 50 -t 2 --hide-histogram --test-time 180
29+
resources:
30+
requests:
31+
cpus: '2'
32+
memory: 2g
33+
34+
priority: 44

‎redis_benchmarks_specification/test-suites/memtier_benchmark-1key-zset-listpack-zrank-100-elements-pipeline-1.yml

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
version: 0.4
2+
name: memtier_benchmark-1key-zset-listpack-zrank-100-elements-pipeline-1
3+
description: Runs memtier_benchmark, for a keyspace length of 1 SORTED SET key. The SORTED SET contains 100 elements (listpack encoded) and we query it using ZRANK.
4+
dbconfig:
5+
configuration-parameters:
6+
save: '""'
7+
check:
8+
keyspacelen: 1
9+
init_commands:
10+
- '"ZADD" "zset:listpack:100" "1" "1" "2" "2" "3" "3" "4" "4" "5" "5" "6" "6" "7" "7" "8" "8" "9" "9" "10" "10" "11" "11" "12" "12" "13" "13" "14" "14" "15" "15" "16" "16" "17" "17" "18" "18" "19" "19" "20" "20" "21" "21" "22" "22" "23" "23" "24" "24" "25" "25" "26" "26" "27" "27" "28" "28" "29" "29" "30" "30" "31" "31" "32" "32" "33" "33" "34" "34" "35" "35" "36" "36" "37" "37" "38" "38" "39" "39" "40" "40" "41" "41" "42" "42" "43" "43" "44" "44" "45" "45" "46" "46" "47" "47" "48" "48" "49" "49" "50" "50" "51" "51" "52" "52" "53" "53" "54" "54" "55" "55" "56" "56" "57" "57" "58" "58" "59" "59" "60" "60" "61" "61" "62" "62" "63" "63" "64" "64" "65" "65" "66" "66" "67" "67" "68" "68" "69" "69" "70" "70" "71" "71" "72" "72" "73" "73" "74" "74" "75" "75" "76" "76" "77" "77" "78" "78" "79" "79" "80" "80" "81" "81" "82" "82" "83" "83" "84" "84" "85" "85" "86" "86" "87" "87" "88" "88" "89" "89" "90" "90" "91" "91" "92" "92" "93" "93" "94" "94" "95" "95" "96" "96" "97" "97" "98" "98" "99" "99" "100" "100"'
11+
resources:
12+
requests:
13+
memory: 1g
14+
tested-groups:
15+
- sorted-set
16+
tested-commands:
17+
- zrank
18+
redis-topologies:
19+
- oss-standalone
20+
build-variants:
21+
- gcc:8.5.0-amd64-debian-buster-default
22+
- dockerhub
23+
clientconfig:
24+
run_image: redislabs/memtier_benchmark:edge
25+
tool: memtier_benchmark
26+
arguments: --command="ZRANK zset:listpack:100 __key__" --key-maximum 100 --key-minimum 1 --command-key-pattern=R --key-prefix "" --hide-histogram --test-time 180 --pipeline 1
27+
resources:
28+
requests:
29+
cpus: '4'
30+
memory: 2g
31+
32+
priority: 72

‎redis_benchmarks_specification/test-suites/memtier_benchmark-50Mkeys-string-set-ex-10-with-precondition-pipeline-10.yml

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
version: 0.4
2+
name: memtier_benchmark-50Mkeys-string-set-ex-10-with-precondition-pipeline-10
3+
description: Multi-step test - Precondition SET EX 10000 for 50M keys, then SET EX 10 for another 50M keys with different prefix. Tests expiration performance with existing long-TTL keys.
4+
dbconfig:
5+
configuration-parameters:
6+
save: '""'
7+
check:
8+
keyspacelen: 50000000
9+
preload_tool:
10+
run_image: redislabs/memtier_benchmark:edge
11+
tool: memtier_benchmark
12+
arguments: '--command "SET __key__ BBB EX 10000" -c 1 -t 1 --command-key-pattern=P --pipeline 10 --hide-histogram --key-maximum 50000000 -n allkeys --key-prefix=1'
13+
resources:
14+
requests:
15+
memory: 10g
16+
tested-commands:
17+
- set
18+
tested-groups:
19+
- string
20+
redis-topologies:
21+
- oss-standalone
22+
build-variants:
23+
- gcc:8.5.0-amd64-debian-buster-default
24+
- dockerhub
25+
clientconfig:
26+
run_image: redislabs/memtier_benchmark:edge
27+
tool: memtier_benchmark
28+
arguments: '--command "SET __key__ BBB EX 10" -c 1 -t 1 --command-key-pattern=P --pipeline 10 --hide-histogram --key-maximum 50000000 -n allkeys --key-prefix=2'
29+
resources:
30+
requests:
31+
cpus: '2'
32+
memory: 2g
33+
34+
priority: 1

‎redis_benchmarks_specification/test-suites/memtier_benchmark-50Mkeys-string-set-ex-10years-pipeline-10.yml

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
version: 0.4
2+
name: memtier_benchmark-50Mkeys-string-set-ex-10years-pipeline-10
3+
description: Runs memtier_benchmark, for a keyspace of 50M keys with SET EX 311040000 (10 years) commands and pipeline 10. Tests very long TTL performance.
4+
dbconfig:
5+
configuration-parameters:
6+
save: '""'
7+
check:
8+
keyspacelen: 0
9+
resources:
10+
requests:
11+
memory: 10g
12+
tested-commands:
13+
- set
14+
tested-groups:
15+
- string
16+
redis-topologies:
17+
- oss-standalone
18+
build-variants:
19+
- gcc:8.5.0-amd64-debian-buster-default
20+
- dockerhub
21+
clientconfig:
22+
run_image: redislabs/memtier_benchmark:edge
23+
tool: memtier_benchmark
24+
arguments: '--command "SET __key__ BBB EX 311040000" -c 1 -t 1 --command-key-pattern=P --pipeline 10 --hide-histogram --key-maximum 50000000 -n allkeys'
25+
resources:
26+
requests:
27+
cpus: '2'
28+
memory: 2g
29+
30+
priority: 1

‎redis_benchmarks_specification/test-suites/memtier_benchmark-50Mkeys-string-set-ex-3-pipeline-10.yml

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
version: 0.4
2+
name: memtier_benchmark-50Mkeys-string-set-ex-3-pipeline-10
3+
description: Runs memtier_benchmark, for a keyspace of 50M keys with SET EX 3 commands and pipeline 10. Tests short TTL expiration performance.
4+
dbconfig:
5+
configuration-parameters:
6+
save: '""'
7+
check:
8+
keyspacelen: 0
9+
resources:
10+
requests:
11+
memory: 10g
12+
tested-commands:
13+
- set
14+
tested-groups:
15+
- string
16+
redis-topologies:
17+
- oss-standalone
18+
build-variants:
19+
- gcc:8.5.0-amd64-debian-buster-default
20+
- dockerhub
21+
clientconfig:
22+
run_image: redislabs/memtier_benchmark:edge
23+
tool: memtier_benchmark
24+
arguments: '--command "SET __key__ BBB EX 3" -c 1 -t 1 --command-key-pattern=P --pipeline 10 --hide-histogram --key-maximum 50000000 -n allkeys --key-prefix=1'
25+
resources:
26+
requests:
27+
cpus: '2'
28+
memory: 2g
29+
30+
priority: 1

‎redis_benchmarks_specification/test-suites/memtier_benchmark-50Mkeys-string-set-ex-random-range-pipeline-10.yml

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
version: 0.4
2+
name: memtier_benchmark-50Mkeys-string-set-ex-random-range-pipeline-10
3+
description: Runs memtier_benchmark with SET EX using random TTL range from 20 seconds to 30 days. Worst-case test for ebuckets with wide TTL distribution. Note - requires memtier_benchmark modification to support __data__ placeholder for EX values.
4+
dbconfig:
5+
configuration-parameters:
6+
save: '""'
7+
check:
8+
keyspacelen: 0
9+
resources:
10+
requests:
11+
memory: 10g
12+
tested-commands:
13+
- set
14+
tested-groups:
15+
- string
16+
redis-topologies:
17+
- oss-standalone
18+
build-variants:
19+
- gcc:8.5.0-amd64-debian-buster-default
20+
- dockerhub
21+
clientconfig:
22+
run_image: redislabs/memtier_benchmark:edge
23+
tool: memtier_benchmark
24+
arguments: '--command "SET __key__ BBB EX __data__" -c 1 -t 1 --data-size-range=20-2592000 --command-key-pattern=P --pipeline 10 --hide-histogram --key-maximum 50000000 -n allkeys --key-prefix=1'
25+
resources:
26+
requests:
27+
cpus: '2'
28+
memory: 2g
29+
30+
priority: 1

0 commit comments

Comments
 (0)