Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dispatcher thread #2280

Closed
wants to merge 2 commits into from
Closed

Dispatcher thread #2280

wants to merge 2 commits into from

Conversation

gatesn
Copy link
Contributor

@gatesn gatesn commented Feb 8, 2025

Experiment to bring back a dedicated dispatcher thread (we should probably do something like this regardless, but I think we can improve the APIs to not require Send + Sync everywhere with thread-local I/O.

@gatesn gatesn added the benchmark Run benchmarks on this branch label Feb 8, 2025
@github-actions github-actions bot removed the benchmark Run benchmarks on this branch label Feb 8, 2025
Copy link
Contributor

github-actions bot commented Feb 8, 2025

Benchmarks: TPC-H

Table of Results
name PR 14407f4 base a4f00e3 ratio (PR/base) unit
tpch_q01/arrow 536983648 5.40046e+08 0.99433 ns
tpch_q01/parquet 757596025 7.71131e+08 0.982448 ns
tpch_q01/vortex-file-compressed 485248939 5.24847e+08 0.924554 ns
tpch_q02/arrow 142791635 1.40184e+08 1.0186 ns
tpch_q02/parquet 174685874 1.72514e+08 1.01259 ns
tpch_q02/vortex-file-compressed 146490484 1.4634e+08 1.00103 ns
tpch_q03/arrow 171347490 1.75574e+08 0.975927 ns
tpch_q03/parquet 376942517 3.74592e+08 1.00627 ns
tpch_q03/vortex-file-compressed 223854454 2.45961e+08 0.910121 ns
tpch_q04/arrow 179317091 1.82733e+08 0.981308 ns
tpch_q04/parquet 217621286 2.12773e+08 1.02279 ns
tpch_q04/vortex-file-compressed 192705170 1.94987e+08 0.988297 ns
tpch_q05/arrow 326274538 3.33395e+08 0.978643 ns
tpch_q05/parquet 500464751 4.99832e+08 1.00127 ns
tpch_q05/vortex-file-compressed 362512260 3.66636e+08 0.988752 ns
tpch_q06/arrow 25513781 2.6669e+07 0.956683 ns
tpch_q06/parquet 147972547 1.47496e+08 1.00323 ns
tpch_q06/vortex-file-compressed 89922261 8.08067e+07 1.11281 ns
tpch_q07/arrow 609918301 6.25401e+08 0.975244 ns
tpch_q07/parquet 766842223 7.55842e+08 1.01455 ns
tpch_q07/vortex-file-compressed 631847067 6.47801e+08 0.975372 ns
tpch_q08/arrow 270252531 2.67001e+08 1.01218 ns
tpch_q08/parquet 536204818 5.4581e+08 0.982402 ns
tpch_q08/vortex-file-compressed 332492504 3.46347e+08 0.959998 ns
tpch_q09/arrow 472507786 4.71897e+08 1.00129 ns
tpch_q09/parquet 778095129 7.6576e+08 1.01611 ns
tpch_q09/vortex-file-compressed 554240167 5.77743e+08 0.95932 ns
tpch_q10/arrow 262011227 2.65367e+08 0.987355 ns
tpch_q10/parquet 498730925 4.94646e+08 1.00826 ns
tpch_q10/vortex-file-compressed 295633449 3.05425e+08 0.96794 ns
tpch_q11/arrow 144703380 1.41321e+08 1.02393 ns
tpch_q11/parquet 151518798 1.43078e+08 1.059 ns
tpch_q11/vortex-file-compressed 126587126 1.328e+08 0.953214 ns
tpch_q12/arrow 181860971 1.80602e+08 1.00697 ns
tpch_q12/parquet 325144333 3.22239e+08 1.00902 ns
tpch_q12/vortex-file-compressed 302530618 3.02699e+08 0.999445 ns
tpch_q13/arrow 168068094 1.67602e+08 1.00278 ns
tpch_q13/parquet 305860686 3.06592e+08 0.997615 ns
tpch_q13/vortex-file-compressed 184405287 1.73428e+08 1.06329 ns
tpch_q14/arrow 38433929 3.71134e+07 1.03558 ns
tpch_q14/parquet 231946744 2.27465e+08 1.0197 ns
tpch_q14/vortex-file-compressed 91032999 8.85915e+07 1.02756 ns
tpch_q15/arrow 67858539 6.75684e+07 1.00429 ns
tpch_q15/parquet 323764166 3.18543e+08 1.01639 ns
tpch_q15/vortex-file-compressed 155483356 1.57639e+08 0.986328 ns
tpch_q16/arrow 96696884 9.65321e+07 1.00171 ns
tpch_q16/parquet 110787588 1.10557e+08 1.00209 ns
tpch_q16/vortex-file-compressed 103693718 1.02191e+08 1.0147 ns
tpch_q17/arrow 618458735 5.95742e+08 1.03813 ns
tpch_q17/parquet 672984712 6.89617e+08 0.975882 ns
tpch_q17/vortex-file-compressed 568421476 5.92768e+08 0.958927 ns
tpch_q18/arrow 1100841950 1.12992e+09 0.974263 ns
tpch_q18/parquet 1315653161 1.31305e+09 1.00198 ns
tpch_q18/vortex-file-compressed 1139028892 1.13473e+09 1.00379 ns
tpch_q19/arrow 149237287 1.51487e+08 0.985148 ns
tpch_q19/parquet 412273559 4.13382e+08 0.997317 ns
tpch_q19/vortex-file-compressed 161649443 1.63843e+08 0.986615 ns
tpch_q20/arrow 174136003 1.74385e+08 0.998571 ns
tpch_q20/parquet 310111741 3.06606e+08 1.01143 ns
tpch_q20/vortex-file-compressed 227874203 2.30269e+08 0.9896 ns
tpch_q21/arrow 965783319 9.75097e+08 0.990448 ns
tpch_q21/parquet 1106500509 1.08524e+09 1.01959 ns
tpch_q21/vortex-file-compressed 997226979 1.00924e+09 0.988092 ns
tpch_q22/arrow 79057424 7.84817e+07 1.00734 ns
tpch_q22/parquet 108532093 1.09939e+08 0.987202 ns
tpch_q22/vortex-file-compressed 81780869 8.2654e+07 0.989437 ns

Copy link
Contributor

github-actions bot commented Feb 8, 2025

Benchmarks: Clickbench

Table of Results
name PR 14407f4 base a4f00e3 ratio (PR/base) unit
clickbench_q00/parquet 2295463 1.88345e+06 1.21876 ns
clickbench_q01/parquet 57589143 5.97141e+07 0.964415 ns
clickbench_q02/parquet 116356294 1.18497e+08 0.981932 ns
clickbench_q03/parquet 82810824 8.22405e+07 1.00693 ns
clickbench_q04/parquet 636540650 6.51665e+08 0.976791 ns
clickbench_q05/parquet 805878806 8.21962e+08 0.980434 ns
clickbench_q06/parquet 2025667 2.06707e+06 0.979968 ns
clickbench_q07/parquet 61700749 6.09043e+07 1.01308 ns
clickbench_q08/parquet 746851444 7.43829e+08 1.00406 ns
clickbench_q09/parquet 1022976049 1.07439e+09 0.952143 ns
clickbench_q10/parquet 249305312 2.59281e+08 0.961526 ns
clickbench_q11/parquet 297243826 3.09773e+08 0.959553 ns
clickbench_q12/parquet 833360519 8.6506e+08 0.963356 ns
clickbench_q13/parquet 1054600460 1.13261e+09 0.931127 ns
clickbench_q14/parquet 814265163 8.53037e+08 0.954549 ns
clickbench_q15/parquet 716547221 7.72966e+08 0.927009 ns
clickbench_q16/parquet 1568255272 1.6969e+09 0.924187 ns
clickbench_q17/parquet 1424675980 1.46865e+09 0.970055 ns
clickbench_q18/parquet 2964961078 2.93729e+09 1.00942 ns
clickbench_q19/parquet 67122591 6.75202e+07 0.994111 ns
clickbench_q20/parquet 1225441956 1.21807e+09 1.00605 ns
clickbench_q21/parquet 1408380475 1.43498e+09 0.981466 ns
clickbench_q22/parquet 2357370425 2.38803e+09 0.987162 ns
clickbench_q23/parquet 8063467493 8.10103e+09 0.995363 ns
clickbench_q24/parquet 532310689 5.27075e+08 1.00993 ns
clickbench_q25/parquet 515361742 5.1857e+08 0.993813 ns
clickbench_q26/parquet 590241185 5.98644e+08 0.985963 ns
clickbench_q27/parquet 1598139346 1.63469e+09 0.977641 ns
clickbench_q28/parquet 11366766237 1.16856e+10 0.972716 ns
clickbench_q29/parquet 429603658 4.23771e+08 1.01376 ns
clickbench_q30/parquet 767879767 7.63245e+08 1.00607 ns
clickbench_q31/parquet 795131418 7.80024e+08 1.01937 ns
clickbench_q32/parquet 2777200166 2.66892e+09 1.04057 ns
clickbench_q33/parquet 2944133648 2.80858e+09 1.04826 ns
clickbench_q34/parquet 2783810614 2.68922e+09 1.03517 ns
clickbench_q35/parquet 848110017 8.27792e+08 1.02454 ns
clickbench_q36/parquet 176364507 1.7421e+08 1.01237 ns
clickbench_q37/parquet 91387997 9.01018e+07 1.01428 ns
clickbench_q38/parquet 119453361 1.14742e+08 1.04106 ns
clickbench_q39/parquet 334272010 3.29714e+08 1.01382 ns
clickbench_q40/parquet 50223445 4.79908e+07 1.04652 ns
clickbench_q41/parquet 50516157 4.86608e+07 1.03813 ns
clickbench_q42/parquet 71872766 6.49526e+07 1.10654 ns
clickbench_q00/vortex-file-compressed 2152782 2.00721e+06 1.07253 ns
clickbench_q01/vortex-file-compressed 25626970 2.11537e+07 1.21147 ns
clickbench_q02/vortex-file-compressed 99320741 9.49344e+07 1.0462 ns
clickbench_q03/vortex-file-compressed 81180931 7.15316e+07 1.1349 ns
clickbench_q04/vortex-file-compressed 645045305 6.07539e+08 1.06173 ns
clickbench_q05/vortex-file-compressed 682776130 6.45228e+08 1.05819 ns
clickbench_q06/vortex-file-compressed 2060540 2.1044e+06 0.979157 ns
clickbench_q07/vortex-file-compressed 50685895 3.61751e+07 1.40113 ns
clickbench_q08/vortex-file-compressed 814681349 7.59176e+08 1.07311 ns
clickbench_q09/vortex-file-compressed 1009083348 9.4134e+08 1.07197 ns
clickbench_q10/vortex-file-compressed 287274717 2.85342e+08 1.00677 ns
clickbench_q11/vortex-file-compressed 314661886 3.07398e+08 1.02363 ns
clickbench_q12/vortex-file-compressed 542965980 5.2439e+08 1.03542 ns
clickbench_q13/vortex-file-compressed 815803222 7.80913e+08 1.04468 ns
clickbench_q14/vortex-file-compressed 545436603 5.44173e+08 1.00232 ns
clickbench_q15/vortex-file-compressed 732783392 7.1446e+08 1.02565 ns
clickbench_q16/vortex-file-compressed 1408303892 1.37831e+09 1.02176 ns
clickbench_q17/vortex-file-compressed 1341697342 1.32962e+09 1.00908 ns
clickbench_q18/vortex-file-compressed 2744253408 2.91819e+09 0.940397 ns
clickbench_q19/vortex-file-compressed 46703644 4.14579e+07 1.12653 ns
clickbench_q20/vortex-file-compressed 792408495 6.52858e+08 1.21375 ns
clickbench_q21/vortex-file-compressed 918458426 8.20247e+08 1.11973 ns
clickbench_q22/vortex-file-compressed 2139482244 2.06055e+09 1.03831 ns
clickbench_q23/vortex-file-compressed 4314513975 4.24108e+09 1.01732 ns
clickbench_q24/vortex-file-compressed 281244436 2.78502e+08 1.00985 ns
clickbench_q25/vortex-file-compressed 244343976 2.35385e+08 1.03806 ns
clickbench_q26/vortex-file-compressed 350151144 3.50845e+08 0.998022 ns
clickbench_q27/vortex-file-compressed 1375365708 1.31471e+09 1.04614 ns
clickbench_q28/vortex-file-compressed 10448682640 1.06541e+10 0.980718 ns
clickbench_q29/vortex-file-compressed 687311481 6.80009e+08 1.01074 ns
clickbench_q30/vortex-file-compressed 527168199 5.04332e+08 1.04528 ns
clickbench_q31/vortex-file-compressed 530856864 5.37503e+08 0.987635 ns
clickbench_q32/vortex-file-compressed 2779962769 2.84801e+09 0.976105 ns
clickbench_q33/vortex-file-compressed 2323854457 2.19405e+09 1.05916 ns
clickbench_q34/vortex-file-compressed 2312198066 2.19817e+09 1.05187 ns
clickbench_q35/vortex-file-compressed 958772216 9.22187e+08 1.03967 ns
clickbench_q36/vortex-file-compressed 49937886 7.03095e+07 0.710258 ns
clickbench_q37/vortex-file-compressed 52910216 7.04902e+07 0.750604 ns
clickbench_q38/vortex-file-compressed 39985894 5.17138e+07 0.773215 ns
clickbench_q39/vortex-file-compressed 84298687 1.28288e+08 0.657105 ns
clickbench_q40/vortex-file-compressed 33781656 3.15638e+07 1.07027 ns
clickbench_q41/vortex-file-compressed 35954671 3.21981e+07 1.11667 ns
clickbench_q42/vortex-file-compressed 41307685 4.77596e+07 0.864908 ns

Copy link
Contributor

github-actions bot commented Feb 8, 2025

Benchmarks: random_access

Table of Results
name PR 14407f4 base a4f00e3 ratio (PR/base) unit
random-access/vortex-tokio-local-disk 2.35357e+06 2.14399e+06 1.09775 ns
random-access/vortex-local-fs 2.35231e+06 2.20058e+06 1.06895 ns
random-access/parquet-tokio-local-disk 2.40769e+08 2.2806e+08 1.05573 ns

Copy link
Contributor

github-actions bot commented Feb 8, 2025

Benchmarks: compress

Table of Results
name PR 14407f4 base a4f00e3 ratio (PR/base) unit
compress time/wide table cols=10 chunks=1 rows=1000 3.71021e+06 3.74952e+06 0.989518 ns
compress time/wide table cols=10 chunks=1 rows=1000 throughput 0.0323814 0.032042 1.01059 bytes/ns
parquet_rs-zstd compress time/wide table cols=10 chunks=1 rows=1000 717608 720848 0.995506 ns
parquet_rs-zstd compress time/wide table cols=10 chunks=1 rows=1000 throughput 0.16742 0.166668 1.00451 bytes/ns
decompress time/wide table cols=10 chunks=1 rows=1000 243745 243337 1.00167 ns
decompress time/wide table cols=10 chunks=1 rows=1000 throughput 0.492901 0.493726 0.998329 bytes/ns
parquet_rs-zstd decompress time/wide table cols=10 chunks=1 rows=1000 240603 240415 1.00078 ns
parquet_rs-zstd decompress time/wide table cols=10 chunks=1 rows=1000 throughput 0.499337 0.499728 0.999218 bytes/ns
compress time/wide table cols=100 chunks=1 rows=1000 3.74443e+07 3.85838e+07 0.970467 ns
compress time/wide table cols=100 chunks=1 rows=1000 throughput 0.0320802 0.0311328 1.03043 bytes/ns
parquet_rs-zstd compress time/wide table cols=100 chunks=1 rows=1000 7.51227e+06 8.51829e+06 0.881899 ns
parquet_rs-zstd compress time/wide table cols=100 chunks=1 rows=1000 throughput 0.159901 0.141017 1.13392 bytes/ns
decompress time/wide table cols=100 chunks=1 rows=1000 2.52504e+06 2.53773e+06 0.994997 ns
decompress time/wide table cols=100 chunks=1 rows=1000 throughput 0.475724 0.473345 1.00503 bytes/ns
parquet_rs-zstd decompress time/wide table cols=100 chunks=1 rows=1000 2.47668e+06 2.50841e+06 0.98735 ns
parquet_rs-zstd decompress time/wide table cols=100 chunks=1 rows=1000 throughput 0.485012 0.478877 1.01281 bytes/ns
compress time/wide table cols=1000 chunks=1 rows=1000 4.08645e+08 4.17266e+08 0.979339 ns
compress time/wide table cols=1000 chunks=1 rows=1000 throughput 0.0293948 0.0287875 1.0211 bytes/ns
parquet_rs-zstd compress time/wide table cols=1000 chunks=1 rows=1000 9.61219e+07 9.69849e+07 0.991102 ns
parquet_rs-zstd compress time/wide table cols=1000 chunks=1 rows=1000 throughput 0.124967 0.123855 1.00898 bytes/ns
decompress time/wide table cols=1000 chunks=1 rows=1000 2.84566e+07 3.00538e+07 0.946854 ns
decompress time/wide table cols=1000 chunks=1 rows=1000 throughput 0.422118 0.399683 1.05613 bytes/ns
parquet_rs-zstd decompress time/wide table cols=1000 chunks=1 rows=1000 2.69536e+07 2.85225e+07 0.944996 ns
parquet_rs-zstd decompress time/wide table cols=1000 chunks=1 rows=1000 throughput 0.445656 0.421143 1.05821 bytes/ns
compress time/wide table cols=10 chunks=50 rows=1000 7.7579e+06 7.90543e+06 0.981337 ns
compress time/wide table cols=10 chunks=50 rows=1000 throughput 0.0163012 0.015997 1.01902 bytes/ns
parquet_rs-zstd compress time/wide table cols=10 chunks=50 rows=1000 1.00368e+06 1.00269e+06 1.001 ns
parquet_rs-zstd compress time/wide table cols=10 chunks=50 rows=1000 throughput 0.125999 0.126124 0.999006 bytes/ns
decompress time/wide table cols=10 chunks=50 rows=1000 249118 249731 0.997544 ns
decompress time/wide table cols=10 chunks=50 rows=1000 throughput 0.507643 0.506396 1.00246 bytes/ns
parquet_rs-zstd decompress time/wide table cols=10 chunks=50 rows=1000 247170 247524 0.998569 ns
parquet_rs-zstd decompress time/wide table cols=10 chunks=50 rows=1000 throughput 0.511644 0.510912 1.00143 bytes/ns
compress time/wide table cols=100 chunks=50 rows=1000 8.00948e+07 8.91662e+07 0.898264 ns
compress time/wide table cols=100 chunks=50 rows=1000 throughput 0.0157371 0.0141361 1.11326 bytes/ns
parquet_rs-zstd compress time/wide table cols=100 chunks=50 rows=1000 1.13078e+07 1.39855e+07 0.808539 ns
parquet_rs-zstd compress time/wide table cols=100 chunks=50 rows=1000 throughput 0.111468 0.0901263 1.2368 bytes/ns
decompress time/wide table cols=100 chunks=50 rows=1000 2.53081e+06 2.54798e+06 0.993261 ns
decompress time/wide table cols=100 chunks=50 rows=1000 throughput 0.498047 0.494691 1.00678 bytes/ns
parquet_rs-zstd decompress time/wide table cols=100 chunks=50 rows=1000 2.48918e+06 2.50457e+06 0.993855 ns
parquet_rs-zstd decompress time/wide table cols=100 chunks=50 rows=1000 throughput 0.506376 0.503264 1.00618 bytes/ns
compress time/wide table cols=1000 chunks=50 rows=1000 9.07968e+08 9.36426e+08 0.96961 ns
compress time/wide table cols=1000 chunks=50 rows=1000 throughput 0.0138776 0.0134559 1.03134 bytes/ns
parquet_rs-zstd compress time/wide table cols=1000 chunks=50 rows=1000 1.65753e+08 1.78174e+08 0.930283 ns
parquet_rs-zstd compress time/wide table cols=1000 chunks=50 rows=1000 throughput 0.0760197 0.0707198 1.07494 bytes/ns
decompress time/wide table cols=1000 chunks=50 rows=1000 2.84752e+07 3.11767e+07 0.913349 ns
decompress time/wide table cols=1000 chunks=50 rows=1000 throughput 0.442507 0.404163 1.09487 bytes/ns
parquet_rs-zstd decompress time/wide table cols=1000 chunks=50 rows=1000 2.70648e+07 2.85171e+07 0.949074 ns
parquet_rs-zstd decompress time/wide table cols=1000 chunks=50 rows=1000 throughput 0.465566 0.441857 1.05366 bytes/ns
compress time/taxi 1.56884e+09 1.60323e+09 0.978545 ns
compress time/taxi throughput 0.300102 0.293663 1.02193 bytes/ns
parquet_rs-zstd compress time/taxi 1.85125e+09 1.87451e+09 0.987591 ns
parquet_rs-zstd compress time/taxi throughput 0.25432 0.251164 1.01257 bytes/ns
decompress time/taxi 2.19635e+08 2.32727e+08 0.943743 ns
decompress time/taxi throughput 2.14361 2.02302 1.05961 bytes/ns
parquet_rs-zstd decompress time/taxi 2.91478e+08 2.9487e+08 0.988495 ns
parquet_rs-zstd decompress time/taxi throughput 1.61525 1.59667 1.01164 bytes/ns
compress time/AirlineSentiment 291721 291937 0.99926 ns
compress time/AirlineSentiment throughput 0.00699298 0.00698781 1.00074 bytes/ns
parquet_rs-zstd compress time/AirlineSentiment 47499.1 47892.8 0.99178 ns
parquet_rs-zstd compress time/AirlineSentiment throughput 0.0429482 0.0425951 1.00829 bytes/ns
decompress time/AirlineSentiment 102323 103808 0.985695 ns
decompress time/AirlineSentiment throughput 0.0199368 0.0196516 1.01451 bytes/ns
parquet_rs-zstd decompress time/AirlineSentiment 27877.7 28128.9 0.991067 ns
parquet_rs-zstd decompress time/AirlineSentiment throughput 0.0731768 0.0725232 1.00901 bytes/ns
compress time/Arade 3.05658e+09 3.13302e+09 0.975603 ns
compress time/Arade throughput 0.257487 0.251205 1.02501 bytes/ns
parquet_rs-zstd compress time/Arade 3.35726e+09 3.44012e+09 0.975914 ns
parquet_rs-zstd compress time/Arade throughput 0.234426 0.22878 1.02468 bytes/ns
decompress time/Arade 5.32827e+08 5.5362e+08 0.962442 ns
decompress time/Arade throughput 1.47708 1.42161 1.03902 bytes/ns
parquet_rs-zstd decompress time/Arade 6.16101e+08 6.30793e+08 0.976709 ns
parquet_rs-zstd decompress time/Arade throughput 1.27744 1.24768 1.02385 bytes/ns
compress time/Bimbo 1.21794e+10 1.22984e+10 0.990321 ns
compress time/Bimbo throughput 0.584705 0.579046 1.00977 bytes/ns
parquet_rs-zstd compress time/Bimbo 2.20289e+10 2.25314e+10 0.977697 ns
parquet_rs-zstd compress time/Bimbo throughput 0.323274 0.316064 1.02281 bytes/ns
decompress time/Bimbo 4.64684e+09 4.7437e+09 0.97958 ns
decompress time/Bimbo throughput 1.53252 1.50122 1.02085 bytes/ns
parquet_rs-zstd decompress time/Bimbo 2.3986e+09 2.43966e+09 0.98317 ns
parquet_rs-zstd decompress time/Bimbo throughput 2.96896 2.91899 1.01712 bytes/ns
compress time/CMSprovider 1.4963e+10 1.51207e+10 0.989567 ns
compress time/CMSprovider throughput 0.344126 0.340536 1.01054 bytes/ns
parquet_rs-zstd compress time/CMSprovider 1.83108e+10 1.88645e+10 0.970648 ns
parquet_rs-zstd compress time/CMSprovider throughput 0.281209 0.272955 1.03024 bytes/ns
decompress time/CMSprovider 4.79085e+09 4.90457e+09 0.976813 ns
decompress time/CMSprovider throughput 1.07479 1.04987 1.02374 bytes/ns
parquet_rs-zstd decompress time/CMSprovider 3.31841e+09 3.41274e+09 0.972359 ns
parquet_rs-zstd decompress time/CMSprovider throughput 1.55169 1.5088 1.02843 bytes/ns
compress time/Euro2016 2.27806e+09 2.37571e+09 0.958899 ns
compress time/Euro2016 throughput 0.172627 0.165532 1.04286 bytes/ns
parquet_rs-zstd compress time/Euro2016 1.71979e+09 1.76134e+09 0.976408 ns
parquet_rs-zstd compress time/Euro2016 throughput 0.228664 0.22327 1.02416 bytes/ns
decompress time/Euro2016 3.18808e+08 3.20549e+08 0.994567 ns
decompress time/Euro2016 throughput 1.23352 1.22682 1.00546 bytes/ns
parquet_rs-zstd decompress time/Euro2016 4.64341e+08 4.75695e+08 0.976132 ns
parquet_rs-zstd decompress time/Euro2016 throughput 0.846909 0.826695 1.02445 bytes/ns
compress time/Food 1.10092e+09 1.14511e+09 0.961411 ns
compress time/Food throughput 0.302219 0.290556 1.04014 bytes/ns
parquet_rs-zstd compress time/Food 1.11356e+09 1.14457e+09 0.972904 ns
parquet_rs-zstd compress time/Food throughput 0.29879 0.290694 1.02785 bytes/ns
decompress time/Food 2.05801e+08 2.14306e+08 0.960312 ns
decompress time/Food throughput 1.61671 1.55254 1.04133 bytes/ns
parquet_rs-zstd decompress time/Food 2.06308e+08 2.16511e+08 0.952875 ns
parquet_rs-zstd decompress time/Food throughput 1.61273 1.53673 1.04946 bytes/ns
compress time/HashTags 2.92567e+09 3.06083e+09 0.955841 ns
compress time/HashTags throughput 0.27498 0.262837 1.0462 bytes/ns
parquet_rs-zstd compress time/HashTags 2.75358e+09 2.8102e+09 0.979854 ns
parquet_rs-zstd compress time/HashTags throughput 0.292164 0.286279 1.02056 bytes/ns
decompress time/HashTags 4.98532e+08 5.16517e+08 0.965181 ns
decompress time/HashTags throughput 1.61374 1.55755 1.03608 bytes/ns
parquet_rs-zstd decompress time/HashTags 7.1841e+08 7.87174e+08 0.912645 ns
parquet_rs-zstd decompress time/HashTags throughput 1.11983 1.02201 1.09572 bytes/ns
compress time/TPC-H l_comment chunked without fsst 3.03557e+09 3.41956e+09 0.887707 ns
compress time/TPC-H l_comment chunked without fsst throughput 0.0820929 0.0728744 1.1265 bytes/ns
parquet_rs-zstd compress time/TPC-H l_comment chunked without fsst 9.44207e+08 9.67784e+08 0.975638 ns
parquet_rs-zstd compress time/TPC-H l_comment chunked without fsst throughput 0.263924 0.257494 1.02497 bytes/ns
decompress time/TPC-H l_comment chunked without fsst 7.6295e+07 7.95405e+07 0.959197 ns
decompress time/TPC-H l_comment chunked without fsst throughput 3.26625 3.13298 1.04254 bytes/ns
parquet_rs-zstd decompress time/TPC-H l_comment chunked without fsst 2.43592e+08 2.49465e+08 0.976457 ns
parquet_rs-zstd decompress time/TPC-H l_comment chunked without fsst throughput 1.02302 0.998932 1.02411 bytes/ns
compress time/TPC-H l_comment chunked 9.63776e+08 9.77174e+08 0.986289 ns
compress time/TPC-H l_comment chunked throughput 0.258565 0.25502 1.0139 bytes/ns
parquet_rs-zstd compress time/TPC-H l_comment chunked 9.4767e+08 9.73773e+08 0.973194 ns
parquet_rs-zstd compress time/TPC-H l_comment chunked throughput 0.262959 0.25591 1.02754 bytes/ns
decompress time/TPC-H l_comment chunked 1.06697e+08 1.13425e+08 0.940683 ns
decompress time/TPC-H l_comment chunked throughput 2.33557 2.19703 1.06306 bytes/ns
parquet_rs-zstd decompress time/TPC-H l_comment chunked 2.45081e+08 2.49924e+08 0.980623 ns
parquet_rs-zstd decompress time/TPC-H l_comment chunked throughput 1.0168 0.997098 1.01976 bytes/ns
compress time/TPC-H l_comment canonical 9.94607e+08 9.96359e+08 0.998242 ns
compress time/TPC-H l_comment canonical throughput 0.250549 0.250108 1.00176 bytes/ns
parquet_rs-zstd compress time/TPC-H l_comment canonical 9.57467e+08 9.78455e+08 0.97855 ns
parquet_rs-zstd compress time/TPC-H l_comment canonical throughput 0.260268 0.254685 1.02192 bytes/ns
decompress time/TPC-H l_comment canonical 1.06525e+08 1.13353e+08 0.939765 ns
decompress time/TPC-H l_comment canonical throughput 2.33933 2.19842 1.0641 bytes/ns
parquet_rs-zstd decompress time/TPC-H l_comment canonical 2.45175e+08 2.51543e+08 0.974684 ns
parquet_rs-zstd decompress time/TPC-H l_comment canonical throughput 1.01641 0.990676 1.02597 bytes/ns

@gatesn gatesn closed this Feb 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant