feat: added compression at session_log side by antoniomuso · Pull Request #3075 · garden-co/classic-jazz

antoniomuso · 2025-10-20T11:22:32Z

Description

This PR introduces LZ4 compression at the Session Log level.
Compression is applied only to private data, as encrypted data benefits the most, encryption followed by Base64 encoding significantly increases the output size.

The transaction now has a parameter encoding?: "lz4" | "zstd" | "snappy" that specifies the compression algorithm to use. If omitted, the transaction remains uncompressed.

I created two separate benchmarks to evaluate compression performance and data size reduction using realistic data structures.

Additionally, this PR adds a compress parameter to both colist and comap. This parameter is not meant to be exposed; it’s only used internally to benchmark compression more easily.

The benchmark compress.bench.ts, shows the compression performance and import performance from storage. you can run it by:
cd bench && pnpm bench compression.bench.ts

LZ4

 ✓  bench  compression.bench.ts > Import performance - Small payloads (512B)
 > [Small 512B] Private: 33601 bytes, Trusting: 28950 bytes, Compressed: 27211 bytes (19.0% vs private, 6.0% vs trusting)
 1818ms
     name                                            hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · private (encrypted, no compression)       2,426.74  0.3229  2.8500  0.4121  0.4298  0.7208  1.0071  2.8420  ±2.30%     1214
   · trusting (no encryption, no compression)  3,027.17  0.2783  2.5988  0.3303  0.3268  0.4940  0.9106  1.0672  ±1.25%     1514
   · private with compress:true                1,443.24  0.2895  3.8939  0.6929  0.5976  3.6285  3.6825  3.8939  ±6.91%      722

 ✓  bench  compression.bench.ts > Import performance - Moderate payloads (1KB)
 > [1KB] Private: 33601 bytes, Trusting: 28950 bytes, Compressed: 27210 bytes (19.0% savings)
 1831ms
     name                                            hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · private (encrypted, no compression)       2,936.62  0.3184  0.8510  0.3405  0.3396  0.4612  0.7736  0.8355  ±0.64%     1469
   · trusting (no encryption, no compression)  2,004.03  0.2838  3.8926  0.4990  0.5476  3.5743  3.7906  3.8707  ±6.25%     1003
   · private with compress:true                2,907.75  0.2622  2.2025  0.3439  0.3130  1.5315  1.8463  2.1805  ±3.00%     1454

 ✓  bench  compression.bench.ts > Import performance - Medium payloads (4KB)
 > [4KB] Private: 65361 bytes, Trusting: 56251 bytes, Compressed: 46210 bytes (29.3% savings)
 1818ms
     name                                            hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · private (encrypted, no compression)       1,455.32  0.5550  4.2995  0.6871  0.6142  3.7243  3.9995  4.2995  ±4.31%      729
   · trusting (no encryption, no compression)  1,752.23  0.4994  1.0797  0.5707  0.5687  1.0077  1.0491  1.0797  ±0.68%      877
   · private with compress:true                1,991.09  0.4211  3.7427  0.5022  0.4651  1.2332  2.9013  3.7427  ±2.86%      997

 ✓  bench  compression.bench.ts > Import performance - Large payloads (100KB)
 > [100KB] Private: 1500555 bytes, Trusting: 1285384 bytes, Compressed: 726947 bytes (51.6% savings)
 3926ms
     name                                           hz      min      max     mean      p75      p99     p995     p999     rme  samples
   · private (encrypted, no compression)       73.1333  11.8198  19.3565  13.6737  13.0222  19.2769  19.3565  19.3565  ±3.95%      100
   · trusting (no encryption, no compression)  85.0591  11.3070  12.3170  11.7565  11.8863  12.2879  12.3170  12.3170  ±0.34%      100
   · private with compress:true                 104.17   6.6296  14.0746   9.5999  10.3965  13.4445  14.0746  14.0746  ±3.53%      100

 ✓  bench  compression.bench.ts > Import performance with NAPI - Medium payloads (4KB)
 1818ms
     name                                                   hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · NAPI - private (encrypted, no compression)       4,449.37  0.1930  0.7813  0.2248  0.2209  0.5696  0.6864  0.7348  ±0.97%     2225
   · NAPI - trusting (no encryption, no compression)  3,861.28  0.2240  3.1452  0.2590  0.2554  0.6143  0.6623  0.8493  ±1.43%     1931
   · NAPI - private with compress:true                4,879.94  0.1745  3.3075  0.2049  0.1992  0.5560  0.6615  1.3714  ±1.67%     2440

 ✓  bench  compression.bench.ts > Write performance - Transaction creation (4KB payload)
 1827ms
     name                                                               hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · create private transaction (no compression)                  5,221.63  0.1301  0.8217  0.1915  0.2145  0.2465  0.3235  0.6647  ±0.82%     2612
   · create trusting transaction (no encryption, no compression)  5,507.27  0.1145  0.8676  0.1816  0.2092  0.2684  0.3355  0.7018  ±0.95%     2754
   · create private transaction with compress:true                5,273.70  0.1162  0.7490  0.1896  0.2196  0.2534  0.2709  0.5673  ±0.82%     2638

 ✓  bench  compression.bench.ts > Write performance - Small payload (~900 bytes)
 > Payload size: 2171 bytes (realistic user data)
 1815ms
     name                                                    hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · write - private (encrypted, no compression)       5,647.31  0.0995  0.7517  0.1771  0.2107  0.2504  0.2610  0.6017  ±0.92%     2825
   · write - trusting (no encryption, no compression)  5,823.63  0.0926  0.8034  0.1717  0.2053  0.2405  0.2445  0.4439  ±0.91%     2913
   · write - private with compress:true                5,689.19  0.1000  0.7835  0.1758  0.2079  0.2412  0.2436  0.6840  ±0.92%     2845

 ✓  bench  compression.bench.ts > Write performance - Large payload (~5KB)
 > Payload size: 6745 bytes (realistic user/post/comment data)
 1822ms
     name                                                    hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · write - private (encrypted, no compression)       3,762.05  0.1439  6.6087  0.2658  0.2210  0.8306  3.9405  6.2451  ±6.20%     1883
   · write - trusting (no encryption, no compression)  2,260.62  0.1118  6.1258  0.4424  0.4031  3.9583  4.0854  6.0541  ±8.48%     1131
   · write - private with compress:true                2,390.84  0.1270  6.5522  0.4183  0.4287  4.0868  4.1641  6.2470  ±8.38%     1198

 ✓  bench  compression.bench.ts > Bulk operations - Many small updates (100x)
 4620ms
     name                                                          hz      min      max     mean      p75      p99     p995     p999      rme  samples
   · 100 small updates - private (encrypted, no compression)  34.5905  11.7572  43.2176  28.9097  37.3405  43.2176  43.2176  43.2176   ±9.21%       50
   · 100 small updates - trusting (no overhead)               36.2311   9.4591  45.2914  27.6006  34.6565  45.2914  45.2914  45.2914   ±9.64%       50
   · 100 small updates - private with compress:true           34.9493   9.7710  43.8899  28.6129  39.3092  43.8899  43.8899  43.8899  ±10.02%       50

 ✓  bench  compression.bench.ts > Bulk operations - Many large updates (50x, ~5KB each)
 2023ms
     name                                                         hz      min      max     mean      p75      p99     p995     p999     rme  samples
   · 50 large updates - private (encrypted, no compression)  37.0835  14.6557  34.1730  26.9661  28.6074  34.1730  34.1730  34.1730  ±8.21%       20
   · 50 large updates - trusting (no overhead)               41.9314  12.8724  27.3266  23.8485  26.5957  27.3266  27.3266  27.3266  ±8.25%       21
   · 50 large updates - private with compress:true           43.4392  12.4518  27.8455  23.0207  25.1793  27.8455  27.8455  27.8455  ±7.63%       22

 ✓  bench  compression.bench.ts > List operations with compression - comparing data types
 > [List] Private: 9250 bytes, Trusting: 7814 bytes, Compressed: 8987 bytes (2.8% savings)
 1816ms
     name                                                          hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · list import - private (encrypted, no compression)       2,185.40  0.1537  7.8218  0.4576  0.4912  4.4725  4.7564  7.7786  ±9.43%     1093
   · list import - trusting (no encryption, no compression)  4,322.89  0.1177  7.9230  0.2313  0.1489  2.9052  4.2807  4.7290  ±7.68%     2163
   · list import - private with compress:true                4,956.44  0.1471  7.7670  0.2018  0.1597  0.7112  1.1026  4.5645  ±5.75%     2479

 ✓  bench  compression.bench.ts > CoPlainText - Text operations with compression
 > [Text] Private: 27584 bytes (68.0% savings vs 86282 trusting bytes)
 1226ms
     name                                                     hz     min      max    mean     p75      p99     p995     p999     rme  samples
   · text import - private (encrypted + auto-compressed)  794.33  1.0840   2.9056  1.2589  1.1588   2.5151   2.8572   2.9056  ±2.70%      398
   · text import - trusting (no overhead)                 428.70  1.4775  10.4343  2.3326  2.8475  10.1760  10.2803  10.4343  ±9.18%      217

 ✓  bench  compression.bench.ts > CoPlainText - Long document (~10KB)
 > [CoPlainText 10KB] Private: 20749 bytes, Trusting: 507807 bytes, Savings: 95.9%
 2834ms
     name                                                               hz     min      max     mean      p75      p99     p995     p999     rme  samples
   · import long document - private (encrypted + auto-compressed)   577.36  1.5165   3.2766   1.7320   1.6170   2.8063   3.1005   3.2766  ±2.50%      289
   · import long document - trusting (no overhead)                 94.9785  5.5844  17.1783  10.5287  12.7520  16.9029  17.1610  17.1783  ±3.99%      200

 ✓  bench  compression.bench.ts > CoPlainText - Collaborative editing (many small edits)
 > [CoPlainText many edits] Private: 30925 bytes (23.1% savings vs 40194 trusting bytes)
 1209ms
     name                                                                  hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · import with many edits - private (encrypted + auto-compressed)  1,501.43  0.6102  5.0238  0.6660  0.6326  1.3467  1.5399  5.0238  ±2.22%      751
   · import with many edits - trusting (no overhead)                 1,774.59  0.5191  1.5377  0.5635  0.5448  1.3232  1.3719  1.5377  ±1.56%      888

 ✓  bench  compression.bench.ts > CoPlainText - Real-time editing performance
 2416ms
     name                                                              hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · insert single char - private (encrypted + auto-compressed)  4,581.21  0.1125  0.8440  0.2183  0.2515  0.5647  0.6215  0.7530  ±1.35%     2291
   · insert single char - trusting (no overhead)                 4,465.76  0.1140  1.2150  0.2239  0.2639  0.5671  0.6593  0.9146  ±1.51%     2233
   · insert word - private (encrypted + auto-compressed)         2,169.37  0.3325  1.0076  0.4610  0.4917  0.9270  0.9677  0.9882  ±1.48%     1085
   · insert word - trusting (no overhead)                        2,081.81  0.3487  1.4746  0.4804  0.5056  0.9411  0.9689  1.4268  ±1.52%     1041

 ✓  bench  compression.bench.ts > CoPlainText - Large document with NAPI (~15KB)
 > [CoPlainText NAPI 15KB] Private: 35931 bytes, Trusting: 757469 bytes, Savings: 95.3%
 1711ms
     name                                                                     hz     min      max    mean     p75     p99    p995     p999     rme  samples
   · NAPI: import large document - private (encrypted + auto-compressed)  382.16  2.0972   5.1105  2.6167  2.3466  4.4068  4.7468   5.1105  ±3.73%      200
   · NAPI: import large document - trusting (no overhead)                 207.12  4.2218  11.9312  4.8281  5.2012  6.5770  7.6878  11.9312  ±2.31%      200

 ✓  bench  compression.bench.ts > CoPlainText - Single character insertions (real-time typing)
 > CoPlainText single char: simulating real-time typing
 1233ms
     name                                                                                hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · type single char - private (encryption overhead only, too small to compress)  4,515.88  0.1145  1.3555  0.2214  0.2572  0.8028  0.9695  1.1615  ±1.88%     2259
   · type single char - trusting (baseline)                                        3,903.65  0.1067  5.9442  0.2562  0.2490  1.6313  2.5971  3.7286  ±5.44%     2000

 ✓  bench  compression.bench.ts > CoPlainText - Batch insertions with NAPI 4196ms
     name                                   hz     min      max    mean     p75      p99     p995     p999     rme  samples
   · NAPI: insert paragraph - private   251.93  0.9360  24.2005  3.9694  5.7135  10.6024  14.3383  24.2005  ±4.98%      500
   · NAPI: insert paragraph - trusting  250.17  0.8861  27.7552  3.9972  5.6398  14.1652  15.8421  27.7552  ±5.59%      500

 ✓  bench  compression.bench.ts > Compression effectiveness - Realistic data at different sizes
 > === Compression Effectiveness (realistic data, 5 updates each, compress:true) ===
1KB         : Original 10855 bytes -> Compressed 13836 bytes (-27.5% savings)
5KB         : Original 33725 bytes -> Compressed 32621 bytes (3.3% savings)
10KB        : Original 56220 bytes -> Compressed 49481 bytes (12.0% savings)
 4207ms
     name                                hz     min     max    mean     p75     p99    p995    p999     rme   samples
   · baseline - measure only  35,450,921.93  0.0000  1.1814  0.0000  0.0000  0.0000  0.0000  0.0000  ±0.63%  17725461

 BENCH  Summary

   bench  trusting (no encryption, no compression) - compression.bench.ts > Import performance - Small payloads (512B)
 > [Small 512B] Private: 33601 bytes, Trusting: 28950 bytes, Compressed: 27211 bytes (19.0% vs private, 6.0% vs trusting)

    1.25x faster than private (encrypted, no compression)
    2.10x faster than private with compress:true

   bench  private (encrypted, no compression) - compression.bench.ts > Import performance - Moderate payloads (1KB)
 > [1KB] Private: 33601 bytes, Trusting: 28950 bytes, Compressed: 27210 bytes (19.0% savings)

    1.01x faster than private with compress:true
    1.47x faster than trusting (no encryption, no compression)

   bench  private with compress:true - compression.bench.ts > Import performance - Medium payloads (4KB)
 > [4KB] Private: 65361 bytes, Trusting: 56251 bytes, Compressed: 46210 bytes (29.3% savings)

    1.14x faster than trusting (no encryption, no compression)
    1.37x faster than private (encrypted, no compression)

   bench  private with compress:true - compression.bench.ts > Import performance - Large payloads (100KB)
 > [100KB] Private: 1500555 bytes, Trusting: 1285384 bytes, Compressed: 726947 bytes (51.6% savings)

    1.22x faster than trusting (no encryption, no compression)
    1.42x faster than private (encrypted, no compression)

   bench  NAPI - private with compress:true - compression.bench.ts > Import performance with NAPI - Medium payloads (4KB)

    1.10x faster than NAPI - private (encrypted, no compression)
    1.26x faster than NAPI - trusting (no encryption, no compression)

   bench  create trusting transaction (no encryption, no compression) - compression.bench.ts > Write performance - Transaction creation (4KB payload)

    1.04x faster than create private transaction with compress:true
    1.05x faster than create private transaction (no compression)

   bench  write - trusting (no encryption, no compression) - compression.bench.ts > Write performance - Small payload (~900 bytes)
 > Payload size: 2171 bytes (realistic user data)

    1.02x faster than write - private with compress:true
    1.03x faster than write - private (encrypted, no compression)

   bench  write - private (encrypted, no compression) - compression.bench.ts > Write performance - Large payload (~5KB)
 > Payload size: 6745 bytes (realistic user/post/comment data)

    1.57x faster than write - private with compress:true
    1.66x faster than write - trusting (no encryption, no compression)

   bench  100 small updates - trusting (no overhead) - compression.bench.ts > Bulk operations - Many small updates (100x)

    1.04x faster than 100 small updates - private with compress:true
    1.05x faster than 100 small updates - private (encrypted, no compression)

   bench  50 large updates - private with compress:true - compression.bench.ts > Bulk operations - Many large updates (50x, ~5KB each)

    1.04x faster than 50 large updates - trusting (no overhead)
    1.17x faster than 50 large updates - private (encrypted, no compression)

   bench  list import - private with compress:true - compression.bench.ts > List operations with compression - comparing data types
 > [List] Private: 9250 bytes, Trusting: 7814 bytes, Compressed: 8987 bytes (2.8% savings)

    1.15x faster than list import - trusting (no encryption, no compression)
    2.27x faster than list import - private (encrypted, no compression)

   bench  text import - private (encrypted + auto-compressed) - compression.bench.ts > CoPlainText - Text operations with compression
 > [Text] Private: 27584 bytes (68.0% savings vs 86282 trusting bytes)

    1.85x faster than text import - trusting (no overhead)

   bench  import long document - private (encrypted + auto-compressed) - compression.bench.ts > CoPlainText - Long document (~10KB)
 > [CoPlainText 10KB] Private: 20749 bytes, Trusting: 507807 bytes, Savings: 95.9%

    6.08x faster than import long document - trusting (no overhead)

   bench  import with many edits - trusting (no overhead) - compression.bench.ts > CoPlainText - Collaborative editing (many small edits)
 > [CoPlainText many edits] Private: 30925 bytes (23.1% savings vs 40194 trusting bytes)

    1.18x faster than import with many edits - private (encrypted + auto-compressed)

   bench  insert single char - private (encrypted + auto-compressed) - compression.bench.ts > CoPlainText - Real-time editing performance

    1.03x faster than insert single char - trusting (no overhead)
    2.11x faster than insert word - private (encrypted + auto-compressed)
    2.20x faster than insert word - trusting (no overhead)

   bench  NAPI: import large document - private (encrypted + auto-compressed) - compression.bench.ts > CoPlainText - Large document with NAPI (~15KB)
 > [CoPlainText NAPI 15KB] Private: 35931 bytes, Trusting: 757469 bytes, Savings: 95.3%

    1.85x faster than NAPI: import large document - trusting (no overhead)

   bench  type single char - private (encryption overhead only, too small to compress) - compression.bench.ts > CoPlainText - Single character insertions (real-time typing)
 > CoPlainText single char: simulating real-time typing

    1.16x faster than type single char - trusting (baseline)

   bench  NAPI: insert paragraph - private - compression.bench.ts > CoPlainText - Batch insertions with NAPI
    1.01x faster than NAPI: insert paragraph - trusting

   bench  baseline - measure only - compression.bench.ts > Compression effectiveness - Realistic data at different sizes
 > === Compression Effectiveness (realistic data, 5 updates each, compress:true) ===
1KB         : Original 10855 bytes -> Compressed 13836 bytes (-27.5% savings)
5KB         : Original 33725 bytes -> Compressed 32621 bytes (3.3% savings)
10KB        : Original 56220 bytes -> Compressed 49481 bytes (12.0% savings)

LZY

 ✓  bench  compression.bench.ts > Import performance - Small payloads (512B)
 > [Small 512B] Private: 33601 bytes, Trusting: 28951 bytes, Compressed: 27561 bytes (18.0% vs private, 4.8% vs trusting)
 1818ms
     name                                            hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · private (encrypted, no compression)       2,326.76  0.3407  3.1020  0.4298  0.4503  0.7534  1.1756  3.1010  ±2.12%     1164
   · trusting (no encryption, no compression)  2,988.33  0.2896  1.3083  0.3346  0.3304  0.4343  0.9543  1.1655  ±0.98%     1495
   · private with compress:true                1,208.39  0.3360  4.0290  0.8275  0.7532  3.7856  3.9842  4.0290  ±6.64%      605

 ✓  bench  compression.bench.ts > Import performance - Moderate payloads (1KB)
 > [1KB] Private: 33591 bytes, Trusting: 28950 bytes, Compressed: 27570 bytes (17.9% savings)
 1831ms
     name                                            hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · private (encrypted, no compression)       2,804.24  0.3213  0.9713  0.3566  0.3571  0.4817  0.8366  0.9647  ±0.78%     1403
   · trusting (no encryption, no compression)  1,806.30  0.2832  4.1220  0.5536  0.5675  3.8336  3.9371  4.1220  ±7.07%      904
   · private with compress:true                2,989.13  0.2959  0.9236  0.3345  0.3342  0.4398  0.8622  0.8933  ±0.77%     1495

 ✓  bench  compression.bench.ts > Import performance - Medium payloads (4KB)
 > [4KB] Private: 65361 bytes, Trusting: 56249 bytes, Compressed: 47331 bytes (27.6% savings)
 1821ms
     name                                            hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · private (encrypted, no compression)       1,287.07  0.5503  4.5363  0.7770  0.6522  4.0600  4.1137  4.5363  ±5.60%      644
   · trusting (no encryption, no compression)  1,749.29  0.5363  1.2207  0.5717  0.5673  1.0270  1.0460  1.2207  ±0.75%      875
   · private with compress:true                1,636.51  0.4923  6.5728  0.6111  0.5299  3.4827  4.0251  6.5728  ±5.40%      819

 ✓  bench  compression.bench.ts > Import performance - Large payloads (100KB)
 > [100KB] Private: 1500554 bytes, Trusting: 1285398 bytes, Compressed: 731027 bytes (51.3% savings)
 4288ms
     name                                           hz      min      max     mean      p75      p99     p995     p999     rme  samples
   · private (encrypted, no compression)       67.1469  12.0753  21.3381  14.8927  19.1794  20.8149  21.3381  21.3381  ±4.23%      100
   · trusting (no encryption, no compression)  81.4217  11.6660  13.2511  12.2817  12.4128  13.1203  13.2511  13.2511  ±0.54%      100
   · private with compress:true                86.4802   8.0990  15.4226  11.5633  12.5033  15.2028  15.4226  15.4226  ±3.10%      100

 ✓  bench  compression.bench.ts > Import performance with NAPI - Medium payloads (4KB)
 1829ms
     name                                                   hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · NAPI - private (encrypted, no compression)       4,164.49  0.1982  0.9214  0.2401  0.2318  0.7877  0.8606  0.9058  ±1.34%     2083
   · NAPI - trusting (no encryption, no compression)  3,653.45  0.2488  3.7088  0.2737  0.2633  0.7695  0.9511  1.4750  ±1.88%     1827
   · NAPI - private with compress:true                4,113.28  0.1991  1.0094  0.2431  0.2386  0.8008  0.8773  0.9487  ±1.31%     2057

 ✓  bench  compression.bench.ts > Write performance - Transaction creation (4KB payload)
 1836ms
     name                                                               hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · create private transaction (no compression)                  5,060.08  0.1319  3.4771  0.1976  0.2193  0.3170  0.3702  1.1704  ±1.79%     2531
   · create trusting transaction (no encryption, no compression)  5,419.51  0.1159  0.9778  0.1845  0.2111  0.3250  0.3726  0.8843  ±1.03%     2710
   · create private transaction with compress:true                5,029.99  0.1207  0.8002  0.1988  0.2285  0.3342  0.4186  0.7522  ±0.95%     2516

 ✓  bench  compression.bench.ts > Write performance - Small payload (~900 bytes)
 > Payload size: 2171 bytes (realistic user data)
 1813ms
     name                                                    hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · write - private (encrypted, no compression)       5,557.66  0.1097  0.9955  0.1799  0.2096  0.2935  0.3260  0.7390  ±0.99%     2780
   · write - trusting (no encryption, no compression)  5,667.07  0.1033  0.9517  0.1765  0.2092  0.2810  0.3075  0.3912  ±0.95%     2834
   · write - private with compress:true                5,530.72  0.1098  1.0639  0.1808  0.2117  0.2934  0.3236  0.8223  ±1.01%     2766

 ✓  bench  compression.bench.ts > Write performance - Large payload (~5KB)
 > Payload size: 6745 bytes (realistic user/post/comment data)
 1861ms
     name                                                    hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · write - private (encrypted, no compression)       3,837.36  0.1563  7.1799  0.2606  0.2319  0.8452  2.2351  6.5136  ±6.06%     1920
   · write - trusting (no encryption, no compression)  1,850.54  0.1221  7.2418  0.5404  0.4875  4.4988  4.9686  7.0094  ±9.36%     1000
   · write - private with compress:true                3,064.66  0.1476  7.5556  0.3263  0.2242  4.4149  4.6146  4.7625  ±9.06%     1533

 ✓  bench  compression.bench.ts > Bulk operations - Many small updates (100x)
 4597ms
     name                                                          hz      min      max     mean      p75      p99     p995     p999     rme  samples
   · 100 small updates - private (encrypted, no compression)  31.8852  11.3893  45.1590  31.3625  39.4972  45.1590  45.1590  45.1590  ±8.89%       50
   · 100 small updates - trusting (no overhead)               34.4204   9.6485  47.9837  29.0525  37.4951  47.9837  47.9837  47.9837  ±9.59%       50
   · 100 small updates - private with compress:true           41.7908  10.2306  39.0930  23.9287  29.2016  39.0930  39.0930  39.0930  ±8.68%       50

 ✓  bench  compression.bench.ts > Bulk operations - Many large updates (50x, ~5KB each)
 2417ms
     name                                                         hz      min      max     mean      p75      p99     p995     p999      rme  samples
   · 50 large updates - private (encrypted, no compression)  29.8564  28.3701  41.9794  33.4937  37.3975  41.9794  41.9794  41.9794   ±5.50%       20
   · 50 large updates - trusting (no overhead)               29.4714  13.7307  53.8772  33.9313  38.5262  53.8772  53.8772  53.8772  ±12.86%       20
   · 50 large updates - private with compress:true           47.8166  10.0860  37.2287  20.9132  29.0324  37.2287  37.2287  37.2287  ±16.67%       24

 ✓  bench  compression.bench.ts > List operations with compression - comparing data types
 > [List] Private: 9262 bytes, Trusting: 7814 bytes, Compressed: 9028 bytes (2.5% savings)
 1819ms
     name                                                          hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · list import - private (encrypted, no compression)       1,795.55  0.1530  9.3718  0.5569  0.6418  5.1636  5.2038  9.3718  ±9.95%      899
   · list import - trusting (no encryption, no compression)  3,617.95  0.1280  9.0970  0.2764  0.1637  3.5041  4.9613  8.7641  ±9.53%     1812
   · list import - private with compress:true                4,156.74  0.1525  5.5379  0.2406  0.1734  1.0807  3.6402  5.0961  ±6.83%     2081

 ✓  bench  compression.bench.ts > CoPlainText - Text operations with compression
 > [Text] Private: 68669 bytes (40.9% savings vs 116180 trusting bytes)
 1627ms
     name                                                     hz     min      max    mean     p75      p99     p995     p999     rme  samples
   · text import - private (encrypted + auto-compressed)  530.35  1.6254   3.7658  1.8855  1.8195   3.5835   3.5940   3.7658  ±2.83%      266
   · text import - trusting (no overhead)                 222.69  1.8861  13.1859  4.4905  4.5006  12.3217  12.8183  13.1859  ±7.78%      200

 ✓  bench  compression.bench.ts > CoPlainText - Long document (~10KB)
 > [CoPlainText 10KB] Private: 105268 bytes, Trusting: 507806 bytes, Savings: 79.3%
 3248ms
     name                                                               hz     min      max     mean      p75      p99     p995     p999     rme  samples
   · import long document - private (encrypted + auto-compressed)   328.02  2.6523   5.1235   3.0486   2.9513   4.8359   5.0002   5.1235  ±2.50%      200
   · import long document - trusting (no overhead)                 83.4445  5.8416  27.8963  11.9840  15.6105  18.2210  18.8211  27.8963  ±4.70%      200

 ✓  bench  compression.bench.ts > CoPlainText - Collaborative editing (many small edits)
 > [CoPlainText many edits] Private: 33595 bytes (16.4% savings vs 40195 trusting bytes)
 1210ms
     name                                                                  hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · import with many edits - private (encrypted + auto-compressed)  1,359.93  0.6410  5.7673  0.7353  0.7041  1.8704  1.9446  5.7673  ±2.86%      680
   · import with many edits - trusting (no overhead)                 1,660.65  0.5205  2.0377  0.6022  0.5700  1.8576  1.9098  2.0377  ±2.49%      831

 ✓  bench  compression.bench.ts > CoPlainText - Real-time editing performance
 2433ms
     name                                                              hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · insert single char - private (encrypted + auto-compressed)  4,326.60  0.1247  1.4090  0.2311  0.2537  1.0979  1.1854  1.3150  ±2.49%     2164
   · insert single char - trusting (no overhead)                 4,267.63  0.1232  1.5188  0.2343  0.2603  1.1691  1.2760  1.3796  ±2.68%     2134
   · insert word - private (encrypted + auto-compressed)         2,031.99  0.3308  1.8336  0.4921  0.4899  1.4791  1.5178  1.6301  ±2.82%     1016
   · insert word - trusting (no overhead)                        1,935.27  0.3490  1.6691  0.5167  0.5120  1.5598  1.5964  1.6524  ±2.88%     1000

 ✓  bench  compression.bench.ts > CoPlainText - Large document with NAPI (~15KB)
 > [CoPlainText NAPI 15KB] Private: 162132 bytes, Trusting: 757471 bytes, Savings: 78.6%
 2092ms
     name                                                                     hz     min      max    mean     p75      p99     p995     p999     rme  samples
   · NAPI: import large document - private (encrypted + auto-compressed)  251.03  2.9502  24.6753  3.9836  3.5477  13.2175  13.2580  24.6753  ±7.51%      200
   · NAPI: import large document - trusting (no overhead)                 186.15  4.5193  12.5335  5.3719  6.0442   7.6441   8.7055  12.5335  ±2.74%      200

 ✓  bench  compression.bench.ts > CoPlainText - Single character insertions (real-time typing)
 > CoPlainText single char: simulating real-time typing
 1223ms
     name                                                                                hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · type single char - private (encryption overhead only, too small to compress)  4,446.21  0.1193  1.4659  0.2249  0.2570  1.0793  1.1655  1.2980  ±2.25%     2224
   · type single char - trusting (baseline)                                        4,352.09  0.1213  2.2967  0.2298  0.2645  1.1286  1.2241  1.3271  ±2.43%     2177

 ✓  bench  compression.bench.ts > CoPlainText - Batch insertions with NAPI 4666ms
     name                                   hz     min      max    mean     p75      p99     p995     p999     rme  samples
   · NAPI: insert paragraph - private   239.17  0.9615  18.5742  4.1811  6.2012  10.2070  14.3021  18.5742  ±4.97%      500
   · NAPI: insert paragraph - trusting  211.71  0.8780  23.0511  4.7234  6.7004  16.1785  18.1532  23.0511  ±5.39%      500

 ✓  bench  compression.bench.ts > Compression effectiveness - Realistic data at different sizes
 > === Compression Effectiveness (realistic data, 5 updates each, compress:true) ===
1KB         : Original 10855 bytes -> Compressed 14041 bytes (-29.4% savings)
5KB         : Original 33725 bytes -> Compressed 33656 bytes (0.2% savings)
10KB        : Original 56220 bytes -> Compressed 51601 bytes (8.2% savings)
 4327ms
     name                                hz     min     max    mean     p75     p99    p995    p999     rme   samples
   · baseline - measure only  33,798,038.85  0.0000  1.1790  0.0000  0.0000  0.0000  0.0000  0.0001  ±1.11%  16899020

 BENCH  Summary

   bench  trusting (no encryption, no compression) - compression.bench.ts > Import performance - Small payloads (512B)
 > [Small 512B] Private: 33601 bytes, Trusting: 28951 bytes, Compressed: 27561 bytes (18.0% vs private, 4.8% vs trusting)

    1.28x faster than private (encrypted, no compression)
    2.47x faster than private with compress:true

   bench  private with compress:true - compression.bench.ts > Import performance - Moderate payloads (1KB)
 > [1KB] Private: 33591 bytes, Trusting: 28950 bytes, Compressed: 27570 bytes (17.9% savings)

    1.07x faster than private (encrypted, no compression)
    1.65x faster than trusting (no encryption, no compression)

   bench  trusting (no encryption, no compression) - compression.bench.ts > Import performance - Medium payloads (4KB)
 > [4KB] Private: 65361 bytes, Trusting: 56249 bytes, Compressed: 47331 bytes (27.6% savings)

    1.07x faster than private with compress:true
    1.36x faster than private (encrypted, no compression)

   bench  private with compress:true - compression.bench.ts > Import performance - Large payloads (100KB)
 > [100KB] Private: 1500554 bytes, Trusting: 1285398 bytes, Compressed: 731027 bytes (51.3% savings)

    1.06x faster than trusting (no encryption, no compression)
    1.29x faster than private (encrypted, no compression)

   bench  NAPI - private (encrypted, no compression) - compression.bench.ts > Import performance with NAPI - Medium payloads (4KB)

    1.01x faster than NAPI - private with compress:true
    1.14x faster than NAPI - trusting (no encryption, no compression)

   bench  create trusting transaction (no encryption, no compression) - compression.bench.ts > Write performance - Transaction creation (4KB payload)

    1.07x faster than create private transaction (no compression)
    1.08x faster than create private transaction with compress:true

   bench  write - trusting (no encryption, no compression) - compression.bench.ts > Write performance - Small payload (~900 bytes)
 > Payload size: 2171 bytes (realistic user data)

    1.02x faster than write - private (encrypted, no compression)
    1.02x faster than write - private with compress:true

   bench  write - private (encrypted, no compression) - compression.bench.ts > Write performance - Large payload (~5KB)
 > Payload size: 6745 bytes (realistic user/post/comment data)

    1.25x faster than write - private with compress:true
    2.07x faster than write - trusting (no encryption, no compression)

   bench  100 small updates - private with compress:true - compression.bench.ts > Bulk operations - Many small updates (100x)

    1.21x faster than 100 small updates - trusting (no overhead)
    1.31x faster than 100 small updates - private (encrypted, no compression)

   bench  50 large updates - private with compress:true - compression.bench.ts > Bulk operations - Many large updates (50x, ~5KB each)

    1.60x faster than 50 large updates - private (encrypted, no compression)
    1.62x faster than 50 large updates - trusting (no overhead)

   bench  list import - private with compress:true - compression.bench.ts > List operations with compression - comparing data types
 > [List] Private: 9262 bytes, Trusting: 7814 bytes, Compressed: 9028 bytes (2.5% savings)

    1.15x faster than list import - trusting (no encryption, no compression)
    2.32x faster than list import - private (encrypted, no compression)

   bench  text import - private (encrypted + auto-compressed) - compression.bench.ts > CoPlainText - Text operations with compression
 > [Text] Private: 68669 bytes (40.9% savings vs 116180 trusting bytes)

    2.38x faster than text import - trusting (no overhead)

   bench  import long document - private (encrypted + auto-compressed) - compression.bench.ts > CoPlainText - Long document (~10KB)
 > [CoPlainText 10KB] Private: 105268 bytes, Trusting: 507806 bytes, Savings: 79.3%

    3.93x faster than import long document - trusting (no overhead)

   bench  import with many edits - trusting (no overhead) - compression.bench.ts > CoPlainText - Collaborative editing (many small edits)
 > [CoPlainText many edits] Private: 33595 bytes (16.4% savings vs 40195 trusting bytes)

    1.22x faster than import with many edits - private (encrypted + auto-compressed)

   bench  insert single char - private (encrypted + auto-compressed) - compression.bench.ts > CoPlainText - Real-time editing performance

    1.01x faster than insert single char - trusting (no overhead)
    2.13x faster than insert word - private (encrypted + auto-compressed)
    2.24x faster than insert word - trusting (no overhead)

   bench  NAPI: import large document - private (encrypted + auto-compressed) - compression.bench.ts > CoPlainText - Large document with NAPI (~15KB)
 > [CoPlainText NAPI 15KB] Private: 162132 bytes, Trusting: 757471 bytes, Savings: 78.6%

    1.35x faster than NAPI: import large document - trusting (no overhead)

   bench  type single char - private (encryption overhead only, too small to compress) - compression.bench.ts > CoPlainText - Single character insertions (real-time typing)
 > CoPlainText single char: simulating real-time typing

    1.02x faster than type single char - trusting (baseline)

   bench  NAPI: insert paragraph - private - compression.bench.ts > CoPlainText - Batch insertions with NAPI
    1.13x faster than NAPI: insert paragraph - trusting

   bench  baseline - measure only - compression.bench.ts > Compression effectiveness - Realistic data at different sizes
 > === Compression Effectiveness (realistic data, 5 updates each, compress:true) ===
1KB         : Original 10855 bytes -> Compressed 14041 bytes (-29.4% savings)
5KB         : Original 33725 bytes -> Compressed 33656 bytes (0.2% savings)
10KB        : Original 56220 bytes -> Compressed 51601 bytes (8.2% savings)

Manual testing instructions

Tests

Tests have been added and/or updated
Tests have not been updated, because:
I need help with writing tests

Checklist

I've updated the part of the docs that are affected the PR changes
I've generated a changeset, if a version bump is required
I've updated the jsDoc comments to the public APIs I've modified, or added them when missing

vercel · 2025-10-20T11:22:39Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Review	Updated (UTC)
clerk-demo	Ready	Preview, Comment	Jan 17, 2026 5:14am
design-system	Ready	Preview, Comment	Jan 17, 2026 5:14am
file-upload-demo	Ready	Preview, Comment	Jan 17, 2026 5:14am
form-demo	Ready	Preview, Comment	Jan 17, 2026 5:14am
gcmp-homepage	Ready	Preview, Comment	Jan 17, 2026 5:14am
image-upload-demo	Ready	Preview, Comment	Jan 17, 2026 5:14am
jazz-chat	Ready	Preview, Comment	Jan 17, 2026 5:14am
jazz-chat-1	Ready	Preview, Comment	Jan 17, 2026 5:14am
jazz-chat-2	Error		Jan 17, 2026 5:14am
jazz-filestream	Ready	Preview, Comment	Jan 17, 2026 5:14am
jazz-image-upload	Ready	Preview, Comment	Jan 17, 2026 5:14am
jazz-inspector	Ready	Preview, Comment	Jan 17, 2026 5:14am
jazz-multi-cursors	Ready	Preview, Comment	Jan 17, 2026 5:14am
jazz-organization	Ready	Preview, Comment	Jan 17, 2026 5:14am
jazz-paper-scissors	Ready	Preview, Comment	Jan 17, 2026 5:14am
jazz-richtext	Ready	Preview, Comment	Jan 17, 2026 5:14am
jazz-richtext-prosekit	Ready	Preview, Comment	Jan 17, 2026 5:14am
jazz-richtext-tiptap	Ready	Preview, Comment	Jan 17, 2026 5:14am
jazz-todo	Ready	Preview, Comment	Jan 17, 2026 5:14am
jazz-vector-search	Ready	Preview, Comment	Jan 17, 2026 5:14am
jazz-version-history	Canceled		Jan 17, 2026 5:14am
music-demo	Ready	Preview, Comment	Jan 17, 2026 5:14am
passkey-demo	Ready	Preview, Comment	Jan 17, 2026 5:14am
passphrase-auth-demo	Ready	Preview, Comment	Jan 17, 2026 5:14am
quint-ui	Ready	Preview, Comment	Jan 17, 2026 5:14am
reactions-demo	Ready	Preview, Comment	Jan 17, 2026 5:14am

1 Skipped Deployment

Project	Deployment	Review	Updated (UTC)
jazz-homepage	Ignored		Jan 17, 2026 5:14am

blacksmith-sh · 2025-10-20T11:30:16Z

✅ All jobs passed

Summary: 7 successful workflows

✅ End-to-End Tests for React Native (1 job succeeded)
✅ Napi CI (23 jobs succeeded)
✅ Pre-Publish tagged Pull Requests
✅ Playwright Tests (2 jobs succeeded)
✅ Code quality (1 job succeeded)
✅ Jazz Run Tests (1 job succeeded)
✅ Unit Tests (1 job succeeded)

Last updated: 2025-10-20 11:34:54 UTC

vercel · 2026-01-15T20:54:30Z

Deployment failed with the following error:

Creating the Deployment Timed Out.

feat: added compression at session_log side

f30525c

antoniomuso force-pushed the feat/compression-in-changes-side branch from ecd0eb8 to f30525c Compare October 20, 2025 11:23

vercel Bot deployed to Preview – jazz-vector-search October 20, 2025 11:25 View deployment

gdorsi marked this pull request as draft October 20, 2025 18:47

feat: compression with common json files

4778317

vercel Bot deployed to Preview – jazz-vector-search October 20, 2025 20:03 View deployment

vercel Bot deployed to Preview – jazz-vector-search October 21, 2025 06:32 View deployment

antoniomuso force-pushed the feat/compression-in-changes-side branch from bb46417 to 5ea261f Compare October 21, 2025 06:44

vercel Bot deployed to Preview – jazz-vector-search October 21, 2025 06:49 View deployment

antoniomuso force-pushed the feat/compression-in-changes-side branch from 5ea261f to 0adaf44 Compare October 21, 2025 07:03

vercel Bot deployed to Preview – jazz-vector-search October 21, 2025 07:08 View deployment

antoniomuso force-pushed the feat/compression-in-changes-side branch from 0adaf44 to 2a6234c Compare October 21, 2025 07:27

vercel Bot deployed to Preview – jazz-vector-search October 21, 2025 07:32 View deployment

antoniomuso force-pushed the feat/compression-in-changes-side branch from 2a6234c to 41938ab Compare October 21, 2025 12:55

vercel Bot deployed to Preview – jazz-vector-search October 21, 2025 12:59 View deployment

chore: updated benchmarks

52f0dd6

antoniomuso force-pushed the feat/compression-in-changes-side branch from 41938ab to 52f0dd6 Compare October 21, 2025 13:14

vercel Bot deployed to Preview – jazz-vector-search October 21, 2025 13:22 View deployment

feat: added lzy

88938a7

vercel Bot deployed to Preview – jazz-vector-search October 21, 2025 14:06 View deployment

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: added compression at session_log side#3075

feat: added compression at session_log side#3075
antoniomuso wants to merge 4 commits into
mainfrom
feat/compression-in-changes-side

antoniomuso commented Oct 20, 2025 •

edited

Loading

Uh oh!

vercel Bot commented Oct 20, 2025 •

edited

Loading

Uh oh!

blacksmith-sh Bot commented Oct 20, 2025 •

edited

Loading

Uh oh!

vercel Bot commented Jan 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

antoniomuso commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Manual testing instructions

Tests

Checklist

Uh oh!

vercel Bot commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

blacksmith-sh Bot commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vercel Bot commented Jan 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

antoniomuso commented Oct 20, 2025 •

edited

Loading

vercel Bot commented Oct 20, 2025 •

edited

Loading

blacksmith-sh Bot commented Oct 20, 2025 •

edited

Loading