Repository navigation
crl-release-26.2: pebble: add max user key size table property and metric - #6214
Open
RaduBerinde wants to merge 4 commits into
Conversation
Member
When `Compressor.Compress` abandons compression because the block did not
shrink by `MinReductionPercent`, it cleared the returned `Setting`'s
`Algorithm` but left the profile's `Level` intact. The resulting
`{NoAlgorithm, N}` pair does not match `compression.NoCompression`, so
those bytes accumulated in `CompressionStats.others` under a synthetic
bucket (rendered as e.g. `None3`) instead of the no-compression bucket.
Reset the whole setting to `compression.NoCompression`.
`CopySpan` copies data blocks that are not in the block cache verbatim, bypassing the compressor. The columnar writer overwrites the `CompressionStats` property with the compressor's stats, so the stats of a table produced by `CopySpan` only covered the recompressed (cache hit) data blocks and the index blocks. Readers then used these incomplete stats for compression metrics. - Add `block.Compressor.CopiedBlock`, which records the stats for an already-encoded block. The algorithm and decompressed size are derived from the block; the compression level is inferred from the source table's stats when unambiguous. - Call it for every block copied verbatim by `RawColumnWriter.copyDataBlocks`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copy compactions create a new table backing and copy the input table's stats, but never populated the backing's properties. The table stats collector skips tables that have stats, so the properties of these tables were not loaded until the DB was reopened, and the metrics derived from them (e.g. compression metrics, tombstone counts) did not reflect these tables. - `sstable.CopySpan` now also returns the properties of the output table. - External to local copies populate the new backing with the properties returned by `CopySpan`; local to shared copies (identical files) copy the input backing's properties via the new `TableBacking.CopyPropertiesFrom`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was no way to know the size of the largest key in an sstable or in the LSM. - Add the `sstable.Properties.MaxUserKeySize` property (`pebble.max.user-key.size`): the size of the largest user key in the table, including range deletion and range key bounds. It is only written when non-zero, so an absent property means unknown (e.g. tables written by older versions, which read it as a user property). Suffix rewriting and `CopySpan` derive an upper bound from the source table's property. - Keep it in `TableBackingProperties` (in existing padding). - Add `Metrics.Keys.MaxUserKeySize` (max across the LSM's tables) and `Metrics.Keys.MaxUserKeySizeUnknownTables`, computed by the table properties annotator, and show them in the `KEYS` table. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sumeerbhola
approved these changes
Oct 9, 2026
RaduBerinde
force-pushed
the
pebble-add-max-user-key-size-table-property-and-me-26.2
branch
from
October 9, 2026 14:45
3a3d1ad to
2f1d990
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
block: account min-reduction fallbacks as uncompressed
When
Compressor.Compressabandons compression because the block did notshrink by
MinReductionPercent, it cleared the returnedSetting'sAlgorithmbut left the profile'sLevelintact. The resulting{NoAlgorithm, N}pair does not matchcompression.NoCompression, sothose bytes accumulated in
CompressionStats.othersunder a syntheticbucket (rendered as e.g.
None3) instead of the no-compression bucket.Reset the whole setting to
compression.NoCompression.sstable: account for copied blocks in CopySpan compression stats
CopySpancopies data blocks that are not in the block cache verbatim,bypassing the compressor. The columnar writer overwrites the
CompressionStatsproperty with the compressor's stats, so the stats of atable produced by
CopySpanonly covered the recompressed (cache hit) datablocks and the index blocks. Readers then used these incomplete stats for
compression metrics.
block.Compressor.CopiedBlock, which records the stats for analready-encoded block. The algorithm and decompressed size are derived
from the block; the compression level is inferred from the source
table's stats when unambiguous.
RawColumnWriter.copyDataBlocks.Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com
pebble: populate table backing properties in copy compactions
Copy compactions create a new table backing and copy the input table's
stats, but never populated the backing's properties. The table stats
collector skips tables that have stats, so the properties of these tables
were not loaded until the DB was reopened, and the metrics derived from them
(e.g. compression metrics, tombstone counts) did not reflect these tables.
sstable.CopySpannow also returns the properties of the output table.returned by
CopySpan; local to shared copies (identical files) copy theinput backing's properties via the new
TableBacking.CopyPropertiesFrom.Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com
pebble: add max user key size table property and metric
There was no way to know the size of the largest key in an sstable or in
the LSM.
sstable.Properties.MaxUserKeySizeproperty(
pebble.max.user-key.size): the size of the largest user key in thetable, including range deletion and range key bounds. It is only written
when non-zero, so an absent property means unknown (e.g. tables written
by older versions, which read it as a user property). Suffix rewriting
and
CopySpanderive an upper bound from the source table's property.TableBackingProperties(in existing padding).Metrics.Keys.MaxUserKeySize(max across the LSM's tables) andMetrics.Keys.MaxUserKeySizeUnknownTables, computed by the tableproperties annotator, and show them in the
KEYStable.Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com