Skip to content

crl-release-26.2: pebble: add max user key size table property and metric - #6214

Open
RaduBerinde wants to merge 4 commits into
cockroachdb:crl-release-26.2from
RaduBerinde:pebble-add-max-user-key-size-table-property-and-me-26.2
Open

RaduBerinde wants to merge 4 commits into
cockroachdb:crl-release-26.2from
RaduBerinde:pebble-add-max-user-key-size-table-property-and-me-26.2

Conversation

@RaduBerinde

@RaduBerinde RaduBerinde commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

block: account min-reduction fallbacks as uncompressed

When Compressor.Compress abandons compression because the block did not
shrink by MinReductionPercent, it cleared the returned Setting's
Algorithm but left the profile's Level intact. The resulting
{NoAlgorithm, N} pair does not match compression.NoCompression, so
those bytes accumulated in CompressionStats.others under a synthetic
bucket (rendered as e.g. None3) instead of the no-compression bucket.

Reset the whole setting to compression.NoCompression.

sstable: account for copied blocks in CopySpan compression stats

CopySpan copies data blocks that are not in the block cache verbatim,
bypassing the compressor. The columnar writer overwrites the
CompressionStats property with the compressor's stats, so the stats of a
table produced by CopySpan only covered the recompressed (cache hit) data
blocks and the index blocks. Readers then used these incomplete stats for
compression metrics.

  • Add block.Compressor.CopiedBlock, which records the stats for an
    already-encoded block. The algorithm and decompressed size are derived
    from the block; the compression level is inferred from the source
    table's stats when unambiguous.
  • Call it for every block copied verbatim by RawColumnWriter.copyDataBlocks.

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com

pebble: populate table backing properties in copy compactions

Copy compactions create a new table backing and copy the input table's
stats, but never populated the backing's properties. The table stats
collector skips tables that have stats, so the properties of these tables
were not loaded until the DB was reopened, and the metrics derived from them
(e.g. compression metrics, tombstone counts) did not reflect these tables.

  • sstable.CopySpan now also returns the properties of the output table.
  • External to local copies populate the new backing with the properties
    returned by CopySpan; local to shared copies (identical files) copy the
    input backing's properties via the new TableBacking.CopyPropertiesFrom.

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com

pebble: add max user key size table property and metric

There was no way to know the size of the largest key in an sstable or in
the LSM.

  • Add the sstable.Properties.MaxUserKeySize property
    (pebble.max.user-key.size): the size of the largest user key in the
    table, including range deletion and range key bounds. It is only written
    when non-zero, so an absent property means unknown (e.g. tables written
    by older versions, which read it as a user property). Suffix rewriting
    and CopySpan derive an upper bound from the source table's property.
  • Keep it in TableBackingProperties (in existing padding).
  • Add Metrics.Keys.MaxUserKeySize (max across the LSM's tables) and
    Metrics.Keys.MaxUserKeySizeUnknownTables, computed by the table
    properties annotator, and show them in the KEYS table.

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com

@RaduBerinde
RaduBerinde requested a review from a team as a code owner October 9, 2026 03:57
@cockroach-teamcity

Copy link
Copy Markdown
Member

This change is Reviewable

RaduBerinde and others added 4 commits October 8, 2026 21:22
When `Compressor.Compress` abandons compression because the block did not
shrink by `MinReductionPercent`, it cleared the returned `Setting`'s
`Algorithm` but left the profile's `Level` intact. The resulting
`{NoAlgorithm, N}` pair does not match `compression.NoCompression`, so
those bytes accumulated in `CompressionStats.others` under a synthetic
bucket (rendered as e.g. `None3`) instead of the no-compression bucket.

Reset the whole setting to `compression.NoCompression`.
`CopySpan` copies data blocks that are not in the block cache verbatim,
bypassing the compressor. The columnar writer overwrites the
`CompressionStats` property with the compressor's stats, so the stats of a
table produced by `CopySpan` only covered the recompressed (cache hit) data
blocks and the index blocks. Readers then used these incomplete stats for
compression metrics.

- Add `block.Compressor.CopiedBlock`, which records the stats for an
  already-encoded block. The algorithm and decompressed size are derived
  from the block; the compression level is inferred from the source
  table's stats when unambiguous.
- Call it for every block copied verbatim by `RawColumnWriter.copyDataBlocks`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copy compactions create a new table backing and copy the input table's
stats, but never populated the backing's properties. The table stats
collector skips tables that have stats, so the properties of these tables
were not loaded until the DB was reopened, and the metrics derived from them
(e.g. compression metrics, tombstone counts) did not reflect these tables.

- `sstable.CopySpan` now also returns the properties of the output table.
- External to local copies populate the new backing with the properties
  returned by `CopySpan`; local to shared copies (identical files) copy the
  input backing's properties via the new `TableBacking.CopyPropertiesFrom`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was no way to know the size of the largest key in an sstable or in
the LSM.

- Add the `sstable.Properties.MaxUserKeySize` property
  (`pebble.max.user-key.size`): the size of the largest user key in the
  table, including range deletion and range key bounds. It is only written
  when non-zero, so an absent property means unknown (e.g. tables written
  by older versions, which read it as a user property). Suffix rewriting
  and `CopySpan` derive an upper bound from the source table's property.
- Keep it in `TableBackingProperties` (in existing padding).
- Add `Metrics.Keys.MaxUserKeySize` (max across the LSM's tables) and
  `Metrics.Keys.MaxUserKeySizeUnknownTables`, computed by the table
  properties annotator, and show them in the `KEYS` table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@RaduBerinde
RaduBerinde force-pushed the pebble-add-max-user-key-size-table-property-and-me-26.2 branch from 3a3d1ad to 2f1d990 Compare October 9, 2026 14:45

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants