Skip to content

support large jsonb values #2

@wangrunji0408

Description

@wangrunji0408

Risingwave ran into this panic when building a jsonb array.

thread 'risingwave-streaming-actor' panicked at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/jsonbb-0.1.3/src/entry.rs:69:9:
offset too large
...
  20:     0x562e317889f0 - jsonbb::builder::Builder<W>::add_value::hb52ff40cea7658ec
  21:     0x562e3182cecb - <risingwave_common::array::jsonb_array::JsonbArrayBuilder as risingwave_common::array::ArrayBuilder>::append_n::h11d8feb14fa4e818
                               at /risingwave/src/common/src/array/jsonb_array.rs:57:21
  22:     0x562e32d97ab2 - risingwave_common::array::ArrayBuilderImpl::append_n::h29150fb6718c1242
  23:     0x562e327dc789 - risingwave_common::array::ArrayBuilderImpl::append::h07772f518f08d33c
                               at /risingwave/src/common/src/array/mod.rs:492:9
  24:     0x562e327dc789 - risingwave_stream::common::builder::StreamChunkBuilder::append_iter::h0f99e028cdec90d6
                               at /risingwave/src/stream/src/common/builder.rs:97:37
  25:     0x562e327dc8c3 - risingwave_stream::common::builder::StreamChunkBuilder::append_row::hbfb0e0cd55d60cd3
                               at /risingwave/src/stream/src/common/builder.rs:105:9
  26:     0x562e327dc8c3 - risingwave_stream::common::builder::StreamChunkBuilder::append_record::h41d5bc3b9349e0ac
                               at /risingwave/src/stream/src/common/builder.rs:112:43
  27:     0x562e323bfe13 - risingwave_stream::executor::hash_agg::HashAggExecutor<K,S>::flush_data::{{closure}}::h69a40658dc68653d
                               at /risingwave/src/stream/src/executor/hash_agg.rs:505:42
...

I guess the reason is that an accumulated jsonb value exceeded the max size (2^29 = 512MB). We should either increase the pointer size to 8 bytes to support larger jsonb values, or return an error instead of panic in this case.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions