Move ListLikeArray to arrow-array to be shared with json writer and parquet unshredding#9437
Move ListLikeArray to arrow-array to be shared with json writer and parquet unshredding#9437liamzwbao wants to merge 1 commit intoapache:mainfrom
ListLikeArray to arrow-array to be shared with json writer and parquet unshredding#9437Conversation
ListLikeArray to arrow-array to be shared with json writer and…ListLikeArray to arrow-array to be shared with json writer and parquet unshredding
7cc2313 to
419c537
Compare
… parquet unshredding
419c537 to
04680b1
Compare
scovich
left a comment
There was a problem hiding this comment.
Seems reasonable, but would it make sense to actually use it in the JSON writer while we're at it? It's not dead code tho -- variant still uses it -- so this is largely a matter of preference and sequencing that would likely depend on how big the JSON change is.
I believe there's also a StringLikeArray trait -- should we give it the same treatment (not necessarily in this PR)?
I'm going to create a PR for json list encoder/decoder, and this change will help reduce duplication there. I kept it separate so we don’t need to review both changes at once, and can instead focus on the JSON logic in that upcoming PR. Re |
Which issue does this PR close?
ListViewin arrow-json #9340.Rationale for this change
Json writers for ListLike types (List/ListView/FixedSizeList) are pretty similar apart from the element range representation. We already had a good way to abstract this kind of encoder in parquet variant unshredding. Given this, it would be good to move this
ListLikeArraytrait to arrow-array to be shared with json/parquetWhat changes are included in this PR?
Move
ListLikeArraytrait from parquet-variant-compute to arrow-arrayAre these changes tested?
Covered by existing tests
Are there any user-facing changes?
New pub trait in arrow-array