Skip to content

Comments

Move ListLikeArray to arrow-array to be shared with json writer and parquet unshredding#9437

Open
liamzwbao wants to merge 1 commit intoapache:mainfrom
liamzwbao:issue-9340-list-like-array-refactor
Open

Move ListLikeArray to arrow-array to be shared with json writer and parquet unshredding#9437
liamzwbao wants to merge 1 commit intoapache:mainfrom
liamzwbao:issue-9340-list-like-array-refactor

Conversation

@liamzwbao
Copy link
Contributor

@liamzwbao liamzwbao commented Feb 19, 2026

Which issue does this PR close?

Rationale for this change

Json writers for ListLike types (List/ListView/FixedSizeList) are pretty similar apart from the element range representation. We already had a good way to abstract this kind of encoder in parquet variant unshredding. Given this, it would be good to move this ListLikeArray trait to arrow-array to be shared with json/parquet

What changes are included in this PR?

Move ListLikeArray trait from parquet-variant-compute to arrow-array

Are these changes tested?

Covered by existing tests

Are there any user-facing changes?

New pub trait in arrow-array

@liamzwbao liamzwbao marked this pull request as ready for review February 19, 2026 03:01
@github-actions github-actions bot added arrow Changes to the arrow crate parquet-variant parquet-variant* crates labels Feb 19, 2026
@liamzwbao liamzwbao changed the title Move ListLikeArray to arrow-array to be shared with json writer and… Move ListLikeArray to arrow-array to be shared with json writer and parquet unshredding Feb 19, 2026
@liamzwbao liamzwbao force-pushed the issue-9340-list-like-array-refactor branch from 7cc2313 to 419c537 Compare February 19, 2026 03:03
@liamzwbao liamzwbao force-pushed the issue-9340-list-like-array-refactor branch from 419c537 to 04680b1 Compare February 19, 2026 03:22
Copy link
Contributor

@scovich scovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems reasonable, but would it make sense to actually use it in the JSON writer while we're at it? It's not dead code tho -- variant still uses it -- so this is largely a matter of preference and sequencing that would likely depend on how big the JSON change is.

I believe there's also a StringLikeArray trait -- should we give it the same treatment (not necessarily in this PR)?

@liamzwbao
Copy link
Contributor Author

Seems reasonable, but would it make sense to actually use it in the JSON writer while we're at it? It's not dead code tho -- variant still uses it -- so this is largely a matter of preference and sequencing that would likely depend on how big the JSON change is.

I believe there's also a StringLikeArray trait -- should we give it the same treatment (not necessarily in this PR)?

I'm going to create a PR for json list encoder/decoder, and this change will help reduce duplication there. I kept it separate so we don’t need to review both changes at once, and can instead focus on the JSON logic in that upcoming PR.

Re StringLikeArray, we don’t have such a trait, but we do have a StringLikeArrayBuilder in arrow-array that’s already used by parquet-variant. This PR simply mirrors that behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arrow Changes to the arrow crate parquet-variant parquet-variant* crates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants