Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve JSON decoding speed by up to 40% #835

Merged
merged 1 commit into from
Feb 20, 2025
Merged

Improve JSON decoding speed by up to 40% #835

merged 1 commit into from
Feb 20, 2025

Conversation

mwylde
Copy link
Member

@mwylde mwylde commented Feb 20, 2025

Pulls in changes from apache/arrow-rs#7157 (rebased on our arrow-json fork as ArroyoSystems/arrow-rs@9cb387d4f55), which significantly improves JSON decoding performance.

This table shows performance improvements across various JSON documents (from ArroyoSystems/json-benchmarks):

Benchmark BufIter (%) memchr2 (%) simdutf8 (%) Total (%)
logs_json -13.49% -12.61% -0.83% -25.03%
logs_pretty_json -20.63% -10.77% -1.22% -30.04%
nexmark_json -30.01% -16.01% -0.42% -41.46%
nexmark_pretty_json -21.35% -15.76% -1.09% -34.47%
nexmark_bids_json -26.64% -22.20% -3.01% -44.64%
nexmark_bids_pretty_json -26.26% -22.16% -2.64% -44.12%
tweets_json -20.36% -18.97% -14.85% -45.05%
tweets_pretty_json -22.61% -15.10% -11.04% -41.55%
Average -38.29%

@mwylde mwylde enabled auto-merge (squash) February 20, 2025 17:48
@mwylde mwylde changed the title Bump arrow-json to 9cb387d4f55; improves decoding by up to 40% Bump arrow-json; improves decoding by up to 40% Feb 20, 2025
@mwylde mwylde changed the title Bump arrow-json; improves decoding by up to 40% Improve JSON decoding by up to 40% Feb 20, 2025
@mwylde mwylde changed the title Improve JSON decoding by up to 40% Improve JSON decoding speed by up to 40% Feb 20, 2025
@mwylde mwylde merged commit ee5cc7f into master Feb 20, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant