Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cache suffixes in resource tree #1814

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

wyfo
Copy link
Contributor

@wyfo wyfo commented Mar 6, 2025

ROS2 experiments shows that a lot of keyexprs shares common chunks, which can be quite big, for example:

  • 0/pub_20/set_parameters/rcl_interfaces::srv::dds_::SetParameters_/RIHS01_56eed9a67e169f9cb6c1f987bc88f868c14a8fc9f743a263bc734c154015d7e0
  • @ros2_lv/0/5639ab36ef9be8852b663e09962a618d/0/4/SS/%/%/pub_20/%pub_20%set_parameters/rcl_interfaces::srv::dds_::SetParameters_/RIHS01_56eed9a67e169f9cb6c1f987bc88f868c14a8fc9f743a263bc734c154015d7e0/:2:,1000:,:,:,,
  • etc.

In order to reduce the memory consumption, this PR caches the chunks in the resource tree, using a kind of string interning. Also, small chunks like pub_20 are directly stored inlined in the bytes of the chunk pointer (small string optimization).

@wyfo wyfo requested a review from OlivierHecart March 6, 2025 16:35
@wyfo wyfo added the internal Changes not included in the changelog label Mar 6, 2025
Copy link

codecov bot commented Mar 6, 2025

Codecov Report

Attention: Patch coverage is 70.27778% with 214 lines in your changes missing coverage. Please review.

Please upload report for BASE (main@4b035b7). Learn more about missing BASE report.

Files with missing lines Patch % Lines
zenoh/src/net/routing/hat/router/queries.rs 10.90% 49 Missing ⚠️
zenoh/src/net/routing/hat/router/pubsub.rs 18.51% 44 Missing ⚠️
zenoh/src/net/routing/hat/linkstate_peer/pubsub.rs 64.00% 18 Missing ⚠️
...enoh/src/net/routing/hat/linkstate_peer/queries.rs 67.34% 16 Missing ⚠️
zenoh/src/net/routing/hat/linkstate_peer/token.rs 73.07% 14 Missing ⚠️
zenoh/src/net/routing/hat/router/token.rs 75.86% 14 Missing ⚠️
zenoh/src/net/routing/hat/p2p_peer/token.rs 58.33% 10 Missing ⚠️
zenoh/src/net/routing/dispatcher/suffix.rs 93.91% 9 Missing ⚠️
zenoh/src/net/routing/hat/client/token.rs 50.00% 8 Missing ⚠️
zenoh/src/net/routing/hat/p2p_peer/pubsub.rs 61.90% 8 Missing ⚠️
... and 8 more
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1814   +/-   ##
=======================================
  Coverage        ?   71.01%           
=======================================
  Files           ?      361           
  Lines           ?    65293           
  Branches        ?        0           
=======================================
  Hits            ?    46366           
  Misses          ?    18927           
  Partials        ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@wyfo wyfo force-pushed the resource_suffix branch from 74956c3 to 63894ec Compare March 10, 2025 09:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
internal Changes not included in the changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant