Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

inlined data: embed small object to the metadata directly #10

Open
6 of 8 tasks
iwanbk opened this issue Jan 15, 2025 · 1 comment
Open
6 of 8 tasks

inlined data: embed small object to the metadata directly #10

iwanbk opened this issue Jan 15, 2025 · 1 comment
Assignees

Comments

@iwanbk
Copy link
Member

iwanbk commented Jan 15, 2025

  • define the size that considered small: pingcap Titan and fjall consider 1KB is a large value. Garage use 4KB as large value. We will start with 1KB and provide the command line config to change it
  • performance improvement: when getting a key, we have to get the metadata first, which means two gets. with inlined data, we only need one get
  • write reduction: doesn't need to write to the disk storage-> one less write
  • see garage implementation

We could do it after finishing the sled to fjall migration
deliverables:

  • of course the code
  • configurable from the command line:
    - the max size of the object to be inlined
    - option to enable fjall key-value separation, see : fjall: key value separation #40 (comment)
  • test code
  • benchmark code, possibly using cargo bench?

Tasks:

@iwanbk iwanbk changed the title assessment: embed small object to the metadata directly embed small object to the metadata directly Feb 18, 2025
@iwanbk iwanbk changed the title embed small object to the metadata directly inlined data: embed small object to the metadata directly Feb 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant