Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add async or sync for :ledger metadata index's rocksdb write #3368

Closed

Conversation

StevenLuMT
Copy link
Member

Descriptions of the changes in this PR:

Motivation

bookie flush mainly composed of three pieces:
1. flush-entrylog: it's flushing entrylog
2. flush-locations-index: it's flushing entrly location index, use sync mode to flush
3. flush-ledger-index: it's flushing ledger metadata index, this index(LedgerMetadataIndex) use async mode to flush

sync is different from async:

  • sync mode:
    1. create a batch;
    5. add msg to the batch
    6. call method(batch.flush) to flush the batch

  • async mode:
    1. just call method(sync) to write the data
    2. the rocksdb server will be timed to flush the data

Changes

  1. add async or sync for : ledger metadata index rocksdb write
  2. add switch to open this feature, default not open, use default async

@hangc0276 hangc0276 modified the milestones: 4.16.0, 4.17.0 Jul 29, 2022
Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall the patch is good

but I left one comment about concurrency management, it deserves some double check

try {
keyWrapper.set(STORAGE_FLAGS);
newFlagsWrapper.set(newFlags);
synchronized (ledgersDb) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems to be that synchronisation around ledgersDb is not consistent.
sometimes we don't access the variable that way.

I wonder why spotbugs does not complain

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic has not been updated, and the code to move according to #2936
It doesn't seem to be a problem, or if I modify synchronized to the method, I will make a unified correction to the historical code. @eolivelli

@StevenLuMT StevenLuMT requested review from eolivelli and removed request for merlimat, Vanlightly, hangc0276, nicoloboschi and zymap August 19, 2022 17:15
@StevenLuMT StevenLuMT closed this Aug 20, 2022
@StevenLuMT StevenLuMT reopened this Aug 20, 2022
@StevenLuMT
Copy link
Member Author

reopen the pr, run the new workflow

@eolivelli eolivelli removed this from the 4.17.0 milestone May 3, 2024
@StevenLuMT
Copy link
Member Author

Temporarily close first, and reopen after improvement

@StevenLuMT StevenLuMT closed this Feb 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants