Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bulk add directories for initial sync #31

Open
TheDiscordian opened this issue Mar 22, 2021 · 1 comment
Open

Bulk add directories for initial sync #31

TheDiscordian opened this issue Mar 22, 2021 · 1 comment
Labels
performance Strictly performance related

Comments

@TheDiscordian
Copy link
Owner

As directory size grows, calls to MFS like files/cp, files/mkdir, and files/rm get slower and slower. For initial sync at least, adding directories, we could add all the files at once, with the entire directory tree. Then a single files/cp could copy the tree into MFS, solving this problem for initial sync.

Could still be a bit slow for anything other than initial-sync, but I think initial sync is where this will impact the most.
@TheDiscordian TheDiscordian added enhancement New feature or request performance Strictly performance related labels Mar 22, 2021
@TheDiscordian
Copy link
Owner Author

files/cp doesn't seem too slow. Will need more testing, but with #28 and #29 tackled, this issue doesn't impact nearly as much.

@TheDiscordian TheDiscordian mentioned this issue Jul 16, 2021
Open
@TheDiscordian TheDiscordian removed the enhancement New feature or request label Jul 21, 2021
@TheDiscordian TheDiscordian changed the title Adding files to large directories slows down over time Bulk add directories for initial sync Jul 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
performance Strictly performance related
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@TheDiscordian