Faster resumable downloads #2676
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Revert changes to GoogleDriveDownloader
…n excessive number of requests
Revert changes to GoogleDriveDownloader
…kerson/wabbajack into fasterResumableDownloads
…veness. Added dedicated DI extension for ResumableDownloader Added slight delay to download creation to help avoid hitting API limits/DDOS protections Added progress reporting to downloader Added preallocation of file sizes to downloader Fixed error when hashing file due to stream not supporting read
tr4wzified
reviewed
Dec 29, 2024
1. Reverted log stream threshold 2. Removed batching from missing archives download 3. Adjusted threshold for progress reporting in resumable downloader to 10mb 4. Added additional logging to ResumableDownloader 5. Typo
tr4wzified
approved these changes
Dec 29, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Replaces the existing ResumableDownloader with a new implementation that uses native HttpClient rather than the Downloader library.
Reworked how downloads are resumed. If a download is found but does not hash as expected, indicating it was not completed, the length of the file is taken as the starting position for a new download.
Progress reporting working as normal. Adjusted to only report progress on files over a certain size in order to eliminate needless work for the UI.
Added a threshold to the log stream shown in the installation UI. Now only logs of Error or above will be displayed. This significantly improves the responsiveness of the UI.
Removed performance settings that are no longer in use.
Impact
This replaces this PR: #2676 (comment)