Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallel reading of the segments #9

Open
pfbox opened this issue Apr 19, 2024 · 0 comments
Open

Parallel reading of the segments #9

pfbox opened this issue Apr 19, 2024 · 0 comments

Comments

@pfbox
Copy link

pfbox commented Apr 19, 2024

Hello,

Thank you very for the library guys, it's very helpful. Although, we face the latency issues, we need to process around 350GB for 1 day and it takes us about 7h only to read data from AWS, I think it happens because of segments have been reading sequentially. I just wonder if its possible to do parallel reading of those segments.

I was looking at the code but I'm not sure where it has to be implemented.

Regards,
Pavel

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant