Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: very long wait time when you try to use bigger collections #2333

Open
tuehlarsen opened this issue Jan 22, 2025 · 0 comments
Open

[Bug]: very long wait time when you try to use bigger collections #2333

tuehlarsen opened this issue Jan 22, 2025 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@tuehlarsen
Copy link

Browsertrix Version

v1.13.2-a21b2ff

What did you expect to happen? What happened instead?

We have a collection of crawls in our local installation: "Tilvækst_udvalgte_domæner_via_sitemaps" with:
Archived Items 236 items, Total Size 554 GB, Total Pages 210.759 pages
It takes minuts every time you want to use the collection and if you try to replay a specific url it gives you a blanc page. First after a while 2-3 minutes it replay the page...
What can we do to separate this dynamic indexing - and replay issue with collections - from our crawling?
Should we separate which parts to other servers?

Reproduction instructions

see above

Screenshots / Video

No response

Environment

No response

Additional details

No response

@tuehlarsen tuehlarsen added the bug Something isn't working label Jan 22, 2025
@ikreymer ikreymer moved this from Triage to Todo in Webrecorder Projects Feb 5, 2025
@SuaYoo SuaYoo moved this from Todo to Implementing in Webrecorder Projects Feb 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Implementing
Development

No branches or pull requests

2 participants