New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gocomics robots.txt blocking #335

Open

hendersj opened this issue Jan 8, 2025 · 1 comment

hendersj commented Jan 8, 2025

Mostly just calling this out - started yesterday, so those comics no longer download. Probably need to find alternate sources for them.

SKDzd commented Jan 28, 2025

I suggest a --respect-robots option, like --adult

This will have to remove dosagelib/util.py check_robotstxt() IOError

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment