Speeding Up Web Scraping with Parallel Processing

Run Initial Scraper:

$ python3 script.py

Run Parallel Scraper:

$ python3 script_parallel.py

Run Tests:

$ pytest

Name		Name	Last commit message	Last commit date
Latest commit History 6 Commits
scrapers		scrapers
test		test
.gitignore		.gitignore
README.md		README.md
requirements.txt		requirements.txt
script.py		script.py
script_futures.py		script_futures.py
script_parallel.py		script_parallel.py

Navigation Menu