Hide your scrapers IP behind the cloud. Provision proxy servers across different cloud providers to improve your scraping success.
Role in this project:
Cloud Engineer / Infrastructure Engineer Contributions:28 releases, 16 reviews, 158 commits in 1 year 7 months
Contributions summary:Christian primarily focused on building and configuring infrastructure for cloud-based proxy servers. Their contributions involve creating, deploying, and managing proxy instances across multiple cloud providers (DigitalOcean, AWS, and Hetzner), including the use of Docker and tinyproxy for proxy server setup. They also added features such as age limits to automatically recycle proxy instances and implemented CI/CD automation. The user also added support for an UI for the project.
proxy-servercloudproxyscraping
Tool for scraping TripAdvisor and Yelp.
Contributions:4 releases, 28 commits, 18 PRs in 25 days
tripadvisorscraperyelp