Cheap proxies for web scraping
Scraping is the use case where proxy choice swings the bill most. Start with the cheapest line that the target accepts and move up only when you are blocked.
Worked budget: a crawl that needs 20 parallel IPs for a month. On the cheap datacenter proxies line that comes to $20.80 at list price. This is an illustration from your own assumptions, not a measurement of any workload.
Start cheap, escalate on evidence
Run your scraper against the target through a small batch of datacenter IPs. If you get your pages, you are done and the bill is low. If you get 403s, captchas or empty pages, the target blocks hosting ranges and the next step is ISP or residential.
Do this test before buying a large order. A hundred requests tells you more than any provider comparison.
Which line for which target
There is no universal answer, but these starting points save time.
- Public APIs, documentation sites, many news sites: datacenter.
- Retail and travel sites with bot detection: ISP or residential.
- Search engine results: residential, with country targeting through a per-IP line where available.
- Anything behind a login: a static IP (ISP or datacenter), so the session does not hop address.
Control the cost, not just the rate
On a metered line, a scraper that downloads images it does not need can triple the bill. Strip resources you do not parse, cache what does not change and respect rate limits so you are not paying for blocked responses.
Frequently asked questions
How many proxies do I need for scraping?
Enough that no single address exceeds the target’s rate limit. Find the limit by testing, divide your target request rate by it, and add a margin for IPs that get blocked.
Is scraping with a proxy legal?
It depends on the data, the site’s terms and your jurisdiction. Read the acceptable-use policy and get advice for anything beyond public pages.
Try it for $5
Minimum top-up is $5. Run your own target through it before you scale.