Proxy Pool Sizing: A Capacity Planning Framework for Scraping at Scale

"How many IPs do I need" gets answered with a round number pulled from nowhere far more often than it gets actually calculated. Here's a framework for deriving a real number instead of guessing.

The variables that actually determine pool size

Pool size isn't a single input — it's the output of several other decisions you've probably already made without realizing they determine it:

• Target request volume (how many requests you need to make, per unit time)

• Safe requests-per-IP-per-target (how many requests one IP can make to one target before risk rises meaningfully — this varies enormously by target, and needs to come from your own measured data, not a guess)

• IP rest/cooldown period (how long an IP needs between uses on the same target before it's "fresh" again from that target's perspective)

• Target diversity (how many distinct targets you're hitting — a pool sized for one target doesn't need the same math for ten)

The basic formula

pool_size = (requests_per_time_unit * cooldown_period) / safe_requests_per_ip_per_cooldown

Concretely: if you need 10,000 requests/day to one target, a safe rate is 20 requests per IP before needing a 24-hour cooldown, then you need at minimum 10,000 / 20 = 500 IPs actively usable within that rotation — not 500 IPs total if some fraction of your pool is always resting, degraded, or otherwise unavailable.

Why "just buy more IPs" isn't the same as correctly sizing your pool

Oversizing wastes money on capacity you're not using — many providers price by pool size, and IPs sitting idle because your actual usage pattern doesn't need them is pure waste. Undersizing means individual IPs get reused faster than their safe cooldown allows, which is exactly the mechanism that turns a "clean" pool into a degraded one over weeks, even if every IP was genuinely clean on day one.

Accounting for non-uniform target difficulty

A pool sized correctly for your easiest target will be badly undersized for your hardest one. If you're hitting multiple targets with meaningfully different tolerance levels, size for the hardest target you need to hit reliably, and treat capacity for easier targets as a byproduct of that — not the other way around. Sizing for an average across very different targets systematically underprovisions for whichever target actually needs the most IPs.

Building in headroom for degradation over time

No pool stays at 100% "fresh" capacity forever — some fraction of any pool will be resting, some will be temporarily flagged, some genuinely need replacement. A sizing calculation that assumes 100% of your pool is always available at full safe-rate capacity will be undersized in practice within weeks. A working rule of thumb: size for 70-80% effective availability, not 100%, and treat replacement/refresh of degraded IPs as an ongoing operational cost, not a one-time setup decision.

Revisiting the calculation, not just running it once

Safe-requests-per-IP isn't a fixed constant — it drifts as targets update their own detection systems, sometimes in your favor, often not. Re-measure it periodically (monthly is a reasonable cadence for anything running at meaningful volume) rather than trusting a number you derived once and assuming it holds indefinitely.

This is exactly the kind of capacity conversation we have with teams sizing infrastructure on SotaProxy — the right pool size is almost always a derived number specific to your actual traffic pattern and targets, not a tier picked off a pricing page. If you're not sure whether your current pool is over or undersized, working backward from your actual requests-per-IP-per-target data is the fastest way to find out.