diff --git a/sources/platform/proxy/datacenter_proxy.md b/sources/platform/proxy/datacenter_proxy.md index f2806aa6ab..1d51a7783f 100644 --- a/sources/platform/proxy/datacenter_proxy.md +++ b/sources/platform/proxy/datacenter_proxy.md @@ -33,7 +33,7 @@ When using Apify's datacenter proxies, you can either select a proxy group, or t Each user has access to a selected number of proxy servers from a shared pool. These servers are spread into groups (called proxy groups). Each group shares a common feature (location, provider, speed, etc.). -For a full list of plans and number of allocated proxy servers for each plan, see the [pricing page](https://apify.com/pricing). To get access to more servers, you can upgrade your plan in the [subscription settings](https://console.apify.com/billing/subscription); +For a full list of plans and number of allocated proxy servers for each plan, see the [pricing page](https://apify.com/pricing). To check whether your plan includes enough proxy servers, [estimate how many IP addresses you need](./estimate-ip-addresses.md). To get more servers on a paid plan, buy the **Shared datacenter proxies** add-on or upgrade your plan in the [subscription settings](https://console.apify.com/billing/subscription). ### Dedicated proxy groups diff --git a/sources/platform/proxy/estimate-ip-addresses.md b/sources/platform/proxy/estimate-ip-addresses.md new file mode 100644 index 0000000000..0dbc04af19 --- /dev/null +++ b/sources/platform/proxy/estimate-ip-addresses.md @@ -0,0 +1,65 @@ +--- +title: Estimate how many datacenter IP addresses you need +sidebar_label: Estimate IPs +description: Estimate how many datacenter proxy IP addresses a job needs, compare the result with your plan, and get more with an add-on or a plan upgrade. +sidebar_position: 10.25 +slug: /proxy/estimate-ip-addresses +--- + +Your plan includes a fixed number of [datacenter proxy](./datacenter_proxy.md) IP addresses. Before you run a large job through the proxy, estimate how many IP addresses it needs and compare the result with your plan. + +The estimate applies to datacenter proxies only. [Residential proxies](./residential_proxy.md) are billed by traffic, the [Google SERP proxy](./google_serp_proxy.md) by request, and [Unblocker](./unblocker.md) in Unblocker units, so their IP count doesn't limit you. For Google Search results, use the Google SERP proxy instead of sizing datacenter IP addresses for them. + +## Calculate the estimate + +The number of IP addresses a job needs depends on how many pages it visits on a single domain, how long it can take, and how often it repeats. Major websites need 10 times as many IP addresses as other websites. + +| The job can take up to | Divide by | +| ---------------------- | --------- | +| A month | 10,000 | +| A day | 1,000 | +| An hour | 100 | + +| The job repeats every | Multiply by | +| --------------------- | ----------- | +| Month | 1 | +| Day | 10 | +| Hour | 100 | + +To estimate the number of IP addresses: + +1. Take the number of pages the job visits on a single domain. +1. Divide it by the value for how long the job can take. +1. Multiply the result by the value for how often the job repeats. +1. If the target is a major website, such as CNN, multiply by 10. + +### Examples + +| Scenario | Calculation | IP addresses | +| ------------------------------------------------------------------------------------- | ----------------------------- | ------------ | +| 10,000 pages from a local e-commerce website, once a month, finished within a day | `10,000 / 1,000 * 1` | 10 | +| 5M product pages from a major online marketplace, once a month, spread over the month | `5,000,000 / 10,000 * 1 * 10` | 5,000 | +| 100 pages from each of 1,000 websites, every day, finished within a day | `100 / 1,000 * 10` | 1 | + +Each estimate depends on its assumptions: + +- The e-commerce estimate assumes you spread the job across the whole day. A faster job needs more IP addresses. +- The marketplace estimate seems high, but the job visits the same marketplace every day. [IP address rotation](./index.md#ip-address-rotation) spreads those visits across your IP addresses, so more IP addresses mean fewer visits from each one. Long-term reliable access to a major website needs about this many. +- The 1,000-websites estimate uses the number of pages per domain, not the total. Its one IP address covers all 1,000 websites. It assumes you spread the job and the websites don't use Cloudflare or a similar distributed IP protection system. + +## Compare the estimate with your plan + +To see how many datacenter IP addresses your plan includes, go to **Billing** > **Subscription** in Apify Console and look under **Datacenter proxies**. To see the count in each proxy group, open the [Proxy](https://console.apify.com/proxy/groups) page and check the **Available IPs** column. + +## Get more IP addresses + +If you need more IP addresses than your plan includes: + +- On a paid plan, go to **Billing** > **Subscription**, select **Buy add-ons**, and enter the number of IP addresses in **Shared datacenter proxies**. +- Upgrade to a plan that includes more IP addresses. For the IP addresses and add-on price of each plan, see the [Apify pricing page](https://apify.com/pricing). +- For more than 1,999 additional IP addresses, [contact support](https://apify.com/contact). +- For IP addresses that only you use, ask support about [dedicated proxy groups](./datacenter_proxy.md#dedicated-proxy-groups). + +## Adjust the estimate + +The tables give rough estimates. On some websites, they can be far off. If the job repeats without being blocked, try fewer IP addresses. If you see blocking, add more. diff --git a/sources/platform/proxy/index.md b/sources/platform/proxy/index.md index dad87cd46b..8ac76e1a08 100644 --- a/sources/platform/proxy/index.md +++ b/sources/platform/proxy/index.md @@ -18,7 +18,7 @@ Your proxy settings and password are on the [Proxy](https://console.apify.com/pr ## Proxy types -Each proxy type has distinct advantages, disadvantages, and pricing. Use them to access websites from different geographies and with different levels of anonymity. +Each proxy type has distinct advantages, disadvantages, and pricing. Use them to access websites from different geographies and with different levels of anonymity. To check whether your plan includes enough datacenter IP addresses, [estimate how many you need](./estimate-ip-addresses.md).