Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion sources/platform/proxy/datacenter_proxy.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ When using Apify's datacenter proxies, you can either select a proxy group, or t

Each user has access to a selected number of proxy servers from a shared pool. These servers are spread into groups (called proxy groups). Each group shares a common feature (location, provider, speed, etc.).

For a full list of plans and number of allocated proxy servers for each plan, see the [pricing page](https://apify.com/pricing). To get access to more servers, you can upgrade your plan in the [subscription settings](https://console.apify.com/billing/subscription);
For a full list of plans and number of allocated proxy servers for each plan, see the [pricing page](https://apify.com/pricing). To check whether your plan includes enough proxy servers, [estimate how many IP addresses you need](./estimate-ip-addresses.md). To get more servers on a paid plan, buy the **Shared datacenter proxies** add-on or upgrade your plan in the [subscription settings](https://console.apify.com/billing/subscription).

### Dedicated proxy groups

Expand Down
65 changes: 65 additions & 0 deletions sources/platform/proxy/estimate-ip-addresses.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
---
title: Estimate how many datacenter IP addresses you need
sidebar_label: Estimate IPs
description: Estimate how many datacenter proxy IP addresses a job needs, compare the result with your plan, and get more with an add-on or a plan upgrade.
sidebar_position: 10.25
slug: /proxy/estimate-ip-addresses
---

Your plan includes a fixed number of [datacenter proxy](./datacenter_proxy.md) IP addresses. Before you run a large job through the proxy, estimate how many IP addresses it needs and compare the result with your plan.

The estimate applies to datacenter proxies only. [Residential proxies](./residential_proxy.md) are billed by traffic, the [Google SERP proxy](./google_serp_proxy.md) by request, and [Unblocker](./unblocker.md) in Unblocker units, so their IP count doesn't limit you. For Google Search results, use the Google SERP proxy instead of sizing datacenter IP addresses for them.

## Calculate the estimate

The number of IP addresses a job needs depends on how many pages it visits on a single domain, how long it can take, and how often it repeats. Major websites need 10 times as many IP addresses as other websites.

| The job can take up to | Divide by |
| ---------------------- | --------- |
| A month | 10,000 |
| A day | 1,000 |
| An hour | 100 |

| The job repeats every | Multiply by |
| --------------------- | ----------- |
| Month | 1 |
| Day | 10 |
| Hour | 100 |

To estimate the number of IP addresses:

1. Take the number of pages the job visits on a single domain.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This process is hard to follow. Maybe an explicit example in each step?

1. Divide it by the value for how long the job can take.
1. Multiply the result by the value for how often the job repeats.
1. If the target is a major website, such as CNN, multiply by 10.

### Examples

| Scenario | Calculation | IP addresses |
| ------------------------------------------------------------------------------------- | ----------------------------- | ------------ |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do these lengthy tables follow our style guide? I mean all these hyphens and spaces.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually don't think we have any for tables 🤔 rendered they both look the same, but it might be good opportunity to unify this. We have mdlint accepts both styles but requires unity within a page, not project

| 10,000 pages from a local e-commerce website, once a month, finished within a day | `10,000 / 1,000 * 1` | 10 |
| 5M product pages from a major online marketplace, once a month, spread over the month | `5,000,000 / 10,000 * 1 * 10` | 5,000 |
| 100 pages from each of 1,000 websites, every day, finished within a day | `100 / 1,000 * 10` | 1 |

Each estimate depends on its assumptions:

- The e-commerce estimate assumes you spread the job across the whole day. A faster job needs more IP addresses.
- The marketplace estimate seems high, but the job visits the same marketplace every day. [IP address rotation](./index.md#ip-address-rotation) spreads those visits across your IP addresses, so more IP addresses mean fewer visits from each one. Long-term reliable access to a major website needs about this many.
- The 1,000-websites estimate uses the number of pages per domain, not the total. Its one IP address covers all 1,000 websites. It assumes you spread the job and the websites don't use Cloudflare or a similar distributed IP protection system.

## Compare the estimate with your plan

To see how many datacenter IP addresses your plan includes, go to **Billing** > **Subscription** in Apify Console and look under **Datacenter proxies**. To see the count in each proxy group, open the [Proxy](https://console.apify.com/proxy/groups) page and check the **Available IPs** column.

## Get more IP addresses

If you need more IP addresses than your plan includes:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that info deserves its own heading.


- On a paid plan, go to **Billing** > **Subscription**, select **Buy add-ons**, and enter the number of IP addresses in **Shared datacenter proxies**.
- Upgrade to a plan that includes more IP addresses. For the IP addresses and add-on price of each plan, see the [Apify pricing page](https://apify.com/pricing).
- For more than 1,999 additional IP addresses, [contact support](https://apify.com/contact).
- For IP addresses that only you use, ask support about [dedicated proxy groups](./datacenter_proxy.md#dedicated-proxy-groups).

## Adjust the estimate

The tables give rough estimates. On some websites, they can be far off. If the job repeats without being blocked, try fewer IP addresses. If you see blocking, add more.
2 changes: 1 addition & 1 deletion sources/platform/proxy/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Your proxy settings and password are on the [Proxy](https://console.apify.com/pr

## Proxy types

Each proxy type has distinct advantages, disadvantages, and pricing. Use them to access websites from different geographies and with different levels of anonymity.
Each proxy type has distinct advantages, disadvantages, and pricing. Use them to access websites from different geographies and with different levels of anonymity. To check whether your plan includes enough datacenter IP addresses, [estimate how many you need](./estimate-ip-addresses.md).

<CardGrid>
<Card
Expand Down
2 changes: 1 addition & 1 deletion sources/platform/proxy/your_own_proxies.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Using your own proxies
title: Use your own proxies
description: Add your own proxy URLs to Actor runs in Apify Console or configure them in the Apify SDK using the proxy configuration API for JavaScript and Python.
sidebar_position: 10.6
slug: /proxy/using-your-own-proxies
Expand Down
Loading