Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,9 @@ For details, see [Actor developers](/actors/running/store/actor-developers).

It's essential to test your Actors and make sure they work as intended. That's why Apify does it on our side as much as you should do it on yours.

Apify runs automated tests daily to ensure all Actors on Apify Store are functional and reliable. These tests check _if an Actor can successfully run with its default input within 5 minutes_. If an Actor fails for three consecutive days, it’s labeled under maintenance, and the developer is notified. Continuous failures for another 28 days lead to deprecation.
Apify runs automated tests daily to ensure all Actors on Apify Store are functional and reliable. These tests check whether an Actor's run with its default input succeeds within 5 minutes. If an Actor keeps failing, Apify labels it under maintenance, notifies the developer, and later deprecates it. For the schedule, see [How we test](/actors/publishing/test#how-we-test).

To restore an Actor's health, developers should fix and rebuild it. The testing system will automatically recognize the changes within 24 hours. If your Actor requires longer run times or authentication, contact support to explain why it should be excluded from tests. For more control, you can implement your own tests using the Actor Testing tool available on Apify Store.
To restore your Actor's health, fix and rebuild it. The tests use the new build within 24 hours. If your Actor can't pass the tests, for example because it needs more than 5 minutes to finish, you can request to skip the tests. For details, see [Actor testing](/actors/publishing/test#skip-the-automated-tests). For more control, you can write your own tests with the [Actor Testing](https://apify.com/pocesar/actor-testing) tool from Apify Store.

### Actor metrics and reliability score

Expand Down
4 changes: 2 additions & 2 deletions sources/platform/actors/publishing/actor-status.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ To let users know that the Actor might temporarily not work as expected, use the
1. _(Optional)_ In the **Under maintenance message** field, add an additional explanation.
1. Select **Save**.

Note that the **Under maintenance** status makes your Actor rank lower in search results. It's also automatically set when your Actor fails the [automated testing](../development/automated_tests.md) process.
The **Under maintenance** status makes your Actor rank lower in search results. Apify also sets it when your Actor fails the [automated tests](./testing.mdx).

### Mark Actor as deprecated

Expand All @@ -53,4 +53,4 @@ To let your users know that the Actor is no longer being developed, use the **De

Don't delete the Actor as it might break the existing integrations.

Note that the deprecated Actors aren't searchable in Apify Store. The **Deprecated** status is also automatically set when your Actor fails the [automated testing](../development/automated_tests.md) process for 30 days.
Deprecated Actors aren't searchable in Apify Store. Apify also sets the **Deprecated** status when your Actor keeps failing the [automated tests](./testing.mdx).
49 changes: 42 additions & 7 deletions sources/platform/actors/publishing/testing.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ sidebar_position: 3
## Why we test

We want to make sure that all Actors in Apify Store are top-notch, or at least as top-notch as they can be. Since there are many of them, we have an automated testing procedure in place that tests all Actors daily.
This helps us to flag Actors that temporarily don't work as expected `under maintenance`, and to automatically `deprecate` Actors that have been broken for more than a month.
Testing helps us flag Actors that temporarily don't work as expected as **Under maintenance**, and to automatically mark Actors that have been broken for more than a month as **Deprecated**.

If you use an Actor that's under maintenance, see [Actors under maintenance](/actors/running/actors-in-store#actors-under-maintenance).

## How we test

Expand All @@ -18,19 +20,52 @@ and expects it to finish with a **Succeeded** status and non-empty default datas

![Actor page](./images/actor-test.webp)

If the Actor fails to complete successful runs for three consecutive days, the developer will be notified,
and the Actor will be labeled `under maintenance` until it is fixed. After another 14 days of failing runs, you will receive another notification. Finally, if the runs continue to fail after yet another 14 days, the Actor will be `deprecated`.
If more than half of the daily test runs in the last three days fail, Apify notifies you and labels the Actor **Under maintenance** until you fix it. All test runs in those three days must use the same build, so a new build restarts the count. After another 14 days of failing runs, you get another notification. If the runs still fail 14 days after that, Apify marks the Actor as **Deprecated**.

## Find out why the test failed

When Apify labels your Actor **Under maintenance**, you get a notification in Apify Console and by email. The notification tells you whether the test could start the Actor. The fix differs for each case.

### The test couldn't start the Actor

If the test couldn't start the Actor, the cause is usually a problem with the input schema. The test builds its input from the schema's `prefill` values. Every required field needs a `prefill` value, and every value must pass the schema's validation. For example, a `prefill` of `3` fails for a field with `"maximum": 2`.

To see the input the test uses:

1. In Apify Console, open your Actor.
1. Go to **Source** > **Input**.
1. Select **Restore example input**. The form loads the values from the input schema and shows an error next to any value that fails validation.

Fix the input schema, then rebuild the Actor, because the test reads the schema from the build.

## How can I make my Actor healthy again
### The test run failed

When a test run fails, the notification links to it. Check its log for the cause.

If the run timed out, make the test input smaller. Choose `prefill` values that return only a few results, so the run finishes within the 5-minute limit.

## Restore a healthy status

The best course of action is to fix the Actor and rebuild it. The automatic testing system will pick this up within 24 hours
and mark it as healthy. In some cases, your Actor might break because of issues with the target website. In such a case, if your Actor
passes the majority of test runs in the next 7 days, it will be marked as healthy automatically.

## What if my Actor cannot comply with the test logic
## Skip the automated tests

The automated test skips these Actors:

- Actors with [Actor Standby](../running/actor_standby.md) enabled
- Actors in the MCP servers category
- Actors with a required secret input
- Actors with a required token or key input that has no `prefill` value

If your Actor can't pass the test for another reason, such as needing more than 5 minutes to finish, request to skip the tests:

1. In the under-maintenance notification, open the form link.
1. In the **Request to skip Actor checks** dialog, explain why your Actor should skip the tests.
1. Select **Send request**.

Actors that require some sort of authentication will always fail the tests despite being fully functional. Also, some Actors inherently run for longer than 5 minutes.
If that's the case with your Actor, [contact support](http://apify.com/contact) and explain your specific use case that justifies why the Actor should be excluded from the automated tests.
Apify approves requests only in exceptional cases.

## Advanced Actor testing

Expand Down
12 changes: 12 additions & 0 deletions sources/platform/actors/running/store/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,18 @@ _For more information on platform usage cost see the [usage and resources](../us

:::

## Actors under maintenance

An Actor gets the **Under maintenance** label when it fails Apify's [automated tests](/actors/publishing/test), or when its developer sets it. Its page then shows the **This Actor is under maintenance** banner, sometimes with a message from the developer. If you run it, the run can fail or return incomplete results.

If an Actor you use is under maintenance:

- Run it with a small input and check the results.
- On the Actor's **Issues** tab, look for an issue that describes your problem. If there isn't one, create one to ask the developer.
- To find a replacement, select **Show similar Actors** in the banner, or search [Apify Store](https://apify.com/store).

Actors under maintenance rank lower in Apify Store. If an Actor keeps failing the tests, Apify deprecates it.

## Report issues with Actors

Each Actor has an **Issues** tab in Apify Console. There, you can open an issue (ticket) and chat with the Actor's author, platform admins,
Expand Down
Loading