Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion docs/02_concepts/01_actor_lifecycle.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,15 @@ When the Actor exits, either normally or due to an exception, the SDK performs a
</TabItem>
</Tabs>

You can also create an [`Actor`](https://docs.apify.com/sdk/python/reference/class/Actor) instance directly. This does not change its capabilities but allows you to specify optional parameters during initialization, such as disabling automatic `sys.exit()` calls or customizing timeouts. The choice between using a context manager or manual initialization depends on how much control you require over the Actor's startup and shutdown sequence.
You can also create an [`Actor`](https://docs.apify.com/sdk/python/reference/class/Actor) instance directly. This does not change its capabilities but allows you to specify optional parameters during initialization. The key parameters are:

- `configuration` — a custom [`Configuration`](https://docs.apify.com/sdk/python/reference/class/Configuration) instance to control storage paths, API URLs, and other settings.
- `configure_logging` — whether to set up default logging configuration (default `True`). Set to `False` if you configure logging yourself.
- `exit_process` — whether the Actor calls `sys.exit()` when the context manager exits. Defaults to `True`, except in IPython, Pytest, and Scrapy environments.
- `event_listeners_timeout` — maximum time to wait for Actor event listeners to complete before exiting.
- `cleanup_timeout` — maximum time to wait for cleanup tasks to finish (default 30 seconds).

The choice between using a context manager or manual initialization depends on how much control you require over the Actor's startup and shutdown sequence.

<Tabs groupId="request_queue">
<TabItem value="actor_instance_with_context_manager" label="Actor instance with context manager" default>
Expand Down
18 changes: 18 additions & 0 deletions docs/02_concepts/02_actor_input.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ description: Read and validate input data passed to your Actor at runtime.
import RunnableCodeBlock from '@site/src/components/RunnableCodeBlock';

import InputExample from '!!raw-loader!roa-loader!./code/02_input.py';
import RequestListExample from '!!raw-loader!roa-loader!./code/02_request_list.py';
import ApiLink from '@site/src/components/ApiLink';

The Actor gets its [input](https://docs.apify.com/platform/actors/running/input) from the input record in its default [key-value store](https://docs.apify.com/platform/storage/key-value-store).

Expand All @@ -17,3 +19,19 @@ For example, if an Actor received a JSON input with two fields, `{ "firstNumber"
<RunnableCodeBlock className="language-python" language="python">
{InputExample}
</RunnableCodeBlock>

## Loading URLs from Actor input

Actors commonly receive a list of URLs to process via their input. The <ApiLink to="class/ApifyRequestList">`ApifyRequestList`</ApiLink> class (from `apify.request_loaders`) can parse the standard Apify input format for URL sources. It supports both direct URL objects (`{"url": "https://example.com"}`) and remote URL lists (`{"requestsFromUrl": "https://example.com/urls.txt"}`), where the remote file contains one URL per line.

<RunnableCodeBlock className="language-python" language="python">
{RequestListExample}
</RunnableCodeBlock>

## Secret input fields

The Apify platform supports [secret input fields](https://docs.apify.com/platform/actors/development/secret-input) that are encrypted before being stored. When you mark an input field as `"isSecret": true` in your Actor's [input schema](https://docs.apify.com/platform/actors/development/input-schema), the platform encrypts the value with the Actor's public key.

No special handling is needed in your code — when you call [`Actor.get_input`](../../reference/class/Actor#get_input), encrypted fields are automatically decrypted using the Actor's private key, which is provided by the platform via environment variables. You receive the plaintext values directly.

For more details on Actor input and how to define input schemas, see the [Actor input](https://docs.apify.com/platform/actors/running/input) and [input schema](https://docs.apify.com/platform/actors/development/input-schema) documentation on the Apify platform.
23 changes: 22 additions & 1 deletion docs/02_concepts/03_storages.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ description: Use datasets, key-value stores, and request queues to persist Actor
import RunnableCodeBlock from '@site/src/components/RunnableCodeBlock';

import OpeningStoragesExample from '!!raw-loader!roa-loader!./code/03_opening_storages.py';
import OpeningStoragesAliasExample from '!!raw-loader!roa-loader!./code/03_opening_storages_alias.py';
import ApiLink from '@site/src/components/ApiLink';
import DeletingStoragesExample from '!!raw-loader!roa-loader!./code/03_deleting_storages.py';
import DatasetReadWriteExample from '!!raw-loader!roa-loader!./code/03_dataset_read_write.py';
import DatasetExportsExample from '!!raw-loader!roa-loader!./code/03_dataset_exports.py';
Expand Down Expand Up @@ -60,7 +62,7 @@ There are several methods for directly working with the default key-value store
- [`Actor.get_value('my-record')`](../../reference/class/Actor#get_value) reads a record from the default key-value store of the Actor.
- [`Actor.set_value('my-record', 'my-value')`](../../reference/class/Actor#set_value) saves a new value to the record in the default key-value store.
- [`Actor.get_input`](../../reference/class/Actor#get_input) reads the Actor input from the default key-value store of the Actor.
- [`Actor.push_data([{'result': 'Hello, world!'}, ...])`](../../reference/class/Actor#push_data) saves results to the default dataset of the Actor.
- [`Actor.push_data([{'result': 'Hello, world!'}, ...])`](../../reference/class/Actor#push_data) saves results to the default dataset of the Actor. When using the [pay-per-event pricing model](./pay-per-event), `push_data` returns a `ChargeResult` object that indicates whether the charge limit has been reached. You can also pass a `charged_event_name` parameter to charge for a custom event for each pushed item.

## Opening named and unnamed storages

Expand All @@ -70,6 +72,12 @@ The [`Actor.open_dataset`](../../reference/class/Actor#open_dataset), [`Actor.op
{OpeningStoragesExample}
</RunnableCodeBlock>

Besides `id` and `name`, the `open_*` methods also accept an `alias` parameter. An alias creates an unnamed storage scoped to the current Actor run — it does not persist across runs, but lets you reference the same storage within a single run using a human-readable label. The `alias` parameter is mutually exclusive with `id` and `name`.

<RunnableCodeBlock className="language-python" language="python">
{OpeningStoragesAliasExample}
</RunnableCodeBlock>

## Deleting storages

To delete a storage, you can use the [`Dataset.drop`](../../reference/class/Dataset#drop),
Expand Down Expand Up @@ -172,3 +180,16 @@ To check if all the requests in the queue are handled, you can use the [`Request
<RunnableCodeBlock className="language-python" language="python">
{RqExample}
</RunnableCodeBlock>

## Storage clients

Behind the scenes, the SDK uses storage clients to communicate with the storage backend. The SDK automatically selects the appropriate client based on the runtime environment:

- **`SmartApifyStorageClient`** (default on the Apify platform) — a hybrid client that writes to both the Apify API and the local filesystem for resilience.
- **`ApifyStorageClient`** — communicates directly with the Apify platform API for cloud storage.
- **`FileSystemStorageClient`** — stores data on the local filesystem (in the `storage/` directory). Used when running locally.
- **`MemoryStorageClient`** (from Crawlee) — stores data in memory. Useful for testing.

For most use cases, the default storage client selection is sufficient. All storage clients are available from the `apify.storage_clients` module. For details, see the <ApiLink to="class/ApifyStorageClient">`ApifyStorageClient`</ApiLink> API reference.

For comprehensive information about storage on the Apify platform, see the [storage documentation](https://docs.apify.com/platform/storage), including the pages on [datasets](https://docs.apify.com/platform/storage/dataset), [key-value stores](https://docs.apify.com/platform/storage/key-value-store), and [request queues](https://docs.apify.com/platform/storage/request-queue).
24 changes: 24 additions & 0 deletions docs/02_concepts/04_actor_events.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ description: Handle platform events like state persistence and graceful shutdown
import RunnableCodeBlock from '@site/src/components/RunnableCodeBlock';

import ActorEventsExample from '!!raw-loader!roa-loader!./code/04_actor_events.py';
import UseStateExample from '!!raw-loader!roa-loader!./code/04_use_state.py';
import ApiLink from '@site/src/components/ApiLink';

During its runtime, the Actor receives Actor events sent by the Apify platform or generated by the Apify SDK itself.

Expand Down Expand Up @@ -69,6 +71,14 @@ During its runtime, the Actor receives Actor events sent by the Apify platform o
you can achieve the same effect by persisting the state regularly in an interval and listening for the migrating event.
</td>
</tr>
<tr>
<td><code>EXIT</code></td>
<td><code>None</code></td>
<td>
Emitted by the SDK (not the platform) when the Actor is about to exit. You can use this event to perform final cleanup tasks,
such as closing external connections or sending notifications, before the Actor shuts down.
</td>
</tr>
</tbody>
</table>

Expand All @@ -80,3 +90,17 @@ and to remove them, you use the [`Actor.off`](../../reference/class/Actor#off) m
<RunnableCodeBlock className="language-python" language="python">
{ActorEventsExample}
</RunnableCodeBlock>

## Automatic state persistence with use_state

The example above shows how to manually persist state using the `PERSIST_STATE` event. For most use cases, you can use the <ApiLink to="class/Actor#use_state">`Actor.use_state`</ApiLink> method instead, which handles state persistence automatically.

`Actor.use_state` returns a dictionary that is automatically saved to the default key-value store at regular intervals and whenever a migration or shutdown occurs. You can modify the dictionary in place, and changes are persisted without any manual `set_value` calls.

You can optionally specify a `key` (the key-value store key under which the state is stored) and a `kvs_name` (the name of the key-value store to use). By default, the state is stored in the default key-value store under a default key.

<RunnableCodeBlock className="language-python" language="python">
{UseStateExample}
</RunnableCodeBlock>

For more details on platform events and state persistence, see the [system events](https://docs.apify.com/platform/actors/development/programming-interface/system-events) and [state persistence](https://docs.apify.com/platform/actors/development/state-persistence) documentation on the Apify platform.
13 changes: 13 additions & 0 deletions docs/02_concepts/05_proxy_management.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import ApifyProxyConfig from '!!raw-loader!roa-loader!./code/05_apify_proxy_conf
import CustomProxyFunctionExample from '!!raw-loader!roa-loader!./code/05_custom_proxy_function.py';
import ProxyActorInputExample from '!!raw-loader!roa-loader!./code/05_proxy_actor_input.py';
import ProxyHttpxExample from '!!raw-loader!roa-loader!./code/05_proxy_httpx.py';
import TieredProxyExample from '!!raw-loader!roa-loader!./code/05_tiered_proxy.py';
import ApiLink from '@site/src/components/ApiLink';

The Apify SDK provides built-in proxy management through the <ApiLink to="class/ProxyConfiguration">`ProxyConfiguration`</ApiLink> class, supporting both [Apify Proxy](https://apify.com/proxy) and custom proxy servers. Proxies are essential for web scraping to avoid [IP address blocking](https://en.wikipedia.org/wiki/IP_address_blocking) and distribute requests across multiple addresses.
Expand Down Expand Up @@ -81,6 +82,18 @@ Or you can pass it a method (accepting one optional argument, the session ID), t
{CustomProxyFunctionExample}
</RunnableCodeBlock>

### Tiered proxy rotation

<ApiLink to="class/ProxyConfiguration">`ProxyConfiguration`</ApiLink> supports tiered proxy URLs via the `tiered_proxy_urls` parameter. This accepts a list of lists of proxy URLs, where each inner list represents a tier. The proxy rotator starts with the first (cheapest) tier and automatically escalates to higher tiers when lower-tier proxies get blocked. This is useful for optimizing proxy costs — you use cheap datacenter proxies for most requests and only switch to expensive residential proxies when necessary.

:::info
The `tiered_proxy_urls` parameter is only available when constructing `ProxyConfiguration` directly. It is not supported by `Actor.create_proxy_configuration()`.
:::

<RunnableCodeBlock className="language-python" language="python">
{TieredProxyExample}
</RunnableCodeBlock>

### Configuring proxy based on Actor input

To make selecting the proxies that the Actor uses easier, you can use an input field with the editor [`proxy` in your input schema](https://docs.apify.com/platform/actors/development/input-schema#object). This input will then be filled with a dictionary containing the proxy settings you or the users of your Actor selected for the Actor run.
Expand Down
13 changes: 12 additions & 1 deletion docs/02_concepts/06_interacting_with_other_actors.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import InteractingStartExample from '!!raw-loader!roa-loader!./code/06_interacti
import InteractingCallExample from '!!raw-loader!roa-loader!./code/06_interacting_call.py';
import InteractingCallTaskExample from '!!raw-loader!roa-loader!./code/06_interacting_call_task.py';
import InteractingMetamorphExample from '!!raw-loader!roa-loader!./code/06_interacting_metamorph.py';
import InteractingAbortExample from '!!raw-loader!roa-loader!./code/06_interacting_abort.py';
import ApiLink from '@site/src/components/ApiLink';

The Apify SDK lets you start, call, and transform (metamorph) other Actors directly from your Actor code. This is useful for composing complex workflows from smaller, reusable Actors.
Expand Down Expand Up @@ -52,4 +53,14 @@ For example, imagine you have an Actor that accepts a hotel URL on input, and th
{InteractingMetamorphExample}
</RunnableCodeBlock>

For the full list of methods for interacting with other Actors, see the <ApiLink to="class/Actor">`Actor`</ApiLink> API reference.
## Aborting an Actor run

The [`Actor.abort`](../../reference/class/Actor#abort) method aborts a running Actor on the Apify platform. You can use it to cancel a long-running Actor that is no longer needed.

When you set `gracefully=True`, the platform sends `ABORTING` and `PERSIST_STATE` events to the target Actor, giving it time to save its state, and then force-stops it after 30 seconds. Without the `gracefully` flag, the Actor is stopped immediately.

<RunnableCodeBlock className="language-python" language="python">
{InteractingAbortExample}
</RunnableCodeBlock>

For the full list of methods for interacting with other Actors, see the <ApiLink to="class/Actor">`Actor`</ApiLink> API reference. For more details on running Actors and Actor tasks on the platform, see the [Actors](https://docs.apify.com/platform/actors) and [Actor tasks](https://docs.apify.com/platform/actors/tasks) documentation.
20 changes: 19 additions & 1 deletion docs/02_concepts/10_configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ description: Customize Actor behavior through the Configuration class or environ
import RunnableCodeBlock from '@site/src/components/RunnableCodeBlock';

import ConfigExample from '!!raw-loader!roa-loader!./code/10_config.py';
import GetEnvExample from '!!raw-loader!roa-loader!./code/10_get_env.py';
import PlatformDetectionExample from '!!raw-loader!roa-loader!./code/10_platform_detection.py';
import ApiLink from '@site/src/components/ApiLink';

The <ApiLink to="class/Actor">`Actor`</ApiLink> class is configured through the <ApiLink to="class/Configuration">`Configuration`</ApiLink> class, which reads its settings from environment variables. When running on the Apify platform or through the Apify CLI, configuration is automatic — manual setup is only needed for custom requirements.
Expand All @@ -33,4 +35,20 @@ This Actor run will not persist its local storages to the filesystem:
APIFY_PERSIST_STORAGE=0 apify run
```

For the full list of configuration options, see the <ApiLink to="class/Configuration">`Configuration`</ApiLink> API reference.
## Reading the runtime environment

The <ApiLink to="class/Actor#get_env">`Actor.get_env`</ApiLink> method returns a dictionary with all `APIFY_*` environment variables parsed into their typed values. This is useful for inspecting the Actor's runtime context, such as the Actor ID, run ID, or default storage IDs. Variables that are not set or are invalid will have a value of `None`.

<RunnableCodeBlock className="language-python" language="python">
{GetEnvExample}
</RunnableCodeBlock>

## Platform detection

The <ApiLink to="class/Actor#is_at_home">`Actor.is_at_home`</ApiLink> method returns `True` when the Actor is running on the Apify platform, and `False` when running locally. This is useful for branching behavior based on the environment, such as using different storage backends or skipping proxy configuration during local development.

<RunnableCodeBlock className="language-python" language="python">
{PlatformDetectionExample}
</RunnableCodeBlock>

For the full list of configuration options, see the <ApiLink to="class/Configuration">`Configuration`</ApiLink> API reference. For a complete list of environment variables available on the platform, see the [environment variables](https://docs.apify.com/platform/actors/development/programming-interface/environment-variables) documentation.
Loading
Loading