Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ For production use, you should install an adapter package that matches your mess
See the [adapter list](docs/guide/en/adapter-list.md) and follow the adapter-specific documentation for installation and configuration details.

> If you don't have an external broker — whether for development, testing, or because you want to
> design around `QueueInterface` from day one and add a real broker later — you can run the queue
> design around `QueueProducerInterface` from day one and add a real broker later — you can run the queue
> in [synchronous mode](docs/guide/en/synchronous-mode.md) (the adapter argument is optional).
> In this mode messages are processed immediately in the same process, so it won't provide true
> async execution, but the code stays the same when you switch to a real adapter.
Expand Down Expand Up @@ -138,9 +138,11 @@ For setting up all classes manually, see the [Manual configuration](docs/guide/e
To send a message to the queue, get the queue instance and call `push()`. Typically the queue is injected as a dependency:

```php
use Yiisoft\Queue\QueueProducerInterface;

final readonly class Foo
{
public function __construct(private QueueInterface $queue) {}
public function __construct(private QueueProducerInterface $queue) {}

public function bar(): void
{
Expand All @@ -159,11 +161,13 @@ By default, Yii Framework uses [yiisoft/yii-console](https://github.com/yiisoft/
```bash
./yii queue:run # Handle all existing messages in the queue
./yii queue:listen [queueName] # Start a daemon listening for new messages permanently from the specified queue
./yii queue:listen-all [queueName [queueName2 [...]]] # Start a daemon listening for new messages permanently from all queues or specified list of queues (use with caution in production, recommended for dev only)
./yii queue:listen-all [queueName [queueName2 [...]]] # Start a daemon listening for new messages permanently from all consumer-capable queues or specified list of queues (use with caution in production, recommended for dev only)
```

See [Console commands](docs/guide/en/console-commands.md) for more details.

Producers use `Yiisoft\Queue\QueueProducerInterface` (`push()`, `status()`, `getName()`); consumers use `Yiisoft\Queue\QueueConsumerInterface` (`run()`, `listen()`). See [capability configuration](docs/guide/en/queue-capabilities.md) for the strict role map used when named queues are configured.

> In case you're running the queue in synchronous mode (no adapter), `queue:listen` logs an info message and exits. The messages are processed immediately when pushed.

## Documentation
Expand Down
9 changes: 6 additions & 3 deletions config/params.php
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,13 @@
use Yiisoft\Queue\Command\ListenCommand;
use Yiisoft\Queue\Command\RunCommand;
use Yiisoft\Queue\Debug\QueueCollector;
use Yiisoft\Queue\Debug\QueueProviderInterfaceProxy;
use Yiisoft\Queue\Debug\QueueConsumerProviderProxy;
use Yiisoft\Queue\Debug\QueueProducerProviderProxy;
use Yiisoft\Queue\Debug\QueueWorkerInterfaceProxy;
use Yiisoft\Queue\Message\MessageHandlerInterface;
use Yiisoft\Queue\Message\Serializer\MessageSerializer;
use Yiisoft\Queue\Provider\QueueProviderInterface;
use Yiisoft\Queue\Provider\QueueConsumerProviderInterface;
use Yiisoft\Queue\Provider\QueueProducerProviderInterface;
use Yiisoft\Queue\Worker\WorkerInterface;

return [
Expand Down Expand Up @@ -50,7 +52,8 @@
QueueCollector::class,
],
'trackedServices' => [
QueueProviderInterface::class => [QueueProviderInterfaceProxy::class, QueueCollector::class],
QueueProducerProviderInterface::class => [QueueProducerProviderProxy::class, QueueCollector::class],
QueueConsumerProviderInterface::class => [QueueConsumerProviderProxy::class, QueueCollector::class],
WorkerInterface::class => [QueueWorkerInterfaceProxy::class, QueueCollector::class],
],
],
Expand Down
1 change: 1 addition & 0 deletions docs/guide/en/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Yii Queue is a framework-agnostic PHP queue library for running tasks asynchrono
- [Adapter list](adapter-list.md)
- [Synchronous mode](synchronous-mode.md)
- [Queue names](queue-names.md)
- [Producer and consumer capabilities](queue-capabilities.md)

## Build and handle messages

Expand Down
16 changes: 2 additions & 14 deletions docs/guide/en/advanced-map.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Use this index when you need to customize internals: custom middleware, adapters
## Configuration and infrastructure

- [Manual configuration without yiisoft/config](configuration-manual.md) — wiring queues, workers, and middleware factories without `yiisoft/config`.
- [Queue provider registry](#queue-provider-registry) — selecting and extending adapter factories.
- [Advanced queue names and providers](queue-names-advanced.md) — resolving named producer and consumer capabilities, composing providers, and implementing custom registries.
- [Loops and worker processes](loops.md) — implementing custom runners, heartbeat hooks, and graceful shutdown (requires `pcntl`).
- [Worker](worker.md) — resolving worker dependencies and starting workers.
- [Performance tuning](performance-tuning.md) — profiling handlers, envelopes, and adapters.
Expand All @@ -20,7 +20,7 @@ Use this index when you need to customize internals: custom middleware, adapters

## Queue adapters and interoperability

- [Custom queue provider implementations](queue-names-advanced.md#extending-the-registry) — bespoke selection logic, tenant registries, and fallback strategies.
- [Custom queue provider implementations](queue-names-advanced.md#combining-and-extending-providers) — bespoke selection logic, tenant registries, and fallback strategies.
- [Consuming messages from external systems](consuming-messages-from-external-systems.md) — contract for third-party producers.

## Tooling and diagnostics
Expand All @@ -30,15 +30,3 @@ Use this index when you need to customize internals: custom middleware, adapters
## Internals and contribution

- [Internals guide](../../internals.md) — local QA tooling (PHPUnit, Infection, Psalm, Rector, ComposerRequireChecker).

## Queue provider registry

When multiple queue names share infrastructure, rely on `QueueProviderInterface`:

- A queue name is passed to `QueueProviderInterface::get($queueName)` and resolved into a configured `QueueInterface` instance.
- Default implementation (`AdapterFactoryQueueProvider`) enforces a strict registry defined in `yiisoft/queue.queues`. Unknown names throw `QueueNotFoundException`.
- Alternative providers include:
- `PredefinedQueueProvider` — accepts a pre-built map of queue name → `QueueInterface` instance.
- `QueueFactoryProvider` — creates queue objects lazily from [`yiisoft/factory`](https://github.com/yiisoft/factory) definitions.
- `CompositeQueueProvider` — aggregates multiple providers and selects the first that knows the queue name.
- Implement `QueueProviderInterface` to introduce custom registries or fallback strategies, then register the implementation in DI.
30 changes: 20 additions & 10 deletions docs/guide/en/configuration-manual.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ To use the queue, you need to create instances of the following classes:

1. **Adapter** - handles the actual queue backend like AMQP, Redis, etc.
2. **Worker** - processes messages from the queue
3. **Queue** - the main entry point for pushing messages
3. **QueueProducer** - pushes messages; **QueueConsumer** consumes them when needed

### Example

Expand All @@ -24,7 +24,8 @@ use Yiisoft\Queue\Middleware\FailureHandling\FailureMiddlewareDispatcher;
use Yiisoft\Queue\Middleware\FailureHandling\FailureMiddlewareFactory;
use Yiisoft\Queue\Middleware\Push\PushMiddlewareConfig;
use Yiisoft\Queue\Middleware\Push\PushMiddlewareFactory;
use Yiisoft\Queue\Queue;
use Yiisoft\Queue\QueueConsumer;
use Yiisoft\Queue\QueueProducer;
use Yiisoft\Queue\Worker\Worker;

// A PSR-11 container is required for resolving dependencies of middleware and handlers.
Expand Down Expand Up @@ -70,16 +71,16 @@ $loop = new SimpleLoop();

// Create queue. Without an adapter the queue runs in synchronous mode (messages are processed
// immediately on push). Pass an adapter (e.g., AMQP, Redis) for asynchronous processing.
$queue = new Queue(
$worker,
$loop,
$producer = new QueueProducer(
$logger,
$pushMiddlewareConfig,
worker: $worker,
);
$consumer = new QueueConsumer($worker, $loop, $logger);

// Now you can push messages
// Now you can push messages. With no adapter, the producer dispatches directly to the worker.
$message = new DownloadFileMessage(url: 'https://example.com/file.pdf', destinationPath: '/tmp/file.pdf');
$queue->push($message);
$producer->push($message);
```

## Using Queue Provider
Expand All @@ -89,25 +90,34 @@ For multiple queue names, use `PredefinedQueueProvider` (maps queue names to pre
```php
use Yiisoft\Queue\Provider\PredefinedQueueProvider;

// PredefinedQueueProvider: pass fully built queue instances.
// PredefinedQueueProvider: pass fully built role instances in a strict role map.
$provider = new PredefinedQueueProvider([
'queue1' => $queue1,
'queue2' => $queue2,
'queue1' => ['producer' => $producer1, 'consumer' => $consumer1],
'queue2' => ['producer' => $producer2],
]);
```

## Running the queue

Message consumption methods are available on `Yiisoft\Queue\QueueConsumerInterface`.
`QueueProducer` and `QueueConsumer` are separate capabilities. Obtain or construct the consumer role before calling these methods.

### Processing existing messages

```php
use Yiisoft\Queue\QueueConsumerInterface;

/** @var QueueConsumerInterface $queue */
$queue->run(); // Process all messages
$queue->run(10); // Process up to 10 messages
```

### Listening for new messages

```php
use Yiisoft\Queue\QueueConsumerInterface;

/** @var QueueConsumerInterface $queue */
$queue->listen(); // Run indefinitely
```

Expand Down
6 changes: 3 additions & 3 deletions docs/guide/en/console-commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ If you are using [yiisoft/config](https://github.com/yiisoft/config) and [yiisof

If you are using [symfony/console](https://github.com/symfony/console) directly, you should register the commands manually.

> **Note:** The default queue name list (used when no queue names are passed to a command) is only available when using [yiisoft/config](https://github.com/yiisoft/config) and [yiisoft/yii-console](https://github.com/yiisoft/yii-console). Without them, you must pass the queue name list explicitly to the command constructor.
> **Note:** `queue:run` and `queue:listen-all` use `QueueConsumerProviderInterface::getConsumerNames()` when no queue names are passed. Explicitly passed names are resolved with `getConsumer()` and must have a consumer role.

In [yiisoft/app](https://github.com/yiisoft/app) the `yii` console binary is provided out of the box.
If you are using [yiisoft/yii-console](https://github.com/yiisoft/yii-console) or `symfony/console` without that template, invoke these commands the same way you invoke other console commands in your application.
Expand All @@ -17,7 +17,7 @@ The command `queue:run` obtains and handles messages until the queue is empty, t

You can also narrow the scope of processed messages by specifying queue name(s) and maximum number of messages to process:

- Specify one or more queue names to process. Messages from other queues will be ignored. Defaults to all registered queue names.
- Specify one or more queue names to process. Messages from other queues will be ignored. Defaults to all registered consumer-capable queue names.
- Use `--limit` to limit the number of messages processed. When set, command will exit either when all the messages are processed or when the maximum count is reached.

The full command signature is:
Expand All @@ -39,7 +39,7 @@ yii queue:listen [queueName]

The following command iterates through multiple queues and is meant to be used in development environment only, as it consumes a lot of CPU for iterating through queues. You can pass to it:

- `queueName` argument(s). Specify one or more queue names to process. Messages from other queues will be ignored. Defaults to all registered queue names.
- `queueName` argument(s). Specify one or more queue names to process. Messages from other queues will be ignored. Defaults to all registered consumer-capable queue names.
- `--limit` option to limit the number of messages processed before switching to another queue. E.g. you set `--limit` to 500 and right now you have 1000 messages in `queue1`. This command will consume only 500 of them, then it will switch to `queue2` to see if there are any messages there. Defaults to `0` (no limit).
- `--pause` option to specify the number of seconds to pause between checking queues when no messages are found. Defaults to `1`.

Expand Down
16 changes: 10 additions & 6 deletions docs/guide/en/debug-integration-advanced.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Use this guide when you need to understand which events are tracked by the queue
The integration is based on `Yiisoft\Queue\Debug\QueueCollector` and captures:

- Pushed messages grouped by queue name.
- Message status checks performed via `QueueInterface::status()`.
- Message status checks performed via `QueueProducerInterface::status()`.
- Messages processed by a worker grouped by queue name.

## How it works
Expand All @@ -16,20 +16,23 @@ The collector is enabled by registering it in Yii Debug and wrapping tracked ser

Out of the box (see this package's `config/params.php`), the following services are wrapped:

- `Yiisoft\Queue\Provider\QueueProviderInterface` is wrapped with `Yiisoft\Queue\Debug\QueueProviderInterfaceProxy`. The proxy decorates returned queues with `Yiisoft\Queue\Debug\QueueDecorator` so that `push()` and `status()` calls are reported to the collector.
- `Yiisoft\Queue\Provider\QueueProducerProviderInterface` is wrapped with `Yiisoft\Queue\Debug\QueueProducerProviderProxy`, which returns `QueueProducerDecorator` instances so `push()` and `status()` calls are reported.
- `Yiisoft\Queue\Provider\QueueConsumerProviderInterface` is wrapped with `Yiisoft\Queue\Debug\QueueConsumerProviderProxy`, which returns typed consumer decorators.
- `Yiisoft\Queue\Worker\WorkerInterface` is wrapped with `Yiisoft\Queue\Debug\QueueWorkerInterfaceProxy` to record message processing events.

To see data in the debug panel, obtain `QueueProviderInterface` and `WorkerInterface` from the DI container — the debug proxies are registered there and will not be active if the services are instantiated directly.
To see data in the debug panel, obtain the typed provider dependencies and `WorkerInterface` from the DI container — the proxies are registered there and will not be active if the services are instantiated directly.

## Manual configuration

If you do not rely on the defaults supplied via [yiisoft/config](https://github.com/yiisoft/config), configure the collector and proxies explicitly:

```php
use Yiisoft\Queue\Debug\QueueCollector;
use Yiisoft\Queue\Debug\QueueProviderInterfaceProxy;
use Yiisoft\Queue\Debug\QueueConsumerProviderProxy;
use Yiisoft\Queue\Debug\QueueProducerProviderProxy;
use Yiisoft\Queue\Debug\QueueWorkerInterfaceProxy;
use Yiisoft\Queue\Provider\QueueProviderInterface;
use Yiisoft\Queue\Provider\QueueConsumerProviderInterface;
use Yiisoft\Queue\Provider\QueueProducerProviderInterface;
use Yiisoft\Queue\Worker\WorkerInterface;

return [
Expand All @@ -38,7 +41,8 @@ return [
QueueCollector::class,
],
'trackedServices' => [
QueueProviderInterface::class => [QueueProviderInterfaceProxy::class, QueueCollector::class],
QueueProducerProviderInterface::class => [QueueProducerProviderProxy::class, QueueCollector::class],
QueueConsumerProviderInterface::class => [QueueConsumerProviderProxy::class, QueueCollector::class],
WorkerInterface::class => [QueueWorkerInterfaceProxy::class, QueueCollector::class],
],
],
Expand Down
5 changes: 3 additions & 2 deletions docs/guide/en/error-handling-advanced.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ This document covers advanced internals of the failure handling pipeline, built-

- the message
- the caught exception
- the queue instance
- the logical queue name
- an optional direct retry producer (provided for synchronous producer execution)

4. A failure pipeline is selected by queue name

Expand Down Expand Up @@ -83,7 +84,7 @@ This interface has the only method `processFailure` with these parameters:
- [`FailureHandlingRequest $request`](../../../src/Middleware/FailureHandling/FailureHandlingRequest.php) - a request for a message handling. It consists of
- a [message](../../../src/Message/MessageInterface.php)
- a `Throwable $exception` object thrown on the `request` handling
- a queue the message came from
- the logical queue name the message came from and, when available, a direct retry producer
- `FailureHandlerInterface $handler` - failure strategy pipeline continuation. Your Middleware should call `$handler->handleFailure($request)` when the middleware itself should not interrupt failure pipeline execution.

> Note: your strategy have to check by its own if it should be applied. Look into [`SendAgainMiddleware::suits()`](../../../src/Middleware/FailureHandling/Implementation/SendAgainMiddleware.php#L54) for an example.
Loading
Loading