Laravel Queues

To run Laravel Queues on AWS Lambda using Amazon SQS , we don’t want to run the php artisan queue:work command. Instead, we create a Lambda function that is invoked immediately when there are new jobs to process.

To create the SQS queue (and the permissions for the Lambda functions to read/write to it), we can either do that manually, or use serverless.yml.

To make things simpler, we will use the Serverless Lift  plugin to create and configure the SQS queue.

First install the Lift plugin:

npm install --save-dev serverless-lift

Enable it in the plugins section of serverless.yml (in the file generated for Laravel, uncomment the - serverless-lift line), then use the Queue construct  in serverless.yml:

provider:
    # ...
    environment:
        # ...
        QUEUE_CONNECTION: sqs
        SQS_QUEUE: ${construct:jobs.queueUrl}
 
functions:
    # ...
 
constructs:
    jobs:
        type: queue
        worker:
            handler: Bref\LaravelBridge\Queue\QueueHandler
            runtime: php-84
            timeout: 60 # seconds

We define Laravel environment variables in provider.environment (this could also be done in the deployed .env file):

• QUEUE_CONNECTION: sqs enables the SQS queue connection
• SQS_QUEUE: ${construct:jobs.queueUrl} passes the URL of the created SQS queue

If you want to create the SQS queue manually, you will need to set these variables. AWS credentials (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY) are automatically set up with the appropriate permissions for Laravel to use the SQS queue.

That’s it! Anytime a job is pushed to Laravel Queues, it will be sent to SQS, and SQS will invoke our “worker” function so that it is processed.

How it works

When integrated with AWS Lambda, SQS has a built-in retry mechanism and storage for failed messages. These features work slightly differently than Laravel Queues. The “Bref for Laravel” integration does not use these SQS features.

Instead, “Bref for Laravel” makes all the features of Laravel Queues work out of the box, just like on any server. Read more in the Laravel Queues documentation .

Failed jobs are stored in the failed_jobs database table, like on any server. If your application has no database, set QUEUE_FAILED_DRIVER: 'null' in provider.environment: failed jobs are still logged, but not stored. Otherwise, storing the failed job fails as well, for example with Database file at path [/var/task/database/database.sqlite] does not exist.

Multiple queues

On a server, one worker can process several queues by priority (php artisan queue:work --queue=high,default). On Lambda, each SQS queue has its own worker function: all queues are processed at the same time, and each worker scales on its own. There is no priority between queues.

Multiple queues are useful when some jobs need different settings, for example:

• a longer timeout or more memory, for jobs like report generation,
• a limited concurrency, to stay below the rate limit of an API or to avoid overloading the database,
• a separate queue for urgent jobs, so that a burst of bulk jobs does not delay them,
• a FIFO queue , for jobs that must be processed in order.

If all jobs can use the same settings, a single queue is simpler.

Jobs are dispatched to a queue by name, like on any Laravel application:

SendWelcomeEmail::dispatch($user)->onQueue('emails');

Add a queue construct for each queue in serverless.yml, and set SQS_PREFIX:

provider:
    # ...
    environment:
        # ...
        QUEUE_CONNECTION: sqs
        # The default queue
        SQS_QUEUE: ${construct:jobs.queueUrl}
        # Base URL of the other queues
        SQS_PREFIX: https://sqs.${aws:region}.amazonaws.com/${aws:accountId}
 
constructs:
    jobs:
        type: queue
        worker:
            handler: Bref\LaravelBridge\Queue\QueueHandler
            runtime: php-84
            timeout: 60 # seconds
 
    emails:
        type: queue
        # At most 5 jobs of this queue run at the same time
        maxConcurrency: 5
        worker:
            handler: Bref\LaravelBridge\Queue\QueueHandler
            runtime: php-84
            timeout: 20 # seconds

All workers use the same QueueHandler class, without extra configuration: a worker processes the jobs of the queue that invokes it. Settings like maxConcurrency, batchSize or fifo are set on each construct, see the Lift queue documentation .

Lift names SQS queues <stack>-<construct>, for example my-app-prod-emails. Send the jobs of the emails queue to that SQS queue with Queue::forward() (Laravel 13.26 and later, see below for older versions), in AppServiceProvider::boot():

use Illuminate\Support\Facades\Queue;
 
if ($stack = config('bref.stack_name')) {
    Queue::forward([
        'emails' => "$stack-emails",
    ], connection: 'sqs');
}

config('bref.stack_name') contains the name of the CloudFormation stack on AWS Lambda, and is null in local development. Keep the if: connection: 'sqs' also moves the jobs of the forwarded queues to the SQS connection, which would break local development.

For a FIFO queue, for example an orders construct with fifo: true, forward 'orders.fifo' => "$stack-orders.fifo": Lift adds .fifo to the name of FIFO queues.

On Laravel 13 and later, Queue::route() sends all the jobs of a class to a queue, without calling onQueue() on each dispatch:

Queue::route(SendWelcomeEmail::class, queue: 'emails');

Jobs dispatched without a queue go to the default queue (SQS_QUEUE), even when they are dispatched by a job that runs in another queue.

Jobs stay in their queue: a job that throws an exception is retried in the same queue. Failed jobs also remember their queue, so php artisan queue:retry (or the “Retry” button in the Bref Cloud dashboard) sends them back to the queue they came from.

Before Laravel 13.26

Queue::forward() is available since Laravel 13.26. On older versions, including Laravel 12, pass the URL of each queue to Laravel:

provider:
    environment:
        SQS_QUEUE_EMAILS: ${construct:emails.queueUrl}

return [
    // ...
 
    'queues' => [
        // SQS queue URL on Lambda, queue name in local development
        'emails' => env('SQS_QUEUE_EMAILS', 'emails'),
    ],
];

Then dispatch jobs with that URL:

SendWelcomeEmail::dispatch($user)->onQueue(config('queue.queues.emails'));