Introduction
The Laravel queue service provides a unified API across a variety of different queue back-ends. Queues allow you to defer the processing of a time consuming task, such as sending an e-mail, until a later time which drastically speeds up web requests to your application.
Configuration
The queue configuration file is stored in
config/queue.php. In this file you will
find connection configurations for each of the queue
drivers that are included with the framework, which
includes a database, Beanstalkd,
Amazon SQS, Redis, and synchronous
(for local use) driver.
A null queue driver is also included which
simply discards queued jobs.
Driver Prerequisites
Database
In order to use the database queue driver,
you will need a database table to hold the jobs. To
generate a migration that creates this table, run the
queue:table Artisan command. Once the
migration is created, you may migrate your database
using the migrate command:
php artisan queue:table
php artisan migrate
Other Queue Dependencies
The following dependencies are needed for the listed queue drivers:
- Amazon SQS:
aws/aws-sdk-php ~3.0 - Beanstalkd:
pda/pheanstalk ~3.0 - Redis:
predis/predis ~1.0
Writing Job Classes
Generating Job Classes
By default, all of the queueable jobs for your
application are stored in the app/Jobs
directory. You may generate a new queued job using the
Artisan CLI:
php artisan make:job SendReminderEmail
This command will generate a new class in the
app/Jobs directory, and the class will
implement the
Illuminate\Contracts\Queue\ShouldQueue
interface, indicating to Laravel that the job should be
pushed onto the queue instead of run synchronously.
Job Class Structure
Job classes are very simple, normally containing only a
handle method which is called when the job
is processed by the queue. To get started, let's take a
look at an example job class:
<?php
namespace App\Jobs;
use App\User;
use App\Jobs\Job;
use Illuminate\Contracts\Mail\Mailer;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
class SendReminderEmail extends Job implements ShouldQueue
{
use InteractsWithQueue, SerializesModels;
protected $user;
/**
* Create a new job instance.
*
* @param User $user
* @return void
*/
public function __construct(User $user)
{
$this->user = $user;
}
/**
* Execute the job.
*
* @param Mailer $mailer
* @return void
*/
public function handle(Mailer $mailer)
{
$mailer->send('emails.reminder', ['user' => $this->user], function ($m) {
//
});
$this->user->reminders()->create(...);
}
}
In this example, note that we were able to pass an Eloquent model directly
into the queued job's constructor. Because of the
SerializesModels trait that the job is
using, Eloquent models will be gracefully serialized and
unserialized when the job is processing. If your queued
job accepts an Eloquent model in its constructor, only
the identifier for the model will be serialized onto the
queue. When the job is actually handled, the queue
system will automatically re-retrieve the full model
instance from the database. It's all totally transparent
to your application and prevents issues that can arise
from serializing full Eloquent model instances.
The handle method is called when the job is
processed by the queue. Note that we are able to
type-hint dependencies on the handle method
of the job. The Laravel service
container automatically injects these
dependencies.
When Things Go Wrong
If an exception is thrown while the job is being
processed, it will automatically be released back onto
the queue so it may be attempted again. The job will
continue to be released until it has been attempted the
maximum number of times allowed by your application. The
number of maximum attempts is defined by the
--tries switch used on the
queue:listen or queue:work
Artisan jobs. More information on running the queue
listener can be
found below.
Manually Releasing Jobs
If you would like to release the job
manually, the InteractsWithQueue trait,
which is already included in your generated job class,
provides access to the queue job release
method. The release method accepts one
argument: the number of seconds you wish to wait until
the job is made available again:
public function handle(Mailer $mailer)
{
if (condition) {
$this->release(10);
}
}
Checking The Number Of Run Attempts
As noted above, if an exception occurs while the job is
being processed, it will automatically be released back
onto the queue. You may check the number of attempts
that have been made to run the job using the
attempts method:
public function handle(Mailer $mailer)
{
if ($this->attempts() > 3) {
//
}
}
Pushing Jobs Onto The Queue
The default Laravel controller located in
app/Http/Controllers/Controller.php uses a
DispatchesJobs trait. This trait provides
several methods allowing you to conveniently push jobs
onto the queue, such as the dispatch
method:
<?php
namespace App\Http\Controllers;
use App\User;
use Illuminate\Http\Request;
use App\Jobs\SendReminderEmail;
use App\Http\Controllers\Controller;
class UserController extends Controller
{
/**
* Send a reminder e-mail to a given user.
*
* @param Request $request
* @param int $id
* @return Response
*/
public function sendReminderEmail(Request $request, $id)
{
$user = User::findOrFail($id);
$this->dispatch(new SendReminderEmail($user));
}
}
The DispatchesJobs Trait
Of course, sometimes you may wish to dispatch a job from
somewhere in your application besides a route or
controller. For that reason, you can include the
DispatchesJobs trait on any of the classes
in your application to gain access to its various
dispatch methods. For example, here is a sample class
that uses the trait:
<?php
namespace App;
use Illuminate\Foundation\Bus\DispatchesJobs;
class ExampleClass
{
use DispatchesJobs;
}
The dispatch Function
Or, you may use the dispatch global
function:
Route::get('/job', function () {
dispatch(new App\Jobs\PerformTask);
return 'Done!';
});
Specifying The Queue For A Job
You may also specify the queue a job should be sent to.
By pushing jobs to different queues, you may
"categorize" your queued jobs, and even
prioritize how many workers you assign to various
queues. This does not push jobs to different queue
"connections" as defined by your queue
configuration file, but only to specific queues within a
single connection. To specify the queue, use the
onQueue method on the job instance. The
onQueue method is provided by the
Illuminate\Bus\Queueable trait, which is
already included on the App\Jobs\Job base
class:
<?php
namespace App\Http\Controllers;
use App\User;
use Illuminate\Http\Request;
use App\Jobs\SendReminderEmail;
use App\Http\Controllers\Controller;
class UserController extends Controller
{
/**
* Send a reminder e-mail to a given user.
*
* @param Request $request
* @param int $id
* @return Response
*/
public function sendReminderEmail(Request $request, $id)
{
$user = User::findOrFail($id);
$job = (new SendReminderEmail($user))->onQueue('emails');
$this->dispatch($job);
}
}
Note: The
DispatchesJobstrait pushes jobs to queues within the default queue connection.
Specifying The Queue Connection For A Job
If you are working with multiple queue connections, you
may specify which connection to push a job to. To
specify the connection, use the
onConnection method on the job instance.
The onConnection method is provided by the
Illuminate\Bus\Queueable trait, which is
already included on the App\Jobs\Job base
class:
<?php
namespace App\Http\Controllers;
use App\User;
use Illuminate\Http\Request;
use App\Jobs\SendReminderEmail;
use App\Http\Controllers\Controller;
class UserController extends Controller
{
/**
* Send a reminder e-mail to a given user.
*
* @param Request $request
* @param int $id
* @return Response
*/
public function sendReminderEmail(Request $request, $id)
{
$user = User::findOrFail($id);
$job = (new SendReminderEmail($user))->onConnection('alternate');
$this->dispatch($job);
}
}
Of course, you can also chain the
onConnection and onQueue
methods to specify the connection and the queue for a
job:
public function sendReminderEmail(Request $request, $id)
{
$user = User::findOrFail($id);
$job = (new SendReminderEmail($user))
->onConnection('alternate')
->onQueue('emails');
$this->dispatch($job);
}
Delayed Jobs
Sometimes you may wish to delay the execution of a queued
job. For instance, you may wish to queue a job that
sends a customer a reminder e-mail 5 minutes after
sign-up. You may accomplish this using the
delay method on your job class, which is
provided by the Illuminate\Bus\Queueable
trait:
<?php
namespace App\Http\Controllers;
use App\User;
use Illuminate\Http\Request;
use App\Jobs\SendReminderEmail;
use App\Http\Controllers\Controller;
class UserController extends Controller
{
/**
* Send a reminder e-mail to a given user.
*
* @param Request $request
* @param int $id
* @return Response
*/
public function sendReminderEmail(Request $request, $id)
{
$user = User::findOrFail($id);
$job = (new SendReminderEmail($user))->delay(60 * 5);
$this->dispatch($job);
}
}
In this example, we're specifying that the job should be delayed in the queue for 5 minutes before being made available to workers.
Note: The Amazon SQS service has a maximum delay time of 15 minutes.
Job Events
Job Lifecycle Events
The Queue::before and
Queue::after methods allow you to register
a callback to be executed before a queued job is started
or when it executes successfully. The callbacks are
great opportunity to perform additional logging, queue a
subsequent job, or increment statistics for a dashboard.
For example, we may attach a callback to this event from
the AppServiceProvider that is included
with Laravel:
<?php
namespace App\Providers;
use Queue;
use Illuminate\Support\ServiceProvider;
use Illuminate\Queue\Events\JobProcessed;
class AppServiceProvider extends ServiceProvider
{
/**
* Bootstrap any application services.
*
* @return void
*/
public function boot()
{
Queue::after(function (JobProcessed $event) {
// $event->connectionName
// $event->job
// $event->data
});
}
/**
* Register the service provider.
*
* @return void
*/
public function register()
{
//
}
}
Running The Queue Listener
Starting The Queue Listener
Laravel includes an Artisan command that will run new
jobs as they are pushed onto the queue. You may run the
listener using the queue:listen
command:
php artisan queue:listen
You may also specify which queue connection the listener should utilize:
php artisan queue:listen connection-name
Note that once this task has started, it will continue to run until it is manually stopped. You may use a process monitor such as Supervisor to ensure that the queue listener does not stop running.
Queue Priorities
You may pass a comma-delimited list of queue connections
to the listen job to set queue
priorities:
php artisan queue:listen --queue=high,low
In this example, jobs on the high queue will
always be processed before moving onto jobs from the
low queue.
Specifying The Job Timeout Parameter
You may also set the length of time (in seconds) each job should be allowed to run:
php artisan queue:listen --timeout=60
Specifying Queue Sleep Duration
In addition, you may specify the number of seconds to wait before polling for new jobs:
php artisan queue:listen --sleep=5
Note that the queue only "sleeps" if no jobs are on the queue. If more jobs are available, the queue will continue to work them without sleeping.
Processing The First Job On The Queue
To process only the first job on the queue, you may use
the queue:work command:
php artisan queue:work
Supervisor Configuration
Supervisor is a process monitor for the Linux operating
system, and will automatically restart your
queue:listen or queue:work
commands if they fail. To install Supervisor on Ubuntu,
you may use the following command:
sudo apt-get install supervisor
Supervisor configuration files are typically stored in
the /etc/supervisor/conf.d directory.
Within this directory, you may create any number of
configuration files that instruct supervisor how your
processes should be monitored. For example, let's create
a laravel-worker.conf file that starts and
monitors a queue:work process:
[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /home/forge/app.com/artisan queue:work sqs --sleep=3 --tries=3 --daemon
autostart=true
autorestart=true
user=forge
numprocs=8
redirect_stderr=true
stdout_logfile=/home/forge/app.com/worker.log
In this example, the numprocs directive will
instruct Supervisor to run 8 queue:work
processes and monitor all of them, automatically
restarting them if they fail. Of course, you should
change the queue:work sqs portion of the
command directive to reflect your chosen
queue driver.
Once the configuration file has been created, you may update the Supervisor configuration and start the processes using the following commands:
sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl start laravel-worker:*
For more information on configuring and using Supervisor, consult the Supervisor documentation. Alternatively, you may use Laravel Forge to automatically configure and manage your Supervisor configuration from a convenient web interface.
Daemon Queue Listener
The queue:work Artisan command includes a
--daemon option for forcing the queue
worker to continue processing jobs without ever
re-booting the framework. This results in a significant
reduction of CPU usage when compared to the
queue:listen command:
To start a queue worker in daemon mode, use the
--daemon flag:
php artisan queue:work connection-name --daemon
php artisan queue:work connection-name --daemon --sleep=3
php artisan queue:work connection-name --daemon --sleep=3 --tries=3
As you can see, the queue:work job supports
most of the same options available to
queue:listen. You may use the php
artisan help queue:work job to view all of
the available options.
Coding Considerations For Daemon Queue Listeners
Daemon queue workers do not restart the framework before
processing each job. Therefore, you should be careful to
free any heavy resources before your job finishes. For
example, if you are doing image manipulation with the GD
library, you should free the memory with
imagedestroy when you are done.
Deploying With Daemon Queue Listeners
Since daemon queue workers are long-lived processes, they will not pick up changes in your code without being restarted. So, the simplest way to deploy an application using daemon queue workers is to restart the workers during your deployment script. You may gracefully restart all of the workers by including the following command in your deployment script:
php artisan queue:restart
This command will gracefully instruct all queue workers
to "die" after they finish processing their
current job so that no existing jobs are lost. Remember,
the queue workers will die when the
queue:restart command is executed, so you
should be running a process manager such as Supervisor
which automatically restarts the queue workers.
Note: This command relies on the cache system to schedule the restart. By default, APCu does not work for CLI jobs. If you are using APCu, add
apc.enable_cli=1to your APCu configuration.
Dealing With Failed Jobs
Since things don't always go as planned, sometimes your
queued jobs will fail. Don't worry, it happens to the
best of us! Laravel includes a convenient way to specify
the maximum number of times a job should be attempted.
After a job has exceeded this amount of attempts, it
will be inserted into a failed_jobs table.
The name of the table can be configured via the
config/queue.php configuration file.
To create a migration for the failed_jobs
table, you may use the queue:failed-table
command:
php artisan queue:failed-table
When running your queue
listener, you may specify the maximum number of
times a job should be attempted using the
--tries switch on the
queue:listen command:
php artisan queue:listen connection-name --tries=3
Failed Job Events
If you would like to register an event that will be
called when a queued job fails, you may use the
Queue::failing method. This event is a
great opportunity to notify your team via e-mail or HipChat. For
example, we may attach a callback to this event from the
AppServiceProvider that is included with
Laravel:
<?php
namespace App\Providers;
use Queue;
use Illuminate\Queue\Events\JobFailed;
use Illuminate\Support\ServiceProvider;
class AppServiceProvider extends ServiceProvider
{
/**
* Bootstrap any application services.
*
* @return void
*/
public function boot()
{
Queue::failing(function (JobFailed $event) {
// $event->connectionName
// $event->job
// $event->data
});
}
/**
* Register the service provider.
*
* @return void
*/
public function register()
{
//
}
}
Failed Method On Job Classes
For more granular control, you may define a
failed method directly on a queue job
class, allowing you to perform job specific actions when
a failure occurs:
<?php
namespace App\Jobs;
use App\Jobs\Job;
use Illuminate\Contracts\Mail\Mailer;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
class SendReminderEmail extends Job implements ShouldQueue
{
use InteractsWithQueue, SerializesModels;
/**
* Execute the job.
*
* @param Mailer $mailer
* @return void
*/
public function handle(Mailer $mailer)
{
//
}
/**
* Handle a job failure.
*
* @return void
*/
public function failed()
{
// Called when the job is failing...
}
}
Retrying Failed Jobs
To view all of your failed jobs that have been inserted
into your failed_jobs database table, you
may use the queue:failed Artisan
command:
php artisan queue:failed
The queue:failed command will list the job
ID, connection, queue, and failure time. The job ID may
be used to retry the failed job. For instance, to retry
a failed job that has an ID of 5, the following command
should be issued:
php artisan queue:retry 5
To retry all of your failed jobs, use
queue:retry with all as the
ID:
php artisan queue:retry all
If you would like to delete a failed job, you may use the
queue:forget command:
php artisan queue:forget 5
To delete all of your failed jobs, you may use the
queue:flush command:
php artisan queue:flush