Configuring Queue Workers

+15 Mana ✨
Running Queue Workers(DOCUMENTATION)Running Queue Workers(DOCUMENTATION)

Introduction

A queue worker is a long-running PHP process that pulls jobs off a connection and executes them, one after the other, until you stop it. Getting the worker configuration right — timeouts, retries, memory limits, restart policy — is the single biggest lever on queue reliability.

Key Concepts

  • Worker: The queue:work artisan command, typically run under a process supervisor.
  • Supervisor: An external process manager (Supervisor, systemd) that keeps workers running and restarts them when they exit.
  • Graceful restart: queue:restart tells workers to finish their current job, then exit — so a fresh process can boot with new code.
  • Failed jobs: Jobs that exceeded $tries or crashed are moved to the failed_jobs table for inspection.

Real World Context

An app that ships twice a day must restart workers after each deploy — old workers have the old code cached in memory. An app with spiky traffic needs queue priorities so a 10,000-item import doesn't starve password-reset emails. Both of these are worker configuration problems, not job code problems.

Deep Dive

Worker Commands

bash
# Basic worker
php artisan queue:work

# Specific connection and queue
php artisan queue:work redis --queue=emails

# Common options
php artisan queue:work \
    --tries=3 \           # Retry failed jobs 3 times
    --timeout=60 \         # Max seconds per job
    --memory=128 \         # Restart if memory exceeds 128MB
    --sleep=3 \            # Seconds to sleep when no jobs
    --max-jobs=1000 \      # Restart after processing 1000 jobs
    --max-time=3600        # Restart after 1 hour

Note: the Laravel 13 queue docs only cover queue:work. Treat it as the canonical worker command — boot it once and rely on queue:restart (and Supervisor) to cycle it after deploys.

Queue Priorities

Process high-priority jobs first:

bash
php artisan queue:work --queue=high,default,low

Dispatch to specific queues:

php
// Dispatch to high priority queue
SendUrgentEmail::dispatch($user)->onQueue('high');

// Normal priority (default queue)
ProcessReport::dispatch($data);

// Low priority
CleanupOldFiles::dispatch()->onQueue('low');

Supervisor Configuration

For production, use Supervisor to keep workers running:

ini
# /etc/supervisor/conf.d/laravel-worker.conf
[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/html/artisan queue:work redis --sleep=3 --tries=3 --max-time=3600
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=www-data
numprocs=8
redirect_stderr=true
stdout_logfile=/var/www/html/storage/logs/worker.log
stopwaitsecs=3600
bash
# Apply configuration
sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl start "laravel-worker:*"

Restarting Workers

After deploying new code:

bash
# Graceful restart (finishes current job)
php artisan queue:restart

Important: Workers cache your application. Always restart after deployments!

Handling Failed Jobs

bash
# View failed jobs
php artisan queue:failed

# Retry a specific job
php artisan queue:retry 5

# Retry all failed jobs
php artisan queue:retry all

# Delete a failed job
php artisan queue:forget 5

# Delete all failed jobs
php artisan queue:flush

Failed jobs are stored in the failed_jobs table:

php
Schema::create('failed_jobs', function (Blueprint $table) {
    $table->id();
    $table->string('uuid')->unique();
    $table->text('connection');
    $table->text('queue');
    $table->longText('payload');
    $table->longText('exception');
    $table->timestamp('failed_at')->useCurrent();
});

Monitoring with Horizon

For Redis queues, Laravel Horizon provides a beautiful dashboard:

bash
composer require laravel/horizon
php artisan horizon:install
php artisan migrate

Access dashboard at /horizon.

Worker Health Checks

php
// In a scheduled command
$schedule->command('queue:monitor redis:default,redis:emails --max=100')
    ->everyMinute();

Alert when queues have too many jobs waiting.

Common Pitfalls

  1. Forgetting queue:restart after deploy — workers cache your application bytecode. A fresh deploy with no restart means workers keep running the old code and new bug fixes never take effect.
  2. Running one worker without --queue priorities — a single queue:work picks jobs in FIFO order from the default queue. A 5-minute import job will block every 100ms notification behind it. Use --queue=high,default,low and dispatch urgent work to high.

Best Practices

  1. Always run workers under Supervisor (or systemd) — if the worker crashes, the process manager respawns it. A worker running unattended via nohup is a production incident waiting to happen.
  2. Set --max-time=3600 and --memory=128 — workers accumulate memory over long runs (model hydration, event listeners). Letting Supervisor recycle a worker every hour keeps the footprint bounded.

Summary

  • queue:work is the canonical worker command — run it under Supervisor in production.
  • Use --queue=high,default,low to process urgent jobs first.
  • Run queue:restart after every deploy to load new code.
  • Monitor failed jobs with queue:failed and resurrect them with queue:retry.
✓ Completed