Resque

Resque is a Redis-backed library for creating background jobs, placing those jobs on one or more queues, and processing them later.

Features:

• Workers can be distributed between multiple machines
• Includes support for priorities (queues)
• Resilient to memory leaks (forking)
• Expects failure
• Has the ability to track the status of jobs
• Will mark a job as failed, if a forked child running a job does not exit with a status code as 0
• Has built in support for setUp and tearDown methods, called pre and post jobs

Jobs

Queueing

Resque::enqueue('default', 'My_Job', $args);

Defining Job

class My_Job
{
 public function perform()
 {
	 // Work work work
	 echo $this->args['name'];
 }
}

Dequeueing Jobs

This method can be used to conveniently remove a job from a queue.

// Removes job class 'My_Job' of queue 'default' Resque::dequeue('default', ['My_Job']);

Tracking Job Statuses

php-resque has the ability to perform basic status tracking of a queued job. The status information will allow you to check if a job is in the queue, is currently being run, has finished, or has failed.

$token = Resque::enqueue('default', 'My_Job', $args, true); $status = new Resque_Job_Status($token);

Workers

$ QUEUE=file_serve php bin/resque

Priorities $ QUEUE=file_serve,warm_cache bin/resque

Running all queues $ QUEUE='*' bin/resque

Running multiple workers $ COUNT=5 bin/resque

Forking

Similarly to the Ruby versions, supported platforms will immediately fork after picking up a job. The forked child will exit as soon as the job finishes.

The difference with php-resque is that if a forked child does not exit nicely (PHP error or such), php-resque will automatically fail the job.

Signals

Signals also work on supported platforms exactly as in the Ruby version of Resque:

• QUIT - Wait for job to finish processing then exit
• TERM / INT - Immediately kill job then exit
• USR1 - Immediately kill job but don't exit
• USR2 - Pause worker, no new jobs will be processed
• CONT - Resume worker.

---

Worker: PHP process that will run indefinitely, always monitoring for new jobs to execute. Pseudocode

while  (true)  { 
	$jobs = pullData();  # Pull jobs from the queues  
	foreach  ($jobs as $class => $args)  {  # For each jobs found 
		$job =  new $class(); $job->perform($args);  # Execute them  
	} 
	sleep(300);  # Then Sleep for 5 minutes (300 seconds), and retry  
}

Workers have to be started in the CLI. You can’t ceate worker through a browser, because :

• You can’t do background jobs through a browser
• PCNTL extension works only in CLI mode

QUEUE=notification VVERBOSE=1 php resque.php

nohup QUEUE=notification php resque.php & &: Daemon-ify the worker nohup: enables the commnad to keep running even if the user logged out

nohup QUEUE=notification php resque.php >> /path/to/your/logfile.log 2>&1 &

tail -F /path/to/your/logfile.log

Forking

when a Resque worker reserves a job it immediately forks a child process. The child processes the job then exits. When the child has exited successfully, the worker reserves another job and repeats the process. Resque assumes chaos => a job uses too much memory, a job delays

Thanks to Resque’s parent / child architecture, jobs that use too much memory release that memory upon completion. No unwanted growth.

With Resque’s parent / child architecture you can tell the parent to forcefully kill the child then immediately start processing more jobs.

Jobs

A job is a written order to tell the workers to execute a particular task.

Resque::enqueue('default', 'Mail',  array('dest@mail.com', 'hi!', 'this is a test content'));
// queue name, class name, arguments
// -------
class Mail  {  
	public  function perform()  {}
}

Locating the job

All your jobs classes must be discoverable by the workers. There’s more than one way to do that

Using APP_INCLUDE

QUEUE=default APP_INCLUDE=/path/file.php php resque

Track your jobs

All workers activities are logged. You’ll find in the logs:

• When a job is enqueued with Resque::enqueue
• When a worker detects a new job in the queue
• When a worker begins executing a job
• The status of the job execution (success/fail)

Depending on your verbose mode, they come with more or less details.

Bundle

Features:

• Creating a Job, with container access in order to leverage your Symfony services
• Enqueue a Job with parameters on a given queue
• Creating background worker on a given queue
• A UX to monitor your queues, workers and job statuses
• ability to schedule jobs to run at a specific time or after a number of seconds delay
• ability to auto re-queue failed jobs, with back-off strategies

TODOs:

• Log management
• Job status tracking
• Redis configuration
• Localisation
• Tests

Running worker on a queueu

• the default queue : app/console bcc:resque:worker-start default
• the q1 and q2 queue : app/console bcc:resque:worker-start q1,q2

Supervisor

[program:myapp_phpresque_default]
command = /usr/bin/php /home/sites/myapp/prod/current/vendor/bcc/resque-bundle/BCC/ResqueBundle/bin/resque
user = myusername
environment = APP_INCLUDE='/home/sites/myapp/prod/current/vendor/autoload.php',VERBOSE='1',QUEUE='default'
stopsignal=QUIT