Laravel and Workers, Microservices

Posted: 2015-03-09 00:01:19

We are starting to use and their workers for a lot of the tasks that our apps need to do. For example one app needs to scan websites for images and text and report on them. In our case that is 2 workers, one with the code needed to get the text we want and the other images. Another worker runs behat tests to take screenshots and reports back to the called with the results.

Using has made this whole process easy and scalable. One request can be for say 100 urls and with we can run one worker per url or using the Symfony Process library we can even use a worker to run a multi-threaded processes.

Some of the resources out there like iron`s example are great. And using this library has made it super easy. Below I cover how exactly to set this up. (hopefully this week we will have a Laravel 5 version of it out)

Step 1 Install

Install 4.2 work. (5 might be ready soon)

composer create-project laravel/laravel=4.2 example_worker --prefer-dist

Set your minimum stability in your composer.json

    "config": {
        "preferred-install": "dist"
    "minimum-stability": "dev"

Then pull in the library

composer require iron-io/laraworker

And add this one patch for PHP 5.6 TODO add code snippet


And of course as the notes for Laraworker

php vendor/iron-io/laraworker/LaraWorker.php -i true

As the developer notes this makes a new folder and file

/worker/libs/worker_boot.php and /worker/ExampleLaraWorker.php

Step 2 Configure

We will use the .env to do configuration not the way noted in the laraworker docs so lets install that. Just use this post to set that up.

So after you are done your, as in the Laraworker docs, we need to set the queue config.

Set credentials in app/config/queue.php and set default to iron --> 'default' => 'iron',

So yours will look like

    'default' => getenv('QUEUE_DRIVER'),

    'connections' => array(

        'iron' => array(
            'driver'  => 'iron',
            'host'    => '',
            'token'   => getenv('IRON_TOKEN'),
            'project' => getenv('IRON_PROJECT_ID'),
            'queue'   => 'your-queue-name',
            'encrypt' => true,


Then make your project on Iron and get the Token and Project ID

Step 3 See if Example Worker works

Lets see if the Example works before we move forward.

php artisan ironworker:upload --worker_name=ExampleLaraWorker --exec_worker_file_name=ExampleLaraWorker.php

If it worked you will see


This will upload a worker related queue


Step 4 Make our own worker!

The goal of this worker

  • It will get a JSON object of the info needed to do a job
  • It will do the job by getting the json file from the S3 file system where it lives (it could live in a db or other location)
  • Using the JSON object's callback it will send back the results to the caller

That is it.

This example will be used in real life to later on parse say 100 urls for already created json render tree objects of the urls data including images and text. This job only cares about the text. Cause the job is fairly easy I will be sending to each worker 5 urls to process.

Copy the worker in /workers folder to the new Worker name

Due to bad naming abilities I am calling this RenderTreeTextGrepper.php

So now my worker folder has


But I do not want that class to have all my code so I will start to build out a namespace for all of this and the 2 classes I want to manage ALL of this work.

Class 1 @fire

So the worker will fire the class I have to handle all of this.

    "autoload": {
        "classmap": [
      "psr-4": {
        "AlfredNutileInc\\RenderTreeTextGrepperWorker\\": "app/"


composer dump

Then in app/RenderTreeTextGrepperWorker folder I have


/projects/example_worker/app/RenderTreeTextGrepperWorker/RenderTreeGrepperHandler.php is the class to handle the incoming request and process it.

Class 2 Event Listener

Then I register the event listener with the app/config/app.php to make it easier to handle the results of the output. You can do all of this in class 1 as well.


And that is it.

What is it?

So we are going to upload and run this and here is what will happen. NO WAIT!

First lets make a test so we can see locally if all the logic is there.

Local Test

Just a quick test to see if the handler will handle things and pass results


class RenderTreeTextTest extends \TestCase {

     * @test
    public function should_populate_results()
        $handle = new \AlfredNutileInc\RenderTreeTextGrepperWorker\RenderTreeGrepperHandler();
        $payload = new \AlfredNutileInc\RenderTreeTextGrepperWorker\RenderTreeTextDTO(
            ['foo', 'bar', 'baz'],
            ['text1', 'text2'],
                'caller'     => '',
                'params'     => ['foo', 'bar']
        $results = $handle->handle($payload);


Running this

phpunit --filter=should_populate_results

Produces this

 class AlfredNutileInc\RenderTreeTextGrepperWorker\RenderTreeTextDTO#334 (6) {
    public $uuid =>
    string(7) "foo-bar"
    public $urls =>
    array(3) {
      [0] =>
      string(3) "foo"
      [1] =>
      string(3) "bar"
      [2] =>
      string(3) "baz"
    public $text =>
    array(2) {
      [0] =>
      string(5) "text1"
      [1] =>
      string(5) "text2"
    public $callback =>
    array(2) {
      'caller' =>
      string(41) ""
      'params' =>
      array(2) {
    public $results =>
    array(1) {
      [0] =>
      string(21) "Listener is listening"
    public $status =>

Of course I need to go into more testing for the two classes to see how they react to different data going in but just to see that there are not obvious issues before I upload the worker.

Upload the worker we just made

php artisan ironworker:upload --worker_name=RenderTreeTextGrepper --exec_worker_file_name=RenderTreeTextGrepper.php

And then we see on

new worker

Then we run it

php artisan ironworker:run --queue_name=RenderTreeTextGrepper

Before that though I updated app/commands/RunWorker.php:26 to make a better payload

    public function fire()
        $queue_name = $this->option('queue_name');
        $payload = "This is Hello World payload :)";

        if($queue_name == 'RenderTreeTextGrepper')
            $payload = new \AlfredNutileInc\RenderTreeTextGrepperWorker\RenderTreeTextDTO(
                ['foo', 'bar', 'baz'],
                ['text1', 'text2'],
                    'caller'     => '',
                    'params'     => ['foo', 'bar']

We then see the Task


And the example log output


Guzzle and the Callback

How to format the callback?

Let's require guzzle

composer require guzzlehttp/guzzle

At this point we have a working example. The queue takes the json and the worker processes it!


Thanks to the library and it really is that simple.