Question

How to setup Laravel queue for app

Posted October 27, 2020 5.7k views
LaravelDigitalOcean App Platform

Hi,

I am looking for a way to get our Laravel queue working on the app platform.
For this we’ve created a Redis DB and want to run the php artisan queue:work command to process the jobs.

For this I have a few questions.

  • Do we need to set the queue:work command as a Run Command in the app?
  • Can it be done in the Web Service App or does a Worker Component has any benefits for this?
  • How can the process be monitored and kept running. Do we still need to configure Supervise for it (as Laravel documentation suggests) and if so how can this be done in an app service?

Would be great to hear your suggestions and experiences with this.

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
Submit an Answer
1 answer

Hi 👋🏼

The recommended way to do this is to run the queue runner as a worker component. You would set the run command to php artisan queue:work like you said.

Running it as a worker component removes the need for supervisor because then App Platform will manage it for you and ensure that it is always running, restarted on crashes, etc. You will also be able to see analytics for it separate from your actual Laravel app.

The other option is to run it alongside your Service component, but you will need to replace the run command with one that starts both the webserver (heroku-php-apache2 public/) and the queue runner (php artisan queue:work) at the same time and ensures that both are kept running and restarted if they were to crash.

That’s what supervisor does however it might be a little complicated to set up on App Platform as it is not available in the run environment so you will need to work around that as a non-root user in the run command. If you go this route I would recommend searching for a different process manager that is easier to run as non-root. I’m sure there’s a composer or npm CLI that does that out there.

So, in short, it’s much easier to run it as an App Platform Worker :)

  • Thanks for your answer, that’s all I need to know about the worker!

    But now I am doubting whether to use Redis for the queue due to its job persistence.
    By default it stores the jobs in memory so if Redis crashes the jobs are gone. As far as I can find it’s not possible to save the data to disk (AOF persistence) at DO.

    Can you tell me what’s the best practice for keeping the job data safe?

    • We use database queues for mission critical operations, and use Redis for caching and stuff that just has to happen fast. So I just spawn several queue workers on several connections.

      Raises the question @kamaln7 though how to do that on App Platform without buying a separate worker for each connection?

      • It looks like this thread has been dormant a month and I’m curious if ya’ll came with a solution regarding

        how to do that on App Platform without buying a separate worker for each connection?

        • Hi @VoordeMensen @schoeyfield ,

          The only solution we could find, was to create a custom Dockerfile for php/laravel that includes the supervisor. Like this:

          FROM php:7.4-fpm
          
          COPY . /var/www
          
          RUN apt-get update && apt-get install -y \
              git \
              curl \
              libpng-dev \
              libonig-dev \
              libxml2-dev \
              libzip-dev \
              zip \
              unzip \
              supervisor
          
          # composer
          RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
          
          RUN cp /var/www/docker/workers/php.ini /usr/local/etc/php/php.ini
          
          # extensions
          RUN docker-php-ext-configure pdo_mysql
          RUN docker-php-ext-configure mysqli
          RUN docker-php-ext-install pdo_mysql
          RUN docker-php-ext-install zip mbstring
          
          RUN cp /var/www/docker/workers/laravel-worker.conf /etc/supervisor/conf.d/laravel-worker.conf
          RUN cp /var/www/docker/workers/supervisord.conf /etc/supervisord.conf
          
          WORKDIR /var/www
          
          RUN composer install
          

          And then have a folder for the supervisord.conf, <your_worker_for_the_queue>.conf and php.ini in your repo at docker/workers/....

          To start the supervisor on the worker, set-up this run command:

          supervisord -n -c /etc/supervisord.conf
          
          • Hello, @bartf

            Thank you for your solution.

            Can you, please, help with more info about the files from docker/workers?

            I assume the <your_worker_for_the_queue>.conf is a file constructed as specified in the Laravel Queue docs.

            But I am not sure where I get the supervisord.conf and php.ini from as a safe, minimal example for what should be there.

            Thank you again,