Does the way you write cronjob affect the performance?

May 3, 2017 4.5k views
Nginx PHP MySQL Linux Basics Linux Commands Ubuntu 16.04
koushik355
By:
koushik355

Hi guys, I am not so good in understanding the depth of cronjob. So, I have a question regarding cronjob.
Please look at the below cronjobs and kindly tell me if they would affect the performance or the performance will be same in both cases
1st case

* * * * * executer /path/to/job1
* * * * * executer /path/to/job2
* * * * * executer /path/to/job3
* * * * * executer /path/to/job4
* * * * * (sleep 30; executer /path/to/job1)
* * * * * (sleep 30; executer /path/to/job2)
* * * * * (sleep 30; executer /path/to/job3)
* * * * * (sleep 30; executer /path/to/job4)

2nd case

  • * * * * executer /path/to/job1
  • * * * * (sleep 30; executer /path/to/job1)
  • * * * * executer /path/to/job2
  • * * * * (sleep 30; executer /path/to/job2)
  • * * * * executer /path/to/job3
  • * * * * (sleep 30; executer /path/to/job3)
  • * * * * executer /path/to/job3
  • * * * * (sleep 30; executer /path/to/job3)

Will the performance on both be same or they can be different?
Thank you very much in advance for your help guys.

Log In to Comment
2 Answers
jtittle1 May 3, 2017

@koushik355

It really depends on what the cron jobs are doing.

Using sleep to delay the start of the next cron job will allow resource usage to wind down if you're doing something that's resource intensive, which may provide better performance if the cron job will be using quite a bit of CPU.

For example, if one of the cron jobs is copying a large number of files from location X to location Y, it may be beneficial to delay the next cron job from running for a short period of time. Likewise, if you're doing something really intensive that would cause CPU usage to rise, I/O wait to rise, etc -- then yes, delaying the execution of the next cron job would be wise.

That being said, you may be better off simply setting up the delay in the cron job itself as opposed to in the command(s) being executed by the cron job.

Using:

* * * * *

... is equal to run every minute. You could setup more of a waterfall effect with the cron jobs and use the built in functionality of cron to simply manage the delay.

For example, you could run one every two minutes, one every four minutes, one every eight minutes, etc -- you'd simply need to modify the first * and add a /# to it.

i.e. to run every second minute, you could use:

*/2 * * * * /path/to/script

The above will run 30x every 60 minutes (1 hour) -- or every second minute.

For every 5 minutes, you could use:

*/5 * * * * /path/to/script

That'll run 12x every 60 minutes (1 hour) -- or every fifth minute.

Reply
koushik355 May 3, 2017

Hi Jonathan, thanks a lot for taking time to write the answer.
Much appreciated, brother.
Here is what I am trying to achieve. I want to run the same script more than within a minute.
Running it 3 times would be better.
And yes, the process can be intensive because it will be sending emails to a bunch of users using postfix.
What you think I should do to achieve that?
Thank you very much for your help.

Reply
  • jtittle1 May 3, 2017

    @koushik355

    Since cron jobs are limited to a minimum run interval of 60 seconds / 1 minute, it wouldn't be possible to further reduce the frequency (i.e. it wouldn't be possible to do every 20 seconds), so the only work around, using cron, would be to do what you're already doing.

    A better alternative may be to use an actual programming language or create a service that will handle the execution (via systemctl or init). You'd have a bit more flexibility there, though it'd be something you'd have to design as I can't think of an OOB solution that would just work off the top of my head.

    • koushik355 May 4, 2017

      Hi Jonathan, thanks for the reply, brother.
      I am using Php. So, running the cron in any of the either way won't affect the performance and it's both the same. Right? 

      * * * * * executer /path/to/job1
* * * * * executer /path/to/job2
* * * * * (sleep 30; executer /path/to/job1)
* * * * * (sleep 30; executer /path/to/job2)

      * * * * executer /path/to/job1
* * * * (sleep 30; executer /path/to/job1)
* * * * executer /path/to/job2
* * * * (sleep 30; executer /path/to/job2)
      • jtittle1 May 4, 2017

        @koushik355

        The cron jobs themselves aren't going to be an issue, it's the scripts. It all boils down to what they do.

        If you're sending 10k e-mails per run, for example, and you're trying to do runs every thirty seconds, you may very well run in to issues and that'd be beyond the scope of the cron system. You'd need to find a way to better optimize the scripts being ran to account for the potential delays and resource usage.

        Without knowing what the script is doing or seeing the script(s), it's hard to really say how performance would be impacted, though just running the cron jobs themselves every minute isn't going to be an issue nor is delaying them.

        Ultimate, the scripts will be the cause of the issues, if there are any along the way.

        • koushik355 May 4, 2017

          That's what it would do in the future though. It will send a bunch of emails per second.
          Seems like optimizing the script is the best bet here.
          You mentioned about running into issues. Can you please tell what kinda issues?

          • jtittle1 May 4, 2017

            @koushik355

            When it comes to PHP (or really any language) and sending large quantities of e-mail, it's often best to use some sort of queue instead of sending everything at once.

            For example (this is very basic), if you're wanting to send 10,000 e-mails, it'd be best to pass those through to a queue class, or function, that will handle a number of e-mails per X seconds or Y minutes.

            Using a queue, you'd send 500 of the 10,000 at once, wait a specified period of time, send another 500, etc.

            Using such a class or function (depending on your style of programming) may allow you to further reduce the load that would normally come with sending a large quantity of mail all at once.

            Beyond that, I would consider looking in to MailGun or SendGrid as those are services that are designed to handle large quantities of mail at once and will do a similar method of queueing on their end. They're free for a number of e-mails per month, though if you're planning to send 30-100k e-mails, they would cost a bit.

Have another answer? Share your knowledge.

Related Questions