Unable to execute a python script via crontab, but can execute it manually. What gives?

Posted March 2, 2016 90.7k views
UbuntuGetting StartedPythonSystem Tools

I’ve written a python script for python 2.7 - its main purpose is to search the directory that it is in for any CSVs that begin with “contact” and end in “.CSV” - it then parses those CSVs and makes some API calls to send the data elsewhere.

It does this successfully using a few modules; namely glob2, requests and CSVmapper.

I’ve installed the dependancies either via pip or easy install successfully - this is proven by the fact that I can execute the script manually from with in the scripts folder on Digital Ocean by typing python

Initially, the cron job executed the script, but came back with an error which alerted me to the fact that blob or glob weren’t installed by default, which prompted me to install glob2.

When I fixed this, nothing happened. No logs. No execution. No expected results.

I even tried executing the cron command on the command line:

/usr/bin/python /var/www/client_folder/ &> /var/www/client_folder/log.txt

This printed the expected output to the log file with no error.

In summary: My script works without error when manually executed, but for some reason the cron job is either not executing it, or finding it, or something.

I am accessing the crontab via crontab -e, and this was my original command (i’ve set it for every minute just to test it out) - i’ve also confirmed that /usr/bin/python is indeed where python lives:

* * * * * /usr/bin/python /var/www/client_folder/ &> /var/www/client_folder/log.txt

As an addendum, the python script itself has a chmod of 755 and the csv it’s looking at has a chmod of 644. I am logged in as root, and all of the files regarding this issue were created under root.

I have also tried adding root in the cron command itself, to no avail.

  • Ended up getting it to work - turned out to be an environmental variable issue - Crontab couldn’t see things from the script’s perspective I guess. Came across the solution here: - basically just add a simple cd /var/www/your_script_directory - this probably isn’t best practice, but for a quick fix it does the job; here is a sample of my current command:

    * * * * * cd /var/www/clientfolder/ && /usr/bin/python /var/www/clientfolder/ >> /var/www/clientfolder/your_python_script.log
  • Glad to hear it Mark - adding your fix to be the answer for this question as well

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

Submit an Answer
5 answers

Perfectly adding the folder location using “‘cd”’ really helped me to solve the issue on Mac

@markorapaic Thanks. It helped me as well. March 2, 2016 answer still helping in 2019. Thank You.

From Markorapaic: Add a simple cd /var/www/yourscriptdirectory - this probably isn’t best practice, but for a quick fix it does the job; here is a sample of my current command:

  • * * * * cd /var/www/clientfolder/ && /usr/bin/python /var/www/clientfolder/yourpythonscript.

I had the same problem as outlined by @markorapaic and followed his instructions in the follow-up comment. Cron working perfectly now. Thanks Mark!

  • I have a python script for pulling the data from SQS URL and i tried the same thing as suggested above. But above suggestion isn’t working for me.
    Also, when i execute that script individually it gets executed.

    Secondly, when i am doing ps -ef | grep ’<myscriptname>’

    I am getting ? instead of pts/0 or pts/1 or pts/2

    please help

Hello, I have the same problem, cron is not recognizing the jupyter command, i tried:

cd /home/pakin/anaconda3/bin && jupyter nbconvert --execute --to html /home/pakin/Documents/desarrolloWeb/PakinV3o2/chispazo.ipynb

but it does not work. Do you have any suggestion? Greetings.