Unable to execute a python script via crontab, but can execute it manually. What gives?

March 2, 2016 55.8k views
System Tools Ubuntu Getting Started Python

I’ve written a python script for python 2.7 - its main purpose is to search the directory that it is in for any CSVs that begin with “contact” and end in “.CSV” - it then parses those CSVs and makes some API calls to send the data elsewhere.

It does this successfully using a few modules; namely glob2, requests and CSVmapper.

I’ve installed the dependancies either via pip or easy install successfully - this is proven by the fact that I can execute the script manually from with in the scripts folder on Digital Ocean by typing python my_python_script.py.

Initially, the cron job executed the script, but came back with an error which alerted me to the fact that blob or glob weren’t installed by default, which prompted me to install glob2.

When I fixed this, nothing happened. No logs. No execution. No expected results.

I even tried executing the cron command on the command line:

/usr/bin/python /var/www/client_folder/my_python_script.py &> /var/www/client_folder/log.txt

This printed the expected output to the log file with no error.

In summary: My script works without error when manually executed, but for some reason the cron job is either not executing it, or finding it, or something.

I am accessing the crontab via crontab -e, and this was my original command (i’ve set it for every minute just to test it out) - i’ve also confirmed that /usr/bin/python is indeed where python lives:

* * * * * /usr/bin/python /var/www/client_folder/my_python_script.py &> /var/www/client_folder/log.txt

As an addendum, the python script itself has a chmod of 755 and the csv it’s looking at has a chmod of 644. I am logged in as root, and all of the files regarding this issue were created under root.

I have also tried adding root in the cron command itself, to no avail.

  • Ended up getting it to work - turned out to be an environmental variable issue - Crontab couldn’t see things from the script’s perspective I guess. Came across the solution here: http://askubuntu.com/a/24527 - basically just add a simple cd /var/www/your_script_directory - this probably isn’t best practice, but for a quick fix it does the job; here is a sample of my current command:

    * * * * * cd /var/www/clientfolder/ && /usr/bin/python /var/www/clientfolder/your_python_script.py >> /var/www/clientfolder/your_python_script.log
  • Glad to hear it Mark - adding your fix to be the answer for this question as well

4 Answers

From Markorapaic: Add a simple cd /var/www/yourscriptdirectory - this probably isn’t best practice, but for a quick fix it does the job; here is a sample of my current command:

  • * * * * cd /var/www/clientfolder/ && /usr/bin/python /var/www/clientfolder/yourpythonscript.

I had the same problem as outlined by @markorapaic and followed his instructions in the follow-up comment. Cron working perfectly now. Thanks Mark!

Hello, I have the same problem, cron is not recognizing the jupyter command, i tried:

cd /home/pakin/anaconda3/bin && jupyter nbconvert --execute --to html /home/pakin/Documents/desarrolloWeb/PakinV3o2/chispazo.ipynb

but it does not work. Do you have any suggestion? Greetings.

Perfectly adding the folder location using “‘cd”’ really helped me to solve the issue on Mac

Have another answer? Share your knowledge.