Flask program giving module not found

February 10, 2018 4.2k views
Python Deployment CentOS

Im building a python web application with flask and uWSGI following this lovely guide https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-uwsgi-and-nginx-on-centos-7 and it worked marvels, the basic hello does indeed appear in the website hen loaded. I have installed every single module and dependency in the project file. Im trying now to build on the working script and I now my init.py file looks like this:

from flask import Flask
import pylab as pl
import numpy as np
import pandas as pd

from sklearn import svm
from sklearn import tree
import matplotlib.pyplot as plt
from sklearn import linear_model
from sklearn.pipeline import Pipeline
from sklearn.metrics import confusion_matrix
from sklearn.naive_bayes import MultinomialNB
from sklearn.linear_model import SGDClassifier
from mlxtend.plotting import plot_decision_regions
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.feature_extraction.text import TfidfTransformer

app = Flask(__name__)

@app.route("/")


def hello():
    data = pd.read_csv('test1.csv', error_bad_lines=False, delimiter=',')
        numpy_array = data.as_matrix()
        #print numpy_array

        #text in column 1, classifier in column 2.
        X = numpy_array[:,0]
        Y = numpy_array[:,1]
        Y=Y.astype(np.str)

        #divide the test set and set the variable to their correct label/text
        X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.4, random_state=42)

        #MultinomialNB
        text_clf = Pipeline([('vect', CountVectorizer(stop_words='english')), ('tfidf', TfidfTransformer()),('clf', MultinomialNB()),])

        text_clf = text_clf.fit(X_train.astype('U'),Y_train.astype('U'))
        predicted = text_clf.predict(X_test)
        # print the actual accuracy
        print "MNB accuracy: ", np.mean(predicted == Y_test)

        #make the confusion matrix
        y_actu = pd.Series(Y_test, name='Actual')
        y_pred = pd.Series(predicted, name='Predicted')
        df_confusion = pd.crosstab(y_actu, y_pred)
        print df_confusion

        print"-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------$

       #SVM
    vect = CountVectorizer(min_df=0., max_df=1.0)
       X = vect.fit_transform(X_train.astype('U'))
       min_frequency = 22

       text_clf_svm = Pipeline([('vect', CountVectorizer(min_df=min_frequency, stop_words='english')), ('tfidf', TfidfTransformer()),('clf-svm', SGDClassifier(loss='hinge', penalty='l2', alpha=1e-03, n_iter=1000, random_state=21))])

       text_clf_svm = text_clf_svm.fit(X_train.astype('U'),Y_train.astype('U'))
       predicted_svm = text_clf_svm.predict(X_test)
       # print the actual accuracy
       print "svm accuracy: ", np.mean(predicted_svm == Y_test)

       #make the confusion matrix
       y_actu = pd.Series(Y_test, name='Actual')
       y_pred = pd.Series(predicted_svm, name='Predicted')
       df_confusion = pd.crosstab(y_actu, y_pred)

       print df_confusion


if __name__ == "__main__":
   app.run()

All is good with this as far as im concerned made sure to install all dependencies and module sin the folder from which im running the code. but when I run it I get the following error

[root@python-political-bias-app fyp]# semodule -i mynginx.pp
[root@python-political-bias-app fyp]# env/bin/uwsgi --socket 127.0.0.1:8080 -w WSGI:app &
[1] 1710
[root@python-political-bias-app fyp]# *** Starting uWSGI 2.0.15 (64bit) on [Wed Feb  7 01:16:21 2018] ***
compiled with version: 4.8.5 20150623 (Red Hat 4.8.5-16) on 06 February 2018 20:03:13
os: Linux-3.10.0-693.17.1.el7.x86_64 #1 SMP Thu Jan 25 20:13:58 UTC 2018
nodename: python-political-bias-app
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 1
current working directory: /root/fyp
detected binary path: /root/fyp/env/bin/uwsgi
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) *** 
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 3807
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address 127.0.0.1:8080 fd 3
Python version: 2.7.5 (default, Aug  4 2017, 00:39:18)  [GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x74bba0
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 72768 bytes (71 KB) for 1 cores
*** Operational MODE: single process ***
Traceback (most recent call last):
  File "./WSGI.py", line 1, in <module>
    from app import app
  File "./app/__init__.py", line 2, in <module>
    import pylab as pl
ImportError: No module named pylab
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***

Im pretty lost as to why, any pointer would really help out, the code itself runs perfectly well inside locally, so Im not sure whats going on .

2 comments
  • The key part of the error message is:

      File "./app/__init__.py", line 2, in <module>
        import pylab as pl
    ImportError: No module named pylab
    

    Python can't find the pylab module, which means you may not have installed / properly installed the matplotlib/numpy/scipy combo of modules.

    Starting from a brand new CentOS instance, the process would look like:

    1. Get all the python/pip stuff required to install and build matplotlib/numpy/scipy

      yum install -y epel-release
      yum install -y python-pip
      yum install -y gcc gcc-c++ python-argparse python-devel freetype-devel libpng-devel python-tools
      
    2. Pip install matplotlib/numpy/scipy:

      pip install numpy scipy matplotlib
      

    Once all that succeeds, try running your app again, see what happens!

  • I have installed all the dependencies correctly and it still returns the error, I belive that the program is unable to find the module cause its not in its path, how would I go about modifying the path to find the modules?

2 Answers

This question was answered by @aha:

The key part of the error message is:

 File "./app/__init__.py", line 2, in <module>
   import pylab as pl
ImportError: No module named pylab

Python can't find the pylab module, which means you may not have installed / properly installed the matplotlib/numpy/scipy combo of modules.

Starting from a brand new CentOS instance, the process would look like:

  1. Get all the python/pip stuff required to install and build matplotlib/numpy/scipy

    yum install -y epel-release
    yum install -y python-pip
    yum install -y gcc gcc-c++ python-argparse python-devel freetype-devel libpng-devel python-tools
    
  2. Pip install matplotlib/numpy/scipy:

    pip install numpy scipy matplotlib
    

Once all that succeeds, try running your app again, see what happens!

View the original comment

  • In addition to what @aha has already suggested, as you are running the app from a virtualenv, you must make sure that the dependencies are installed inside of it, not system-wide. Remember to activate it before running the pip install like so:

    • source env/bin/activate

    If you really need to modify the Python path when launching the app, uWSGI supports a --pythonpath run time argument.

Have another answer? Share your knowledge.