Trouble installing NumPy in virtualenv on Ubuntu 14.04

May 5, 2014 7.6k views
Yesterday was my first day using DigitalOcean. I set up Ubuntu v. 14.04, cloned the repository my colleague and I are working on, and tried to install all the required Python modules and non-Python dependencies, but there were problems with NumPy that I could not resolve. I have not used Ubuntu in some months, but I normally use pip for almost all Python installations, within a virtualenv virtual environment, and install the many dependencies using apt-get. I remember that NumPy often presents problems. This time it seemed impossible to make NumPy compile. Installing with root privileges succeeded, but then NumPy was not available within the virtualenv environment. I have now installed the Anaconda distribution, instead, so that I can move forward, but I'd be glad to hear of suggestions for effective NumPy installation using pip within a virtual environment.
2 Answers
Without seeing the error, it's hard to say exactly what went wrong. Though one thing to remember is that installing numpy with pip requires compiling
C code, so you need gcc and Python's developmental headers around:

apt-get install python-dev build-essential

Also, if you want your virtualenv to be able to access packages installed through apt, create it using:

virtualenv --system-site-packages your_enev
  • (My droplet: 1GB Ram 30GB SSD Disk New York 3 Ubuntu 14.04 x64).

    I am having the same issue. When I try to install scipy, I get a LOT of output. I'm not sure where to post it because it's probably a few thousand lines of output, but it begins and ends in this (I'm not sure what's relevant but here's a gist of my entire console output):

    Collecting scipy (from seaborn==0.5.1->-r requirements.txt (line 25))
      Using cached scipy-0.15.1.tar.gz
    Building wheels for collected packages: scipy
      Running setup.py bdist_wheel for scipy
    ^@^@^@^@^@^@  Complete output from command /home/matt/.virtualenvs/airbnb/bin/python3 -c "import setuptools;__file__='/tmp/pip-build-556p2ube/scipy/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /tmp/tmpp7nc_u95pip-wheel-:
      lapack_opt_info:
      openblas_lapack_info:
        libraries openblas not found in ['/home/matt/.virtualenvs/airbnb/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
        NOT AVAILABLE
    
      lapack_mkl_info:
      mkl_info:
        libraries mkl,vml,guide not found in ['/home/matt/.virtualenvs/airbnb/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
        NOT AVAILABLE
    
        NOT AVAILABLE
    
      atlas_3_10_threads_info:
      Setting PTATLAS=ATLAS
        libraries tatlas,tatlas not found in /home/matt/.virtualenvs/airbnb/lib
        libraries lapack_atlas not found in /home/matt/.virtualenvs/airbnb/lib
        libraries tatlas,tatlas not found in /usr/local/lib
        libraries lapack_atlas not found in /usr/local/lib
        libraries tatlas,tatlas not found in /usr/lib/atlas-base
        libraries tatlas,tatlas not found in /usr/lib
        libraries tatlas,tatlas not found in /usr/lib/x86_64-linux-gnu
        libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
      <class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
        NOT AVAILABLE
    
      atlas_3_10_info:
        libraries satlas,satlas not found in /home/matt/.virtualenvs/airbnb/lib
        libraries lapack_atlas not found in /home/matt/.virtualenvs/airbnb/lib
        libraries satlas,satlas not found in /usr/local/lib
        libraries lapack_atlas not found in /usr/local/lib
        libraries satlas,satlas not found in /usr/lib/atlas-base
        libraries satlas,satlas not found in /usr/lib
        libraries satlas,satlas not found in /usr/lib/x86_64-linux-gnu
        libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
      <class 'numpy.distutils.system_info.atlas_3_10_info'>
        NOT AVAILABLE
    
      atlas_threads_info:
      Setting PTATLAS=ATLAS
        libraries ptf77blas,ptcblas,atlas not found in /home/matt/.virtualenvs/airbnb/lib
        libraries lapack_atlas not found in /home/matt/.virtualenvs/airbnb/lib
        libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
        libraries lapack_atlas not found in /usr/local/lib
        libraries ptf77blas,ptcblas,atlas not found in /usr/lib/atlas-base
        libraries ptf77blas,ptcblas,atlas not found in /usr/lib
        libraries ptf77blas,ptcblas,atlas not found in /usr/lib/x86_64-linux-gnu
        libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
      <class 'numpy.distutils.system_info.atlas_threads_info'>
        NOT AVAILABLE
    
      atlas_info:
        libraries f77blas,cblas,atlas not found in /home/matt/.virtualenvs/airbnb/lib
        libraries lapack_atlas not found in /home/matt/.virtualenvs/airbnb/lib
        libraries f77blas,cblas,atlas not found in /usr/local/lib
        libraries lapack_atlas not found in /usr/local/lib
      <class 'numpy.distutils.system_info.atlas_info'>
      customize Gnu95FCompiler
      Found executable /usr/bin/gfortran
      customize Gnu95FCompiler
      customize Gnu95FCompiler using config
      compiling '_configtest.c':
    ...
        customize Gnu95FCompiler
        customize Gnu95FCompiler
        customize Gnu95FCompiler using build_ext
        building 'scipy.sparse._sparsetools' extension
        compiling C++ sources
        C compiler: x86_64-linux-gnu-g++ -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC
    
        compile options: '-D__STDC_FORMAT_MACROS=1 -Iscipy/sparse/sparsetools -I/home/matt/.virtualenvs/airbnb/lib/python3.4/site-packages/numpy/core/include -I/usr/include/python3.4m -I/home/matt/.virtualenvs/airbnb/include/python3.4m -c'
        x86_64-linux-gnu-g++: scipy/sparse/sparsetools/bsr.cxx
        In file included from /home/matt/.virtualenvs/airbnb/lib/python3.4/site-packages/numpy/core/include/numpy/ndarraytypes.h:1804:0,
                         from /home/matt/.virtualenvs/airbnb/lib/python3.4/site-packages/numpy/core/include/numpy/ndarrayobject.h:17,
                         from scipy/sparse/sparsetools/sparsetools.h:5,
                         from scipy/sparse/sparsetools/bsr.cxx:4:
        /home/matt/.virtualenvs/airbnb/lib/python3.4/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
         #warning "Using deprecated NumPy API, disable it by " \
          ^
        virtual memory exhausted: Cannot allocate memory
        Running from scipy source directory.
        error: Command "x86_64-linux-gnu-g++ -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -D__STDC_FORMAT_MACROS=1 -Iscipy/sparse/sparsetools -I/home/matt/.virtualenvs/airbnb/lib/python3.4/site-packages/numpy/core/include -I/usr/include/python3.4m -I/home/matt/.virtualenvs/airbnb/include/python3.4m -c scipy/sparse/sparsetools/bsr.cxx -o build/temp.linux-x86_64-3.4/scipy/sparse/sparsetools/bsr.o" failed with exit status 1
        In file included from /home/matt/.virtualenvs/airbnb/lib/python3.4/site-packages/numpy/core/include/numpy/ndarraytypes.h:1804:0,
                         from /home/matt/.virtualenvs/airbnb/lib/python3.4/site-packages/numpy/core/include/numpy/ndarrayobject.h:17,
                         from scipy/sparse/sparsetools/sparsetools.h:5,
                         from scipy/sparse/sparsetools/bsr.cxx:4:
        /home/matt/.virtualenvs/airbnb/lib/python3.4/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
         #warning "Using deprecated NumPy API, disable it by " \
          ^
        virtual memory exhausted: Cannot allocate memory
    
        ----------------------------------------
    Command "/home/matt/.virtualenvs/airbnb/bin/python3 -c "import setuptools, tokenize;__file__='/tmp/pip-build-556p2ube/scipy/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-sjzugg9m-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/matt/.virtualenvs/airbnb/include/site/python3.4/scipy" failed with error code 1 in /tmp/pip-build-556p2ube/scipy
    
  • thanks a lot , that solves my problems just right.
    After installing the c packages , i have the right result

Have another answer? Share your knowledge.