pip install scrapy error with python 3 on Ubuntu 16.04

October 6, 2016 5.6k views
Python Ubuntu 16.04
rufio
By:
rufio

When starting this tutorial on using scrapy to crawl a site on a fresh Ubuntu 16.04 droplet with only python installed via apt install python python-pip - I ran the scrapy install command:

pip install scrapy

and got the following errors:

Failed building wheel for cryptography

and

Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-uMrPxH/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-w7GcS0-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-uMrPxH/cryptography/

The Solution

I was missing some vital cryptography libs that scrapy depends on, to install them I ran:

sudo apt-get install build-essential libssl-dev libffi-dev python-dev
pip install cryptography

once that executed I could install scrapy with a pip install scrapy and get on with the excellent tutorial!

3 Answers

No need to answer - just posting this for others who have the same issue.

Thanks for your comment, I added

  • sudo apt-get install build-essential libssl-dev libffi-dev python-dev

to the local Ubuntu 16.04 set-up tutorial in case others run into this issue.

This tutorial will get you up and running with a local Python 3 programming environment in Ubuntu 16.04. It will guide you through installing Python 3 on your local Linux machine and setting up a programming environment via the command line. Though it will explicitly cover the installation procedures for Ubuntu 16.04, the general principles apply to any other Debian-based distributions.

For some reasons, it doesn't work on ubuntu 14. Any more solutions?

Exact error is

error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------

Command "/home/work/Workspace/venv/tmtext/bin/python2.7 -u -c "import setuptools, tokenize;file='/tmp/pip-install-12TB2u/cryptography/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-record-dfm5Vh/install-record.txt --single-version-externally-managed --compile --install-headers /home/work/Workspace/venv/tmtext/include/site/python2.7/cryptography" failed with error code 1 in /tmp/pip-install-12TB2u/cryptography/

Have another answer? Share your knowledge.