pip install scrapy error with python 3 on Ubuntu 16.04

October 6, 2016 145 views
Python Ubuntu 16.04

When starting this tutorial on using scrapy to crawl a site on a fresh Ubuntu 16.04 droplet with only python installed via apt install python python-pip - I ran the scrapy install command:

pip install scrapy

and got the following errors:

Failed building wheel for cryptography


Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-uMrPxH/cryptography/';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-w7GcS0-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-uMrPxH/cryptography/

The Solution

I was missing some vital cryptography libs that scrapy depends on, to install them I ran:

sudo apt-get install build-essential libssl-dev libffi-dev python-dev
pip install cryptography

once that executed I could install scrapy with a pip install scrapy and get on with the excellent tutorial!

2 Answers

No need to answer - just posting this for others who have the same issue.

Thanks for your comment, I added

  • sudo apt-get install build-essential libssl-dev libffi-dev python-dev

to the local Ubuntu 16.04 set-up tutorial in case others run into this issue.

This tutorial will get you up and running with a local Python 3 programming environment in either Ubuntu 16.04 or Debian 8. Python is a versatile programming language that can be used for many different projects. First published in 1991 with a name inspired by the British comedy group Monty Python, the development team wanted to make Python a language that was fun to use.
Have another answer? Share your knowledge.