By Andy H
When starting this tutorial on using scrapy to crawl a site on a fresh Ubuntu 16.04 droplet with only python installed via apt install python python-pip - I ran the scrapy install command:
pip install scrapy
and got the following errors:
Failed building wheel for cryptography
and
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-uMrPxH/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-w7GcS0-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-uMrPxH/cryptography/
The Solution
I was missing some vital cryptography libs that scrapy depends on, to install them I ran:
sudo apt-get install build-essential libssl-dev libffi-dev python-dev
pip install cryptography
once that executed I could install scrapy with a pip install scrapy and get on with the excellent tutorial!
Become a contributor for community
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
DigitalOcean Documentation
Full documentation for every DigitalOcean product.
Resources for startups and AI-native businesses
The Wave has everything you need to know about building a business, from raising funding to marketing your product.
The developer cloud
Scale up as you grow — whether you're running one virtual machine or ten thousand.
Start building today
From GPU-powered inference and Kubernetes to managed databases and storage, get everything you need to build, scale, and deploy intelligent applications.
