Question

403 status code for request sent from DO droplet

Posted December 26, 2021 83 views
PythonHTML

I run the same short code on my local desktop and on DigitalOcean droplet, in both cases with python3.8:

import requests
page = requests.get(“https://bitinfocharts.com/top-100-richest-bitcoin-addresses.html”)
page.status_code

On my local computer I got code 200 - OK, but when I run it from droplet I got code 403 - forbidden. I both cases pip3 shows the same version of requests module: 2.25.1.

What can be the reason for this difference?

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
Submit an Answer
1 answer

Hello,

It looks like that the website that you are trying to scrape uses Cloudflare protection meaning that Cloudflare checks your user agent/browser and denies access to the website.

What you could try doing is to use chromedriver rather than requesting the website directly. That way Cloudflare should be able to detect an actual browser and let your requests go through.

Hope that this helps.
Regards,
Bobby