Question

403 status code for request sent from DO droplet

I run the same short code on my local desktop and on DigitalOcean droplet, in both cases with python3.8:

import requests page = requests.get(“https://bitinfocharts.com/top-100-richest-bitcoin-addresses.html”) page.status_code

On my local computer I got code 200 - OK, but when I run it from droplet I got code 403 - forbidden. I both cases pip3 shows the same version of requests module: 2.25.1.

What can be the reason for this difference?

Submit an answer

This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

Sign In or Sign Up to Answer

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

Bobby Iliev • December 28, 2021

Hello,

It looks like that the website that you are trying to scrape uses Cloudflare protection meaning that Cloudflare checks your user agent/browser and denies access to the website.

What you could try doing is to use chromedriver rather than requesting the website directly. That way Cloudflare should be able to detect an actual browser and let your requests go through.

Hope that this helps. Regards, Bobby