Question
403 status code for request sent from DO droplet
I run the same short code on my local desktop and on DigitalOcean droplet, in both cases with python3.8:
import requests page = requests.get(“https://bitinfocharts.com/top-100-richest-bitcoin-addresses.html”) page.status_code
On my local computer I got code 200 - OK, but when I run it from droplet I got code 403 - forbidden. I both cases pip3 shows the same version of requests module: 2.25.1.
What can be the reason for this difference?
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.
Want to learn more? Join the DigitalOcean Community!
Join our DigitalOcean community of over a million developers for free! Get help and share knowledge in Q&A, subscribe to topics of interest, and get courses and tools that will help you grow as a developer and scale your project or business.
Hello,
It looks like that the website that you are trying to scrape uses Cloudflare protection meaning that Cloudflare checks your user agent/browser and denies access to the website.
What you could try doing is to use
chromedriverrather than requesting the website directly. That way Cloudflare should be able to detect an actual browser and let your requests go through.Hope that this helps. Regards, Bobby