Report this

What is the reason for this report?

Issues with ip address causing 403 error on web scraping tool

Posted on June 21, 2016

Hi there

So i have a nokogiri web scrape on my website. It works lovely and fine on my local dev machine but when putting it live on here or heroku. I get a 403 forbidden error code.

Don’t suppose anyone knows how to fix this?

Thanks Sam



This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

Hey Sam,

It could be the web properties you are trying to scrape have blocked DigitalOcean or Heroku’s IP ranges. Lots of places do this to prevent cloud servers from scraping their site.

Regards,

–Andrew

The developer cloud

Scale up as you grow — whether you're running one virtual machine or ten thousand.

Get started for free

Sign up and get $200 in credit for your first 60 days with DigitalOcean.*

*This promotional offer applies to new accounts only.