Related

Join the DigitalOcean Community Community

Join 1M+ other developers and:

  • Get help and share knowledge in Q&A
  • Subscribe to topics of interest
  • Get courses & tools that help you grow as a developer or small business owner
 Join Now
How do I keep my Python script running? Question Question
What droplet do I choose for a discord Python bot? Question Question

Question

Why is the collection of website data by python crawler often restricted by ip access?

Posted March 1, 2021 195 views
Python

When I was collecting website data, I was often prompted to restrict ip access. I am using the ip address of the tomatoip.io free trial. Should I buy more ips to make a list?

Add a comment
leyte948
By:
leyte948

Related

Join the DigitalOcean Community Community

Join 1M+ other developers and:

  • Get help and share knowledge in Q&A
  • Subscribe to topics of interest
  • Get courses & tools that help you grow as a developer or small business owner
 Join Now
How do I keep my Python script running? Question Question
What droplet do I choose for a discord Python bot? Question Question

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
Submit an Answer
1 answer
KFSys March 1, 2021

Hi @leyte948,

Can you please give a little bit more information? Were you the one that was doing the restriction or other websites that you were crawling were restricting the IP from which you were executing the script?

If other websites were restricting your IP Address it’s possible they’ve decided your crawler is a sort of DDOS thus blocking it.

Regards,
KFSys

Reply Report

Related

Looking for something else?

Ask a new question Search for more help