How to block website from crawling my server?

This looks like a googlebot, have you tried disallowing google via your robots.txt file?

I tried that , also tried to block the ip it self using sudo ufw deny from 66.249.64.1/24

Heya,

In your robots.txt file, you can use the Disallow directive to specify which bots or user-agents should be blocked.

For example, to block all access from the user-agent crawl-66-249-64-3, you can add the following lines:

User-agent: crawl-66-249-64-3
Disallow: /

This instructs the bot “crawl-66-249-64-3” not to crawl any pages on your site.

To make sure the changes take effect, you may need to restart your web server. Use the appropriate command for your web server software.

For Apache, you can use:

sudo systemctl restart apache2

For Nginx:

sudo systemctl restart nginx

To test if the robots.txt file is correctly blocking the bot, you can use various online tools or access logs to see if the bot’s access to your site decreases or stops.

Remember that well-behaved bots usually respect the rules in the robots.txt file, but malicious bots might not. It’s an effective way to communicate your preferences to web crawlers, but it doesn’t guarantee that all bots will obey. If you continue to experience issues, you might consider implementing additional security measures or IP blocking as mentioned earlier.

Hope that this helps!