I have updated my robots.txt and removed all restrictions for google-bot.
I saw this question and they suggested it might be cdn cache. But I dont have that activated on my digitalocean project.
Why doesnt my robots.txt update? It has been updated for over 2 weeks now and google still doesnt find my new robots.txt.
Become a contributor for community
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
DigitalOcean Documentation
Full documentation for every DigitalOcean product.
Resources for startups and AI-native businesses
The Wave has everything you need to know about building a business, from raising funding to marketing your product.
The developer cloud
Scale up as you grow — whether you're running one virtual machine or ten thousand.
Start building today
From GPU-powered inference and Kubernetes to managed databases and storage, get everything you need to build, scale, and deploy intelligent applications.
