I have updated my robots.txt and removed all restrictions for google-bot.
I saw this question and they suggested it might be cdn cache. But I dont have that activated on my digitalocean project.
Why doesnt my robots.txt update? It has been updated for over 2 weeks now and google still doesnt find my new robots.txt.
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
Hello there,
Have you submitted a new request so the site can be re-crawled via the Google Developer tools so you can have the suite re-indexed?
I believe this link should help you to complete the process:
https://developers.google.com/search/docs/advanced/crawling/ask-google-to-recrawl
Hope that this helps!
Become a contributor for community
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
DigitalOcean Documentation
Full documentation for every DigitalOcean product.
Resources for startups and AI-native businesses
The Wave has everything you need to know about building a business, from raising funding to marketing your product.
The developer cloud
Scale up as you grow — whether you're running one virtual machine or ten thousand.
Get started for free
Sign up and get $200 in credit for your first 60 days with DigitalOcean.*
*This promotional offer applies to new accounts only.