Hi,
I have a site hosted on digital ocean APP platform, website is available but if I want to index it by google it fails to index it due to a 5xx error. I am using google search console. Since the site is up, only thing that I can think of is DNS issues, does anybody know what this could be?
I am using laravel, inertia ssr and vuejs.
My robots.txt looks like this and it is available by other crawlers.
User-agent: *
Disallow: /login
Disallow: /register
Disallow: /images
Disallow: /*.png$
Disallow: /*.svg$
Disallow: /*.jpg$
Allow: /
Sitemap: https://mysite/sitemap.xml
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.
Enter your email to get $200 in credit for your first 60 days with DigitalOcean.
New accounts only. By submitting your email you agree to our Privacy Policy.
Hi there,
I could suggest a couple of things:
curl
with providing the Google bot as the agent, that way you will verify if the error is caused by the user agent:Let me know how it goes!
Best,
Bobby