How i can setup robots.txt for my blog?

Posted December 11, 2013 10.5k views
Hello! I have successfully created my blog instance with Ghost droplet and already do some customization. Now i need to create robots.txt file. I googled, that i need to create robots.txt under my theme folder. But when i try to access it i get 404 error and i see in my adress line hostname/robots.txt/ (with slash). I read that i need to configure nginx to serve this file correctly. But i have no experience in this, so if somebody can help me with that - it will be very good :)

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

Submit an Answer
10 answers
Does it work without the slash?
No, it's adding slash when i try to access robots.txt (without slash)
Where did you store robots.txt? Please pastebin your nginx config as well.
I have same issue.

I place my robots.txt in the root folder of my current theme.

The nginx config is in default state.

Here is the default.conf file:
server {
listen 80;
server_name ;

client_max_body_size 10M;

location = /robots.txt {
allow all;
log_not_found off;
access_log off;

Maybe i need to add some lines?
Are you sure this is the correct file? There doesn't seem to be anything that's telling nginx to proxy requests to Ghost.

What's listening on port 80?
sudo netstat -plutn | grep 80
tcp 0 0* LISTEN 3124/nginx
Odd. Are there any files in the conf.d directory?
ls /etc/nginx/conf.d
Yes. There are file /etc/nginx/conf.d/default.conf
I'm used this solution
Now it's work fine.

how to create sitemap