After getting flask running with nginx and uwsgi with the help of this eminent community, and with the guide found here https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-uwsgi-and-nginx-on-ubuntu-16-04.
I am confused since I can not understand how to add robot.txt to a site running flask. Robot.txt is a static file, but how to configure to be able to run flask with robots.txt?
If I am not misstaken shouldn’t nginx handle static files and robot.txt in production? How? Any guide toward correct answer is more than appreciated.
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
I ran into this issue and was trying to make it work with static directories that were loading from a blueprint.
This solution works well for me because it shows how to modify headers and control the response more granularly for a variety of scenarios.
from flask import Flask, Response
app = Flask(__name__)
@app.route('/robots.txt')
def noindex():
r = Response(response="User-Agent: *\nDisallow: /\n", status=200, mimetype="text/plain")
r.headers["Content-Type"] = "text/plain; charset=utf-8"
return r
Cheers.
NGINX will serve pretty much anything you configure it to – whether static or dynamic.
Generally, you want to place the robots.txt file in the web root where you’re serving your app, but flask seems to work a little differently, so you’d most likely need to place in the root that you’ve set for your static file path.
You may want to check the following to see if it can help you to get things setup:
https://coderwall.com/p/kc-ewg/flask-handler-for-robots-txt
https://www.pythonanywhere.com/forums/topic/2899/
http://stackoverflow.com/questions/4239825/static-files-in-flask-robot-txt-sitemap-xml-mod-wsgi
Become a contributor for community
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
DigitalOcean Documentation
Full documentation for every DigitalOcean product.
Resources for startups and SMBs
The Wave has everything you need to know about building a business, from raising funding to marketing your product.
The developer cloud
Scale up as you grow — whether you're running one virtual machine or ten thousand.
Get started for free
Sign up and get $200 in credit for your first 60 days with DigitalOcean.*
*This promotional offer applies to new accounts only.