New Django site on Nginx went to 502 few days after setup

February 9, 2017 332 views
Nginx Django Ubuntu 16.04

I set up a django site on a new droplet with Uwsgi and Niginx a few days ago following this tutorial

https://www.digitalocean.com/community/tutorials/how-to-serve-django-applications-with-uwsgi-and-nginx-on-ubuntu-16-04

I had the site set up correctly with the Django 'It worked' message displaying and Nginx reporting everything was setup correctly.

I then came back to the site (nothing else was done to it) 2 days later to see a 502 message and also I was unable to ssh into the server. I had to reset the root pw and login via the console and set port 22 to allowed in ufw which I thought was strange as I had logged in and out via ssh many times when doing the setup a few days before.

Next I went through all the uwsgi and nginx config files and made sure everything was ok, nothing had been changed so I then restarted Nginx, still the site show 502, finally I ran this

sudo systemctl start uwsgi

and then the site was back to normal.

Does anyone know what caused this and is it going to happen again?

In the Nginx error logs I see something about the sock file:

connect() to unix:/run/uwsgi/firstsite.sock failed (2: No such file or directory) while connecting to upstream

but the sock file was created just fine the first time and was still there.

Also it was showing some errors referring to phpmyadmin e.g.

2017/02/09 07:06:49 [crit] 1615#1615: *47 connect() to unix:/run/uwsgi/firstsite.sock failed (2: No such file or directory) while connecting to upstream, client: 139.129.228.236, server: my-ip, request: "GET /phpmyadmin HTTP/1.1", upstream: "uwsgi://unix:/run/uwsgi/firstsite.sock:", host: "my-ip", referrer: "http://my-ip/phpmyadmin"

why is phpmyadmin there when this is a Django site?

thanks

2 Answers

@easycompany80

SSH and Firewall

Not being able to SSH in to the server is, as you discovered, most likely an issue with the firewall and how it was setup. Since you're using Ubuntu, ufw is available and relatively simple to setup in a way that will prevent that from happening again.

What I generally do, or recommend others to do, is to start fresh by resetting the firewall. To do this, we'd disable ufw first (run this even if you believe that ufw is already disabled, better to be safe).

ufw disable

Now we can reset it:

ufw reset

With a clean slate, we configure the firewall to deny incoming and allow outgoing.

ufw default deny incoming \
&& ufw default allow outgoing

As of right now, nothing is allowed in and anything is allowed out. So now we need to configure our services. I normally use port specific allow commands just for confirmation. We need to allow SSH, HTTP, and HTTPS on most configurations, so that's what I start with.

ufw allow 22/tcp \
&& ufw allow 80/tcp \
&& ufw allow 443/tcp

If you have any additional ports that will be accessed by the public, you can extend the above command to include those. Once done, all that you need to do is re-enable ufw by running:

ufw enable

When prompted, type y and hit enter. Since Port 22 is allowed through, your connection won't drop and you'll be able to keep your connection as normal.

NGINX and Django

Receiving a 502 error is generally a failure to connect. In most cases, this is going to be due to misconfiguration and incorrect permissions or a service is failing/crashing. Since you're able to run the restart command and bring service back up, and it works, this most likely isn't an issue with config or permissions, but rather an issue with service(s) crashing.

What size Droplet are you running on and what's the output of free -mh and what's your CPU usage look like when running top?

If the socket is never removed, that to me signals a service crashing because it doesn't have enough resources to maintain it.

Thanks for the detailed answer, I did all that for ufw and now this is whats allowed:

To Action From


22/tcp ALLOW Anywhere

80/tcp ALLOW Anywhere

443/tcp ALLOW Anywhere

Nginx Full ALLOW Anywhere

8000 ALLOW Anywhere

22/tcp (v6) ALLOW Anywhere (v6)

80/tcp (v6) ALLOW Anywhere (v6)

443/tcp (v6) ALLOW Anywhere (v6)

Nginx Full (v6) ALLOW Anywhere (v6)

8000 (v6) ALLOW Anywhere (v6)

'free -mh' gives the following:

total used free shared buff/cache available
Mem: 488M 110M 46M 18M 330M 327M
Swap: 0B 0B 0B

I should add that this is a brand new droplet with nothing but a fresh uwsgi & nginx setup and basic django project with no content, its only the $5 droplet but surely I shouldnt be using my allocated resources?

This is what 'top' gives me btw:

top - 10:37:40 up 1 day, 57 min, 2 users, load average: 0.00, 0.00, 0.00
Tasks: 146 total, 1 running, 145 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.0 us, 0.3 sy, 0.0 ni, 99.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 500136 total, 47248 free, 113608 used, 339280 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 335628 avail Mem

this is my site btw

http://128.199.120.162/

and its still up now 16hrs since I had that problem so thats something.

I contacted support btw and they said about the /phpmyadmin appended on my ip could be someone trying to access it, do they mean hacking bots? Ive worked with Wordpress blogs a lot (although not really on the technical side) and I know that there are lots of bots out there that just attempt to sniff out WP sites and brute force the login. Do I need to worry about that or should I just ignore it?

Have another answer? Share your knowledge.