nginx connection refused

May 25, 2017 906 views
Nginx Node.js Firewall Ubuntu 16.04

We are receiving 'connection refused' on nginx. We run several node apps and server those using nginx reverse proxy on different ports. ports are allowed through ufw. apps are running and doing what they are supposed to do. Still this error. ufw.log showing :

May 24 21:58:42 ubuntu-1gb-nyc2-01 kernel: [70662.714517] [UFW BLOCK] IN=eth0 OUT= MAC=04:01:a7:ee:06:01:4c:96:14:a4:ab:f0:08:00 SRC=61.245.170.207 DST=162.243.243.22 LEN=52 TOS=0x18 PREC=0x00 TTL=47 ID=1656 DF PROTO=TCP SPT=4385 DPT=9121 WINDOW=700 RES=0x00 ACK FIN URGP=0

Where as port 4385 is allowed.

netstat -tnlp

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 1262/nginx
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1040/sshd
tcp 0 0 0.0.0.0:5432 0.0.0.0:* LISTEN 1132/postgres
tcp 0 0 127.0.0.1:27017 0.0.0.0:* LISTEN 1044/mongod
tcp 0 0 127.0.0.1:3306 0.0.0.0:* LISTEN 1095/mysqld
tcp6 0 0 :::22 :::* LISTEN 1040/sshd
tcp6 0 0 :::11000 :::* LISTEN 2113/node
tcp6 0 0 :::5432 :::* LISTEN 1132/postgres
tcp6 0 0 :::11100 :::* LISTEN 2099/node
tcp6 0 0 :::9117 :::* LISTEN 2015/node
tcp6 0 0 :::9118 :::* LISTEN 2029/node
tcp6 0 0 :::9121 :::* LISTEN 2142/node
tcp6 0 0 :::9122 :::* LISTEN 2072/node
tcp6 0 0 :::9125 :::* LISTEN 2085/node
tcp6 0 0 :::9130 :::* LISTEN 2045/node

nginx, in our case shall respond to various port ranges, e.g. 3500-3599.
Now I allowed those ports in ufw. When try to add again, it says skipping rules already added. But not shown when netstat is run.

Does that mean those ports are not allowed and causing the issue? Any help will be greatly appreciated.

3 Answers

@w3resource2011

Please post the output of this command to a code block (using the </> icon)

tail -50 /var/log/nginx/error.log

Also, while troubleshooting, it may be beneficial to disable the firewall until we find out what the root cause is.

ufw disable

Output of

tail -50 /var/log/nginx/error.log

2017/05/24 07:07:46 [error] 12840#0: *1 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 79.183.50.161, server: 162.243.243.22, request: "GET /favicon.ico HTTP/1.1", host: "162.243.243.22", referrer: "http://162.243.243.22/"
2017/05/24 07:09:19 [error] 12840#0: *6 open() "/usr/share/nginx/html/69da0f30-4071-11e7-b954-47b3e680d863.js" failed (2: No such file or directory), client: 79.183.50.161, server: 162.243.243.22, request: "GET /69da0f30-4071-11e7-b954-47b3e680d863.js HTTP/1.1", host: "162.243.243.22"
2017/05/24 07:09:27 [error] 12840#0: *6 open() "/usr/share/nginx/html/home/students/69da0f30-4071-11e7-b954-47b3e680d863.js" failed (2: No such file or directory), client: 79.183.50.161, server: 162.243.243.22, request: "GET /home/students/69da0f30-4071-11e7-b954-47b3e680d863.js HTTP/1.1", host: "162.243.243.22"
2017/05/24 07:09:32 [error] 12840#0: *6 "/usr/share/nginx/html/home/students/index.html" is not found (2: No such file or directory), client: 79.183.50.161, server: 162.243.243.22, request: "GET /home/students/ HTTP/1.1", host: "162.243.243.22"
2017/05/24 07:09:35 [error] 12840#0: *6 "/usr/share/nginx/html/home/students/index.html" is not found (2: No such file or directory), client: 79.183.50.161, server: 162.243.243.22, request: "GET /home/students/ HTTP/1.1", host: "162.243.243.22"
2017/05/24 07:09:38 [error] 12840#0: *6 "/usr/share/nginx/html/home/index.html" is not found (2: No such file or directory), client: 79.183.50.161, server: 162.243.243.22, request: "GET /home/ HTTP/1.1", host: "162.243.243.22"
2017/05/24 07:15:52 [error] 12840#0: *11 open() "/usr/share/nginx/html/d" failed (2: No such file or directory), client: 79.183.50.161, server: 162.243.243.22, request: "GET /d HTTP/1.1", host: "162.243.243.22"
2017/05/24 07:16:03 [error] 12840#0: *11 open() "/usr/share/nginx/html/d" failed (2: No such file or directory), client: 79.183.50.161, server: 162.243.243.22, request: "GET /d HTTP/1.1", host: "162.243.243.22"
2017/05/24 07:19:01 [error] 12840#0: *12 open() "/usr/share/nginx/html/data" failed (2: No such file or directory), client: 109.64.47.145, server: 162.243.243.22, request: "GET /data HTTP/1.1", host: "162.243.243.22"
2017/05/24 07:19:04 [error] 12840#0: *12 open() "/usr/share/nginx/html/www" failed (2: No such file or directory), client: 109.64.47.145, server: 162.243.243.22, request: "GET /www HTTP/1.1", host: "162.243.243.22"
2017/05/24 07:19:34 [error] 12840#0: *12 open() "/usr/share/nginx/html/some" failed (2: No such file or directory), client: 109.64.47.145, server: 162.243.243.22, request: "GET /some HTTP/1.1", host: "162.243.243.22"
2017/05/24 07:19:42 [error] 12840#0: *12 open() "/usr/share/nginx/html/some/example.html" failed (2: No such file or directory), client: 109.64.47.145, server: 162.243.243.22, request: "GET /some/example.html HTTP/1.1", host: "162.243.243.22"
2017/05/24 07:24:48 [error] 12840#0: *13 open() "/usr/share/nginx/html/home/students/78ad4ed0-4073-11e7-ae2c-1b2b59c23cf4.js" failed (2: No such file or directory), client: 109.64.47.145, server: 162.243.243.22, request: "GET /home/students/78ad4ed0-4073-11e7-ae2c-1b2b59c23cf4.js HTTP/1.1", host: "162.243.243.22"
2017/05/24 08:34:21 [error] 17135#0: *1 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 103.43.80.42, server: 162.243.243.22, request: "GET /favicon.ico HTTP/1.1", host: "162.243.243.22", referrer: "http://162.243.243.22/"
2017/05/24 13:47:33 [error] 19987#0: *3 open() "/usr/share/nginx/html/checkproxy" failed (2: No such file or directory), client: 180.97.106.37, server: 162.243.243.22, request: "HEAD http://180.163.113.82/checkproxy HTTP/1.1", host: "180.163.113.82"
2017/05/24 15:38:58 [error] 19987#0: *4 open() "/usr/share/nginx/html/phpmyadmin" failed (2: No such file or directory), client: 45.77.41.29, server: 162.243.243.22, request: "GET /phpmyadmin HTTP/1.1", host: "162.243.243.22", referrer: "http://162.243.243.22/phpmyadmin"
2017/05/24 18:19:33 [error] 19987#0: *7 open() "/usr/share/nginx/html/muieblackcat" failed (2: No such file or directory), client: 221.193.241.36, server: 162.243.243.22, request: "GET /muieblackcat HTTP/1.1", host: "162.243.243.22"
2017/05/24 18:19:33 [error] 19987#0: *8 open() "/usr/share/nginx/html/phpMyAdmin/scripts/setup.php" failed (2: No such file or directory), client: 221.193.241.36, server: 162.243.243.22, request: "GET //phpMyAdmin/scripts/setup.php HTTP/1.1", host: "162.243.243.22"
2017/05/24 18:19:34 [error] 19987#0: *9 open() "/usr/share/nginx/html/phpmyadmin/scripts/setup.php" failed (2: No such file or directory), client: 221.193.241.36, server: 162.243.243.22, request: "GET //phpmyadmin/scripts/setup.php HTTP/1.1", host: "162.243.243.22"
2017/05/24 18:19:36 [error] 19987#0: *10 open() "/usr/share/nginx/html/pma/scripts/setup.php" failed (2: No such file or directory), client: 221.193.241.36, server: 162.243.243.22, request: "GET //pma/scripts/setup.php HTTP/1.1", host: "162.243.243.22"
2017/05/24 18:19:37 [error] 19987#0: *11 open() "/usr/share/nginx/html/myadmin/scripts/setup.php" failed (2: No such file or directory), client: 221.193.241.36, server: 162.243.243.22, request: "GET //myadmin/scripts/setup.php HTTP/1.1", host: "162.243.243.22"
2017/05/24 18:19:37 [error] 19987#0: *12 open() "/usr/share/nginx/html/MyAdmin/scripts/setup.php" failed (2: No such file or directory), client: 221.193.241.36, server: 162.243.243.22, request: "GET //MyAdmin/scripts/setup.php HTTP/1.1", host: "162.243.243.22"
2017/05/24 20:56:21 [error] 19987#0: *14 open() "/usr/share/nginx/html/muieblackcat" failed (2: No such file or directory), client: 221.193.241.36, server: 162.243.243.22, request: "GET /muieblackcat HTTP/1.1", host: "162.243.243.22"
2017/05/24 20:56:21 [error] 19987#0: *15 open() "/usr/share/nginx/html/phpMyAdmin/scripts/setup.php" failed (2: No such file or directory), client: 221.193.241.36, server: 162.243.243.22, request: "GET //phpMyAdmin/scripts/setup.php HTTP/1.1", host: "162.243.243.22"
2017/05/24 20:56:23 [error] 19987#0: *16 open() "/usr/share/nginx/html/phpmyadmin/scripts/setup.php" failed (2: No such file or directory), client: 221.193.241.36, server: 162.243.243.22, request: "GET //phpmyadmin/scripts/setup.php HTTP/1.1", host: "162.243.243.22"
2017/05/24 20:56:23 [error] 19987#0: *17 open() "/usr/share/nginx/html/pma/scripts/setup.php" failed (2: No such file or directory), client: 221.193.241.36, server: 162.243.243.22, request: "GET //pma/scripts/setup.php HTTP/1.1", host: "162.243.243.22"
2017/05/24 20:56:25 [error] 19987#0: *18 open() "/usr/share/nginx/html/myadmin/scripts/setup.php" failed (2: No such file or directory), client: 221.193.241.36, server: 162.243.243.22, request: "GET //myadmin/scripts/setup.php HTTP/1.1", host: "162.243.243.22"
2017/05/24 20:56:25 [error] 19987#0: *19 open() "/usr/share/nginx/html/MyAdmin/scripts/setup.php" failed (2: No such file or directory), client: 221.193.241.36, server: 162.243.243.22, request: "GET //MyAdmin/scripts/setup.php HTTP/1.1", host: "162.243.243.22"
2017/05/25 02:21:25 [error] 1265#0: *1 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 86.18.82.39, server: 162.243.243.22, request: "GET /favicon.ico HTTP/1.1", host: "162.243.243.22", referrer: "http://162.243.243.22/"
2017/05/25 05:06:08 [error] 1265#0: *3 open() "/usr/share/nginx/html/phpMyAdmin" failed (2: No such file or directory), client: 218.93.206.139, server: 162.243.243.22, request: "GET /phpMyAdmin HTTP/1.1", host: "162.243.243.22", referrer: "http://162.243.243.22/phpMyAdmin"
2017/05/25 05:43:37 [error] 1263#0: *2 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 103.10.67.174, server: 162.243.243.22, request: "GET /favicon.ico HTTP/1.1", host: "162.243.243.22", referrer: "http://162.243.243.22/"
2017/05/25 06:00:25 [error] 1263#0: *5 open() "/usr/share/nginx/html/cgi/common.cgi" failed (2: No such file or directory), client: 201.52.56.181, server: 162.243.243.22, request: "GET /cgi/common.cgi HTTP/1.0", host: "162.243.243.22"
2017/05/25 06:00:26 [error] 1263#0: *6 open() "/usr/share/nginx/html/stssys.htm" failed (2: No such file or directory), client: 201.52.56.181, server: 162.243.243.22, request: "GET /stssys.htm HTTP/1.0", host: "162.243.243.22"
2017/05/25 06:00:26 [error] 1263#0: *8 open() "/usr/share/nginx/html/command.php" failed (2: No such file or directory), client: 201.52.56.181, server: 162.243.243.22, request: "POST /command.php HTTP/1.0", host: "162.243.243.22"
2017/05/25 06:37:50 [error] 1263#0: *9 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 103.43.80.42, server: 162.243.243.22, request: "GET /favicon.ico HTTP/1.1", host: "162.243.243.22", referrer: "http://162.243.243.22/"
2017/05/25 07:35:26 [error] 1263#0: *12 open() "/usr/share/nginx/html/currentconfig/Account1" failed (2: No such file or directory), client: 60.191.38.77, server: 162.243.243.22, request: "GET /currentconfig/Account1 HTTP/1.1", host: "162.243.243.22"
2017/05/25 08:01:08 [error] 1263#0: *14 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 186.151.77.138, server: 162.243.243.22, request: "GET /favicon.ico HTTP/1.1", host: "162.243.243.22"
2017/05/25 08:01:08 [error] 1263#0: *14 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 186.151.77.138, server: 162.243.243.22, request: "GET /favicon.ico HTTP/1.1", host: "162.243.243.22"

nginx, in this case is used as a reverse proxy and does not have a root.

@w3resource2011

Since the error log isn't really providing too much, at least right now. You can run:

nginx -t

To test your configuration to see if it spits out any errors which may be useful. If not, then we'd need to look at each server block to see if there's an issue there.

You would need to post them in a code block though to keep the formatting intact. To do that, you can click on the </> icon in the toolbar of the message box and then paste them between the the back ticks on a new line, resulting in:

server {
    ...
    ...
    ...
    ...
}

etc.

Have another answer? Share your knowledge.