Question
connect() failed (111: Connection refused) while connecting to upstream
- Posted August 14, 2019
- NginxAPILet's EncryptDigitalOceanUbuntu 18.04
Hi all,
Followed this tutorial to get Let’s Encrypt working on my Nest.js/Node server.
Everything works - running sudo nginx -t returns:
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful
However, accessing my domain gives me a 502 Bad Gateway error. Specifically (from /var/log/nginx):
15004#15004: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 77.98.141.183, server: ww2.zone, request: "GET / HTTP/2.0", upstream: "http://127.0.0.1:8000/", host: "ww2.zone"
Here is my /etc/nginx/sites-available file:
Default server configuration
server {
listen 80;
listen 127.0.01;
listen [::]:80 default_server ipv6only=on;
return 301 https://$host$request_uri;
}
server {
# Enable HTTP/2
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name ww2.zone;
# Use the Let’s Encrypt certificates
ssl_certificate /etc/letsencrypt/live/ww2.zone/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/ww2.zone/privkey.pem;
# Include the SSL configuration from cipherli.st
include snippets/ssl-params.conf;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://localhost:8000/;
proxy_ssl_session_reuse off;
proxy_set_header Host $http_host;
proxy_cache_bypass $http_upgrade;
proxy_redirect off;
}
}
And my server is within a git Repo within home/username.
The code to listen on the port is:
await app.listen(process.env.PORT || 5000);
This used to be process.env.PORT || 80 but that would throw:
Error: listen EADDRINUSE: address already in use :::80
Any ideas? Pulling my hair out atm.
These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.
Hello,
You can only have 1 service listening on 1 port. So as you have Nginx listening on port 80 at the moment, you can not have your Node on the same port.
Also as you currently have your Nginx set to work as a reverse proxy for port 8000, what you could do is to just set Node to listen on port 8000.
This would fix your 502 error.
Here is a link to an answer on how to troubleshoot common Nginx problems:
Here is also a quick video demo on how to do that:
Hope that this helps! Regards, Bobby
I have the same problem.
I get the errors (I change the name of my ip to the word server for security reasons):
[crit] 21357#0: *15 connect() to 127.0.0.1:5000 failed (13: Permission denied) while connecting to upstream, client: 186.86.225.27, server: server, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:5000/", host: "server"
[crit] 21357#0: *15 connect() to [::1]:5000 failed (13: Permission denied) while connecting to upstream, client: 186.86.225.27, server: server, request: "GET / HTTP/1.1", upstream: "http://[::1]:5000/", host: "server"
[error] 21357#0: *15 no live upstreams while connecting to upstream, client: 186.86.225.27, server: server, request: "GET /favicon.ico HTTP/1.1", upstream: "http://localhost/favicon.ico", host: "server", referrer: "server"
The service is running on port 5000 with the command netstat -plant
tcp 0 0 127.0.0.1:5000 0.0.0.0:* LISTEN 9046/dotnet
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/systemd
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 19716/nginx: master
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1001/sshd
tcp 0 1049 server:22 221.181.185.140:14596 FIN_WAIT1 -
tcp 0 1049 server:22 221.181.185.140:26923 FIN_WAIT1 -
tcp 1 0 127.0.0.1:53162 127.0.0.1:80 CLOSE_WAIT 9864/amplify-agent
tcp 0 0 server:22 186.86.225.27:49672 ESTABLISHED 1747/sshd: root [pr
tcp 0 0 server:22 121.4.85.107:52210 FIN_WAIT2 -
tcp 0 1049 server:22 221.181.185.140:40999 FIN_WAIT1 -
tcp 0 36 server:22 186.86.225.27:65194 ESTABLISHED 1354/sshd: root [pr
tcp 0 1049 server:22 221.181.185.140:52399 FIN_WAIT1 -
tcp 0 0 server:47592 35.155.143.94:443 ESTABLISHED 9864/amplify-agent
tcp 0 0 server:22 186.86.225.27:51921 ESTABLISHED 8789/sshd: root [pr
tcp 0 1048 server:22 221.181.185.159:33491 ESTABLISHED 21120/sshd: [accept
tcp 0 0 server:80 186.86.225.27:61003 FIN_WAIT2 -
tcp 0 0 server:80 186.86.225.27:54720 FIN_WAIT2 -
tcp6 0 0 ::1:5000 :::* LISTEN 9046/dotnet
tcp6 0 0 :::111 :::* LISTEN 1/systemd
tcp6 0 0 :::22 :::* LISTEN 1001/sshd
nginx.conf
# For more information on configuration, see:
# * Official English Documentation: http://nginx.org/en/docs/
# * Official Russian Documentation: http://nginx.org/ru/docs/
user root;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
client_max_body_size 8M;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
#include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*.*;
server {
listen 80;
server_name _;
location / {
proxy_pass http://localhost:5000/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection keep-alive;
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
}
Hello, looks like I am also facing the same issue. And this has suddenly come up. Haven’t done any changes to server unless DO has done at their end.
Kindly help. The error fron nginx/error.log:
2020/10/12 11:05:44 [error] 915#915: *1 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 122.177.172.254, server: test.uxhack.co, request: “GET /favicon.ico HTTP/1.1”, upstream: “http://127.0.0.1:8000/favicon.ico”, host: “test.uxhack.co”, referrer: “https://test.uxhack.co/” 2020/10/12 11:08:07 [error] 915#915: *4 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 122.177.172.254, server: test.uxhack.co, request: “GET / HTTP/1.1”, upstream: “http://127.0.0.1:8000/”, host: “test.uxhack.co” 2020/10/12 11:08:07 [error] 915#915: *4 connect() failed (111: Connection refused) while connecting to upstream, client: 122.177.172.254, server: test.uxhack.co, request: “GET /favicon.ico HTTP/1.1”, upstream: “http://127.0.0.1:8000/favicon.ico”, host: “test.uxhack.co”, referrer: “https://test.uxhack.co/”
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 914/nginx: master p
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 693/systemd-resolve
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 893/sshd
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 914/nginx: master p
tcp 0 0 127.0.0.1:8000 0.0.0.0:* LISTEN 2165/python3
tcp 0 0 139.59.5.173:80 52.84.150.39:13100 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:16550 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:4970 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:24841 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:40249 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:27163 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:31014 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:40948 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:46110 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:38434 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:17559 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:27500 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:18538 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:51304 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:44313 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:31117 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:60028 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:23283 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:39763 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:2837 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:54697 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:28925 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:12038 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:23878 SYN_RECV -
tcp 0 5140 139.59.5.173:22 122.177.172.254:53190 ESTABLISHED 1132/sshd: root@pts
tcp 0 0 139.59.5.173:80 52.84.150.39:34837 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:60232 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:45250 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:24603 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:34587 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:59458 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:160 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:46887 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:825 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:7317 SYN_RECV -
tcp 0 0 139.59.5.173:80 52.84.150.39:60776 SYN_RECV -
tcp6 0 0 :::3306 :::* LISTEN 925/mysqld
tcp6 0 0 :::22 :::* LISTEN 893/sshd
Kindly help
Hi there @moisesalejandro,
What I could suggest in your case is to check your Docker container’s logs for more information on why the connections might be failing.
To do that, you can run the following:
- First get your Docker container ID:
- docker ps -a
- Then check the logs with:
- docker logs your_container_id
Feel free to share the logs here! Regards, Bobby
hello bobbyiliev thank you for your reply.
i dont know what you mean with proxy 4001 and backend service, i have never heard of them. I got never issues like this, how i can i install and set up this kind of proxy? btw i run a shop system with admin interface.
these lines are only to stop nginx redirecting my other network devices to localhost
proxy_pass http://127.0.0.1:4001;
proxy_set_header Host $server_name;
hope you can help regards amino
have the same problem since a few days…
nginx conf
server {
listen 80;
root /var/www;
index index.php index.html index.htm;
server_name_in_redirect off;
server_name localhost;
proxy_redirect off;
proxy_set_header Host $http_host;
location / {
proxy_pass http://127.0.0.1:4001;
proxy_set_header Host $server_name;
try_files $uri $uri/ /index.php;
}
# error_page 404 /404.html;
# error_page 500 502 503 504 /50x.html;
# location = /50x.html {
# root /var/www;
# }
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
location ~ \.php$ {
try_files $uri =404;
#fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
}
}
ufw
-- ------ ---
80 ALLOW Anywhere
Nginx HTTP ALLOW Anywhere
80 (v6) ALLOW Anywhere (v6)
Nginx HTTP (v6) ALLOW Anywhere (v6)
netstat -plant
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 783/systemd-resolve
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 901/cupsd
tcp 0 0 127.0.0.1:9000 0.0.0.0:* LISTEN 77280/php-fpm: mast
tcp 0 0 127.0.0.1:3306 0.0.0.0:* LISTEN 68490/mysqld
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 78190/nginx: master
tcp6 0 0 ::1:631 :::* LISTEN 901/cupsd
tcp6 0 0 :::33060 :::* LISTEN 68490/mysqld
php www.conf
;listen = /var/run/php/php7.4-fpm.sock
listen = 127.0.0.1:9000
i dont know, even the people from stackoverflow (dont get me wrong)
hope you guys got the solution, thanks
hello, got same problem. But mine is very simple. My Webserver runs on my laptop, which is my editor for the www files. But i cannot visit my website from another Device in same network. it change the the local ip to localhost on my other device (?) or it gives me a 502 site
Since 4 hours i have been looking on the internet, did a lot of changes but nothing helped. Have you guys any tips for me ?
thanks and sorry for my bad english
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:5000 0.0.0.0:* LISTEN 19783/dotnet
tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 26606/mysqld
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 14585/nginx: master
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 614/systemd-resolve
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 736/sshd
tcp 0 0 104.248.238.219:22 3.131.7.66:53227 ESTABLISHED 19114/sshd: root@no
tcp 0 1081 104.248.238.219:22 222.186.180.142:37107 FIN_WAIT1 -
tcp 0 1081 104.248.238.219:22 222.186.180.142:44469 FIN_WAIT1 -
tcp 0 1 104.248.238.219:22 222.186.169.192:47172 FIN_WAIT1 -
tcp 0 0 104.248.238.219:80 98.209.15.49:64734 SYN_RECV -
tcp 0 0 104.248.238.219:22 113.57.170.50:18180 ESTABLISHED 19826/sshd: unknown
tcp 0 0 104.248.238.219:22 188.165.169.238:59054 ESTABLISHED 19829/sshd: unknown
tcp 0 464 104.248.238.219:22 3.131.7.66:53230 ESTABLISHED 19224/sshd: root@pt
tcp6 0 0 127.0.0.1:8005 :::* LISTEN 23842/java
tcp6 0 0 ::1:5000 :::* LISTEN 19783/dotnet
tcp6 0 0 :::8080 :::* LISTEN 23842/java
tcp6 0 0 :::22 :::* LISTEN 736/sshd
See this is my result after hit that command. I think service is starting there.
**Error:** connect() failed (111: Connection refused) while connecting to upstream,
client: 157.32.151.133, server: _, request: "GET /api/jinping/getdetails HTTP/1.1",
upstream: "http://127.0.0.1:5000/api/jinping/getdetails", host: "104.248.238.219"
**NgInx Config File(/etc/nginx/sites-available): **
upstream backend {
server 127.0.0.1:5000
keepalive 32;
}
server {
server_name _;
listen 80;
location / {
proxy_pass http://127.0.0.1:5000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection keep-alive;
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
#listen [::]:443; # managed by Certbot
#listen 443 ssl; # managed by Certbot
#ssl_certificate /etc/letsencrypt/live/your-domain/fullchain.pem; # managed by Certbot
#ssl_certificate_key /etc/letsencrypt/live/your-domain/privkey.pem; # managed by Certbot
#include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
#ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
Service (.Net Core web API):
[Unit]
Description=HealthCare .NET Web API App running on Ubuntu
[Service]
WorkingDirectory=/var/www/healthcare/
ExecStart=/usr/bin/dotnet /var/www/healthcare/bin/Debug/netcoreapp2.0/publish/Session1.dll --urls="http://104.248.238.219:5000;https://104.248.238.219:5001"
Restart=always
# Restart service after 10 seconds if the dotnet service crashes:
RestartSec=10
SyslogIdentifier=Health
User=root
Environment=ASPNETCORE_ENVIRONMENT=Production
Environment=DOTNET_PRINT_TELEMETRY_MESSAGE=false
[Install]
WantedBy=multi-user.target
Please help me to solve this issue.
I am having same issue however on browser, i am having this issue and logs doesn’t have such issues. I ran the command “netstat -plant”. Here’s the result.
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN -
tcp 0 612 <IP>:22 106.204.206.144:3884 ESTABLISHED -
tcp 0 1 <IP>:42432 <IP>:3306 SYN_SENT -
tcp 0 1 <IP>:42434 <IP>:3306 SYN_SENT -
tcp6 0 0 :::21 :::* LISTEN -
tcp6 0 0 :::22 :::* LISTEN -
I have the same problem, when trying to access the nginx test page
2020/05/23 17:03:43 [error] 9037#9037: *28 connect() failed (111: Connection refused) while connecting to upstream, client: 179.126.139.213, server: hellonode, request: "GET /favicon.ico HTTP/1.1", upstream: "http://127.0.0.1:3000/favicon.ico", host: "165.22.43.156", referrer: "http://165.22.43.156/"
2020/05/23 17:03:43 [error] 9037#9037: *27 connect() failed (111: Connection refused) while connecting to upstream, client: 179.126.139.213, server: hellonode, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "165.22.43.156"
run the comand netstat -plant
root@know-how-cloud:/# netstat -plant
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 165.22.43.156:3333 0.0.0.0:* LISTEN 6385/node /home/dep
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 9029/nginx: master
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 722/systemd-resolve
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 926/sshd
tcp 0 0 165.22.43.156:22 179.126.139.213:5270 ESTABLISHED 8011/sshd: root@not
tcp 0 0 165.22.43.156:37592 161.35.127.202:25060 ESTABLISHED 6385/node /home/dep
tcp 0 340 165.22.43.156:22 179.126.139.213:1979 ESTABLISHED 5495/sshd: deploy [
tcp 0 1081 165.22.43.156:22 112.85.42.195:41424 FIN_WAIT1 -
tcp 0 0 165.22.43.156:22 222.186.42.7:61212 ESTABLISHED 9455/sshd: [accepte
tcp 0 0 165.22.43.156:80 179.126.139.213:65126 ESTABLISHED 9037/nginx: worker
tcp 0 0 165.22.43.156:80 179.126.139.213:6467 ESTABLISHED 9037/nginx: worker
tcp6 0 0 :::80 :::* LISTEN 9029/nginx: master
tcp6 0 0 :::22 :::* LISTEN 926/sshd
Do you know how I can fix it?
Hi @bobbyiliev How have you been? I am facing a same problem like the the issue above.
So I have a MERN stack app which always worked like charm until this morning when I uploaded some code changes and then (without touching anything) a 502 Bad Gateway. I’ve just seen this post above which is resolved but any of the steps in that post didn’t help me anything at all.
Not sure, how can I find out?
Expected behavior:
Actual behavior:
Hi Bobby,
One more to the list having this issue.
nginx.conf
Node.js server is running correctly on PORT 4000 because I can to a POST to it, get a response and see data in my MySQL
This is my error: