Question
Question
searx metasearch instance - refused to connect
Hi Community,
the good old “refused to connect” is driving me nuts in my particular setup and I hope somebody does see what I simply can’t anymore after fiddling around with almost everything I could possibly think off…
Here we go, with a system description as well as with all the glory details of the issue I’m facing…
System Details:
[user@host ~]$ head -1 /etc/*release*
# PRETTY_NAME="Debian GNU/Linux 10 (buster)"
## nginx version: nginx/1.14.2
built with OpenSSL 1.1.1c 28 May 2019 (running with OpenSSL 1.1.1d 10 Sep 2019)
I’m trying to figure out why on earth my “searx metasearch” instance suddenly stopped working, although I’ve even reverted back to a working snapshot 5 days back, when the same configuration was working as set up initially. Haven’t changed anything within my configuration at all… I even re-installed the entire application without any success.
Since I’m trying to access that searx instance running locally on my nginx on TCP_Port 8888 through uwsgi as described within this install wiki:
proxy or uWSGI is the relevant entry for my setup.
The uWSGI setup reflects this one:
1:1, no changes at all…
[user@host ~]$ sudo -H service uwsgi status searx
● uwsgi.service - LSB: Start/stop uWSGI server instance(s)
Loaded: loaded (/etc/init.d/uwsgi; generated)
Active: active (running) since Tue 2020-07-14 03:17:37 CEST; 2min 52s ago
Docs: man:systemd-sysv-generator(8)
Process: 1626 ExecStart=/etc/init.d/uwsgi start (code=exited, status=0/SUCCESS)
Tasks: 4 (limit: 1167)
Memory: 124.0M
CGroup: /system.slice/uwsgi.service
├─1728 /usr/bin/uwsgi --ini /usr/share/uwsgi/conf/default.ini --ini /etc/uwsgi/apps-enabled/searx.ini --daemonize /var/log/uwsgi/app/searx.log
├─1735 /usr/bin/uwsgi --ini /usr/share/uwsgi/conf/default.ini --ini /etc/uwsgi/apps-enabled/searx.ini --daemonize /var/log/uwsgi/app/searx.log
├─1736 /usr/bin/uwsgi --ini /usr/share/uwsgi/conf/default.ini --ini /etc/uwsgi/apps-enabled/searx.ini --daemonize /var/log/uwsgi/app/searx.log
└─1737 /usr/bin/uwsgi --ini /usr/share/uwsgi/conf/default.ini --ini /etc/uwsgi/apps-enabled/searx.ini --daemonize /var/log/uwsgi/app/searx.log
Jul 14 03:17:36 host systemd[1]: Starting LSB: Start/stop uWSGI server instance(s)...
Jul 14 03:17:37 host uwsgi[1626]: Starting app server(s): uwsgi -> . done.
Jul 14 03:17:37 host systemd[1]: Started LSB: Start/stop uWSGI server instance(s).
The service is running as verified above and also accessible
on the internal host listening on Port 8888
[user@host ~]$ curl --verbose --insecure --location http://127.0.0.1:8888
* Expire in 0 ms for 6 (transfer 0x5645278c2f50)
* Trying 127.0.0.1...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x5645278c2f50)
* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)
> GET / HTTP/1.1
> Host: 127.0.0.1:8888
> User-Agent: curl/7.64.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=utf-8
< Content-Length: 10535
< Server-Timing: total;dur=11.569
<
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
--- cut for brevity ---
</noscript>
</body>
* Connection #0 to host 127.0.0.1 left intact
</html>%
The same test again, this time around with my subdomain;
[user@host ~]$ curl --verbose --insecure --location https://subdomain.domain
* Expire in 0 ms for 6 (transfer 0x560e66e73f50)
* Expire in 1 ms for 1 (transfer 0x560e66e73f50)
* Connected to subdomain.domain (xxxx:xxxx:x:xx::xxxx:xxxx) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=subdomain.domain
* start date: Jul 1 17:16:32 2020 GMT
* expire date: Sep 29 17:16:32 2020 GMT
* issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x560e66e73f50)
> GET / HTTP/2
> Host: subdomain.domain
> User-Agent: curl/7.64.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 200
< server: nginx
< date: Tue, 14 Jul 2020 01:25:33 GMT
--- cut for brevity ---
</noscript>
</body>
* Connection #0 to host subdomain.domain left intact
</html>%
Firewall allowing connections to 8888 and 443
[user@host ~]$ sudo ufw status numbered | grep 8888
[10] 8888/tcp ALLOW IN Anywhere
[32] 8888/tcp (v6) ALLOW IN Anywhere (v6)
[user@host ~]$ sudo ufw status numbered | grep 443
[19] 443/tcp ALLOW IN Anywhere
[23] 443/tcp (v6) ALLOW IN Anywhere (v6)
TCP sockets also hanging in there:
[user@host ~]$ sudo netstat -anp | grep 8888
tcp 0 0 127.0.0.1:8888 0.0.0.0:* LISTEN 1728/uwsgi
[user@host ~]$ sudo netstat -anp | grep :443
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 615/nginx: master p
tcp 32 0 138.197.209.92:44094 99.84.174.86:443 CLOSE_WAIT 1735/uwsgi
tcp 32 0 138.197.209.92:47040 99.84.243.60:443 CLOSE_WAIT 1735/uwsgi
tcp 32 0 138.197.209.92:55226 140.177.18.77:443 CLOSE_WAIT 1736/uwsgi
tcp 32 0 138.197.209.92:44096 99.84.174.86:443 CLOSE_WAIT 1736/uwsgi
tcp 32 0 138.197.209.92:47042 99.84.243.60:443 CLOSE_WAIT 1736/uwsgi
tcp 32 0 138.197.209.92:42792 140.177.16.37:443 CLOSE_WAIT 1735/uwsgi
tcp6 0 0 :::443 :::* LISTEN 615/nginx: master p
and lastly, the relevant nginx config for the searx engine;
[user@host ~]$ sudo cat /etc/nginx/sites-available/searx
server {
server_name searx.domain www.searx.domain;
root /usr/local/searx;
access_log /dev/null;
error_log /dev/null;
location / {
proxy_pass http://127.0.0.1:8888;
proxy_set_header Host $host;
proxy_set_header Connection $http_connection;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Scheme $scheme;
proxy_buffering off;
}
listen 443 ssl http2; # managed by Certbot
listen [::]:443 ssl http2;
ssl_certificate
/etc/letsencrypt/live/searx.rsec.systems/fullchain.pem; # managed by Certbot
ssl_certificate_key
/etc/letsencrypt/live/searx.rsec.systems/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
if ($host = www.searx.domain) {
return 301 https://$host$request_uri;
} # managed by Certbot
if ($host = searx.domain) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
server_name searx.domain www.searx.domain;
return 404; # managed by Certbot
}
DNS resolution works just fine;
[user@host ~]$ dig searx.domain A,CNAME
; <<>> DiG 9.11.5-P4-5.1+deb10u1-Debian <<>> searx.domain A,CNAME
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24508
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;searx.domain. IN A
;; ANSWER SECTION:
searx.domain. 278 IN CNAME domain.
domain. 278 IN A xxx.xxx.xxx.xxx
;; Query time: 21 msec
;; SERVER: 67.207.67.3#53(67.207.67.3)
;; WHEN: Tue Jul 14 03:40:59 CEST 2020
;; MSG SIZE rcvd: 77
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 54492
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;A,CNAME. IN A
;; AUTHORITY SECTION:
. 3600 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2020071301 1800 900 604800 86400
;; Query time: 4 msec
;; SERVER: 67.207.67.3#53(67.207.67.3)
;; WHEN: Tue Jul 14 03:40:59 CEST 2020
;; MSG SIZE rcvd: 111
Any help pointing me into the right direction would be much appreciated.
Add a comment
edited by MattIPv4