habbywall
By:
habbywall

Can't load IP address or domain, but SSH and Ping is working.

May 12, 2017 793 views
DigitalOcean Ubuntu 16.04

I am running a LAMP stack on Ubuntu 16.04. On it I have three wordpress instances installed, one is on a subdomain, and the other two are on TLDs. Earlier today everything was working fine.

Just now when I go to one of the domains it does not load and then gives me this error:
ERRCONNECTIONTIMED_OUT

I have tried restarting Apache and turning my droplet on and off, still it does not work.
I am able to SFTP into the IP using Filezilla, I can SSH into the IP on a command line, I ran a ping test on the IP and it came back fine. I am only on the 1GB Memory 1 Core Processor, is it possible that's an issue? My usage of the site seems to be minimal, I just installed the wordpress instances and only had a bare bones site on one of them, the rest were blank.

9 Answers

@habbywall

Please post the output of of the following command in a code block.

tail -50 /var/log/apache2/error.log
[Fri May 12 06:25:03.452513 2017] [mpm_prefork:notice] [pid 21495] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g configured -- resuming normal operations
[Fri May 12 06:25:03.452597 2017] [core:notice] [pid 21495] AH00094: Command line: '/usr/sbin/apache2'
[Fri May 12 22:59:58.252453 2017] [mpm_prefork:notice] [pid 21495] AH00169: caught SIGTERM, shutting down
[Fri May 12 22:59:59.232313 2017] [mpm_prefork:notice] [pid 29167] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g configured -- resuming normal operations
[Fri May 12 22:59:59.232383 2017] [core:notice] [pid 29167] AH00094: Command line: '/usr/sbin/apache2'
[Fri May 12 23:03:55.347648 2017] [mpm_prefork:notice] [pid 29167] AH00169: caught SIGTERM, shutting down
[Fri May 12 23:04:34.042060 2017] [mpm_prefork:notice] [pid 1581] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g configured -- resuming normal operations
[Fri May 12 23:04:34.043473 2017] [core:notice] [pid 1581] AH00094: Command line: '/usr/sbin/apache2'
[Fri May 12 23:28:29.930075 2017] [mpm_prefork:notice] [pid 1581] AH00169: caught SIGTERM, shutting down
[Fri May 12 23:28:31.058566 2017] [mpm_prefork:notice] [pid 2226] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g configured -- resuming normal operations
[Fri May 12 23:28:31.058652 2017] [core:notice] [pid 2226] AH00094: Command line: '/usr/sbin/apache2'

Here's that response:

To                         Action      From
--                         ------      ----
22                         LIMIT       Anywhere                  
443                        ALLOW       Anywhere                  
80                         ALLOW       Anywhere                  
8787                       ALLOW       Anywhere                  
3838                       ALLOW       Anywhere                  
22 (v6)                    LIMIT       Anywhere (v6)             
443 (v6)                   ALLOW       Anywhere (v6)             
80 (v6)                    ALLOW       Anywhere (v6)             
8787 (v6)                  ALLOW       Anywhere (v6)             
3838 (v6)                  ALLOW       Anywhere (v6)             

Note: I previously opened ports 8787 and 3838 as that's where Shiny Server and R Studio install to, which I have active.

It's also worth noting, when I go to my IP:8787 or IP:3838 they both work fine, it just seems to be my main IP that isn't working properly. Could it have to do with me having installed an SSL certificate on the domain names yesterday?

  • @habbywall

    The IP is working fine. It has something to do with the ports of your web server, so yes, this is very likely something to do with you activating a SSL certificate.

    Did you use Let's Encrypt or did you buy the certificate?
    How did you install the certificate?

    Can you connect to your server through https? Try https://Your-IP-or-Domain/

    And run this command too to list which ports are being listened on:

    sudo lsof -iTCP -sTCP:LISTEN -P
    
    • I installed using certbot and Let's Encrypt with these directions:
      https://www.digitalocean.com/community/tutorials/how-to-secure-apache-with-let-s-encrypt-on-ubuntu-14-04

      I only encrypted the domain itself, not the IP, is there a way to do both, or should I have done both? When I try using https:// before my domain and IP it still doesn't work.

      data is my subdomain, after running that command it looks like there's a lot going on with it? Could this be part of the cause of it running through my ram?

      When I run that command I get this:

      COMMAND    PID           USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
      shiny-ser 1364           root   11u  IPv4  17077      0t0  TCP *:3838 (LISTEN)
      sshd      1391           root    3u  IPv4  14173      0t0  TCP *:22 (LISTEN)
      sshd      1391           root    4u  IPv6  14175      0t0  TCP *:22 (LISTEN)
      rserver   1441 rstudio-server    6u  IPv4  14755      0t0  TCP *:8787 (LISTEN)
      master    1773           root   12u  IPv4  16565      0t0  TCP localhost:25 (LISTEN)
      master    1773           root   13u  IPv6  16566      0t0  TCP ip6-localhost:25 (LISTEN)
      mysqld    2439          mysql   36u  IPv4  21607      0t0  TCP localhost:3306 (LISTEN)
      apache2   2527           root    4u  IPv6  22029      0t0  TCP *:80 (LISTEN)
      apache2   2527           root    6u  IPv6  22033      0t0  TCP *:443 (LISTEN)
      apache2   2530       www-data    4u  IPv6  22029      0t0  TCP *:80 (LISTEN)
      apache2   2530       www-data    6u  IPv6  22033      0t0  TCP *:443 (LISTEN)
      apache2   2531       www-data    4u  IPv6  22029      0t0  TCP *:80 (LISTEN)
      apache2   2531       www-data    6u  IPv6  22033      0t0  TCP *:443 (LISTEN)
      apache2   2532       www-data    4u  IPv6  22029      0t0  TCP *:80 (LISTEN)
      apache2   2532       www-data    6u  IPv6  22033      0t0  TCP *:443 (LISTEN)
      apache2   2533       www-data    4u  IPv6  22029      0t0  TCP *:80 (LISTEN)
      apache2   2533       www-data    6u  IPv6  22033      0t0  TCP *:443 (LISTEN)
      apache2   2534       www-data    4u  IPv6  22029      0t0  TCP *:80 (LISTEN)
      apache2   2534       www-data    6u  IPv6  22033      0t0  TCP *:443 (LISTEN)
      apache2   2608       www-data    4u  IPv6  22029      0t0  TCP *:80 (LISTEN)
      apache2   2608       www-data    6u  IPv6  22033      0t0  TCP *:443 (LISTEN)
      
      
      This tutorial will show you how to set up a free TLS/SSL certificate from Let’s Encrypt on a Ubuntu 14.04 server running Apache as web server. SSL certificates are used within web servers to encrypt the traffic between server and client, providing extra security for users accessing your application.
      • @habbywall I've found the problem. Apache is only listening on IPv6.
        @jtittle I must admit my Apache skills are too bad to figure this out. Do you know what it could be?

        • @habbywall @hansen

          I'd check to see how Apache is setup with the Listen. If IPv6 is specified before IPv4, or if IPv4 isn't specified at all, then that could be the issue.

          Normally, you'd have something binding to 0.0.0.0 or to a specific IP, depending on the setup.

          i.e.

          Listen 0.0.0.0
          
          • I'm trying to run that command, both with 0.0.0.0 and my IP(107.170.1.26) and it's coming back with command not found, am I doing something wrong to enter it?

@habbywall

If you're seeing:

caught SIGTERM, shutting down

There's really only two reasons for that -- that's either from a restart (i.e. you restarted Apache from the command line with something such as service apache2 restart), or Apache is crashing and the service is handling an auto-restart for you.

Since it doesn't appear to be overly frequent, I wouldn't think that Apache is crashing, but it could very well be. Accessing Apache over the non-standard ports probably won't cause it to crash since primary traffic would be routed to 80/443 by default unless you make the other ports known to the public in some way.

...

I'd take a look at the output of top. I'd run top from the CLI and then hit SHIFT + M, then c, then e. That'll sort the process listing and make the resource usage human-readable.

The fourth line from the top of top shows memory usage -- which you could also get by using:

free -mh

If you can post the output of that command above, while Apache is running, we can take a look at the RAM usage and see if anything looks odd there.

...

You may very well need to upgrade so that you have more RAM available. Anytime RAM is limited and can not be freed through normal channels, services can begin to crash. Some will try to restart, some won't -- it really depends on the service script.

If it is the service script doing the restarts, that's not a bad thing, it's working as intended, but it may be doing it because of a lack of RAM.

  • I did free -mh and got this:

                  total        used        free      shared  buff/cache   available
    Mem:           992M        312M        307M         22M        371M        504M
    Swap:            0B          0B          0B
    
    
  • @jtittle Excellent point, I actually didn't consider RAM, but looking at the amount of things running on 1GB, it might very well be the problem.

    • Is it possible for me to just remove the wordpress files/databases for two of the sites, and then see if that clears some of it up? Since they don't have any actual content yet it wouldn't really hurt anything to try.

@habbywall
You need to edit your Apache Vhost configuration, which is located in /etc/apache/sites-enabled/.
In the files, you should find the Listen parameter and change it to Listen 0.0.0.0.
And then restart Apache with this command sudo service apache restart

@habbywall

The Listen directive would be in your Apache configuration.

It may be set as one of the following, it all depends on the repository or, if you modified the config, what you set it as:

Listen 0.0.0.0
Listen 80

In the first example, it'll only listen on IPv4, in the second, on both IPv4 and IPv6.

@habbywall

I deployed a fresh Ubuntu 16.04 Droplet to check the configuration for Apache as it tends to change from time to time.

The Listen directive should be defined in /etc/apache2/ports.conf, which should look like:

Listen 80

<IfModule ssl_module>
        Listen 443
</IfModule>

<IfModule mod_gnutls.c>
        Listen 443
</IfModule>

If any of those are setup with an IP, or if any Listen directive is setup with an IP elsewhere, then that may be the issue. Only specifying the port leads to listening on both IPv4 and IPv6 IP's. If you specify an IP address, then it will only listen on the protocol for that IP.

You can run:

grep "Listen" /etc/apache2/*.conf

To see if any files, other than ports.conf, have a Listen directive setup. If they did, check that file and that should lead to us getting this fixed.

@jtittle
I checked the ports.conf and did the grep command, but it was only in the ports.conf. it looks identical to your file too. I changed it to 0.0.0.0 to see if that would do anything and it didn't.

Another note, is now my error has changed, it's no longer timing out when I try to visit the domain/IP, not it just says connection is being refused.

@jtittle @hansen

I just restarted apache to fix the connection being refused. I have absolutely no idea at what point it was fixed, but a tremendous thank you to you both for helping me with this!!

  • @habbywall

    Always happy to help, though I don't know if I really did anything :-). Either way, I'm happy to hear that it's working -- that's the most important thing!

Have another answer? Share your knowledge.