Installed LetsEncrypt and redirects to https work but web pages are no longer reachable

August 1, 2017 226 views
Let's Encrypt Apache

I installed Let's Encrypt per the following tutorial:

https://www.digitalocean.com/community/tutorials/how-to-secure-apache-with-let-s-encrypt-on-ubuntu-16-04

Works to require a redirect to https but I can no longer reach the index.html web page or any other page on the site. I deleted the lines added by let's encrypt to the .conf file for my site and now the site is reachable. I was trying to setup a password protection of the site per the following tutorial:

https://www.digitalocean.com/community/tutorials/how-to-set-up-password-authentication-with-apache-on-ubuntu-16-04

But instead of receiving a prompt for a username and password, I just received the following message in Google Chrome (and also could not reach pages in IE):


Not Found

The requested URL / was not found on this server.

Apache/2.4.18 (Ubuntu) Server at mysitename.com Port 443


the "mysitename.com" is just edited to not include my actual domain.

Any suggestions will be appreciated.

3 Answers
hansen August 5, 2017
Accepted Answer

@redsteamsoftware

The Let's Encrypt install shouldn't mess with the listeners, but if the configuration was a bit off to begin with, then the wizard sometimes does strange things.

Run this command to list which configuration files Apache is using:

sudo apache2ctl -S

And then post each configuration file it lists and /etc/apache2/apache.conf

  • Below is the printout from the command:

    VirtualHost configuration:
    *:80                   is a NameVirtualHost
             default server sitename1.com (/etc/apache2/sites-enabled/sitename1.com.conf:1)
             port 80 namevhost sitename1.com (/etc/apache2/sites-enabled/sitename1.com.conf:1)
                     alias www.sitename1.com
             port 80 namevhost sitename2.com (/etc/apache2/sites-enabled/sitename2.com.conf:1)
                     alias www.sitename2.com
                     alias sitename2a.com
                     alias www.sitename2a.com
             port 80 namevhost sitename3.com (/etc/apache2/sites-enabled/sitename3.com.conf:1)
                     alias www.sitename3.com
    *:443                  is a NameVirtualHost
             default server sitename3.com (/etc/apache2/sites-enabled/sitename3.com-le-ssl.conf:2)
             port 443 namevhost sitename3.com (/etc/apache2/sites-enabled/sitename3.com-le-ssl.conf:2)
                     alias www.sitename3.com
             port 443 namevhost sitename3.com (/etc/apache2/sites-enabled/sitename3.com.conf:34)
                     alias www.sitename3.com
    ServerRoot: "/etc/apache2"
    Main DocumentRoot: "/var/www/html"
    Main ErrorLog: "/var/log/apache2/error.log"
    Mutex default: dir="/var/lock/apache2" mechanism=fcntl
    Mutex mpm-accept: using_defaults
    Mutex watchdog-callback: using_defaults
    Mutex rewrite-map: using_defaults
    Mutex ssl-stapling-refresh: using_defaults
    Mutex ssl-stapling: using_defaults
    Mutex ssl-cache: using_defaults
    PidFile: "/var/run/apache2/apache2.pid"
    Define: DUMP_VHOSTS
    Define: DUMP_RUN_CFG
    User: name="www-data" id=33
    Group: name="www-data" id=33
    

    Following is the /etc/apache2/sites-enabled/sitename3.com.conf config file:
    (The other .conf files for this site that have a .com-le-ssl.conf at the end I did not make)

    <VirtualHost *:80>
            # The ServerName directive sets the request scheme, hostname and port t$
            # the server uses to identify itself. This is used when creating
            # redirection URLs. In the context of virtual hosts, the ServerName
            # specifies what hostname must appear in the request's Host: header to
            # match this virtual host. For the default virtual host (this file) this
            # value is not decisive as it is used as a last resort host regardless.
            # However, you must set it for any further virtual host explicitly.
            #ServerName www.example.com
    
            ServerAdmin admin@sitename2.com
            ServerName sitename3.com
            ServerAlias www.sitename3.com
            DocumentRoot /var/www/sitename3.com/html
    
            # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
            # error, crit, alert, emerg.
            # It is also possible to configure the loglevel for particular
            # modules, e.g.
            #LogLevel info ssl:warn
    
            ErrorLog ${APACHE_LOG_DIR}/error.log
            CustomLog ${APACHE_LOG_DIR}/access.log combined
    
            # For most configuration files from conf-available/, which are
            # enabled or disabled at a global level, it is possible to
            # include a line for only one particular virtual host. For example the
            # following line enables the CGI configuration for this host only
            # after it has been globally disabled with "a2disconf".
            #Include conf-available/serve-cgi-bin.conf
    
    </VirtualHost>
    
    <VirtualHost *:443>
    
            ServerAdmin admin@sitename2.com
            ServerName sitename3.com
            ServerAlias www.sitename3.com
            DocumentRoot /var/www/sitename3.com/html
    
            ErrorLog ${APACHE_LOG_DIR}/error.log
            CustomLog ${APACHE_LOG_DIR}/access.log combined
    
    </VirtualHost>
    
    # vim: syntax=apache ts=4 sw=4 sts=4 sr noet
    
    • @redsteamsoftware

      Your sitename3.com.conf should not have the <VirtualHost *:443> section in the bottom, since it's probably being done by sitename3.com-le-ssl.conf
      And I still need to see /etc/apache2/apache.conf

      Otherwise I would probably recommend that you remove all the Let's Encrypt stuff and the new Apache configurations, and make sure your Apache works on IPv4 port 80.
      Then you can try doing Let's Encrypt again.

      • @hansen

        So you were right. The problem was that in the .conf file called:
        sitename3.com-le-ssl.conf ,which was made by let's encrypt, the DocumentRoot was:

        DocumentRoot /var/www/public_html

        and not:

        DocumentRoot /var/www/sitename3.com/html

        as it needed to be since I'm doing virtual hosts.

        So it is resolved now and works. Just to recap, I removed the <VirtualHost> section from my sitename3.com.conf since it is in the .conf file that Let's Encrypt made which is:

        sitename3.com-le-ssl.conf

        but in the sitename3.com-le-ssl.conf file, I just changed the document root to the correct one which is not the default one.

        Thanks very much for your assistance with this. As you can tell, I'm new to Linux and apache and much appreciate the guidance given.

Hi @redsteamsoftware

Did you check if HTTPS worked before redirecting? If not, then it could just be your firewall.

sudo ufw status

But without seeing your VirtualHost configurations (both), then it's difficult to know the exact problem. Post them and hide your domain if you don't want to share it.

  • Hi @hansen

    Thank you for the reply. I did the command you listed and it says:

    Status: inactive

    I have not configured any firewalls yet in the Networking section of the control panel, so this might be the cause of the problem. I'll get up to speed on the firewalls and see if that rectifies the issue and post back.

    EDIT: I created a FW in the Control Panel and applied it to my droplet. Also enabled the Apache FW through my console. I do need to get up to speed on the FW features of Apache before I can ascertain what settings are required. I'll do that and post back.

    • @redsteamsoftware

      I would highly recommend not using two firewalls, since you might forget one. I would probably recommend that you use the firewall in the control panel, since it's a bit easier to use.

      EDIT: So disable the one on the droplet:

      sudo ufw disable
      
      • Thank you for the reply @Hansen .

        I did get it working partially by changing from "Configuring Access Control within the Virtual Host Definition" to "Configuring Access Control with .htaccess Files"

        I followed the instructions for that in the following Tutorial; I believe these instructions are the same as in the original tutorial I was following. I will disable the apache firewall though as you have suggested and stick with the firewall in the control panel. One thing to note for anyone following this, is that it is still not working for HTTPS traffic, just HTTP, which for my needs are fine for now. I believe I have the firewall allowing in HTTPS traffic however.

        https://www.digitalocean.com/community/tutorials/how-to-set-up-password-authentication-with-apache-on-ubuntu-14-04

        When setting up a web server, there are often sections of the site that you wish to restrict access to. Web applications often provide their own authentication and authorization methods, but the web server itself can be used to restrict access if these are inadequate or...
        • @redsteamsoftware

          You need to allow port 443 for https traffic. Once you've done that, then let's check if Apache is even listening on that port by running this command:

          sudo lsof -iTCP -sTCP:LISTEN -P
          
          • Hi @hansen

            Port 443 is allowed through my firewall and I ran the above listed command.

            Port 443 is listed on there multiple times. One example line is as follows:

            COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
            apache2 12322 www-data 6u IPv6 32901260 0t0 TCP *:443 (LISTEN)

            Thank you very much for your generous assistance with this. I'm new to Linux in general and apache. I do appreciate the time you are taking with this.

@redsteamsoftware

But the line you've posted says that Apache is only listening on 443 with IPv6.
Use the </> button in the comment editor to insert console output, which doesn't lose it's formatting.

  • @hansen

    Can't figure out how to copy the entire listing of the command from the console window but I can confirm the only port listed for IPv4 is 22 when I run that command.

    • @redsteamsoftware

      Are you using PuTTY? If yes, then when you select something with your mouse, then it automatically copies it.

      But that means that your VirtualHost configuration for 443 is probably specified to listen to IPv6-only, but you can change it to this:

      <VirtualHost *:443>
      
      • @hansen

        Not using PuTTY here, I use it often at work though, just using the DO console.

        Anyway, I did modify the .conf file and added an appropriate section for <VirtualHost *:443>, but this might be more complicated, because it is still not working. I think it is correct that the apache2 is only listening for IPv6 on ports 80 and 443. I found this documentation at the apache site and will dig into this a bit further and let you know what I find:

        https://httpd.apache.org/docs/2.4/bind.html

        • @redsteamsoftware

          Okay, then connect with PuTTY. The web console should only be used if you have no other means to connect, since it's very limited for doing anything.

          If you have defined your Bind interface from the default, then yes, that might be the problem. Just leave it at the default, since it will use all interfaces.
          And if you've specified Listen to something other than the default, then revert that too.

          By default, it should just work, so you must have defined something somewhere to limit it to your IPv6.

          Otherwise you need to post all your Apache configurations here, so we can narrow down the problem.

          • COMMAND   PID     USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
            sshd     1511     root    3u  IPv4    15570      0t0  TCP *:22 (LISTEN)
            sshd     1511     root    4u  IPv6    15584      0t0  TCP *:22 (LISTEN)
            apache2 22234     root    4u  IPv6 32945707      0t0  TCP *:80 (LISTEN)
            apache2 22234     root    6u  IPv6 32945711      0t0  TCP *:443 (LISTEN)
            apache2 22237 www-data    4u  IPv6 32945707      0t0  TCP *:80 (LISTEN)
            apache2 22237 www-data    6u  IPv6 32945711      0t0  TCP *:443 (LISTEN)
            apache2 22238 www-data    4u  IPv6 32945707      0t0  TCP *:80 (LISTEN)
            apache2 22238 www-data    6u  IPv6 32945711      0t0  TCP *:443 (LISTEN)
            apache2 22239 www-data    4u  IPv6 32945707      0t0  TCP *:80 (LISTEN)
            apache2 22239 www-data    6u  IPv6 32945711      0t0  TCP *:443 (LISTEN)
            apache2 22240 www-data    4u  IPv6 32945707      0t0  TCP *:80 (LISTEN)
            apache2 22240 www-data    6u  IPv6 32945711      0t0  TCP *:443 (LISTEN)
            apache2 22241 www-data    4u  IPv6 32945707      0t0  TCP *:80 (LISTEN)
            apache2 22241 www-data    6u  IPv6 32945711      0t0  TCP *:443 (LISTEN)
            apache2 22244 www-data    4u  IPv6 32945707      0t0  TCP *:80 (LISTEN)
            apache2 22244 www-data    6u  IPv6 32945711      0t0  TCP *:443 (LISTEN)
            apache2 22255 www-data    4u  IPv6 32945707      0t0  TCP *:80 (LISTEN)
            apache2 22255 www-data    6u  IPv6 32945711      0t0  TCP *:443 (LISTEN)
            apache2 22257 www-data    4u  IPv6 32945707      0t0  TCP *:80 (LISTEN)
            apache2 22257 www-data    6u  IPv6 32945711      0t0  TCP *:443 (LISTEN)
            apache2 22259 www-data    4u  IPv6 32945707      0t0  TCP *:80 (LISTEN)
            apache2 22259 www-data    6u  IPv6 32945711      0t0  TCP *:443 (LISTEN)
            apache2 22260 www-data    4u  IPv6 32945707      0t0  TCP *:80 (LISTEN)
            apache2 22260 www-data    6u  IPv6 32945711      0t0  TCP *:443 (LISTEN)
            mysqld  27509    mysql   24u  IPv4 27897985      0t0  TCP localhost:3306 (LISTEN)
            
            

            Ok, connected with PuTTY and pasted in the entire contents (listed above), although I did not specify any LISTEN directives as of yet, I did install LetsEncrypt and obtained the certificate so perhaps something was done on that end.

Have another answer? Share your knowledge.