NGINX reverse proxy to Apache with Wordpress bad gateway

March 14, 2017 5.1k views
Apache Nginx WordPress CentOS

I have followed the guide here (https://www.digitalocean.com/community/tutorials/how-to-configure-nginx-as-a-web-server-and-reverse-proxy-for-apache-on-one-ubuntu-16-04-server)

While they are using Ubuntu we are using Centos 7.2.

Going directly to apache on port 8080 or 8443 (ssl) works. Pulling static files from Nginx works as well on port 80 and 443. However once the configuration is updated to use pass php traffic to apache I get 502 Bad Gateway. I

Nginx Config

server {
    listen 80;
    listen 443 ssl;
    server_name www.domain.com domain.com;  

    ssl on;
    ssl_protocols       TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers         ECDHE-RSA-AES256-SHA384:AES256-SHA256:RC4:HIGH:!MD5:!aNULL:!eNULL:!NULL:!DH:!EDH:!AESGCM;
    ssl_session_cache   shared:SSL:5m;
    ssl_session_timeout 10m;
    ssl_prefer_server_ciphers on;
    ssl_certificate /etc/nginx/ssl/domain_com.pem;
    ssl_certificate_key /etc/nginx/ssl/domain_com.key;

    root /data/web/corp/domain;
    index index.php index.htm index.html;

    location / {
        try_files $uri $uri/ /index.php;
    }

    location ~ \.php$ {
        proxy_pass https://127.0.0.1:8443;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    location ~ /\. {
        deny all;
    }
}

I have tried to set the proxy_pass to the internal IP, Public IP, domain name, and loopback ip. Seems to all be the same issue.

Apache is set to listen on 8080 and 443.

The Nginx error logs show this:

2017/03/14 23:07:52 [error] 24934#0: *306 upstream prematurely closed connection while reading response header from upstream, client: 71.25.9.17, server: www.domain.com, request: "GET / HTTP/1.1", upstream: "https://127.0.0.1:8443/index.php", host: "www.domain.com"

Apache SSL error log:

[Tue Mar 14 23:12:36.955268 2017] [ssl:error] [pid 63873] [client 127.0.0.1:45822] AH02261: Re-negotiation handshake failed: Not accepted by client!?, referer: https://www.domain.com/favicon.ico

Apache SSL access log:

127.0.0.1 - - [14/Mar/2017:23:12:16 +0000] "GET /index.php HTTP/1.0" 403 211

I am not sure where to start as I am new to Nginx but from my reading it seems to be much better/faster at serving up static content than Apache. While our development group currently requires .htaccess we have to keep using Apache.

10 comments
  • Ok so I rebuilt the server from scratch and followed the guide exactly including using the same OS.

    I was able to install wordpress and I can get to the backend (admin panel) but I am unable to view any pages if I make them. The error is too many redirects.

    I have added in the wordpress suggested by @stephenafamo and even updated the proxy_pass line into the nginx config suggested by @hansen.

    I am unable to figure out is why I am unable to see any pages or posts that are made.

    Here is the new configs:
    Nginx

    server {
        listen 80;
        listen 443 ssl;
        server_name miosystems.com www.miosystems.com;
    
        ssl on;
        ssl_certificate /etc/nginx/ssl/miosystems_com.pem;
        ssl_certificate_key /etc/nginx/ssl/miosystems_com.key.pem;
        ssl_ciphers         ECDHE-RSA-AES256-SHA384:AES256-SHA256:RC4:HIGH:!MD5:!aNULL:!eNULL:!NULL:!DH:!EDH:!AESGCM;
        ssl_session_cache   shared:SSL:5m;
        ssl_session_timeout 10m;
        ssl_prefer_server_ciphers on;
    
        root /data/web/mobeus/miosystems;
        index index.php index.htm index.html;
    
    
        location / {
            try_files $uri $uri/ /index.php;
        }
    
        location ~ \.php$ {
            proxy_pass http://127.0.0.1:8080;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
    
        location ~ /\. {
            deny all;
        }
    }
    

    Apache vhost config

    <VirtualHost *:8080>
        ServerName miosystems.com
        ServerAlias www.miosystems.com
        DocumentRoot /data/web/mobeus/miosystems
        <Directory /data/web/mobeus/miosystems>
            AllowOverride All
            Options +ExecCGI +FollowSymLinks +Includes +IncludesNOEXEC +MultiViews +SymLinksIfOwnerMatch
            Require all granted
         </Directory>
    
    # compress text, HTML, JavaScript, CSS, and XML
      AddOutputFilterByType DEFLATE text/plain
      AddOutputFilterByType DEFLATE text/html
      AddOutputFilterByType DEFLATE text/xml
      AddOutputFilterByType DEFLATE text/css
      AddOutputFilterByType DEFLATE text/javascript
      AddOutputFilterByType DEFLATE text/js
      AddOutputFilterByType DEFLATE text/x-javascript
      AddOutputFilterByType DEFLATE image/jpg
      AddOutputFilterByType DEFLATE image/png
      AddOutputFilterByType DEFLATE image/gif
      AddOutputFilterByType DEFLATE image/jpeg
      AddOutputFilterByType DEFLATE application/xml
      AddOutputFilterByType DEFLATE application/xhtml+xml
      AddOutputFilterByType DEFLATE application/rss+xml
      AddOutputFilterByType DEFLATE application/javascript
      AddOutputFilterByType DEFLATE application/x-javascript
    
    DeflateFilterNote Input instream
    DeflateFilterNote Output outstream
    DeflateFilterNote Ratio ratio
    
    # remove browser bugs
      BrowserMatch ^Mozilla/4 gzip-only-text/html
      BrowserMatch ^Mozilla/4\.0[678] no-gzip
      BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
      Header append Vary User-Agent
    
    </VirtualHost>
    
    

    Nginx error log is empty

    Nginx Access log

    173.8.247.161 - - [16/Mar/2017:19:32:44 -0600] "GET / HTTP/1.1" 301 5 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/602.4.8 (KHTML, like Gecko) Version/10.0.3 Safari/602.4.8"
    

    Apache access log

    miosystems.com:80 127.0.0.1 - - [16/Mar/2017:19:32:44 -0600] "GET /index.php HTTP/1.0" 301 215 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/602.4.8 (KHTML, like Gecko) Version/10.0.3 Safari/602.4.8"
    

    Apache Error log

    [Thu Mar 16 19:31:16.426598 2017] [:notice] [pid 30243:tid 140643587030528] FastCGI: process manager initialized (pid 30243)
    [Thu Mar 16 19:31:16.428237 2017] [mpm_event:notice] [pid 30240:tid 140643587030528] AH00489: Apache/2.4.18 (Ubuntu) mod_fastcgi/mod_fastcgi-SNAP-0910052141 configured -- resuming normal operations
    [Thu Mar 16 19:31:16.428298 2017] [core:notice] [pid 30240:tid 140643587030528] AH00094: Command line: '/usr/sbin/apache2'
    

    php-fpm log:

    mtvmw09:/data/logs# cat php7.0-fpm.log 
    [16-Mar-2017 19:43:19] NOTICE: fpm is running, pid 1500
    [16-Mar-2017 19:43:19] NOTICE: ready to handle connections
    [16-Mar-2017 19:43:19] NOTICE: systemd monitor interval set to 10000ms
    

    I am at a loss to figure out why this is not working. Any Ideas

  • @joshopkins
    Is there any reason why you're using both Nginx and Apache?

    If I visit http://miosystems.com I get the following error:

    400 Bad Request
    The plain HTTP request was sent to HTTPS port
    

    So it seems like Nginx is still using a proxy that ends in HTTPS.
    If I visit https://miosystems.com I get a too many redirects, like you.

    Can you rename/remove the .htaccess in the main WordPress folder? Since that might be causing the extra redirects.

    EDIT: The Apache access log says miosystems.com:80 which indicates it's listening on port 80, not 8080. That's weird.

    @jtittle Have you seen this problem before?

  • @hansen The reason is some of our applications still require the .htaccess rules but are slowing being redeveloped so they can be relaced. Nginx is much faster with serving static content which is why we are using both.

    With the fresh install of wordpress the .htaccess only contains:

    <IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteBase /
    RewriteRule ^index\.php$ - [L]
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteRule . /index.php [L]
    </IfModule>
    

    removing the .htaccess has no affect.

    I haven't figured out how to put the redirect to force ssl only but if you go to https://miosystems.com/wp-admin you get the login page. If you pull up https://miosystems.com/info.php you get the phpinfo page. This only seems to be an issue with any page on the frontend of wordpress. I made a test page https://miosystems.com/test but it will not pull up either.

  • @joshopkins
    The main reason why Apache is slow, is the htaccess and htpasswd files. Nginx doesn't allow dynamic configuration like that, which is why it's much faster.

    If you go to https://miosystems.com/wp-admin/options-general.php and edit both URLs to https://miosystems.com - does that fix it?

    To force HTTPS, just change the top part of the Nginx config to this:

    server {
        listen 80;
        server_name miosystems.com www.miosystems.com;
        return 301 https://$server_name$request_uri;
    }
    server { #This is the current part, but removed "listen 80;"
        listen 443 ssl;
        server_name miosystems.com www.miosystems.com;
    
  • @hansen
    Both urls are already listed as https://miosystems.com on that page.

    Thanks for the code I have put it in place.

    No errors in the logs either. Access logs show it bouncing back and forth. What I don't understand is why the backend works but the frontend doesn't. My only thought would be that nothing on the backend is index.php. Every page is defined as wp-login.php and the like.

    I removed the index.php from the index line and it came up with the index.html test page created during the setup of the systems while following the guide.

    I have also noticed that when trying to go to https://miosystems.com/index.php/test/ to see if it would pull up the test page it just default right back to https://miosystems.com

    It also seems like ssl is not complete as when going to a static file or to the wordpress admin I get the green ssl cert notice but when just going to https://miosystems.com it says it is unsecure.

  • @joshopkins
    Can you install this plugin into WordPress: https://wordpress.org/plugins/better-search-replace/
    Then run a search+replace of http://miosystems.com with https://miosystems.com
    And http://www.miosystems.com with https://miosystems.com

    If the WordPress backend is working, then this problem is very likely related to a plugin, theme or other custom code.

  • @hansen @stephenafamo @jtittle
    This is fresh with no content other than what comes with wordpress other than the test page https://miosystems.com/test and a single image I uploaded to test serving static content https://miosystems.com/wp-content/uploads/haven.png.

    Installed the plugin and ran the searches it found nothing. Manually checked the database directly as well with the same results.

    I was able to make some headway last night by playing with the wp-config.php file.

    I added the following:

    // If WordPress is behind reverse proxy 
    // which proxies https to http
    if ( (!empty( $_SERVER['HTTP_X_FORWARDED_HOST'])) ||
         (!empty( $_SERVER['HTTP_X_FORWARDED_FOR'])) ) { 
    
        // http://wordpress.org/support/topic/wordpress-behind-reverse-proxy-1
        $_SERVER['HTTP_HOST'] = $_SERVER['HTTP_X_FORWARDED_HOST'];
    
        define('WP_HOME', 'https://miosystems.com/');
        define('WP_SITEURL', 'https://miosystems.com');
    
        // http://wordpress.org/support/topic/compatibility-with-wordpress-behind-a-reverse-proxy
        $_SERVER['HTTPS'] = 'on';
    }
    

    My only issue now it seems that while I can see what is set as the default main page I am unable to now navigate to any other page on the frontend such as https://miosystems.com/test (which is the only other page). However any url that is not index.php works i.e. https://miosystems.com/wp-login.php

  • @joshopkins Try replacing your Nginx config from

        location / {
            try_files $uri $uri/ /index.php;
        }
    

    to this

        location / {
            try_files $uri $uri/ /index.php?$args;
        }
    
  • @hansen
    I tried that as well as

    location / {
            try_files $uri $uri/ /index.php?$is_args$args;
        }
    
  • @hansen @jtittle

    I am not sure of this is an apache issue or nginx issue. I added back the .htaccess file with the basic permalink structure that wordpress gives but it doesnt make a difference. At this point I am almost willing to pay someone to get this to work. So aggravating.

    Thanks for all the help you all have already given and for any more ideas you may have for me.

6 Answers

I'm pretty sure your problem comes from using HTTPS through localhost.

Since all the traffic is contained on the same server between Nginx and Apache, then you don't really need to encrypt that traffic.

And remember to setup Apache to listen on 127.0.0.1 (known as localhost), and just HTTP port 8080. You don't want Apache to listen on your public IP - otherwise there would be multiple ways into your system, which wouldn't be good.

Try this instead in your Nginx php-section:

proxy_pass http://127.0.0.1:8080;

I had a battle with this myself.
Add the following lines to wp-config.php

// If we're behind a proxy server and using HTTPS, we need to alert Wordpress of that fact
// see also http://codex.wordpress.org/Administration_Over_SSL#Using_a_Reverse_Proxy
if (isset($_SERVER['HTTP_X_FORWARDED_PROTO']) && $_SERVER['HTTP_X_FORWARDED_PROTO'] === 'https') {
    $_SERVER['HTTPS'] = 'on';
}

It should come just before

/* That's all, stop editing! Happy blogging. */

Getting a little closer. But have a different error now. Now I am getting a forbidden error access denied.

Forbidden

You don't have permission to access /index.php on this server.

I have updated the user and group on the directory and files to apache:apache. I have even for testing given them all 777 permissions. Still no luck. Apache is running as user and group apache.

.htaccess file


# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L] 

#START - WP Hide & Security Enhancer
#WriteCheckString:1489014293_56908
<IfModule mod_env.c>
SetEnv HTTP_MOD_REWRITE On
</IfModule>

RewriteRule ^modules/(.+) /wp-content/plugins/$1 [L,QSA]

RewriteRule ^assets/(.+) /wp-includes/$1 [L,QSA]

RewriteRule ^site_uploads/(.+) /wp-content/uploads/$1 [L,QSA]
RewriteCond %{ENV:REDIRECT_STATUS} ^$
RewriteCond %{REQUEST_FILENAME} -f
RewriteCond %{REQUEST_FILENAME} !wp-activate.php [NC]
RewriteCond %{REQUEST_FILENAME} !wp-cron.php [NC]
RewriteCond %{REQUEST_FILENAME} !wp-signup.php [NC]
RewriteCond %{REQUEST_FILENAME} !wp-comments-post.php [NC]
RewriteCond %{REQUEST_FILENAME} !wp-login.php [NC]
RewriteRule ^wp-([a-z-])+.php /index.php [L]
RewriteCond %{ENV:REDIRECT_STATUS} ^$
RewriteRule ^readme.html /index.php [L]

RewriteRule ^user-input.php /wp-comments-post.php [L,QSA]
RewriteCond %{ENV:REDIRECT_STATUS} ^$
RewriteRule ^license.txt /index.php [L]

RewriteRule ^content/(.+) /wp-content/$1 [L,QSA]

                            <FilesMatch "">
                                <IfModule mod_headers.c>
                                    Header unset X-Powered-By
                                </IfModule>
                            </FilesMatch>

                            <FilesMatch "">
                                <IfModule mod_headers.c>
                                    Header unset X-Pingback
                                </IfModule>
                            </FilesMatch>
#END - WP Hide & Security Enhancer

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

# END WordPress

@joshopkins @hansen

By chance, are you using CloudFlare? When you mentioned the too many redirects error, CloudFlare immediately came to mind as that's a very common error, but extremely easy to fix.

With CloudFlare, you need to login and head over to the SSL page, then set the configuration to use Full (Strict). Allow it to resolve and that should fix the issue.

As far as the 400 error, in your NGINX configuration, comment out:

ssl on;

So that it looks like:

#ssl on;

Once that change is made, reload or restart NGINX and let me know if you're still having issues.

@joshopkins

If you'd like to shoot me an e-mail, you can find it in my profile (click on my username). I can make a few suggestions and we can go from there depending on what you'd like to do.

Did you ever get this resolved? After hours of tinkering I have been able to get this working because of 1 simple missing "s".. yes an S... If anyone else is having this issue, I'll post my configs for review... Once I added that s, it worked like a charm.. One issue I am stil having is with .htaccess files.. Apparently the proxy to apache from nginx does not include the ip traffic is originally coming from and this causes deny,allow issues.. I have not resolved this yet but the site is now accessible!! Anything in bold you should probably be changing to your own settings... :)

-------------- NGINX -> ** /etc/nginx/sites-available/domainx.conf**

server {
listen 80;
servername xxxxx.domainx.com;
return 301 https://$host$request
uri;
}

server {
listen 443 ssl;
servername xxxxx.domainx.com;
ssl
certificate /etc/nginx/ssl/cert_domainx.crt;
sslcertificatekey /etc/nginx/ssl/cert_domainx.key;
location / {
proxypass https://192.168.0.100:6443;
proxy
setheader X-Real-IP $remoteaddr;
proxysetheader X-Forwarded-For $proxyaddxforwardedfor;
proxysetheader X-Forwarded-Proto https;
proxysetheader X-Forwarded-Port 443;
proxysetheader Host $host;
}
}

*-------------- APACHE -> * ports.conf

Listen 8085

<IfModule ssl_module>
Listen 6443
</IfModule>

<IfModule mod_gnutls.c>
Listen 6443
</IfModule>

*-------------- APACHE -> * 000-default.conf

<VirtualHost *:**8085**>
ServerName xxxxx.domainx.com
Redirect permanent / https://xxxxx.domainx.com/
</VirtualHost>

<VirtualHost *:**6443**>
# The ServerName directive sets the request scheme, hostname and port that
# the server uses to identify itself. This is used when creating
# redirection URLs. In the context of virtual hosts, the ServerName
# specifies what hostname must appear in the request's Host: header to
# match this virtual host. For the default virtual host (this file) this
# value is not decisive as it is used as a last resort host regardless.
# However, you must set it for any further virtual host explicitly.
ServerName xxxxx.domainx.com

    ServerAdmin **webmaster@localhost**
    DocumentRoot /var/www/html

    SSLEngine on
    SSLCertificateFile **/etc/apache2/ssl/cert_domainx.crt**
    SSLCertificateKeyFile **/etc/apache2/ssl/cert_domainx.key**
    SSLCACertificateFile **/etc/apache2/ssl/cert_intermediate.crt**

    # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
    # error, crit, alert, emerg.
    # It is also possible to configure the loglevel for particular
    # modules, e.g.
    #LogLevel info ssl:warn

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined

    # For most configuration files from conf-available/, which are
    # enabled or disabled at a global level, it is possible to
    # include a line for only one particular virtual host. For example the
    # following line enables the CGI configuration for this host only
    # after it has been globally disabled with "a2disconf".
    #Include conf-available/serve-cgi-bin.conf

</VirtualHost>

Have another answer? Share your knowledge.