Could it be that IPv6 doesn't work as expected on Digitalocean?

September 17, 2018 947 views
IPv6 Nginx

I'm setting up a new server with Ubuntu 18.04.1 Nginx 1.14.0 and PHP 7.2.7 This is the first server that I try to setup with both IPv4 and IPv6 enabled. Everything works fine until I tested a page with a lot of broken links to missing images. It seems to take forever for the server to realise they are missing and respond to the http request. Some missing files give a HTTP status of "502 bad gateway". When I remove the line "listen [::]:443 ssl http2;" in the nginx server conf file the problem is gone. Is it a DNS problem that causes these delays and 502 errors, could it be that IPv6 doesn't work as expected on Digitalocean?

My conclusion from an earlier discussion on Stackoverflow was, that the script in my php app should not be the problem, because my old server (that is only IPv4 enabled) with the exact same page loads very quick. Also because when I remove that line "listen [::]:443 ssl http2;" in the nginx server conf file the problem is gone.

I noticed that when accessing the server through its IPv6 2a03:b0c0:0:1010::190:6001, there is a certificate mismatch notification. This seems strange to me because the nginx server setup (see contents listed below) leads both IPv4 and IPv6 to the same certificate. Accessing the server through its IPv4 https://37.139.19.66 immediately shows https://test.vuyk.eu

The zone file records:

    AAAA    test.vuyk.eu    directs to 2a03:b0c0:0:1010::190:6001 3600
    A   test.vuyk.eu    directs to 37.139.19.66           3600

The hosts file might be a problem, however the problem persists if the hosts file is emptied all together. Here is the contents:


    127.0.0.1 localhost
    ::1 localhost
    2a03:b0c0:0:1010::190:6001 localhost
    #vuykhost2.vuyk.eu is the hostname of the server
    127.0.1.1 vuykhost2.vuyk.eu 
    ::1 ip6-localhost ip6-loopback
    fe00::0 ip6-localnet
    ff00::0 ip6-mcastprefix
    ff02::1 ip6-allnodes
    ff02::2 ip6-allrouters
    ff02::3 ip6-allhosts

Below is the nginx server configuration, again, when I remove the line "listen [::]:443 ssl http2;" everything works fine:

  server {
            listen 443 ssl http2;
            listen [::]:443 ssl http2;
            ssl_certificate /etc/letsencrypt/live/test.vuyk.eu/fullchain.pem;
            ssl_certificate_key /etc/letsencrypt/live/test.vuyk.eu/privkey.pem;
            include snippets/ssl-params.conf;

            server_name test.vuyk.eu;
            root /var/www/vuyk.eu/webroot;
            index index.php index.html index.htm ;

            location / {
                try_files $uri $uri/ /index.php?$args;
            }

            location ~ \.php$ {
                include fastcgi.conf;
                fastcgi_pass unix:/run/php/php7.2-fpm.sock;
            }
    }

nginx.conf

    user www-data;
    worker_processes auto;
    pid /run/nginx.pid;
    include /etc/nginx/modules-enabled/*.conf;

    events {
        worker_connections 2048;
        multi_accept on;
    }

    http {

        ##
        # Basic Settings
        ##

        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        #   keepalive_timeout 65;
        types_hash_max_size 2048;
        # server_tokens off;

        # server_names_hash_bucket_size 64;
        # server_name_in_redirect off;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        ##
        # SSL Settings
        ##

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
        ssl_prefer_server_ciphers on;

        ##
        # Logging Settings
        ##

        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;

        ##
        # Gzip Settings
        ##

        gzip             on;
        gzip_comp_level  2;
        gzip_min_length  1000;
        gzip_proxied     expired no-cache no-store private auth;
        gzip_types       text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

        ##
        # Virtual Host Configs
        ##

        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;
        client_body_buffer_size 10K;
        client_header_buffer_size 1k;
        client_max_body_size 100m;
        large_client_header_buffers 4 8k;
        fastcgi_buffers 16 16k;
        fastcgi_buffer_size 32k;
        fastcgi_read_timeout 500; #gateway probleem
        client_body_timeout 12;
        client_header_timeout 12;
        keepalive_timeout 25;
        send_timeout 10;
    }

A part of the nginx error.log:

    2018/08/30 16:25:27 [error] 29228#29228: *76 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 2a02:a440:91e3:1:4481:654b:a3e8:9617, server: test.vuyk.eu, request: "GET /images/klanten1/JHoogeveen.gif HTTP/2.0", upstream: "fastcgi://unix:/run/php/php7.2-fpm.sock:", host: "test.vuyk.eu", referrer: "https://test.vuyk.eu/portfolio-2"

Messages from the php7.2-fpm.log (there are a lot similar lines)

[30-Aug-2018 16:16:08] WARNING: [pool www] server reached pm.max_children setting (15), consider raising it
[30-Aug-2018 16:16:27] WARNING: [pool www] child 29026, script '/var/www/vuyk.eu/webroot/index.php' (request: "GET /index.php") execution timed out (22.937711 sec), terminating
[30-Aug-2018 16:16:27] WARNING: [pool www] child 29245 exited on signal 15 (SIGTERM) after 20.490546 seconds from start
[30-Aug-2018 16:16:27] NOTICE: [pool www] child 29263 started

Below is the timeline of HTTP requests and replies.

GET https://test.vuyk.eu/portfolio-2 [HTTP/2.0 200 OK 132ms]
GET https://test.vuyk.eu/templates/purity_iii/css/bootstrap.css [HTTP/2.0 200 OK 40ms]
GET https://test.vuyk.eu/templates/system/css/system.css [HTTP/2.0 200 OK 50ms]
GET https://test.vuyk.eu/templates/purity_iii/css/template.css [HTTP/2.0 200 OK 50ms]
GET https://test.vuyk.eu/templates/purity_iii/fonts/font-awesome/css/font-awesome.min.css [HTTP/2.0 200 OK 50ms]
GET https://test.vuyk.eu/templates/purity_iii/css/layouts/corporate.css [HTTP/2.0 200 OK 50ms]
GET https://test.vuyk.eu/media/jui/js/jquery.min.js?48b6d1b3850bca834b403c58682b4579 [HTTP/2.0 200 OK 60ms]
GET https://test.vuyk.eu/media/jui/js/jquery-noconflict.js?48b6d1b3850bca834b403c58682b4579 [HTTP/2.0 200 OK 60ms]
GET https://test.vuyk.eu/media/jui/js/jquery-migrate.min.js?48b6d1b3850bca834b403c58682b4579 [HTTP/2.0 200 OK 60ms]
GET https://test.vuyk.eu/media/system/js/caption.js?48b6d1b3850bca834b403c58682b4579 [HTTP/2.0 200 OK 70ms]
GET https://test.vuyk.eu/plugins/system/t3/base-bs3/bootstrap/js/bootstrap.js? 8b6d1b3850bca834b403c58682b4579 [HTTP/2.0 200 OK 80ms]
GET https://test.vuyk.eu/plugins/system/t3/base-bs3/js/jquery.tap.min.js [HTTP/2.0 200 OK 80ms]
GET https://test.vuyk.eu/plugins/system/t3/base-bs3/js/script.js [HTTP/2.0 200 OK 70ms]
GET https://test.vuyk.eu/plugins/system/t3/base-bs3/js/menu.js [HTTP/2.0 200 OK 70ms]
GET https://test.vuyk.eu/templates/purity_iii/js/script.js [HTTP/2.0 200 OK 70ms]
GET https://test.vuyk.eu/plugins/system/t3/base-bs3/js/nav-collapse.js [HTTP/2.0 200 OK 70ms]
GET https://test.vuyk.eu/templates/purity_iii/css/custom-vuyk.css [HTTP/2.0 200 OK 70ms]
GET https://test.vuyk.eu/images/klanten1/schipper2.gif [HTTP/2.0 502 Bad Gateway 23988ms]
GET https://test.vuyk.eu/images/klanten1/Kuiper.gif [HTTP/2.0 502 Bad Gateway 24038ms]
GET https://test.vuyk.eu/images/klanten1/WindMatch.gif [HTTP/2.0 502 Bad Gateway 24008ms]
GET https://test.vuyk.eu/images/klanten1/Tuinland.gif [HTTP/2.0 502 Bad Gateway 24018ms]
GET https://test.vuyk.eu/images/klanten1/Wezenberg.gif [HTTP/2.0 502 Bad Gateway 24038ms]
GET https://test.vuyk.eu/images/klanten1/Morgenster.gif [HTTP/2.0 502 Bad Gateway 23998ms]
GET https://test.vuyk.eu/images/klanten1/Harrie-boerhof.gif [HTTP/2.0 502 Bad Gateway 24028ms]
GET https://test.vuyk.eu/images/klanten1/Lococensus.gif [HTTP/2.0 502 Bad Gateway 23998ms]
GET https://test.vuyk.eu/images/klanten1/JHoogeveen.gif [HTTP/2.0 502 Bad Gateway 23978ms]
GET https://test.vuyk.eu/images/klanten1/DeDeur.gif [HTTP/2.0 502 Bad Gateway 23988ms]
GET https://test.vuyk.eu/images/klanten1/Runhaar.gif [HTTP/2.0 502 Bad Gateway 23958ms]
GET https://test.vuyk.eu/images/klanten1/Schunselaar-schildersbedrijf.gif [HTTP/2.0 502 Bad Gateway 23948ms]
GET https://test.vuyk.eu/images/klanten1/Capelle.gif [HTTP/2.0 502 Bad Gateway 23958ms]
GET https://test.vuyk.eu/images/klanten1/Distantlake.gif [HTTP/2.0 502 Bad Gateway 24038ms]
GET https://test.vuyk.eu/images/klanten1/Eikenaar.gif [HTTP/2.0 502 Bad Gateway 24018ms]
GET https://test.vuyk.eu/images/klanten1/FFWD.gif [HTTP/2.0 404 Not Found 26274ms]
GET https://test.vuyk.eu/images/klanten1/Veltec.gif [HTTP/2.0 404 Not Found 26791ms]
GET https://test.vuyk.eu/images/klanten1/Heutink.gif [HTTP/2.0 404 Not Found 26811ms]
GET https://test.vuyk.eu/images/klanten1/Lindeboom.gif [HTTP/2.0 404 Not Found 26777ms]
GET https://test.vuyk.eu/images/klanten1/aataxi.gif [HTTP/2.0 404 Not Found 26828ms]
GET https://test.vuyk.eu/images/klanten1/Aewind.gif [HTTP/2.0 404 Not Found 26811ms]
GET https://test.vuyk.eu/images/klanten1/Praatmaatgroep.gif [HTTP/2.0 404 Not Found 26800ms]
GET https://test.vuyk.eu/media/system/css/system.css [HTTP/2.0 200 OK 20ms]
JQMIGRATE: Migrate is installed, version 1.4.1 jquery-migrate.min.js:2:542
GET https://test.vuyk.eu/images/logo.gif [HTTP/2.0 200 OK 20ms]
GET https://test.vuyk.eu/images/reclame-en-communicatie.gif [HTTP/2.0 200 OK 20ms]
GET https://test.vuyk.eu/fonts/opensans-regular-webfont.woff [HTTP/2.0 200 OK 40ms]
GET https://test.vuyk.eu/templates/purity_iii/fonts/font-awesome/fonts/fontawesome-webfont.woff2?v=4.7.0 [HTTP/2.0 200 OK 70ms]
1 Answer

Hey friend!

There is nothing on the network side at least that would cause oddity. I would address the scenario with IPv6 removed from the thought process, at the end of it IPv6 is just another pathway so if items fail to load you would want to find the codes returned on load attempt and review the relevant logs to determine the backend cause. Note that SMTP is disabled on IPv6 so that could cause timeouts but I doubt your application is attempting an SMTP connect on page load. As for https://37.139.19.66, I'm not sure why you do not see the certificate mismatch there. I do see that, as you should any time you visit SSL over IP as SSL certificates are only signed for hostnames.

This is one error you noted:

    2018/08/30 16:25:27 [error] 29228#29228: *76 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 2a02:a440:91e3:1:4481:654b:a3e8:9617, server: test.vuyk.eu, request: "GET /images/klanten1/JHoogeveen.gif HTTP/2.0", upstream: "fastcgi://unix:/run/php/php7.2-fpm.sock:", host: "test.vuyk.eu", referrer: "https://test.vuyk.eu/portfolio-2"

Note that I do not have IPv6 enabled on my computer right now, and I visited https://test.vuyk.eu/portfolio-2 to find that an image failed to load. Copying the URL of an image that failed to load, I entered this URL: https://test.vuyk.eu/images/klanten1/schipper2.gif

From there it took a while to load a 404 page. I would suggest treating it as a normal 404, finding out the what and why of being unable to reach that file at that path.

Jarland

Have another answer? Share your knowledge.