Question
Help tracking down bandwidth issue
- Posted on December 5, 2019
- NginxUbuntu 18.04
Asked by Kyle Stubbins
Hi all;
I’m not even sure exactly how to describe my issue… I have an Ubuntu 18.04 server running Nginx. About once a month my local computer (also Ubuntu 18.04) seems to get “stuck” with a connection to my DO server. I didn’t notice it this week, and it has used 8GB of combined upload / download to my server. To what? I don’t know.
I use tcptrack, but I only see my server IP and port 443. I don’t know what it’s connected to on my server. At first I though it was my SSH sessions as sometimes they freeze and I close the window, but when I saw port 443 I knew it was a website. I’ve rebooted my local computer multiple times with no effect. Run tcptrack and there is the connection, using 28kb/s up and down.
Today, I did a reload of the Nginx server and my connection immediately showed “RESET” as the state and then disappeared completely. It’s been OK since. I need some help with what to use to actually track down the issue. It’s my server and it happened to me. I could reset Nginx. What if it’s happening to others that connect to my server, or is it during some of my administration? I left my local computer off all day and when I turned it on, the connection persisted. If it’s happening to others and they have no way of resetting the connection. The reason I even know is I’m in the country on very limited bandwidth and got a text notification, otherwise I might not have even noticed. It ends up costing me a small fortune.
Any help in pointing my to how to go about solving this would be appreciated. I’m going to dig through sever logs and see if I can find clues.
Thank you; Kyle
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.
Want to learn more? Join the DigitalOcean Community!
Join our DigitalOcean community of over a million developers for free! Get help and share knowledge in Q&A, subscribe to topics of interest, and get courses and tools that will help you grow as a developer and scale your project or business.
So, I spent the most of the weekend tracking down this issue and, ultimately, redoing my entire network. It ended up being a DNS issue.
Not sure why or how this all happened as I’ve been running the same DNS for the past few years, but a recent update must have triggered something (I recently upgraded my desktop to Ubuntu 18.04 and created a new droplet, also 18.04).
The issue, was my domain at home was kylestubbins.com (single sign on SSSD - LDAP / Kerberos), which was authoritive, so no requests for kylestubbins.com ever left my LAN. When I put www.kylestubbins.com on DO, I simply setup DNS records for it on my LAN. Everything was golden. What I’ve noticed now is that when I mistype any domain, it redirects to my DO server, which has records for my kylestubbins.com domain. Why? I haven’t yet figured that out. The same settings, with my new DNS domain, behaves exactly as it should and says “hey, can’t find that!” when I mistype something.
The cause: one of my printers kept posting to ipp/port1, which was being expanded to kylestubbins.com/ipp/port1 (I’m assuming by the expand-hosts setting of dnsmasq) and simply using bandwidth between my LAN and my DO server. 28kB/s on average.
The solution: was to redo the DNS for my LAN by adding the subdomain home.kylestubbins.com. (Yeah, I know there are those of you shaking your head right now… live and learn.)
On the plus side, I also solved an NFSv4 issue I was having, where I was getting long wait times opening NFS shares (this was happening with 16.04, but not as prevalent – it was painful in 18.04 for a while).
It was DNS. I should have known. It’s always DNS.
My LAN has never worked so well. Silver lining, I guess.
Kyle
Hi @kylestubbins,
I think that going through the server logs would be the best first step.
What I could suggest is using this script here to summarize your access logs.
The output that it would provide you with should look something like this:
This should give you some more information on what files and what IPs are hitting your server the most.
Regards, Boby