By Rahul Shettigar and Anish Singh Walia
Network connectivity issues can be challenging to diagnose, especially when dealing with asymmetric performance, latency variations, and unexpected bandwidth limitations. Identifying the root cause requires a structured approach using diagnostic tools and network analysis techniques. This tutorial walks you through the process of detecting and resolving network asymmetry issues by analyzing performance metrics, running network tests, and applying optimization techniques.
This tutorial is for system administrators, network engineers, DevOps professionals, and anyone responsible for maintaining network infrastructure. Whether you’re managing cloud resources, on-premises servers, or hybrid environments, you’ll learn practical skills to identify and fix asymmetric network issues that can impact application performance and user experience. By following this tutorial, you’ll be able to restore balanced network operations, reduce troubleshooting time, and ensure optimal performance across your infrastructure.
Please ensure you have the following prerequisites before you begin:
When diagnosing network performance issues, the initial step is to establish a baseline. For demonstration purposes you can use a pure bare metal GPU node and a Premium Dedicated Droplet (PDD) in the same AMS region.
This involves measuring the maximum throughput of the network between the nodes.
The expected throughput is 10 Gbps, which is the limit on a Premium Dedicated Droplet (PDD).
Test Procedure:
iperf3
for Bandwidth TestingTo measure real-time bandwidth, use iperf3
:
On the bare metal GPU node (acting as the iperf3
server) use the following command to start the server:
On Premium Dedicated Droplet (PDD) (acting as the iperf3
client) run the following command to start the client:
Where:
-c <server-b-ip>
→ Client mode: Specifies the target server for the test.
-P 10
→ Parallel connections: Runs 10 parallel streams for the test, simulating multiple simultaneous connections.
-t 30
→ Test duration: Runs the test for 30 seconds.
This will provide a detailed view of available bandwidth and potential bottlenecks.
mtr
for Network TracingTo further investigate, use My Traceroute to identify network routing issues.
Example: From bare metal GPU node → Premium Dedicated Droplet (PDD)
Where:
-r
→ Report mode: Runs mtr
in report mode, meaning it collects statistics and exits instead of running interactively.
-w
→ Wide report format: Displays additional statistics in the output.
-b
→ Show both IPs and hostnames: Includes both IP addresses and resolved hostnames in the output.
-z
→ Show AS numbers: Displays the Autonomous System (AS) numbers for each hop (if available).
-c100
→ Run 100 queries: Sends 100 probe packets to each hop and averages the results.
Look for packet loss or high latency at intermediate hops, which could indicate congestion or misrouting.
Example: From Premium Dedicated Droplet (PDD) → bare metal GPU node
If a loss occurs at a specific hop and continues beyond that point, it may indicate a problematic router in the path.
TCP Connection Pooling and Thread Limits: Speedtest clients often use more threads for downloading than for uploading, which can affect test results. For example, the default configuration of speedtest-cli uses 34 parallel TCP connections for downloads but only 6 for uploads.
MTU (Maximum Transmission Unit) Misconfiguration: A high MTU setting can cause packet fragmentation, leading to performance degradation. Additionally, VPN tunnels like WireGuard introduce overhead, making an MTU reduction necessary.
Network Provider Policies & Peering Agreements: Some ISPs and cloud providers implement traffic shaping that prioritizes downloads over uploads, while interconnection agreements between ISPs can also impact routing performance.
What is MTU?
MTU stands for Maximum Transmission Unit, which is the maximum size of a data packet that can be transmitted over a network interface. It is a critical parameter in network communication as it determines the maximum amount of data that can be sent in a single packet. A higher MTU allows for larger packets, which can improve network performance by reducing the number of packets sent. However, if the MTU is set too high, it can lead to packet fragmentation, which can negatively impact network performance.
What is WireGuard?
WireGuard is a fast, modern, and secure VPN (Virtual Private Network) solution that aims to replace traditional VPN solutions like OpenVPN and IPSec. It is designed to be easy to use, highly performant, and secure. WireGuard uses state-of-the-art cryptography and is optimized for modern network architectures. It is widely used for creating secure, point-to-point connections between devices over the internet.
To set the MTU for WireGuard, you need to run the following commands on both the bare metal GPU node and the Premium Dedicated Droplet (PDD).
This command sets the Maximum Transmission Unit (MTU) of the network interface wg0
to 1400
. The MTU is the maximum size of a data packet that can be transmitted over a network interface. Setting the MTU to 1400
ensures that packets sent over this interface do not exceed this size, which can help prevent packet fragmentation and improve network performance.
Or lower if necessary:
This command sets the MTU of the network interface wg0
to 1300
, which is a lower value than the previous command. This may be necessary if the network infrastructure or VPN tunnel requires a lower MTU to function properly.
speedtest-cli
, use iperf3
for more controlled testing. Run the following command on the Premium Dedicated Droplet (PDD) as the client and the bare metal GPU node as the server:On the client (PDD):
This command runs iperf3
to perform a network bandwidth test. The options used are:
-c <server-b-ip>
: specifies the server to connect to, where <server-b-ip>
is the IP address of the server.-P 10
: sets the number of parallel streams to 10, which can help to saturate the network link and provide a more accurate measurement of bandwidth.-t 60
: sets the duration of the test to 60 seconds, allowing for a longer test period to gather more data and ensure a stable measurement.On the server (bare metal GPU node):
This provides more reliable results than commercial speed test servers.
After applying the fixes, run the tests again and observe:
mtr
results should be minimized.The ideal MTU for WireGuard depends on the network infrastructure and VPN tunnel requirements. A common value is 1400, but it may need to be lowered to 1300 or even lower in some cases.
If you’re experiencing packet fragmentation or network performance issues, adjusting the MTU might be necessary. A high MTU can lead to packet fragmentation, which can cause performance issues, while a low MTU can result in increased overhead due to smaller packet sizes. The optimal value for MTU is typically between 1300 and 1400, as this range balances packet size and fragmentation avoidance for optimal network performance and bottleneck minimization. However, consult your network infrastructure and VPN tunnel documentation for specific requirements to determine the optimal MTU for your setup.
speedtest-cli
and iperf3
for network testing?speedtest-cli
is a tool for testing internet bandwidth using commercial speed test servers, which is ideal for measuring internet connectivity and speed. On the other hand, iperf3
is a more controlled tool for measuring network bandwidth between two points, making it suitable for internal network testing, such as measuring the performance of a VPN tunnel or a local network. iperf3
provides more reliable results for internal network testing due to its ability to control the test environment and eliminate external factors that might affect the results.
iperf3
for network testing?Run iperf3 -c <server-b-ip> -P 10 -t 60
on the client side and iperf3 -s
on the server side. This will perform a 60-second network bandwidth test with 10 parallel streams.
If the issue persists, contact your ISP to investigate potential traffic shaping policies that might be impacting upload speeds. Request a route optimization check to ensure the most efficient path between data centers.
After applying the fixes, re-run the tests and observe improvements in upload speeds, minimized packet loss in mtr
results, and reduced network asymmetry. This can be done by comparing the results before and after the fixes. If the upload speeds have significantly improved, the packet loss has been minimized, and the network asymmetry has been reduced, the fixes have been successful.
This comprehensive tutorial has walked you through a structured approach to identify and address asymmetric network performance issues. By following the steps outlined, you’ve learned how to analyze network performance using tools like iperf3
and mtr
, diagnose common issues such as packet loss and network asymmetry, and apply targeted fixes to improve upload speeds and overall network reliability. With these skills, you’re now equipped to tackle network performance problems with confidence, ensuring smoother data transfer and more efficient communication across your network.
For more in-depth knowledge on networking concepts and troubleshooting, check out these DigitalOcean tutorials:
These resources provide valuable insights into network configuration, troubleshooting, and optimization techniques that complement the strategies discussed in this guide.
Continue building with DigitalOcean Gen AI Platform.
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!