Our application calls multiple external APIs (other DO droplets, paypal API, aws instances). We log any timeout or exception that happens on these calls.

Every one hour for a few minutes, we get some timeouts calling external APIs. It happens consistently every hour between HH:01 and HH:04 . Connectivity is not completely lost, but the rate of error increases significantly. Before this issue started, we used to get 1-2 timeout timeouts per day, which is ok. But now we’re getting between 10 and 35 timeouts every first few minutes of the hour. The timeouts happen to a variety of hosts including other droplets (to which we connect by hostname, not IP)

We started having this issue seemingly out of nowhere. Without installing any updates to our software or changing anything on the droplet. If anyone can point me in the right direction, I’d much appreciate the help.

Here is an image that shows the successful vs unsuccessful external calls on a timeline. https://drive.google.com/file/d/1ESHyylePOYTYk2bgQJ5fq5dhupj3NELB/view

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
1 answer

Hello,

It will be the best to check the connection using traceroute or mtr to see if there is a packet loss and on which exact hop it happens as well.

The timeouts can be caused from poorly configured router or switch or caused from the end host where the APIs are hosted/called.

The command will look like this:

On Windows open cmd and run:

tracert example.com

On Linux open a terminal window and type:

traceroute example.com

You can also use any site that offers online traceroute checks.

Hope this helps!

Alex

Submit an Answer