do-agent - Client.Timeout exceeded while awaiting headers

January 13, 2018 2.2k views
Monitoring Ubuntu 16.04
lukegripenberg
By:
lukegripenberg

Hi,

Our droplets in AMS3 are getting constant timeouts when trying to send metrics. This is really filling up our logs.

Is anybody else experiencing this?

All outbound traffic is being allowed from the droplets.

Jan 13 12:25:36 hostx syslog do-agent[1451]: 2018/01/13 11:25:35 Sending metrics to DigitalOcean: Post https://ams3.sonar.digitalocean.com/v1/metrics/droplet_id/XXXXXXX: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
Jan 13 12:26:50 hostx syslog do-agent[1451]: 2018/01/13 11:26:49 Sending metrics to DigitalOcean: Post https://ams3.sonar.digitalocean.com/v1/metrics/droplet_id/XXXXXXX: net/http: request canceled (Client.Timeout exceeded while awaiting headers)

Log In to Comment
3 Answers
miroslavmimokuhajda January 13, 2018

i saw something similar when we had problems with DNS or IPv6 settings.
try dig -4 ams3.sonar.digitalocean.com for DNS query,
and ping -c 5 ams3.sonar.digitalocean.com for ping with DNS query,
and ping -c 5 162.243.188.200 for ping without DNS query.

IP 162.243.188.200 stands for ams3.sonar.digitalocean.com .

Reply
lukegripenberg January 13, 2018

Certainly no problems here, except for a pretty big delay in pings:


; <<>> DiG 9.10.3-P4-Ubuntu <<>> -4 ams3.sonar.digitalocean.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43050
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;ams3.sonar.digitalocean.com.   IN  A

;; ANSWER SECTION:
ams3.sonar.digitalocean.com. 126 IN A   162.243.188.200

;; Query time: 0 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sat Jan 13 14:49:59 UTC 2018
;; MSG SIZE  rcvd: 72


PING ams3.sonar.digitalocean.com (162.243.188.200) 56(84) bytes of data.
64 bytes from 162.243.188.200: icmp_seq=1 ttl=59 time=80.4 ms
64 bytes from 162.243.188.200: icmp_seq=2 ttl=59 time=85.1 ms
64 bytes from 162.243.188.200: icmp_seq=3 ttl=59 time=80.6 ms
64 bytes from 162.243.188.200: icmp_seq=4 ttl=59 time=79.4 ms
64 bytes from 162.243.188.200: icmp_seq=5 ttl=59 time=79.4 ms


PING 162.243.188.200 (162.243.188.200) 56(84) bytes of data.
64 bytes from 162.243.188.200: icmp_seq=1 ttl=59 time=86.8 ms
64 bytes from 162.243.188.200: icmp_seq=2 ttl=59 time=84.5 ms
64 bytes from 162.243.188.200: icmp_seq=3 ttl=59 time=79.4 ms
64 bytes from 162.243.188.200: icmp_seq=4 ttl=59 time=79.4 ms
64 bytes from 162.243.188.200: icmp_seq=5 ttl=59 time=91.6 ms
Reply
Manju January 13, 2018

Looks like this was related to the incident here - https://status.digitalocean.com/incidents/y7dl42h08t4x .

If so, you should not be seeing the issue anymore.

Reply
Have another answer? Share your knowledge.

Related