Hello,
I’ve recently migrated my Slack bot app to DigitalOcean App Platform and I’m seeing confusing CDN ingress latency spikes that I can’t explain.
Problem
- CDN Ingress Average Latency occasionally spikes to exactly 500ms or 1000ms (not 0.9s or 1.1s - exactly those values) and stay there for around 4-10 mins
- These spikes happen irregularly but consistently
- During spikes, status codes are still 2xx (successful)
- These requests are NOT reaching my application. My app logs show normal <100ms latency during the same time periods
What I’ve tried:
- Disabled edge cache - no impact
- Added health check to
/pingendpoint - working fine, less than 1 ms responses - Checked app logs during spikes - no corresponding slow requests
My confusion:
- How can requests timeout (exact 1000ms suggests timeout) but still return 2xx status codes?
- If requests aren’t reaching my app, what layer is timing out?
- Why does latency sometimes show as 0 despite visible traffic?
App details:
- Go application, single instance
- Health check configured on
/ping - Edge cache disabled
- Custom domain
Here’s a screenshot from low-traffic weekend showing the pattern:
This makes the Insights tab unreliable for monitoring. Any ideas what could cause exactly 1000ms delays in DO’s infrastructure before requests reach the app?
Thanks for any help!
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
Latency spikes accompanied by 2xx responses, where the requests never actually hit your app, often point to issues outside your application layer. A few common causes to check:
-
Load balancer or CDN caching: Sometimes the load balancer or CDN might serve cached 2xx responses without forwarding the request to your backend, causing perceived latency if their cache or routing is delayed.
-
Network-level retries or health checks: Infrastructure components might be retrying requests or performing health checks that respond early or with cached results, showing 2xx but never reaching your app.
-
Proxy or API gateway delays: If you’re using a proxy or API gateway, it might be processing or buffering requests before sending them to your backend, causing spikes.
-
Firewall or security layers: Occasionally, security rules might intercept and respond to requests directly.
To diagnose further:
-
Check logs and metrics at each infrastructure layer (load balancer, CDN, API gateway).
-
Enable detailed tracing or use distributed tracing tools (e.g., OpenTelemetry).
-
Verify caching and health check configurations.
-
Look for spikes in network retries or dropped packets.
Understanding where the 2xx response originates will help pinpoint whether it’s a caching, routing, or infrastructure timing issue causing the latency spikes.
Heya, @a595b5ef530a4516a5cc2723e93a64
This may be outside your application itself. You can make a simple test endpoint in your app that always returns a special header, like X-App-Origin: true, and log when it’s used. If you see 2xx traffic in the CDN metrics during a spike but your app never logs that header, it means those requests were answered at the CDN edge, not by your app.
Regards
Become a contributor for community
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
DigitalOcean Documentation
Full documentation for every DigitalOcean product.
Resources for startups and SMBs
The Wave has everything you need to know about building a business, from raising funding to marketing your product.
The developer cloud
Scale up as you grow — whether you're running one virtual machine or ten thousand.
Get started for free
Sign up and get $200 in credit for your first 60 days with DigitalOcean.*
*This promotional offer applies to new accounts only.