By slow, how slow are we talking about -- seconds, minutes? Generally requests won't just slow to a crawl without a reason, so it'd either be a network issue or a configuration issue. While you may not see a load or issue with New Relic, that doesn't mean there isn't one.
The first thing I'd do is test latency by pinging the server (if you've not disabled ping responses). You can do this by:
ping -c 5 DROPLET_IP
On Windows 8/10
DROPLET_IP is your Droplets IPv4 IP Address. Unless you're pretty far from your server, you should see 20-40ms responses. I'm in Tennessee and those are the response times I get from NY. If you happen to be farther away, those will likely increase and that's normal as distance plays a role.
If you're seeing 200-400ms on a US Data Center and your in the US, for example, that's probably an issue DigitalOcean support needs to address as that's definitely network related.
Beyond that, we can do simple tests like response time. If you'll visit:
... and pop your sites URI in to the input box and let that run, that'll give a general indication as to how long it takes for your site to load, while identifying potential issues. If networking isn't an issue, there's a good change your site itself could be slow (re: HTTP) due to lack of optimization -- i.e. compression (GZIP, Brotli), large number of requests (too many CSS/JS files), large images, etc.
When it comes to SSH, that's pretty much all networking as it's just a connection to the server, so ping -- latency, jitter, etc -- all affect that -- and that could also be a local issue (ISP).