Google Page Speed reporting "slow server response time" on a new server, how to improve?

December 20, 2013 4.3k views
I'm switching from Site5 due to slow server response times that I've experienced and seen reported by Google Page Speed reports. Slow being more than 200 milliseconds according to Google. I'm seeing 500 milliseconds on a clean Droplet and I was surprised, particularly because it has the well advertised SSD. Can anyone help me understand what it would take to get 200 milliseconds consistently? More RAM? CPU? I'm running a 512Mb WordPress Droplet.
3 Answers
varnish, caching, minifying assets?

or do you mean latency?
Forgive me, but I'm not sure how to answer your question. Specifically, I'm referring to Google Page Speed reports which specifically use the phrase "server response time". They define it here as the amount of time it takes to load a rendered page after latency is subtracted:

Does that answer your question?
"I'm running a 512Mb ... Droplet."

Did you install a 32-bit operating system?
Have another answer? Share your knowledge.