This depends on your location, and your droplet location, and actually a lot more things. I was wondering the same as you, so I just got one to test it and I am pretty satisfied, but I have been testing for just two days.
It really depends on the latency between your droplet and visitors, how many visitors your site gets, and how many plugins/themes you have. I suggest trying out a droplet for a few days and seeing if it fits you :]
@santz: Would you mind testing it out yourself? There's currently a promo for a free month on their 512MB droplet (I can't find the promo code in my e-mail, right now, but I'm sure you could find it on Google or on DigitalOcean's Twitter feed).
According to D.O.'s FAQs, they use "Intel Hex-Core" processors and all of their "CPUs are b/w 2 and 3 GHz." That, coupled w/SSD drives & a Tier 1 data center ... I'd say their resources are sufficient for "a small freelancer's site."