While 440 visitors to a normal website isn't a lot, 440 people running the ownCloud client against your server could be. I would recommend first checking on your droplet's resource usage.
top and make a note both of the load numbers shown at the top and which processes you might see many instances of. To exit, press Ctrl+c. You can also see how much available memory you have by running
free -m to see if that is a potential issue.
If you are not using up your available memory there may be adjustments (like the number of apache instances listening) that can be adjusted to improve performance without upgrading to a larger droplet though that might be necessary.