use only DNS at Digital Ocean, you don't have to setup your DNS in VestaCP. I had 4 domain names on one vestacp and all have a standalone website.
Vesta installs its own DNS server, it cannot add records to Digital Oceans automatically. You will need to manually copy the DNS records that Vesta creates into your DNS settings with Digital Ocean.
In Vesta, select the appropriate account, then click on DNS, and then click the "list X records" link for the domain. The go in to DO's DNS settings, add a domain, and the copy each of the records vesta creates as equivalent records in the DO panel.