You’ll need to register your DNS server(s) with your domain registrar. Generally this is done by going in to the domain settings for the given domain that you are using for the name servers and finding a menu option for it. You’ll only need to do this once per server; even if you host multiple domains on the same server, as long as you always use the same DNS server names for your other domains (i.e. ns[1-2].domainone.com can serve as the DNS server for www.domaintwo.com, smtp-server.domainthree.com, etc.).
Note that for redundancy purposes, the DNS specs technically require that you have a minimum of two DNS servers. If you are running everything from one server, you can get away with only having one server, but your registrar may require you to list two (with separate IP addresses). One trick that some hosts use is to give a client two IPs on the same server. DO doesn’t give us this option.
For the sake of simplicity, it may be easier to use DO’s DNS servers instead of trying to use your own. The downside is that each time you add a domain to Vesta you will also have to add it to your DO control panel as well, as there is no easy way to import it from Vesta currently. The good news is that this is pretty much a one-time thing for most sites.
Another option would be to set up a second droplet as a slave DNS server, but if you only have a couple of domains that you’re hosting, this is probably overkill.