Do I still need to point to the DigitalOcean name servers from my registrar if I will not manage the records in DO?

No you don’t need to. You only point at DO if you’re going to use their DNS manager.

Or I just need to point the domain from the domain registrar to my droplet’s IPv4? If yes then is there other benefits of adding the domain in DO?

That’s it. Just point the A record(s) to your droplet’s IP and you’re all set. There’s no benefit besides the convenience of having it at DO.