like @kely says it isn’t duplicate content however it does affect your SEO score once people join your website by IP (wich is most likely not going to happen since its alot more difficult). However you can have the google crawlers to change this IP to the domain name with a 301 redirect and as well have people redirected properly. For this you need to edit your apache VHOST / .htaccess .
Make sure the 000-default.conf has AllowOverride All set for the wordpress directory, if you didn’t change this then it should be set on default. Now you want to create a .htaccess file inside the directory of wordpress ( default /var/www/html ) with the following:

RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_HOST} ^12\.34\.56\.789$
RewriteRule ^(.*)$ http://www.domainname.com/$1 [L,R=301]

You need to replace the IP and domain to your own setup, once done make sure rewrite module is enabled:

a2enmod rewrite

then restart apache and the rewrite should work:

service apache2 restart

Hope you will figure it out with this, if you still have any questions don’t hesistate to ask them.