If both domains are pointed to your WordPress site the change does need to be made in wp-admin unless you have a redirect configured in your nginx configuration. What error are you seeing?

Since nginx does not do .htaccess files like Apache does for rewrites you need to configure things a bit differently. I would recommend checking your configuration against what is shown in the official WordPress Nginx documentation.

How to configure WordPress to use new domain name Introduction

Changing domain for WordPress is easy job. You just go to WP-Admin -> Settings -> General and change WordPress Address (URL) and Site Address (URL).
But in case you use Certificate on your WordPress site, changing URL will cause certificate mismatch error. You probably saw errors like these, you will be asked to add security exception before opening site and it will have red lock instead of green.

In short lines, this is what we will do today:

• Reconfigure nginx to reflect domain changes
• Generate Let's Encrypt certificate for your new domain
• Configure WordPress to use new domain So, let's start. :)

This is based on How to secure Nginx with Let's Encrypt on Ubuntu 16.40 DigitalOceans tutorial. As this is second time, you don't need to follow whole tutorial, follow what I wrote down.

We will use Webroot Plugin to generate certificate for your new domain. In case it is www.keyman.com.hk you should execute:

sudo letsencrypt certonly -a webroot --webroot-path=/var/www/html -d keyman.com.hk -d keyman.com.hk

Take care of --webroot-path, make sure it matches location of your html root folder (where WordPress is installed).
This will generate certificate for keyman.com.hk that can also be used for www.keyman.com.hk.
Successful output should look like:

Output
IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at
   /etc/letsencrypt/live/keyman.com.hk/fullchain.pem. Your cert will
   expire on 2016-12-29. To obtain a new version of the certificate in
   the future, simply run Let's Encrypt again.
 - If you like Let's Encrypt, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

Most important informations are when it expire, and location of it. This should do the job in case of certificates.

We will start with creating snippet for pointing to new cert. Create it as ssl-keyman.com.hk.conf and open it with favorite text editor. We are going to use nano as it is most easiest.

sudo nano /etc/nginx/snippets/ssl-keyman.com.hk.conf

Content of file should be:

ssl_certificate /etc/letsencrypt/live/keyman.com.hk/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/keyman.com.hk/privkey.pem;

You shuold have Nginx server block is located at /etc/nginx/sites-available/default, if you didn't changed anything yourself. Open it with your favorite text editor.

sudo nano /etc/nginx/sites-available/default

Let's say it have two parts:

• Redirector
• WordPress config

You will know it is redirector by looking at its port. Redirector always listens on port 80 and redirects all traffic to 443/https.
It should look like:

server {
        listen 80 default_server;
        listen [::]:80 default_server;
        server_name hkdirector.tech www.hkdirector.tech;
        return 301 https://www.$server_name$request_uri;
}

Change server_name hkdirector.tech www.hkdirector.tech; to

server_name keyman.com.hk www.keyman.com.hk;

This is it for redirector, now let's look for WordPress config. You will see it is listening on port 443 instead of 80.
First part should look like these:

server {
        listen 443 ssl default_server;
        listen [::]:443 ssl default_server;

        include snippets/ssl-hkdirector.tech.conf;
        include snippets/ssl-params.conf;

        root /var/www/html;
        index index.php index.html index.htm index.nginx-debian.html;

        server_name hkdirector.tech www.hkdirector.tech;
       ...
}

Lines red highlighted should be changed.
First change include snippets/ssl-hkdirector.tech.conf; to

include snippets/ssl-keyman.com.hk.conf;

After that change server_name hkdirector.tech www.hkdirector.tech; to

server_name keyman.com.hk www.keyman.com.hk;

After every nginx change, you should verify and test config files. Nginx have very powerful command for it:

sudo nginx -t

It will test all your Nginx configuration and report status. You should see something like:

Output
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

In case of problem arise, report it here, I will try to help you fix it.

Wordpress is most easiest part. Just go to your WP-Admin, Settings, General and update WordPress Address and Site Address to new domain.
If problem arises here, let's first analyzes what does it say. Post it here, and I will try to resolve it with you.

And... Here we goes! If everything goes well, you should have fully working WordPress site on new domain.
In case of any problem, comment/post, I or any other member here will try to resolve it for you.
I hope this will be helpful for you. :)

Btw, it seems my website will have problem sometimes, and it is caused by Let's Encrypt. It is doing more harm than good?

Can I remove it at this stage now?