I’ve been building a system on DigitalOcean and would like some advice on how to proceed.

We sell customized accounting systems. Originally we delivered our system on a web server. Now it is in the cloud, and instead of each customer having their own web server, we have a service that can handle any number of customers.

There are two main interfaces to the service. One is for employees of the company who need to use their accounting system. The other is for the people who customize and maintain the accounting system. They might be employees of the company, or might be our employees.

The first interface is through the web. There is a login page which authenticates users. Usually a user will be able to access several systems, such as the production system and a test system that shows upcoming features. They will select the system and be forwarded to it. All interaction is through standard web interfaces. We currently use Tomcat to host the interfaces.

The system is divided into infrastructure servers and worker servers. There are a fixed number of infrastructure servers and a variable number of worker servers. All these servers are on a
VPN. One of the infrastructure servers is nginx, which hides the VPN from the outside world. Another is the login server. We have servers for Kafka, Zookeeper, nfs, and openVpn. openVpn lets the people customizing and maintaining the accounting system interact with Kafka, Zookeeper and nfs.

When I said there was a variable number of worker servers, I lied. We actually have a fixed number now, but I want to make it a variable number. I can add a new worker fairly easily. I make a new droplet, running a particular image, which mounts the nfs directory. I have to add an entry to the Zookeeper database about the new droplet. I have to add an entry into the nginx configuration file about the web server on this droplet. The droplet can already access Kafka and Zookeeper because they are all on the VPN.

I want to automate adding a new worker. One approach is to use the DigitalOcean API to create the new droplet and change the various databases. That is what I was planning to do until I started to look at Kubernates. I’ve never used Kubernates (I have used Docker a little) but it seems that it is ideally suited for this purpose. I assume that I can continue to make my infrastructure servers
be normal droplets and that it is easy for processes inside Kubernates to talk to them.

Can I mount nfs directories in Kubernates? I allow any server on the VPN to mount my nfs volume. Will this continue to work in Kubernates? Or should I instead use Volumes to do something similar?

I have to tell nginx the URL of each worker. This is the URL on the VPN, it is internal and not external. How do I find the URL (IP address) of a web server that I spin up in Kubernates? Will I have a problem creating thousands of them? Is there a limit?

Can you think of any problems I would run into when I try to run my workers in Kubernates?