I’ve been building a system on DigitalOcean and would like some advice on how to proceed.
We sell customized accounting systems. Originally we delivered our system on a web server. Now it is in the cloud, and instead of each customer having their own web server, we have a service that can handle any number of customers.
There are two main interfaces to the service. One is for employees of the company who need to use their accounting system. The other is for the people who customize and maintain the accounting system. They might be employees of the company, or might be our employees.
The first interface is through the web. There is a login page which authenticates users. Usually a user will be able to access several systems, such as the production system and a test system that shows upcoming features. They will select the system and be forwarded to it. All interaction is through standard web interfaces. We currently use Tomcat to host the interfaces.
The system is divided into infrastructure servers and worker servers. There are a fixed number of infrastructure servers and a variable number of worker servers. All these servers are on a VPN. One of the infrastructure servers is nginx, which hides the VPN from the outside world. Another is the login server. We have servers for Kafka, Zookeeper, nfs, and openVpn. openVpn lets the people customizing and maintaining the accounting system interact with Kafka, Zookeeper and nfs.
When I said there was a variable number of worker servers, I lied. We actually have a fixed number now, but I want to make it a variable number. I can add a new worker fairly easily. I make a new droplet, running a particular image, which mounts the nfs directory. I have to add an entry to the Zookeeper database about the new droplet. I have to add an entry into the nginx configuration file about the web server on this droplet. The droplet can already access Kafka and Zookeeper because they are all on the VPN.
I want to automate adding a new worker. One approach is to use the DigitalOcean API to create the new droplet and change the various databases. That is what I was planning to do until I started to look at Kubernates. I’ve never used Kubernates (I have used Docker a little) but it seems that it is ideally suited for this purpose. I assume that I can continue to make my infrastructure servers be normal droplets and that it is easy for processes inside Kubernates to talk to them.
Can I mount nfs directories in Kubernates? I allow any server on the VPN to mount my nfs volume. Will this continue to work in Kubernates? Or should I instead use Volumes to do something similar?
I have to tell nginx the URL of each worker. This is the URL on the VPN, it is internal and not external. How do I find the URL (IP address) of a web server that I spin up in Kubernates? Will I have a problem creating thousands of them? Is there a limit?
Can you think of any problems I would run into when I try to run my workers in Kubernates?
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
Accepted Answer
My advice, which doesn’t answer your question directly, would be to model your planned kubernetes deployment using droplets. I know this sounds weird, but it will save you a tonn of time, money and heartache. Droplets are cheap. $5 each. Put each of your applications that you think would be in a pod and put them on a single droplet. Then if you can make it all work with logins and shared file systems etc, then you have a blue print for how to build your cluster. If you can’t make it work using droplets, then kubernetes will not magicly fix it. This way as you can add one piece to your plan at a time without getting stuck on a kubernetes issue. Learning kubernetes and designing your infrastructure at the same is not a good receipe for success, don’t ask me how I know this. :(
That is what I have been doing. Only I create new droplets by hand. I am at the point where I want to create them automatically and my question was whether I should learn the DigitalOcean APIs and use them to create the new workers (droplets) or should I switch to Kubernates. I take it you are advising me to stick with droplets. There is probably less for me to learn that way.
At the moment, I am not using Kubernates at all. I do it all directly with droplets and it is working OK.
Become a contributor for community
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
DigitalOcean Documentation
Full documentation for every DigitalOcean product.
Resources for startups and SMBs
The Wave has everything you need to know about building a business, from raising funding to marketing your product.
The developer cloud
Scale up as you grow — whether you're running one virtual machine or ten thousand.
Get started for free
Sign up and get $200 in credit for your first 60 days with DigitalOcean.*
*This promotional offer applies to new accounts only.