Question
Kubernetes readwritemany (or the same effect)
- Posted February 21, 2019
- KubernetesBlock StorageScalingCoreOS
I’ve been using compose for years - k8s for days. I’m building a specialized app on DO k8s and it’s going great.
Thinking ahead, though, I’d like to use DO k8s for our entire infrastructure. We have hundreds of hosted client sites, “common” stuff - there’s a lot of legacy WordPress, but the newer ones are OctoberCMS, Gatsby, React+Headless.
[I’m thinking out loud a little here, bit still mostly looking for advice/help]
I’d like to run all that here, but I have some bumps to get over first:
-
the DO storageclass is readwriteonce. So what happens if a WP site scales? Its uploads directory isn’t in the image, of course, so it’s lost? Can we use a different storageclass that persists, or is there some way (very doubtful) to have a volume readwriteonce by a “master” and readonlymany by others? …or maybe an initContainer that pulls the files from a central locale?
-
This loosely ties into another concern re: initial deployment. Dev happens in a local environment, and deploy involves a one-time ssh/ftp of site-specific files (theme, plugins, imgs, etc). Nodes - naturally - have no direct access, so… I’m lost there. (but the initContainer idea I spitballed above may fix it?)
I would be okay with building a new image for all updates, but a fair amount of work is done by non-tech. My options there are a) pull all admin/updates into tech dept. or b) train design/content departments on building a docker imagine and pushing to a private registry. Both are equally terrifying lol.
These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.
Hey friend!
Great questions. I was just talking to Nicholas, part of our team over here, and he had some thoughts to share on this that I’d like to translate over.
The readwriteonce is a limitation of block storage. Our block storage is just like a USB hard drive you might have lying around. You can’t plug it into two computers at once, as you can’t have two systems fighting over the file system. You could possibly do an NFS server inside of your cluster, or use something like object storage (Spaces) instead. When you need to read/write to storage from multiple servers, object storage tends to be a popular way to do that, most people move beyond block devices for that purpose as it just doesn’t serve it well.
You may be able to use initcontainers to fetch assets, but they still have to be hosted somewhere so it makes more sense to use a remote storage system for it, like object storage.
Jarland
I am also going to give a try to this approach of DO spaces.