My Infrastructure is becoming messy and expensive

Question

I’m not very experienced with DevOps and I get frustrated very easily when things aren’t working. I wish Digital Ocean offered consulting services so I can just pay someone to set things up correctly…

Anyways, I’m hoping someone can provide guidance on what I should do with my infrastructure since I have multiple Apps running and multiple databases and multiple droplets and It’s frustrating to say the least.

I have a “Production” APP that has two servers (frontend,backend) and a managed DB cluster.
I have “Development” APP that is pretty much a duplicate of the production APP for my remote development team to access for testing.
2 Droplets that are running a chat engine (One for each of the Development and Production APP). Each Droplet runs it’s own Database

And now I’m in a situation where I need ElasticSearch engine for production and and development and DO only offers 1-click deployment for another APP. So that will be 2 more Apps. And none of theses can even communicate over VPS so it’s just getting bulky. Seems like I should get rid of the APPs and just manage everything myself but that’s beyond my experience and could take me weeks to get.

Does this setup sound normal enough or am I just digging myself into a hole by continuing to build more stuff onto this already bulky-looking infrastructure?

Bobby · Accepted Answer

Hey! One option that you could consider here is to transfer over your dev environment to a single Droplet. That way you will cut your infrastructure costs for your dev environment, but still keep your production app running on the App Platform so you can take advantage of the scalability and the operational efficiency. For your dev environment, you could use Docker to containerize your applications. Docker containers can run anywhere, making it easier to manage dependencies and reduce the number of servers you need. So you will be able to have all of your services running on a single Droplet as Docker containers. You could even use Docker compose for easier management of all containers. DigitalOcean offers 1-Click installation image for Docker: https://marketplace.digitalocean.com/apps/docker Here is a simplified Docker compose file that packages all of your services together: version: '3.8' services: frontend: image: node:14 working_dir: /app volumes: - ./frontend:/app - /app/node_modules ports: - "3000:3000" command: sh -c "npm install && npm start" depends_on: - backend backend: image: python:3.9 working_dir: /app volumes: - ./backend:/app - /app/venv ports: - "5000:5000" command: sh -c "pip install -r requirements.txt && flask run --host=0.0.0.0" environment: DATABASE_URL: "postgres://user:password@db:5432/devdb" ELASTICSEARCH_URL: "http://elasticsearch:9200" depends_on: - db - elasticsearch db: image: postgres:13 volumes: - db_data:/var/lib/postgresql/data environment: POSTGRES_DB: devdb POSTGRES_USER: user POSTGRES_PASSWORD: password ports: - "5432:5432" elasticsearch: image: docker.elastic.co/elasticsearch/elasticsearch:7.10.0 environment: - discovery.type=single-node ports: - "9200:9200" volumes: db_data: Overview: frontend : This service uses a Node.js image and mounts the local frontend directory to the container’s /app directory. It also exposes port 3000 and installs dependencies before starting the application. backend : This service uses a Python image, mounts the local backend directory to the container’s /app directory, and exposes port 5000. It sets environment variables for the database and ElasticSearch URLs and installs dependencies before starting the Flask application. db : This service uses a PostgreSQL image, sets up a database named devdb, and mounts a Docker volume for persistent storage. It exposes port 5432. elasticsearch : This service uses an ElasticSearch image, sets it to run in single-node mode, and exposes port 9200. volumes : Defines a named volume for PostgreSQL data storage. And for your production environment, you could keep things as they are as that way you can take advantage of all of the production-ready features that the App Platform offers along with the managed database. Let me know if you have any questions! - Bobby

KFSys · Answer

Heya @happygrayangler , Depending on how big is the App and it’s Database, it might be worth it moving your Databases to a Droplet and taking backups of the Droplet to prevent data loss. Additionally, again depending on the Application and how heavy it is, it might be worth having one server for both frontend and backend to save cost and the time to manage all by them being in different places. If however your App is quite big, the solution you’ve come up with is a standart one where it’s almost everywhere used liked that.

alexdo · Answer

Heya, You can consider using*Kubernetes to orchestrate and manage your applications. Kubernetes allows you to manage both your development and production environments on the same platform, providing benefits such as: Automated Scaling: Scale your applications automatically based on traffic and load. Simplified Deployment: Use CI/CD pipelines to automate deployments and rollbacks. Resource Efficiency: Efficiently allocate resources to your applications to reduce costs. High Availability: Easily achieve high availability by running multiple replicas of your services. DigitalOcean Kubernetes (DOKS) is a managed Kubernetes service. Deploy Kubernetes clusters with a fully managed control plane, high availability, autoscaling, and native integration with DigitalOcean Load Balancers and volumes. DOKS clusters are compatible with standard Kubernetes toolchains and the DigitalOcean API and CLI. It also offers integration with their other services, like Load Balancers, Block Storage, and more. It can be a strategic move to optimize and potentially reduce your infrastructure costs. Hope that this helps!

Become a contributor for community

DigitalOcean Documentation

Resources for startups and AI-native businesses

The developer cloud

Start building today

Report this

My Infrastructure is becoming messy and expensive

Related questions

Where to Buy Dedicated Server in India

shh add to existing project

Websocket server hosted on a VM keeps dropping websocket sonnections to peers after random time intervals

Prestashop site & Managed service

How can I optimize a low-cost affiliate dashboard for faster performance on a VPS?

After I reboot my droplet, I can't access my website

www cname not working after 12+ hours still getting 1001 error

Deploying app from digital ocean using github education

Why can I not access my Domain through my web browser, but can access the server via IP?

How do I invite someone as a team member on an existing project(my only project)? I tried doing it but it created a new team and project.

Transfer static IP from droplet to app platform?

Plesk error 500

Report this

My Infrastructure is becoming messy and expensive

Related questions

Where to Buy Dedicated Server in India

shh add to existing project

Websocket server hosted on a VM keeps dropping websocket sonnections to peers after random time intervals

Prestashop site & Managed service

How can I optimize a low-cost affiliate dashboard for faster performance on a VPS?

After I reboot my droplet, I can't access my website

www cname not working after 12+ hours still getting 1001 error

Deploying app from digital ocean using github education

Why can I not access my Domain through my web browser, but can access the server via IP?

How do I invite someone as a team member on an existing project(my only project)? I tried doing it but it created a new team and project.

Transfer static IP from droplet to app platform?

Plesk error 500