Question

I want my app to start working with digital ocean and to query around 10k api's in 10-30seconds

Posted November 18, 2021 95 views
APIKubernetesCloud Computing

I have an app that queries a lot of APIs, it relies on speed and it needs to be super fast.

I want to start running all the processing on the cloud instead of the client-side to make the app faster but I don’t know what will be the best solution.

Atm, it’s run in JS meaning it’s all run client-side so it’s very slow. Some APIs can take 30seconds (very good) and some can take 3-5mins.

My goal is to take most APIs and query them in a range of 10-30seconds or so (If the API is good to semi-good).

After the API requests are received there is very little parsing/calculating so that isn’t heavy and then the data should be sent to a database.

What will be the best solution to get those API queries as fast as possible? I thought of these options: Multiple very small computes. 1-2 big computes Managed Kubernetes (Kinda complicated since not very familiar with it) Computes w/ docker (Kinda complicated since not very familiar with it).

Of course, I will need to also buy proxy servers to route the requests through in case of rate-limiting on the server-side.

Ty very much for helping :)

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
Submit an Answer
1 answer

Hello,

I believe that all suggestions have their benefits. What I would personally do is to do it step by step, so first create a backend service that is capable of consuming the APIs and testing it out on a Droplet directly.

You can take a look at this video on how to deploy a Node.js application on an Ubuntu server:

Once you have the backend service ready, you can take it a step further and dockerize it as per the instructions here:

https://www.digitalocean.com/community/tutorials/how-to-build-a-node-js-application-with-docker

That way you will be able to easily start multiple instances of your Node.js app on the same server.

As you mentioned, the next step would be to go for a container orchestration like Kubernetes. Here are some useful resources:

Hope that this helps!
Best,
Bobby

by Kathleen Juell
Interested in Node.js but not sure where to start with Docker? This tutorial will walk you through the first step: building an image for a Node.js application and creating a container from it. We will also walk you through pushing that image to Docker Hub and using the saved image to recreate your application container.