Tutorial

Node JS Architecture - Single Threaded Event Loop

Published on August 3, 2022
Default avatar

By Rambabu Posa

Node JS Architecture - Single Threaded Event Loop

While we believe that this content benefits our community, we have not yet thoroughly reviewed it. If you have any suggestions for improvements, please let us know by clicking the “report an issue“ button at the bottom of the tutorial.

Today we will look into Node JS Architecture and Single Threaded Event Loop model. In our previous posts, we have discussed about Node JS Basics, Node JS Components and Node JS installation.

Node JS Architecture

Before starting some Node JS programming examples, it’s important to have an idea about Node JS architecture. We will discuss about “How Node JS works under-the-hood, what type of processing model it is following, How Node JS handles concurrent request with Single-Threaded model” etc. in this post.

Node JS Single Threaded Event Loop Model

As we have already discussed, Node JS applications uses “Single Threaded Event Loop Model” architecture to handle multiple concurrent clients. There are many web application technologies like JSP, Spring MVC, ASP.NET, HTML, Ajax, jQuery etc. But all these technologies follow “Multi-Threaded Request-Response” architecture to handle multiple concurrent clients. We are already familiar with “Multi-Threaded Request-Response” architecture because it’s used by most of the web application frameworks. But why Node JS Platform has chosen different architecture to develop web applications. What is the major differences between multithreaded and single threaded event loop architecture. Any web developer can learn Node JS and develop applications very easily. However without understanding Node JS Internals, we cannot design and develop Node JS Applications very well. So before starting developing Node JS Applications, first we will learn Node JS Platform internals.

Node JS Platform

Node JS Platform uses “Single Threaded Event Loop” architecture to handle multiple concurrent clients. Then how it really handles concurrent client requests without using multiple threads. What is Event Loop model? We will discuss these concepts one by one. Before discussing “Single Threaded Event Loop” architecture, first we will go through famous “Multi-Threaded Request-Response” architecture.

Traditional Web Application Processing Model

Any Web Application developed without Node JS, typically follows “Multi-Threaded Request-Response” model. Simply we can call this model as Request/Response Model. Client sends request to the server, then server do some processing based on clients request, prepare response and send it back to the client. This model uses HTTP protocol. As HTTP is a Stateless Protocol, this Request/Response model is also Stateless Model. So we can call this as Request/Response Stateless Model. However, this model uses Multiple Threads to handle concurrent client requests. Before discussing this model internals, first go through the diagram below. Request/Response Model Processing Steps:

  • Clients Send request to Web Server.
  • Web Server internally maintains a Limited Thread pool to provide services to the Client Requests.
  • Web Server is in infinite Loop and waiting for Client Incoming Requests
  • Web Server receives those requests.
    • Web Server pickup one Client Request
    • Pickup one Thread from Thread pool
    • Assign this Thread to Client Request
    • This Thread will take care of reading Client request, processing Client request, performing any Blocking IO Operations (if required) and preparing Response
    • This Thread sends prepared response back to the Web Server
    • Web Server in-turn sends this response to the respective Client.

Server waits in Infinite loop and performs all sub-steps as mentioned above for all n clients. That means this model creates one Thread per Client request. If more clients requests require Blocking IO Operations, then almost all threads are busy in preparing their responses. Then remaining clients Requests should wait for longer time. Request Response Model, Multithreaded request response architecture Diagram Description:

  • Here “n” number of Clients Send request to Web Server. Let us assume they are accessing our Web Application concurrently.

  • Let us assume, our Clients are Client-1, Client-2… and Client-n.

  • Web Server internally maintains a Limited Thread pool. Let us assume “m” number of Threads in Thread pool.

  • Web Server receives those requests one by one.

    • Web Server pickup Client-1 Request-1, Pickup one Thread T-1 from Thread pool and assign this request to Thread T-1

      • Thread T-1 reads Client-1 Request-1 and process it
      • Client-1 Request-1 does not require any Blocking IO Operations
      • Thread T-1 does necessary steps and prepares Response-1 and send it back to the Server
      • Web Server in-turn send this Response-1 to the Client-1
    • Web Server pickup another Client-2 Request-2, Pickup one Thread T-2 from Thread pool and assign this request to Thread T-2

      • Thread T-2 reads Client-1 Request-2 and process it
      • Client-1 Request-2 does not require any Blocking IO Operations
      • Thread T-2 does necessary steps and prepares Response-2 and send it back to the Server
      • Web Server in-turn send this Response-2 to the Client-2
    • Web Server pickup another Client-n Request-n, Pickup one Thread T-n from Thread pool and assign this request to Thread T-n

      • Thread T-n reads Client-n Request-n and process it
      • Client-n Request-n require heavy Blocking IO and computation Operations
      • Thread T-n takes more time to interact with external systems, does necessary steps and prepares Response-n and send it back to the Server
      • Web Server in-turn send this Response-n to the Client-nIf “n” is greater than “m” (Most of the times, its true), then server assigns Threads to Client Requests up to available Threads. After all m Threads are utilized, then remaining Client’s Request should wait in the Queue until some of the busy Threads finish their Request-Processing Job and free to pick up next Request. If those threads are busy with Blocking IO Tasks (For example, interacting with Database, file system, JMS Queue, external services etc.) for longer time, then remaining clients should wait longer time.
  • Once Threads are free in Thread Pool and available for next tasks, Server pickup those threads and assign them to remaining Client Requests.

  • Each Thread utilizes many resources like memory etc. So before going those Threads from busy state to waiting state, they should release all acquired resources.

Drawbacks of Request/Response Stateless Model:

  • Handling more and more concurrent client’s request is bit tough.
  • When Concurrent client requests increases, then it should use more and more threads, finally they eat up more memory.
  • Sometimes, Client’s Request should wait for available threads to process their requests.
  • Wastes time in processing Blocking IO Tasks.

Node JS Architecture - Single Threaded Event Loop

Node JS Platform does not follow Request/Response Multi-Threaded Stateless Model. It follows Single Threaded with Event Loop Model. Node JS Processing model mainly based on Javascript Event based model with Javascript callback mechanism. You should have some good knowledge about how Javascript events and callback mechanism works. If you don’t know, Please go through those posts or tutorials first and get some idea before moving to the next step in this post. As Node JS follows this architecture, it can handle more and more concurrent client requests very easily. Before discussing this model internals, first go through the diagram below. I tried to design this diagram to explain each and every point of Node JS Internals. The main heart of Node JS Processing model is “Event Loop”. If we understand this, then it is very easy to understand the Node JS Internals. Single Threaded Event Loop Model Processing Steps:

  • Clients Send request to Web Server.
  • Node JS Web Server internally maintains a Limited Thread pool to provide services to the Client Requests.
  • Node JS Web Server receives those requests and places them into a Queue. It is known as “Event Queue”.
  • Node JS Web Server internally has a Component, known as “Event Loop”. Why it got this name is that it uses indefinite loop to receive requests and process them. (See some Java Pseudo code to understand this below).
  • Event Loop uses Single Thread only. It is main heart of Node JS Platform Processing Model.
  • Even Loop checks any Client Request is placed in Event Queue. If no, then wait for incoming requests for indefinitely.
  • If yes, then pick up one Client Request from Event Queue
    • Starts process that Client Request
    • If that Client Request Does Not requires any Blocking IO Operations, then process everything, prepare response and send it back to client.
    • If that Client Request requires some Blocking IO Operations like interacting with Database, File System, External Services then it will follow different approach
      • Checks Threads availability from Internal Thread Pool
      • Picks up one Thread and assign this Client Request to that thread.
      • That Thread is responsible for taking that request, process it, perform Blocking IO operations, prepare response and send it back to the Event Loop
      • Event Loop in turn, sends that Response to the respective Client.

NodeJS-Single-Thread-Event-Model Diagram Description:

  • Here “n” number of Clients Send request to Web Server. Let us assume they are accessing our Web Application concurrently.
  • Let us assume, our Clients are Client-1, Client-2… and Client-n.
  • Web Server internally maintains a Limited Thread pool. Let us assume “m” number of Threads in Thread pool.
  • Node JS Web Server receives Client-1, Client-2… and Client-n Requests and places them in the Event Queue.
  • Node JS Even Loop Picks up those requests one by one.
    • Even Loop pickups Client-1 Request-1
      • Checks whether Client-1 Request-1 does require any Blocking IO Operations or takes more time for complex computation tasks.
      • As this request is simple computation and Non-Blocking IO task, it does not require separate Thread to process it.
      • Event Loop process all steps provided in that Client-1 Request-1 Operation (Here Operations means Java Script’s functions) and prepares Response-1
      • Event Loop sends Response-1 to Client-1
    • Even Loop pickups Client-2 Request-2
      • Checks whether Client-2 Request-2does require any Blocking IO Operations or takes more time for complex computation tasks.
      • As this request is simple computation and Non-Blocking IO task, it does not require separate Thread to process it.
      • Event Loop process all steps provided in that Client-2 Request-2 Operation and prepares Response-2
      • Event Loop sends Response-2 to Client-2
    • Even Loop pickups Client-n Request-n
      • Checks whether Client-n Request-n does require any Blocking IO Operations or takes more time for complex computation tasks.
      • As this request is very complex computation or Blocking IO task, Even Loop does not process this request.
      • Event Loop picks up Thread T-1 from Internal Thread pool and assigns this Client-n Request-n to Thread T-1
      • Thread T-1 reads and process Request-n, perform necessary Blocking IO or Computation task, and finally prepares Response-n
      • Thread T-1 sends this Response-n to Event Loop
      • Event Loop in turn, sends this Response-n to Client-n

Here Client Request is a call to one or more Java Script Functions. Java Script Functions may call other functions or may utilize its Callback functions nature. So Each Client Request looks like as shown below: Node JS Architecture, Single Threaded Event Loop For Example:

function1(function2,callback1);
function2(function3,callback2);
function3(input-params);

NOTE: -

  • If you don’t understand how these functions are executed, then I feel you are not familiar with Java Script Functions and Callback mechanism.
  • We should have some idea about Java Script functions and Callback mechanisms. Please go through some online tutorial before starting our Node JS Application development.

Node JS Architecture - Single Threaded Event Loop Advantages

  1. Handling more and more concurrent client’s request is very easy.
  2. Even though our Node JS Application receives more and more Concurrent client requests, there is no need of creating more and more threads, because of Event loop.
  3. Node JS application uses less Threads so that it can utilize only less resources or memory

Event Loop Pseudo Code

As I’m a Java Developer, I will try to explain “How Event Loop works” in Java terminology. It is not in pure Java code, I guess everyone can understand this. If you face any issues in understanding this, please drop me a comment.

public class EventLoop {
while(true){
        	if(Event Queue receives a JavaScript Function Call){
        		ClientRequest request = EventQueue.getClientRequest();
                            If(request requires BlokingIO or takes more computation time)
                                    Assign request to Thread T1
                            Else
                                  Process and Prepare response
                  }
            }
} 

That’s all for Node JS Architecture and Node JS single threaded event loop.

Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.

Learn more about us


About the authors
Default avatar
Rambabu Posa

author

Still looking for an answer?

Ask a questionSearch for more help

Was this helpful?
 
JournalDev
DigitalOcean Employee
DigitalOcean Employee badge
July 8, 2021

Please explain that what if an another request arises to event loop with blocking I/O operation and thread t1 is busy in executing earlier blocking I/O request. The in the case what will event loop decide to perform? Thank You

- Neelesh

    JournalDev
    DigitalOcean Employee
    DigitalOcean Employee badge
    June 28, 2021

    This is one of the best articles on Node.js single-thread architecture. So helpful specially for a newbie like me.

    - Josh

      JournalDev
      DigitalOcean Employee
      DigitalOcean Employee badge
      February 23, 2021

      That means the only deference between multi thread request response and single thread (node.js) is before handling request node js check if the request is blocking operation or not if blocking then it will apply thread otherwise not Am I right? And another thing is though node js is single thread it internally use multi thread to handle request right?

      - Shuvo Khalid

        JournalDev
        DigitalOcean Employee
        DigitalOcean Employee badge
        December 31, 2020

        In the native model, if there is no free thread pool then request should be wait right as per ur blog. But u didnt emntion this case for the nodejs single threaded.

        - Rabeesh V

          JournalDev
          DigitalOcean Employee
          DigitalOcean Employee badge
          December 25, 2020

          Great Explanation. I have a question i.e When Blocking code/file processing is taking care of Thread, who is taking care of simple code/ task which involves calculations/ etc . I mean processer/threads ?

          - Maruthi

            JournalDev
            DigitalOcean Employee
            DigitalOcean Employee badge
            August 30, 2020

            thank you very much, it is the best Node.js architecture explanation I have ever read

            - abdullatif jamous

              JournalDev
              DigitalOcean Employee
              DigitalOcean Employee badge
              June 24, 2020

              Very well explained. This my first time in Node JS and read many articles on the architecture, but your article made the concept so clear. Thanks.

              - Sonal Agarwal

                JournalDev
                DigitalOcean Employee
                DigitalOcean Employee badge
                April 17, 2020

                Good article. I should have read earlier. There is a clear clarification on the high level Thanks.

                - Saranya

                  JournalDev
                  DigitalOcean Employee
                  DigitalOcean Employee badge
                  April 13, 2020

                  Hi All, I am confused about how event loop decide which request is blocking or nonblocking? Jatin Raghv

                  - Jatin

                    JournalDev
                    DigitalOcean Employee
                    DigitalOcean Employee badge
                    February 26, 2020

                    beautiful explanation. everyone can understand this. so clear. thank you so much.

                    - Abhishek

                      Try DigitalOcean for free

                      Click below to sign up and get $200 of credit to try our products over 60 days!

                      Sign up

                      Join the Tech Talk
                      Success! Thank you! Please check your email for further details.

                      Please complete your information!

                      Get our biweekly newsletter

                      Sign up for Infrastructure as a Newsletter.

                      Hollie's Hub for Good

                      Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.

                      Become a contributor

                      Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.

                      Welcome to the developer cloud

                      DigitalOcean makes it simple to launch in the cloud and scale up as you grow — whether you're running one virtual machine or ten thousand.

                      Learn more
                      DigitalOcean Cloud Control Panel