Related
Join 1M+ other developers and:
- Get help and share knowledge in Q&A
- Subscribe to topics of interest
- Get courses & tools that help you grow as a developer or small business owner
Question
Question
Question
Nginx & Passenger stopping / crashing on Ubuntu droplet intermittently
I have a small Rails app that I’m running on a $5/mo droplet. At the moment it gets almost no traffic, me and a few other people. I first deployed it back in early December and for about two months it worked without issue, however for the last month or so Nginx has been shutting down intermittently. I’ll restart and within 2-6 hours the site will be down.
This is the nginx error log:
2021/03/18 01:21:55 [info] 283837#283837: Using 32768KiB of shared memory for nchan in /etc/nginx/nginx.conf:63
2021/03/18 01:21:56 [notice] 283841#283841: signal process started
[ N 2021-03-18 01:21:56.4329 269553/T6 age/Cor/CoreMain.cpp:670 ]: Signal received. Gracefully shutting down... (send signal 2 more time(s) to force shutdown)
[ N 2021-03-18 01:21:56.4329 269553/T1 age/Cor/CoreMain.cpp:1245 ]: Received command to shutdown gracefully. Waiting until all clients have disconnected...
[ N 2021-03-18 01:21:56.4330 269553/T1 age/Cor/CoreMain.cpp:1146 ]: Checking whether to disconnect long-running connections for process 269652, application /srv/project/current (production)
[ N 2021-03-18 01:21:56.4332 269553/T9 Ser/Server.h:901 ]: [ApiServer] Freed 0 spare client objects
[ N 2021-03-18 01:21:56.4332 269553/T9 Ser/Server.h:558 ]: [ApiServer] Shutdown finished
[ N 2021-03-18 01:21:56.4332 269553/T6 Ser/Server.h:901 ]: [ServerThr.1] Freed 0 spare client objects
[ N 2021-03-18 01:21:56.4333 269553/T6 Ser/Server.h:558 ]: [ServerThr.1] Shutdown finished
[ N 2021-03-18 01:21:56.4372 269553/T1 age/Cor/CoreMain.cpp:1146 ]: Checking whether to disconnect long-running connections for process 269652, application /srv/project/current (production)
[ N 2021-03-18 01:21:56.5257 283842/T1 age/Wat/WatchdogMain.cpp:1373 ]: Starting Passenger watchdog...
[ N 2021-03-18 01:21:56.5981 283848/T1 age/Cor/CoreMain.cpp:1340 ]: Starting Passenger core...
[ N 2021-03-18 01:21:56.5983 283848/T1 age/Cor/CoreMain.cpp:256 ]: Passenger core running in multi-application mode.
[ N 2021-03-18 01:21:56.6164 283848/T1 age/Cor/CoreMain.cpp:1015 ]: Passenger core online, PID 283848
[ N 2021-03-18 01:21:57.1347 269553/T1 age/Cor/CoreMain.cpp:1325 ]: Passenger core shutdown finished
2021/03/18 01:21:58 [notice] 283869#283869: signal process started
[ N 2021-03-18 01:21:58.7808 283848/T6 age/Cor/CoreMain.cpp:670 ]: Signal received. Gracefully shutting down... (send signal 2 more time(s) to force shutdown)
[ N 2021-03-18 01:21:58.7808 283848/T1 age/Cor/CoreMain.cpp:1245 ]: Received command to shutdown gracefully. Waiting until all clients have disconnected...
[ N 2021-03-18 01:21:58.7810 283848/Tb Ser/Server.h:901 ]: [ApiServer] Freed 0 spare client objects
[ N 2021-03-18 01:21:58.7810 283848/Tb Ser/Server.h:558 ]: [ApiServer] Shutdown finished
[ N 2021-03-18 01:21:58.7810 283848/T6 Ser/Server.h:901 ]: [ServerThr.1] Freed 0 spare client objects
[ N 2021-03-18 01:21:58.7810 283848/T6 Ser/Server.h:558 ]: [ServerThr.1] Shutdown finished
I also did a grep through the logs to find out if I was running out of memory, I know very little about devops and frankly if that’s the issue I’m not sure where I would start. I did find this:
/var/log/kern.log:Mar 17 01:51:45 kernel: [8741879.980107] Out of memory: Killed process 243117 (node) total-vm:976588kB, anon-rss:232900kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:6136kB oom_score_adj:0
/var/log/kern.log:Mar 17 01:52:37 kernel: [8741931.472999] Out of memory: Killed process 227427 (ruby) total-vm:501472kB, anon-rss:114644kB, file-rss:892kB, shmem-rss:0kB, UID:1000 pgtables:432kB oom_score_adj:0
/var/log/kern.log:Mar 17 02:44:46 kernel: [8745060.656866] Out of memory: Killed process 250331 (node) total-vm:1017780kB, anon-rss:258104kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:6980kB oom_score_adj:0
None of these timestamps line up exactly with when the site actually went down (or even the 5-minute window between UptimeRobot checking on it), but it does seem like Ubuntu is killing things due to a memory issue.
I’m looking of course for a solution that will keep Nginx running smoothly all the time and not crashing every few hours. Short of that I’m wondering if there are any clues or logs I should be looking into.
Related
Join 1M+ other developers and:
- Get help and share knowledge in Q&A
- Subscribe to topics of interest
- Get courses & tools that help you grow as a developer or small business owner
Question
Question
These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.
×