JS Logging in post webhook async function

Question

I have a javascript server that exposes a webhook endpoint that pushes the request body onto a message queue and returns a response code immediately. Asynchronously, the message is then processed off the queue. For some reason, logging output of the async process isn’t always being displayed. After a fresh restart of the App (flush cache), logging occurs for the first 2 events, then stops. When I run the process locally, logging is not interrupted after any amount of events processed. What could be going on here? Any ideas or suggestions would be appreciated.

Larry Singleton · Accepted Answer

So, it turns out that logging wasn’t the issue.The queue was the culprit. I am using “fastq” and declared the queue globally like this: const fastq = require("fastq"); const queue = fastq(processMessage, 2); Notice the “2” - this is supposed to be the number of concurrency threads.However, after the 2nd message, no further processing was occurring (not even logging). Perhaps an “undocumented feature”. I replaced the above two lines with the below and sanity was returned, allowing >2 messages to be processed. const queue = require("fastq").promise(processMessage, 2); #fastq

Bobby · Answer

Hey! This is quite interesting, one guess here would be that the logging lib that you are using might be buffering output and not flushing it immediately. Can you confirm what is the logging library that you are using? Usually you could try forcing a flush after each log write. For example if you are using winston , you can set { exitOnError: false } and use transports that flush logs promptly: If false, handled exceptions will not cause process.exit One other suggestion here could be to make sure that your async functions are properly handling errors and not silently failing, for example you can try to wrap your async code in try/catch blocks and log any errors: async function processMessage(message) { try { // Your processing logic here console.log('Processing complete:', message); } catch (error) { console.error('Error processing message:', error); } } Feel free to share more information about the exact setup or a code snippet and I will be happy to try and reproduce this on my end! - Bobby

Matt Arnold · Answer

This sounds like it could be a caching issue with the logging mechanism. Since local runs don’t show the problem, it might be related to your deployment environment. Try increasing the log verbosity or disabling any log rotation happening after the first few messages.

Become a contributor for community

DigitalOcean Documentation

Resources for startups and AI-native businesses

The developer cloud

Start building today

Report this

JS Logging in post webhook async function

Related questions

In the App Platform, the deployment failed during deploy phase - no deployment logs !

Unescaping message field in forwarded log message from App Platform to managed OpenSearch

How to use doctl apps logs with --type=run_restarted

How to make doctl apps logs work using --type=run_restarted

Session log not viewable when using the agent API

Does a DO app platform service log requests to the console by default?

Activation records and log messages not appearing for my Serverless Function

Generate an Error Log Digest Email

App platform worker hangs, sometimes giving exit code 12

Custom log format for DO App Platform

Postgres DB logs are not forwarded to the Open Search

Should the logtail token be treated as a secret?

Report this

JS Logging in post webhook async function

Related questions

In the App Platform, the deployment failed during deploy phase - no deployment logs !

Unescaping message field in forwarded log message from App Platform to managed OpenSearch

How to use doctl apps logs with --type=run_restarted

How to make doctl apps logs work using --type=run_restarted

Session log not viewable when using the agent API

Does a DO app platform service log requests to the console by default?

Activation records and log messages not appearing for my Serverless Function

Generate an Error Log Digest Email

App platform worker hangs, sometimes giving exit code 12

Custom log format for DO App Platform

Postgres DB logs are not forwarded to the Open Search

Should the logtail token be treated as a secret?