My server instance is a 2GB, 1 vCPU.
I’m attempting to diagnose an issue with very high load average on my managed MySQL database. Every hour, my app dispatches a collection of jobs that perform an “upsert” of anywhere from 100s to 1000s of records.
These records are fetched from an external API in batches of 100, processed and then added to the database. Each “page” from the API, is generally processed in ~10 seconds. So for each job, one DB query every 10 seconds or so.
This process is identical every hour.
During the first 1-3 minutes, the load average spikes to as much as 10, then as the import process continues, the load average calms down to normal levels.
What is strange:
See the monitoring screenshot for more detail: https://ibb.co/S5Qr50C
Notice how at 1pm, the load is high, but at 2 and 3pm, it is much smaller, despite the fact that the import process is identical. Meanwhile, CPU load is lower at 1pm, but higher at 2pm.
Also note that under connection status, the threads in use is well below the limit.
Since the DB is managed, I’m not really able to do much other than point out what I’m seeing.
Any ideas? Thanks so much!
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.
Join our DigitalOcean community of over a million developers for free! Get help and share knowledge in Q&A, subscribe to topics of interest, and get courses and tools that will help you grow as a developer and scale your project or business.
Click below to sign up and get $200 of credit to try our products over 60 days!