High Private Bandwidth & CPU Load on Laravel Servers — Need Help Identifying Root Cause

Hey folks, I’m dealing with a strange performance issue across my Laravel infrastructure and would appreciate expert eyes on this.

🧱 Infrastructure Overview:

I have three separate servers on DigitalOcean:

1. Backend Server

   • 2 vCPU, 4GB RAM

   • Laravel app handling: queues, scheduled jobs, email notifications, Laravel Horizon, admin panel

   • Horizon processes ~17,000 jobs/hour on sometimes

2. Frontend Server

   • 4 vCPU, 8GB RAM

   • Laravel + WordPress serving the public web/app interface

   • No cron jobs or queue workers here — it’s just for public traffic

3. Database Server

   • Monitored using DigitalOcean metrics

🧪 Symptoms & Observations:

1. Backend Server Issues

• CPU usage is constantly 100% even when no jobs are actively processing

• Horizon dashboard shows idle workers

• top and ps show multiple php-fpm: pool www processes using 5%+ CPU each

• Private inbound bandwidth spikes to 100–160 Mbps

• it happens in sequence, after 5-6 hours.

2. Frontend Server

• Experiences private outbound bandwidth spikes (same time as backend spikes)

• No cron jobs, but still shows significant bandwidth usage

3. Database Server

• Private outbound traffic (to backend) also spikes up to 160+ Mbps

• Coincides with backend server inbound private bandwidth spikes

🧩 Investigations So Far:

• Checked Nginx logs: Only minimal legitimate traffic (~5 requests in the past hour from any IP)

• Detected a research scanner from Ruhr University Bochum hitting .env paths (harmless but noted)

• No cron jobs or unusual shell processes on the frontend

• Enabled MySQL slow query log — will monitor if Laravel is overloading DB

• Laravel log doesn’t show any failed jobs or errors

---

🧠 My Theory (but unsure)

It seems like:

• The frontend server may be initiating requests to the backend (APIs or queue triggering?)

• Backend may be doing internal API polling or syncing causing load loops

• Database is returning large responses, adding to outbound bandwidth and backend CPU load

• PHP-FPM is stuck in high CPU usage loops possibly because of Laravel jobs or WordPress plugin issues

💭 Questions I’m Stuck On:

1. What could cause such high private traffic between servers if no visible external traffic or jobs are running?

2. Could there be Laravel-to-Laravel internal calls misfiring, especially between frontend and backend?

3. How to trace which requests (internal or external) are causing this load — ideally in real time?

4. Are there Laravel debugging tools (beyond Telescope) that can help me pinpoint this traffic pattern?

I’ve attached Backend Server usage graphs from DO.