I upgraded my cluster to use 4vCPU and 8GB memory nodes. The cluster has two of these nodes. I only added the NGINX Ingress Controller as an addon. I deployed two pods, each with 400M cpu limit and request. On the K8s Dashboard it shows that the nodes are each using about 1 CPU and 500M CPU worth of request/limit respectively. So there are still 3 CPUs worth of resources on each of them.

When I try to run a 3rd pod, with 400M CPU limit/request, I get insufficient CPU error.

Here is the request/limit that all three pods have configured.
resources:
limits:
cpu: 400M
memory: 400M
requests:
cpu: 400M
memory: 400M

Resource and limit of the two nodes.
1.00 (25.05%)
502.00m (12.55%)

902.00m (22.55%)
502.00m (12.55%)

Error on the pod.
Events:
Type Reason Age From Message

Warning FailedScheduling <unknown> default-scheduler 0/2 nodes are available: 2 Insufficient cpu.
Warning FailedScheduling <unknown> default-scheduler 0/2 nodes are available: 2 Insufficient cpu.

I upgraded my cluster to these more expensive nodes. I feel that the cheaper nodes should have handled my small load, but upgraded anyways. I am not going to upgrade again, 4 CPU and 8GB of ram of more sufficient (and pricey) than what I need, and yet I am still getting insufficient CPU.