Related

Product Managed Kubernetes on DigitalOcean Product
Will kubernetes have autoscaling by default? Question Question
Firewall blocks hostPort Question Question

Question

System.OutOfMemoryException .net core docker in k8s

Posted January 18, 2021 160 views
Kubernetes

Hello ,

Not sure how to go about troubleshooting this issue and thought maybe someone could point me in the right direction.
I’m running a .net 5 ( tried with 3.1 as well) on k8s with a docker running the 5.0-buster-slim.
When performing continuous request operations on large datasets I get a System.OutOfMemoryException.
The node I have has 4gb ram with 2.5 usable.
The deployment allows for 2gb ram for my pod.
I tried setting
“System.GC.Server”: false
to work in a workstation GC mode.
as well as setting the System.GC.HeapHardLimit

Appreciate any reply.

Thank you.

Add a comment
nnh
By:
nnh

Related

Product Managed Kubernetes on DigitalOcean Product
Will kubernetes have autoscaling by default? Question Question
Firewall blocks hostPort Question Question
Add a comment
1 comment
  • nabsul January 18, 2021
    Reply Report

    Are you sure you don’t have a memory leak somewhere? It’s hard to give concrete advice without knowing more detail, but I would check if there might be a part of the code that’s holding onto objects indefinitely, causing you to eventually run out of memory.

    If you’re using EntityFramework or some other ORM, it might be worth looking into the docs to see how it manages objects and when it destroys.

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

×
1 answer
hejeroaz January 18, 2021

Hello,

the solution simple is to add more resources to your pods: CPU and memory
the elaborate solution: refactor your code to reduce async and await calls

the GC is a headache …

you can get more details running your app with this large dataset running in localhost with Docker/docker-compose and controlled resources, maybe monitoring container resources with DockStation …

currently, we handle the extraction, transformation, and loading of more than 1M rows per request smoothly with 8G ram and 2 vCPU assigned to the pod, ending with more than 60M of rows in the Postgresql database server in 20 minutes …

my recommendation: assign more resources to your pods …

best regards ;)

Reply Report
  • nnh January 18, 2021

    Thank you for your reply.
    something doesn’t add up .
    the request completes successfully first x times ,
    it’s just that the memory keeps accumulating,
    adding more resources only means I’ll push my OOM exception further away in time , e.g. 2x request instead of 1x .
    I also must be doing something wrong since I don’t have nearly as many records as you do (about a million or so and I’m pulling 10th of a 1/3rd I need each time)

    Reply Report
Submit an Answer

Related

Looking for something else?

Ask a new question Search for more help