Hello ,
Not sure how to go about troubleshooting this issue and thought maybe someone could point me in the right direction. I’m running a .net 5 ( tried with 3.1 as well) on k8s with a docker running the 5.0-buster-slim. When performing continuous request operations on large datasets I get a System.OutOfMemoryException. The node I have has 4gb ram with 2.5 usable. The deployment allows for 2gb ram for my pod. I tried setting “System.GC.Server”: false to work in a workstation GC mode. as well as setting the System.GC.HeapHardLimit
Appreciate any reply.
Thank you.
These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.
Hello,
the solution simple is to add more resources to your pods: CPU and memory the elaborate solution: refactor your code to reduce async and await calls
the GC is a headache …
you can get more details running your app with this large dataset running in localhost with Docker/docker-compose and controlled resources, maybe monitoring container resources with DockStation …
currently, we handle the extraction, transformation, and loading of more than 1M rows per request smoothly with 8G ram and 2 vCPU assigned to the pod, ending with more than 60M of rows in the Postgresql database server in 20 minutes …
my recommendation: assign more resources to your pods …
best regards ;)
Are you sure you don’t have a memory leak somewhere? It’s hard to give concrete advice without knowing more detail, but I would check if there might be a part of the code that’s holding onto objects indefinitely, causing you to eventually run out of memory.
If you’re using EntityFramework or some other ORM, it might be worth looking into the docs to see how it manages objects and when it destroys.