Question
How to determine how memory and CPU is needed for knowledge bases
- Posted on April 30, 2025
- Managed DatabasesDatabaseAI/MLDatabase Dev/Management
Asked by setec
How can I determine how many knowledgebases (and krowledegebase sources) can be stored on a database server of a given size?
Obviously if the disk space runs out the server needs to be upgraded, however support has suggested that intermittent problems with knowledge bases are caused by a lack of memory but I’m not seeing >80% memory usage when these problems occur.
Adding new database servers for every knowledgebase gets expensive but it isn’t clear how many knowledgebases (and of what size) and single server can handle.
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.
Become a contributor for community
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
DigitalOcean Documentation
Full documentation for every DigitalOcean product.
Resources for startups and SMBs
The Wave has everything you need to know about building a business, from raising funding to marketing your product.
The developer cloud
Scale up as you grow — whether you're running one virtual machine or ten thousand.
Get started for free
Sign up and get $200 in credit for your first 60 days with DigitalOcean.*
*This promotional offer applies to new accounts only.
Hey @setec,
Yeah, this part’s a bit tricky. From what I can tell, there’s no hard limit published anywhere, I think it mostly depends on how big your sources are and how often queries are running. Even if memory usage doesn’t look high overall, it might be hitting limits during spikes or when multiple things happen at once (like indexing and querying).
I think that memory issues can cause odd behavior, but like you, I didn’t see anything obvious in the usage stats. Would be nice if there were some kind of sizing guide or even just example setups. Definitely worth sharing that feedback with support!
Until then, I’m kind of playing it by feel, testing a few knowledgebases per instance and scaling when stuff starts getting weird.
Would love to hear if you hear back from support for a better way to estimate this.
- Bobby