Your Embedded Vector DB Will Run Out of RAM: Size It Before Production Does

The demo was flawless. You loaded ten thousand support tickets into Chroma running in-process, ran a few semantic searches, and the answers came back in milliseconds. The product owner signed off. Six weeks later, someone ingested the full document archive, and at three in the morning the container got OOM-killed mid-request. No warning, no log line that says "you are about to run out of memory". Just Killed in dmesg and a service that will not stay up.

This is the single most common way an embedded vector database goes to production and then falls over. The problem is not that Chroma, LanceDB, FAISS-in-memory or any of the others are bad. The problem is that "embedded" and "in-memory" are marketing conveniences for prototyping, and nobody puts a number on the ceiling. There is no config key called max_ram that warns you. The ceiling is your machine's RAM, and you hit it as a production incident rather than a build error.

The honest version of the advice most articles skip: scaling an embedded vector store from a demo to a real corpus is not a config change, it is an architecture change. Let me show you how to see the wall before you drive into it.

The error you actually see: nothing useful

The frustrating part is that there is rarely a clean vector-database error. The database does not say "out of memory". The Linux kernel kills the process first. What you find afterwards looks like this:

$ dmesg | tail
Out of memory: Killed process 4812 (python) total-vm:14980112kB, anon-rss:11821340kB

Or, if you are running in a container with a memory limit, the orchestrator reports it as an exit and restart:

State:          Running
Last State:     Terminated
  Reason:       OOMKilled
  Exit Code:    137

Exit code 137 is 128 + 9, meaning the process received signal 9 (SIGKILL) from the OOM killer. There is no stack trace pointing at your vector store because the kernel does not do stack traces. This is why teams chase it for days as an "application memory leak" when the real answer is simpler and more uncomfortable: the index does not fit in RAM, and it never will at this corpus size on this machine.

Why an embedded vector DB scales with corpus size, not traffic

A normal relational database keeps most of your data on disk and pulls pages into a cache as needed. You can have a 500 GB Postgres table on a box with 8 GB of RAM and it works, because the working set in memory is a fraction of the total.

An in-memory vector index is the opposite. To answer "which of these vectors is nearest to my query vector", most approximate-nearest-neighbour indexes (HNSW is the common one) need the whole graph resident in memory. HNSW builds a multi-layer graph of connections between vectors, and traversing it means random-access hops all over that structure. If it lived on disk, every hop would be a disk seek and your millisecond query would become a multi-second one. So the embedded, in-memory design keeps everything in RAM by construction. That is what makes it fast, and that is what makes it a ticking clock.

The consequence: your memory usage tracks the number of documents you have indexed, not the number of queries per second. Doubling your traffic barely moves the RAM line. Doubling your corpus doubles it. Teams instinctively load-test query throughput before launch, watch memory stay flat, and conclude they are safe. They tested the wrong axis.

The sizing math, before it is a 3am incident

You can estimate the RAM you need with arithmetic you can do on the back of a receipt. The dominant cost is the raw vectors:

vector bytes = number_of_vectors x dimensions x bytes_per_float

A standard 32-bit float is 4 bytes. So for common embedding models:

Corpus (chunks)	Dimensions	Raw vectors	With HNSW overhead (~1.5x)
100,000	1536	~0.6 GB	~0.9 GB
1,000,000	1536	~6.1 GB	~9.2 GB
10,000,000	1536	~61 GB	~92 GB
1,000,000	384	~1.5 GB	~2.3 GB

The 1.5x is a rough allowance for the HNSW graph links, the id-to-vector mapping and any metadata you store alongside each vector. It is deliberately a "roughly" number, because real overhead depends on your M parameter (links per node) and how much metadata you attach. If you store a big text field per vector, add that too.

Two things fall out of this table immediately. First, embedding dimension matters enormously: dropping from a 1536-dim model to a 384-dim model cuts your memory by four. If you are memory-bound and your retrieval quality holds up with a smaller model, that is the cheapest win available. Second, ten million chunks at 1536 dimensions wants roughly 90 GB of RAM. On the 4 GB VPS a lot of us start clients on in Nairobi, that is not a tuning problem, it is a different machine and a different design. And renting the 128 GB box to hold it all in memory is often the wrong answer economically, which is the whole point of the next section.

The migration path: embedded to client-server, in-memory to disk-backed

There are two different transitions hiding under "my vector DB got too big", and people conflate them. Keep them separate.

Step 1: Establish where you are and where the wall is

Measure the actual per-vector footprint on your data rather than trusting my table. Load a known number of vectors and read the process RSS:

# after loading N vectors into your embedded index
ps -o rss= -p $(pgrep -f your_app) | awk '{print $1/1024 " MB"}'

Divide by N to get bytes per vector, multiply by your projected corpus, and compare to the box's RAM minus what the rest of the app needs. If the projection is within, say, 60 percent of physical RAM, you have no headroom for growth and you should plan the move now, not later.

Step 2: Move from embedded to client-server if you only need to share the index

If the index still fits in RAM but you need multiple app processes or web workers to query it, the fix is to run the vector DB as its own server process instead of in-process. Chroma, for example, runs in client-server mode; Qdrant, Weaviate and Milvus are servers by design. This does not reduce total memory, it just puts the index in one process that owns the RAM, so your web workers stay small and you stop loading a copy of the index per worker. That last point catches people: running four Gunicorn workers each with an embedded index is four copies of the same memory.

Step 3: Move to a disk-backed index if the corpus genuinely will not fit

When the corpus exceeds RAM no matter how you slice it, you need an index that keeps vectors on disk and holds only a working set in memory. Qdrant supports on-disk vectors and memory-mapped storage; Milvus is built for disk-backed segments; LanceDB stores vectors in an on-disk columnar format and reads what it needs. You trade some query latency for a corpus that scales past your RAM. This is the real "architecture change": your retrieval layer is now a service with its own storage, capacity planning and backups, not a library you imported.

Step 4: Consider quantization before you rent a bigger box

Scalar or product quantization stores each vector dimension in fewer bytes (for example int8 instead of a 4-byte float), cutting memory by roughly four with a small, measurable recall cost. Qdrant and Milvus both support it. Quantization can turn a 90 GB requirement into something that fits in a 32 GB box. Measure the recall drop on your own eval set before committing, because the acceptable trade-off depends entirely on your retrieval quality bar.

What people get wrong

"Just add swap." Adding swap to an OOM-killed vector index is the worst common advice. HNSW traversal is random access across the whole graph; if that graph lives in swap, every query pages from disk in a pattern designed to defeat caching. Your service will not crash, it will do something worse: it will get so slow that requests time out while the disk thrashes at 100 percent. You have converted a clean crash into a mystery brownout. Do not do this.

"Set a memory limit and it will handle it." A container memory limit does not make the index smaller. It just changes who kills the process from the host kernel to the cgroup. The index still needs the RAM it needs.

"We load-tested at 500 requests per second, we are fine." As covered above, query load barely touches the memory line. Testing throughput proves nothing about whether next quarter's corpus fits. Load-test ingestion: script loading your projected document count and watch RSS climb.

Verification: prove it fits before you ship

Do not launch on faith. Run the projected corpus through ingestion on a staging box that mirrors production RAM, and watch memory the whole time:

# run in another terminal while you bulk-ingest the projected corpus
watch -n 5 'ps -o rss= -p $(pgrep -f your_app) | awk "{print \$1/1024/1024 \" GB\"}"'

You want to see the memory curve flatten with clear headroom below physical RAM once the full corpus is loaded. If it climbs steadily toward the ceiling and the box has 16 GB while the curve is heading past 12, you have your answer before a customer ever does. This is the same discipline that keeps a RAG system from falling apart on real production data: you test at production scale, not demo scale.

When it is still broken

If you have sized correctly and still hit OOM, check these in order:

• Multiple index copies. Count your worker processes. Each Gunicorn/Uvicorn worker with an embedded index holds its own full copy. Move to client-server so the index lives once.
• Metadata bloat. If you store the full document text as metadata on every vector, you are holding your entire corpus in RAM twice. Store an id and fetch the text from Postgres or object storage at read time.
• A genuine leak during ingestion. If memory climbs without the corpus growing, the batch loader may be holding references. This is a different problem; the technique for chasing it is the same one covered in what actually helps a worker memory leak.
• Concurrent writers. If two jobs write to the same embedded index at once you can get corruption on top of memory pressure. That failure mode is covered in why two jobs writing to the same vector store can corrupt it.

The core lesson is worth repeating because it is the thing nobody puts on the box: an embedded vector database is a library that trades RAM for speed, and its ceiling is your machine. Do the arithmetic before launch, load-test ingestion rather than queries, and when the corpus outgrows the box, accept that you are making an architecture change and not flipping a flag. The teams that plan this transition ship calmly. The teams that discover it via exit code 137 do not.