Vector databases

A vector index answers “which stored vectors are closest to this query vector?” Approximate nearest-neighbor algorithms avoid scanning every row. The usual ones in interviews are HNSW (a layered graph you walk greedily; high recall, more memory, slower builds) and IVF (cluster the space, search a few clusters; faster builds, more tuning of lists and probes). Distance is cosine, dot product, or L2, and it must match how the model was trained.

What you store

The source of truth is the document, the profile, or the chunk, in a database you can rebuild from. The vector store (pgvector, a dedicated engine, the vector type in your search cluster) holds the embedding, the model version, and the id. Text you will show to the user is fetched from the source by that id, not trusted as the only copy inside the index.

Chunking decides quality more than the brand of index. Too long and the embedding averages unrelated ideas. Too short and you lose context. Overlap chunks if sentences cross a boundary. Store the chunk’s source offset so you can cite it.

Filters

Real queries are “closest, among documents this tenant can see, of this type, newer than T.” That is a filtered ANN. Two strategies:

• Post-filter. Search wide, drop mismatches. Simple, and recall falls off a cliff when the filter matches a small fraction of the corpus.
• Pre-filter. Restrict the candidate set first, then search. Better when the filter is selective. Harder if the index structure does not support it, and the remaining set may be too small for a good graph walk.

Say which you are doing. Measure recall on a labeled set with the filter on, not on a vendor’s unfiltered benchmark.

Hybrids and operations

Exact tokens (a name, a SKU, an error code) should also hit a lexical index. Merge with a simple rank fusion if you do not have a learned reranker, then rerank the top few dozen with a slower model if quality matters.

Updating one vector is cheap. Changing the model is a full re-embed: write a second index, dual-write new documents, backfill, then flip the query path. Keep the model version on every row so you can prove you are not mixing spaces.

Latency: a warmed HNSW query is milliseconds to tens of milliseconds for a million vectors, before you add a network hop and a rerank. Do not put a 200 ms rerank on a typeahead keystroke. Do put it on a search results page if it moves relevance.