A query lands among kurtas without sharing a word
A model maps each item to 384–3,072 floats; meaning lives in directions. Query and documents share the model, so related becomes nearby.
Cosine 0.82: a match?
Bad: Yes, above 0.8
An old model’s cut-off yields confident nonsense.
Good: Depends on the model
multilingual-e5 scores mostly sit at 0.7–1.0. Rank; calibrate cut-offs on labelled pairs per model.
Opposite intent, same neighbourhood
Negation, codes and numbers barely move a vector. Turn constraints into filters; keep BM25 for exact tokens.
$2.22
to embed 10,000 company documents
222,000 chunks of 500 tokens make 1.4 GB of float32 vectors. That’s a pgvector-sized problem.
Fixed windows can cut the answer in half
Chunk 7 says “carry forward”, chunk 8 says “45 days”; neither answers. Split on headings and prefix the heading path.
You upgrade the embedding model and embed only new documents with it.
The trap
Old vectors live in another space: plausible nonsense, no error. Version every vector; re-embed everything into a new index.
Pick it, or not
- Pick: users describe what they want, not name it
- Add BM25: SKUs, order IDs, error codes
- Filter, don’t embed: “under ₹500, size M”
- Skip: under ~200,000 tokens? Put it in the prompt
Say it like this
Pick embeddings when users describe, not name, what they want. They blur codes, numbers and negation, so run BM25 beside kNN and rerank. Version every vector; a model change means re-embedding everything.