Layer 3 · Go deeper · 7 min
Go deeper
Size it, build it, then make the calls an interviewer will push you on.
Back of the envelope
Size it up
Tap through each step of the arithmetic.
How many GPUs does a cross-encoder reranker need at 100 queries a second?
What does re-embedding 50 million product titles cost after a model change?
Build-up · 6 steps
Index a policy, then answer from it
The standard RAG retrieval path. Chunk and embed documents once, offline; then embed each question with the same model, retrieve wide and rerank to five.
Step 1 of 6
Chunk by headings
A 30-page PDF as one vector won’t surface its earned-leave paragraph. Split on headings, a few hundred tokens a chunk, and prefix the heading path so each chunk names its subject.
4 interview moments
You’re the candidate
The interviewer pushes on the hard parts. Pick an approach and see how it holds up.
Deep dive 1 of 4
Interviewer
“An HR assistant answers questions from 30-page policy PDFs. How do you split them before embedding?”
Your move. Pick an approach:
Deep dive 2 of 4
Interviewer
“Vector search finds decent chunks, but the LLM reads only the top 5, and they’re often slightly off. What do you add?”
Your move. Pick an approach:
Deep dive 3 of 4
Interviewer
“Employees ask in Hindi, Tamil and romanised Hinglish over English documents. How do you choose the embedding model?”
Your move. Pick an approach:
Deep dive 4 of 4
Interviewer
“Users say the RAG bot’s answers got worse. The team wants a bigger embedding model. What do you do first?”
Your move. Pick an approach: