Choosing search
0-day streakReview

Your progress

Level 1

0 XP100 XP to level 2
0day streak
0best streak
0topics started
0cards reviewed
    Review cards

    Progress is stored only in this browser. No account is required.

    Choosing a search engine: the 60-second story

    Search series · part 4

    Bazaar has 50 million products. Which engine should run its search?

    Postgres, Elasticsearch, Typesense, Algolia, Vespa… Don’t start from a favourite brand. Start from requirements; each one rules something out.

    Any engine, same shape: the index is a rebuildable copy

    Writes go only to the database; CDC and Kafka feed the index. Search takes the top 20 IDs and confirms price and stock.

    Which engine is best?

    Bad: Compare brands

    Vendors publish fast numbers on someone else’s data.

    Good: Answer the requirements

    Size in two years, query types, freshness, facets, vectors, ops, bill, licence, team. Each rules something out.

    Ask in this order. Each answer rules out a family.

    Workload first, then whether you need a new system, then who runs it, then RAM. Only then compare ranking features.

    200 MB

    read to rank “kurta” once in Postgres

    At 5 million products, “kurta” matches 200,000 rows. Assume 1 KB of tsvector each: ranking reads them all. At 300 queries a second, 60 GB/s.

    Postgres full-text search is enough while…

    1. The table and GIN index fit in RAM
    2. Head queries match under about 10,000 rows
    3. A handful of facets and stemming-level linguistics
    4. Search fits on a replica or two

    At 5 million products, Typesense is simpler and cheaper than Elasticsearch. Easy pick?

    Growth decides

    If 50 million products and vectors are two years away, go straight to Elasticsearch or OpenSearch. Skip the second migration.

    269 GB

    of RAM per Typesense node, vectors alone

    Assume 768 dimensions for 50 million products. Elasticsearch’s int8 HNSW needs 42.4 GB of page cache per copy; binary BBQ, 8.7 GB.

    Remember

    Keep the database as truth and search as a rebuildable copy fed by CDC. Postgres at 50,000 products, a dedicated engine at 5 million, Elasticsearch or OpenSearch at 50 million with vectors.