Design People You May Know

People You May Know looks like a small widget: three faces, a “Connect” button and an “×”. Underneath, it is one of the hardest recommendation problems in a social product, because the thing being recommended is another person and the evidence lives in a graph with a billion nodes. Three things make it interesting: generating candidates from friends-of-friends when a few members have tens of thousands of connections, choosing what to compute overnight, what to update within minutes and what to compute in the 200 ms of a page load, and doing all of it without exposing things people never meant to share.

Understand the problem

PYMK is link prediction turned into a product: given the graph today, which edges are most likely to form next, and which of those are worth suggesting? Unlike a news feed, which ranks content from people you already follow, PYMK changes the graph itself — every accepted suggestion becomes an edge that shapes everyone’s next suggestions.

Functional requirements

1. Members should be able to see a ranked list of people they may know, on a dedicated page and in small widgets elsewhere in the app, each with a short reason such as “14 mutual connections”.
2. Members should be able to send an invitation or dismiss a suggestion, and never see a dismissed person, an existing connection, or someone either of them has blocked.
3. Members should see suggestions that reflect recent changes — a new connection, a new job — within minutes.
4. New members should get useful suggestions within a minute of signing up, before they have a single connection.

Non-functional requirements

• Scale. Assume 1 billion members, 300M monthly active and 100M daily active. Active members have a few hundred connections each; assume 250, the example LinkedIn’s graph team uses for a typical first-degree network.
• Latency. A suggestions request returns in under 200 ms at p99, so a widget never delays the page it sits on.
• Freshness. The base list may be up to a day old; a new connection should change both people’s suggestions within about 5 minutes.
• Availability. 99.9%. PYMK is never critical: if anything behind it fails, serve yesterday’s list or hide the widget.
• Consistency. Eventual for candidates and scores. Hard filters are not negotiable: a blocked or dismissed person must never appear, even if the candidate list is stale.
• Privacy. Never reveal something the viewer could not otherwise know: who uploaded whose phone number, who viewed whom, or anything that hints at health, legal or personal circumstances.
• Cost. The nightly computation over the whole graph must fit a fixed compute budget.

Out of scope

• The invitation and notification flow after someone presses “Connect”, apart from rate limits.
• Model training infrastructure. We design which labels and features exist and how models are served.
• Search, the feed and messaging — separate systems that we read signals from.
• Group, page and job recommendations — same funnel, different items.

The set-up

Plan the approach

Start with the obvious design — compute friends-of-friends when the page loads — and show where it breaks. Then the deep dives cover candidate generation, scoring, the batch/nearline/online split, the serving path, cold start, privacy and abuse, and evaluation. Say the shape early: “a recommendation funnel over a graph: cheap and wide offline, narrow and fresh online, with hard filters at the very end.”

Core entities

• Member. ID, profile fields, privacy settings, activity level.
• Connection. An undirected edge (a, b, created_at), stored in both directions as adjacency lists.
• Affiliation. Company, school or city with start and end years: “Infosys, Pune, 2016–2020”.
• Invitation. Sender, recipient, time, status (pending, accepted, ignored).
• Dismissal and block. Viewer, other member, time. Dismissals expire after months; blocks never do and apply in both directions.
• Candidate list. For each viewer, a few hundred (candidate, source, score, features) rows, versioned by the job that produced them.
• Feedback event. Impression, invitation, dismissal, acceptance — each tagged with the impression ID it came from. These become training labels.
• Hashed contact. A normalised, hashed phone number or email from an address-book upload, with the uploader and their consent.

API

Read suggestions. The surface lets the widget ask for 3 and the page for 20; tracking_id ties every later action back to this impression.

GET /v1/pymk?surface=widget&count=3
→ 200
{
  "suggestions": [
    { "member_id": "m_4410", "name": "Priya Nair", "headline": "Data engineer at a fintech",
      "reason": { "type": "mutual_connections", "count": 14 },
      "tracking_id": "imp_8c2f…" }
  ],
  "next_cursor": "c_2"
}

Act on a suggestion.

POST /v1/invitations
{ "to": "m_4410", "tracking_id": "imp_8c2f…" }
→ 201 Created                      or   429 Too Many Requests (weekly invite cap)

POST /v1/pymk/dismissals
{ "member_id": "m_4410", "tracking_id": "imp_8c2f…" }
→ 204 No Content

Match contacts (only with explicit consent). The app normalises phone numbers and emails and sends hashes; the server never stores the raw address book.

POST /v1/contacts:match
{ "consent": true, "hashed_ids": ["h_91c0…", "h_7a1e…"] }
→ 202 Accepted

Back-of-envelope estimates

These numbers decide what can run on request and what must run overnight. (Method: back-of-envelope estimation.)

The graph. Across all members the average degree is lower than for active ones; assume 100. 1B × 100 ÷ 2 = 50B connections, stored in both directions as 100B adjacency entries. At 8 bytes per ID that is 800 GB; Twitter’s Who to Follow team measured about 5 bytes per edge in compact adjacency lists, about 500 GB. Either way, more than one comfortable machine: 16 partitions of about 50 GB, three replicas each, 48 graph nodes.

Friends-of-friends per member. An active member with 250 connections, each with 250 of their own, has 250 × 250 = 62,500 two-hop paths. Many paths reach the same person, which is the point — LinkedIn’s graph team describes a typical second-degree network as at least 50,000 entities.

The nightly job, naively. 300M monthly-active members × 62,500 paths ≈ 1.9 × 10¹³ path records. At 16 bytes per (viewer, candidate) record before aggregation, that is about 300 TB to shuffle every night — before supernodes, which make it much worse. Pruning is mandatory.

Batch output. Keep the top 500 candidates per active member at about 40 bytes each (ID, score and a few compact features): 300M × 500 × 40 B = 6 TB per night to bulk-load into a key-value store.

Requests. 100M daily members × 3 suggestion loads (widget, page, after accepting an invitation) = 300M/day ≈ 3.5k/s on average, about 10k/s at peak. Re-ranking 500 candidates each means 5M candidate scores per second at peak.

Feedback. 300M loads × 10 suggestions seen ≈ 3B impressions/day, about 35k events/s. Assume 1% lead to an invitation (30M/day, about 350/s) and half are accepted: 15M new connections/day, about 170/s. Assume 2% are dismissed: 60M/day; kept 180 days, 60M × 180 ≈ 10.8B rows × 16 B ≈ 170 GB.

New members. Assume 300k sign-ups a day. If every one shared an address book of 400 entries, that would be at most 120M hash lookups/day, about 1.4k/s.

High-level design

Start with the simplest thing that works for a few million members, one requirement at a time.

Members should be able to see people they may know

The first version computes everything on request. A graph service holds every member’s connections as adjacency lists in memory, partitioned by member ID. The PYMK service asks it for the viewer’s friends, then for each friend’s friends, counts how many paths reach each person, removes the viewer’s existing connections, and returns the top 20 by mutual-connection count — the simplest link predictor there is. It works because most new friendships close a triangle: in a Facebook study, 92% of new friendships in Iceland’s network were between friends-of-friends.

Members should be able to invite or dismiss, and never see the wrong person

“Connect” goes to an invitation service, which records a pending invitation and enforces a weekly cap. “×” writes a row to a dismissals store. Blocks live in their own store. On every request, after ranking, the PYMK service removes anyone the viewer is already connected to, has a pending invitation with, has dismissed, or who has blocked the viewer or been blocked by them.

These filters run at read time against live stores, not inside the candidate computation. That is what lets the candidate list be a day old later in the design: a list built last night may contain someone you connected with this morning, but the filter catches it. LinkedIn’s PYMK worked this way from early on: dismissals lived in a separate store and were subtracted from the precomputed recommendations at page load.

Members should see suggestions that reflect recent changes

When Ravi accepts Asha’s invitation, the graph service writes the new edge in both directions and publishes an edge-added event to Kafka. In the on-request design nothing else is needed — the next request walks the new edge. But once candidates are precomputed (they will be), that event is what keeps them fresh: a nearline job merges Ravi’s strongest connections into Asha’s candidate list and Asha’s into Ravi’s, within minutes.

New members should get useful suggestions within a minute

A member who signed up 30 seconds ago has no edges, so friends-of-friends returns nothing. Their first suggestions come from what they have told us: the company and school they typed (an affiliation index finds members with overlapping years), their address book if they agree to share it (a contact matcher maps hashed phone numbers and emails to members), and the person who invited them, whose own network is a strong prior.

This works for a small network. At a billion members it fails on cost and latency: a single 2-hop request reads 62,500 adjacency entries spread across most of the 16 partitions, a member connected to a few hubs reads far more, and ranking by mutual count alone ignores everything else we know. The deep dives fix candidate generation, scoring and the compute split, then tighten serving, cold start, privacy and evaluation.

Deep dives

How do we generate candidates without the friends-of-friends explosion?

Friends-of-friends is the best single source: most new connections close a triangle. Its cost is the problem. The work of a 2-hop join is not “members × d²” but the sum over every intermediate person z of deg(z)², because each of z’s connections becomes a candidate for each of the others. Degrees are heavy-tailed — recruiters, founders and campus ambassadors have tens of thousands of connections — and hubs appear in many people’s networks. A single member with 30,000 connections contributes 30,000² = 9 × 10⁸ paths on her own; ten thousand such hubs would add 9 × 10¹² paths, almost half again on top of the naive nightly estimate. And they are the weakest evidence: being connected to the same recruiter says almost nothing about whether two people know each other.

Bad: exact 2-hop on every request. Asha’s 250 friends are spread across most of the 16 graph partitions, so one request fans out to nearly all of them and waits for the slowest. LinkedIn’s graph team describes the same join on disk as “effectively 250 random look-ups”. At 10k requests a second that is 625M adjacency reads a second before a single model runs, and any viewer connected to a hub reads tens of thousands more.

Good: precompute 2-hop in batch, with caps. Run the join offline and bound each intermediate’s contribution: sample at most 1,000 of a hub’s connections, preferring recent and frequently-interacting ones, and skip intermediates above a ceiling such as 20,000 altogether. A hub of 30,000 then contributes 30,000 × 1,000 = 3 × 10⁷ paths instead of 9 × 10⁸ — 30 times less — and Adamic–Adar weighting (next deep dive) discounts hubs anyway. Keep the top 1,500 per viewer by weighted score. Twitter’s Who to Follow used the same idea in its random walks, with explicit parameters to sample outgoing edges at high-degree vertices.

Great: several capped sources, merged. Friends-of-friends misses people you know who share no connections with you — a school friend who just joined, a colleague from another office. Add sources that see what the graph cannot, each producing its own top-k with a source score:

• Random walks (personalised PageRank). Walks that restart at the viewer reach 3 and more hops for members with few connections. A plain 3-hop expansion is out of the question — the Facebook study notes some users would have a hundred million nodes at 3 hops — but a walk only visits a few thousand. LinkedIn reports that walk-based candidates 5 or 6 hops away still help under-connected members.
• Embeddings. A two-tower model maps members into vectors from company, school, skills, location and activity; candidates are nearest neighbours of the viewer’s vector. See embeddings and semantic search.
• Affiliation overlap. Same company and overlapping years, same school and graduation year, same small city.
• Contacts and activity. Address-book matches and recent interactions, subject to the privacy rules later in this article.

The pool is about 3,000 candidates — LinkedIn’s first stage keeps 3,000 to 5,000 and measures it by recall: did the people this member went on to connect with make it into the pool? Each row keeps its source and source score, which become ranking features.

How do we score candidates — and avoid learning our own biases?

Ranking by mutual count alone is a decent baseline. The link-prediction literature gives better ones: Liben-Nowell and Kleinberg compared common neighbours, Jaccard, Adamic–Adar, Katz and rooted PageRank as predictors of future links. Adamic–Adar is the one to remember: sum, over each mutual connection z, of 1 / log(deg(z)). A mutual friend with 40 connections is strong evidence; a mutual recruiter with 20,000 is almost none.

Hand-built scores like these become features for a learned model, alongside many others:

• Graph: mutual count, Adamic–Adar, Jaccard, personalised PageRank score, whether the candidate is 2 or 3 hops away.
• Affiliation: same company with overlapping years, same school and graduation year, same city, the size of that company or school (sharing a 50-person startup means far more than sharing a 300,000-person IT services firm).
• Interaction: profile views, messages, comments on each other’s posts, searches for each other’s names.
• Each side’s behaviour: how often the viewer invites and how often the candidate accepts, and how many invitations the candidate received this week.
• Source: which generator produced the candidate and its source score.

Labels. Each impression is a training example: an invitation is a positive, an accepted invitation a stronger one, a dismissal a firm negative, and an impression with no action a weak negative, sampled down because positives are rare. With about 1% of impressions leading to an invitation, class imbalance is the norm. Rather than one score, train several heads — P(invite | shown), P(accept | invited) and the expected value of the connection, such as later messages — and combine them; LinkedIn tunes the combination weights with Bayesian optimisation. Optimising invitations alone is a trap: it rewards suggesting people who will never accept.

A funnel, not one model. Scoring 3,000 candidates with a deep model for 300M members every night is wasteful, and doing it per request is impossible. So stages narrow the list, each with its own metric.

Feedback loops. The model only learns from people it chose to show, so its mistakes reinforce themselves. LinkedIn found exactly this: frequent members are over-represented in training data, so they are recommended more, make more connections, and become even more represented. Four counter-measures:

• Explore. Fill 1–2% of slots with candidates from lower in the ranking and log the probability each was shown with, so training can correct for what the model chose.
• Log position. Train with the slot position as a feature and set it to a constant at serving time, so the model learns who is relevant rather than who happened to be shown first.
• Pick labels with care. LinkedIn trains its embedding retriever with impressions, not invitations, as positives, because infrequent members receive fewer invitations and would otherwise be pushed out of everyone’s candidates.
• Re-rank for fairness. Giving qualified infrequent and frequent members equal representation raised invitations to infrequent members by 5.44% and their new connections by 4.8%, with no loss for frequent members.

What runs in batch, what runs nearline, and what runs online?

This is the central trade-off of the design: freshness against cost. (See also batch vs stream processing.)

Batch only. A nightly Spark job reads a graph snapshot and the day’s logs, runs the capped joins, random walks and embedding lookups, applies the light model and writes the top 500 candidates per active member — about 6 TB. A bulk-load job writes it into a read-optimised key-value store as a new version: loaded in the background, switched over when complete, with the previous version kept so a bad model can be rolled back in minutes. LinkedIn has run PYMK this way for a long time: in 2012 it processed hundreds of terabytes daily and served the results from read-only Voldemort stores at under 5 ms median; today its Venice platform runs more than 1,400 such push jobs a day. You gain cheap reads, heavy features and safe rollback. You pay with lists up to a day old, compute spent on members who will not visit, and new members who are invisible until the next run. Twitter’s Who to Follow hit the same wall: reloading its in-memory graph more than once a day was impractical, which hurt new users most.

Online only. Compute everything per request from a live graph. LinkedIn’s graph system does compute a member’s second-degree network on each visit and caches it, with a hit rate above 90%, to show “2nd” badges and mutual counts. But at 10k requests a second, generating and deep-scoring candidates live means 625M adjacency reads a second plus thousands of model evaluations per request. Fresh, and unaffordable.

Great: hybrid — three lanes over one store.

• Batch, nightly: breadth. All sources, all members, the light model; 6 TB pushed as a new version.
• Nearline, minutes: new connections. Each accepted invitation produces an edge-added event; a stream job reads each new friend’s strongest connections (capped at 250), scores them with the light model and merges them into the other person’s list. 15M new connections a day means 30M list merges, about 350 a second. Only the two endpoints are updated. Updating all their friends too — each of Asha’s 250 friends now has Ravi as a friend-of-friend — is the write amplification LinkedIn’s graph team warns about, roughly 250 times the base write rate: 7.5B writes a day for little gain. They wait for the batch. LinkedIn reports that making N-hop candidates near-real-time gave “disproportionate gains”.
• Online, per request: fresh features and the expensive model. The viewer’s embedding is computed from their latest activity; candidates’ embeddings were precomputed, and the store computes the dot products server-side so 500 large vectors never cross the network. LinkedIn’s PYMK has used Venice’s “read compute” this way since 2019.

One subtle failure deserves a sentence in the interview. The nightly snapshot is taken at midnight and the new version goes live at 05:00. If the swap simply replaces the store, every nearline merge made in those five hours disappears — and Asha’s new colleague vanishes from her suggestions. The fix is to replay recent nearline writes on top of the new version before switching to it, which is how Venice’s hybrid stores work.

How do we serve a suggestions page in under 200 ms?

The serving path is short, but it touches many stores, so it is built around parallel calls and per-call deadlines:

1. Read the candidate list. One key lookup returns about 500 rows with features — around 5 ms.
2. Load the filters, in parallel. The viewer’s live connections from the graph service (about 250 IDs), pending invitations, dismissals, and blocks in both directions. Store blocks under both the blocker and the blocked member, so “who has blocked me?” is also one lookup. Drop matches in memory: about 450 survive.
3. Fetch fresh features. Compute the viewer’s embedding from recent activity and ask the feature store for the dot products and counters for those 450 candidates in one batched call.
4. Score with the rich model and re-rank for fairness, diversity and invitation load.
5. Hydrate the top 20 — name, photo, headline — from the profile cache, and build each reason from facts the viewer is allowed to see.
6. Cache the ranked list for 10 minutes per viewer, so paging is stable and the widget and the page don’t compute twice.

Degrade in layers. If the rich model times out, serve the light model’s order. If the feature store is slow, skip fresh features. If the candidate store is down, serve the cached response from earlier in the day or hide the widget. The one thing that never degrades is the filter step: if the blocks store cannot be read, show nothing rather than risk showing someone who blocked the viewer.

Reasons are part of the response. “14 mutual connections” must count only connections the viewer can see — some members hide their connection list — and a reason must never be built from private signals. More on that in the privacy deep dive.

How do we give a brand-new member good suggestions in the first minute?

Retention is decided early: a new member who finds familiar people in their first session tends to come back; one who sees strangers or an empty page does not. Twitter’s Who to Follow treated this as a separate problem — new users jumped the queue for recomputation, so suggestions were usually ready before they finished the welcome flow, and it built a separate set of real-time algorithms just for them. The batch lane is no help, so this path runs nearline and online.

• Affiliations. Index members by (organisation, year) so “Infosys, 2016–2020” finds people with overlapping years. Size matters: the 12 people from your 50-person startup are better candidates than 12 random people from a 300,000-person firm. A verified work email domain is an even stronger signal.
• Contacts, with consent. Normalise phone numbers to international format and lower-case emails, hash them on the device, and look them up in an index of members’ hashed identifiers. Phone numbers have so few possible values that a plain hash can be reversed by brute force, so the server re-hashes with a secret key before storing and treats the table as personal data. Keep the uploader’s hashed contacts too: when Ravi joins next month, he can be suggested to the people who already had his number — subject to the privacy rules below.
• The inviter. If Ravi’s invitation brought the member in, Ravi’s network is the best prior available.
• The first edge. One accepted invitation makes an online 2-hop affordable: walking one or two edges is a few hundred reads, not 62,500.

New members also get top priority in the nearline queue, so the full light-model scoring catches up within minutes rather than at the next nightly run.

How do we protect privacy and stop abuse?

A suggestion is a statement: “we think you know this person, and here is why.” Each one can leak information, so privacy rules are part of the architecture, not a policy page.

Sensitive inferences. Address books are the riskiest signal. If a psychiatrist’s patients all have her number, contact-based suggestions can introduce patients to each other. A journalist’s sources, a lawyer’s clients, someone avoiding an ex-partner: the same mechanism exposes all of them. So:

• “Who has your number” is never shown as a reason, and never the only reason a candidate qualifies. Use it, at most, as one weak feature among many, and let members opt out of being suggested through other people’s contacts.
• Never infer from places or health. Location history, visits and searches about medical or legal topics are not PYMK signals.
• Reasons come only from facts the viewer can already see: visible mutual connections, a shared public employer or school.
• Blocks are absolute and two-way, and members can choose not to appear in suggestions at all. Minors are never suggested to adult strangers.

Connection spam. Fake accounts and aggressive networkers use PYMK as a target list. Cap invitations per week and return 429 beyond it (see rate limiting); lower the cap for accounts whose invitations are often ignored or answered with “I don’t know this person”; keep new, unverified accounts out of other people’s suggestions until they earn some trust; and rate-limit paging through suggestions, which is otherwise an easy way to scrape the graph.

Over-invited members. The other side of spam is the popular member who receives far more invitations than they can handle. LinkedIn added a re-ranker that discounts a candidate’s score as their recent invitations pile up, using a piecewise-linear decay. It halved the number of overloaded recipients; invitations sent fell 1%, but sessions from recipients rose 1%, because invitations went to people for whom they meant more.

How do we know the suggestions are any good?

Offline metrics, per stage. Candidate generation is judged by recall@k: of the people a member actually connected with next week, how many were in the pool? The rankers are judged by AUC, precision@k and calibration — calibration matters because scores from several models are multiplied together. Diversity metrics, such as the spread of companies or countries among suggestions, catch a pool dominated by one source.

Online A/B tests decide. LinkedIn notes that offline metrics rarely match online results, because of presentation and position bias and the gap between logged data and live traffic. The primary metrics are invitations sent and accepted, not either alone. Twitter measured something closer to the real goal: engagement per impression — how much a user went on to interact with the accounts they followed from a suggestion.

A network breaks the usual A/B assumptions. Treatment members send invitations to control members, so the two groups are not independent. LinkedIn built a receiver-side attribution framework to credit a recipient’s session to the sender whose invitation brought them back. Cluster-randomised designs, which assign whole communities to one arm, reduce the leakage further.

Watch network health over months. Keep a small long-term holdout, and track whether new connections lead to messages and interactions, how many members still have fewer than 10 connections, and whether suggestions concentrate on a few popular members. A system that maximises this week’s invitations can still make the network worse.

The final design

Everything heavy happens off the request path; the request path reads a few hundred precomputed rows, filters them against live stores and spends its budget on ranking.

Follow one request and the updates behind it:

1. Request. The widget or page calls the gateway, which authenticates the member and calls the PYMK service.
2. Read. One lookup returns about 500 candidates with features from the versioned candidate store.
3. Filter. In parallel, live connections from the graph service, pending invitations, dismissals and two-way blocks remove about 50.
4. Score. The viewer’s fresh embedding and server-side dot products feed the L2 model; the re-ranker applies fairness, diversity and invitation-load decay.
5. Hydrate. The top 20 get names and photos from the profile cache and reasons from visible facts; the ranked list is cached for 10 minutes.
6. Connect. “Connect” goes to the invitation service and its weekly cap; on acceptance the graph service writes the edge and publishes edge-added.
7. Nearline. The merge job adds each new friend’s strongest connections to the other person’s list within minutes.
8. Nightly. The batch job reads a snapshot of the graph and logs, runs the capped 2-hop, random walks, embedding retrieval and affiliation joins, applies the light model, and pushes a new version — replaying the day’s nearline writes before the swap.
9. New members. Contact matches, affiliations and the inviter’s network fill a new member’s list within seconds of sign-up.

Check it against the requirements. Requests read a few hundred rows and run one model call, so p99 stays under 200 ms with room to spare. A new connection changes both people’s suggestions within minutes; everything else refreshes nightly, as the freshness target allows. Blocked, dismissed and connected members are removed against live stores on every request, so stale candidates cannot leak them. The nightly job’s cost is bounded by caps on hub fan-out and on candidates per source. If any lane fails, the one below it still serves — at worst, yesterday’s list.

What interviewers look for by level

Follow-up questions

What if the relationship is one-way, like follows?

A follow needs no consent, so “who to follow” is closer to recommending items than people. Celebrities become legitimate candidates rather than hubs to suppress, and the signal is mostly “accounts followed by accounts like you”. Twitter’s Who to Follow ran random walks to build a “circle of trust” and then SALSA, treating the people you follow as hubs and the accounts they follow as authorities. Acceptance drops out of the objective; what you optimise instead is engagement after the follow. See design Twitter.

What if a hub’s adjacency list becomes a hot key?

A member with 30,000 connections appears in the 2-hop expansion of 30,000 others, so the partition holding her list is read far more than its neighbours. Batch jobs are fine; the online and nearline paths are not. Keep a capped, sampled copy of every hub’s list — the same 1,000 used for candidate generation — replicated to every graph node, and cache it in each service. The general pattern is in hot keys.

What if a member deletes their account?

Until the next nightly push, their ID still sits in other members’ candidate lists. Add deleted and deactivated members to a small tombstone set checked in the filter step, so they disappear from every suggestion immediately. Then make sure the deletion reaches every input of the batch — graph snapshots, logs and contact tables — within the legal deadline, and that the next pushed version no longer contains them.

What if a region forbids using uploaded contacts?

Make every candidate source and feature switchable per region. The contacts source turns off, and the model is retrained, or at least recalibrated, without those features for that region’s members. Recall for new members will drop there, so lean harder on affiliations and the inviter, and measure the effect with a region-level A/B test rather than assuming it.

How to say it in the interview

“I’ll design People You May Know for a billion members, about 100 million daily, with suggestions served in under 200 milliseconds.

The core signal is friends-of-friends — most new connections close a triangle — but it costs about d squared per member: 62,500 paths for someone with 250 connections, and far more when hubs with tens of thousands of connections are involved. So I generate candidates offline, cap each hub at about a thousand sampled connections, and weight paths with Adamic–Adar so hubs count for little. I add random walks for sparsely connected members, embedding retrieval, same-company-and-years, and contacts, to get a pool of about 3,000 per member, and a light model keeps 500.

Those 500 are pushed nightly as a new version of a key-value store, with rollback. A nearline job listens for new connections and merges each new friend’s network into the other person’s list within minutes, and those writes are replayed on top of each nightly version so they survive the swap.

At request time I read the 500, remove anyone connected, invited, dismissed or blocked using live stores, score the rest with a richer model on fresh features — chance of invite, chance of acceptance, value of the connection — and re-rank for fairness and to protect members who are drowning in invitations.

New members get suggestions from contacts, company, school and their inviter within seconds. Reasons come only from facts the viewer can see; contact-based signals are never shown. I’d judge it on recall offline, then on accepted invitations and long-term network health in A/B tests that account for treated members inviting control members.”

Concepts used in this design

• Batch vs stream processing — the nightly, nearline and online lanes.
• Flink and stream processing — the nearline merge job.
• Kafka — edge events and feedback logs.
• Embeddings and semantic search — two-tower retrieval of similar members.
• Vector databases — nearest-neighbour search over member embeddings.
• Database sharding — partitioning the graph and the candidate store by member.
• Hot keys, cache stampede and invalidation — hub adjacency lists.
• Caching strategies — the profile cache and the 10-minute ranked-list cache.
• Rate limiting — weekly invitation caps and anti-scraping limits.
• Timeouts, retries and backoff with jitter — per-call deadlines on the serving path.
• Strong vs eventual consistency — eventual for candidates, strict for filters.
• Deployment strategies — rolling out models and rolling back a bad push.

References and further reading

1. Building a large-scale recommendation system: People You May Know (LinkedIn Engineering, 2024)linkedin.comThe L0 → L1 → L2 → re-rank funnel, its recall and precision metrics, Bayesian-optimised weights, and why offline metrics rarely match A/B results.
2. Candidate generation in a large-scale graph recommendation system: People You May Know (LinkedIn Engineering, 2024)linkedin.com2- and 3-hop neighbours scored by common connections, personalised PageRank, two-tower embeddings with impressions as positives, heuristic sources, and the gains from near-real-time N-hop.
3. Optimizing PYMK for equity in network creation (LinkedIn Engineering, 2021)linkedin.comDecaying scores of over-invited recipients (overloaded recipients −50%), the frequent-member feedback loop, fairness re-ranking results, receiver-side attribution.
4. LIquid: The soul of a new graph database, Part 1 (LinkedIn Engineering, 2020)linkedin.comFirst-degree networks of a few hundred, second-degree sets of 50,000+, and why materialising them costs about 250× the base write rate.
5. Using set cover to optimize a large-scale low-latency distributed graph (LinkedIn Engineering, 2013)linkedin.comA partitioned graph database plus a network cache of second-degree sets built on each visit, with a hit ratio above 90%.
6. Open sourcing Venice, LinkedIn’s derived data platform (LinkedIn Engineering, 2022)linkedin.comVersioned full pushes from Hadoop with background swap and rollback, hybrid batch-plus-nearline writes, and PYMK’s server-side embedding maths.
7. Serving large-scale batch computed data with Project Voldemort (Sumbaly et al., FAST 2012)usenix.orgPYMK computed daily over hundreds of terabytes, bulk-loaded into read-only stores; dismissals kept in a separate store and subtracted at page load.
8. WTF: The Who to Follow Service at Twitter (Gupta et al., WWW 2013)web.stanford.eduWhole graph in memory on one server, about 5 bytes per edge, daily snapshots, priority queueing and real-time algorithms for new users, sampling at high-degree vertices.
9. Supervised Random Walks: Predicting and Recommending Links in Social Networks (Backstrom and Leskovec, WSDM 2011)arxiv.org92% of new Facebook Iceland friendships closed a triangle; why 3-hop neighbourhoods are impractical; class imbalance in link prediction.
10. The Link Prediction Problem for Social Networks (Liben-Nowell and Kleinberg)cs.cornell.eduDefinitions of common neighbours, Jaccard, Adamic–Adar, Katz and rooted PageRank as link predictors.