API composition: aggregation, BFF and GraphQL federation

API composition turns one client request into one answer built from many services. A layer close to the services calls each one that holds a piece of the data, in parallel where it can, merges the results and returns a single response shaped for the screen. That layer can be a gateway aggregator, a backend for frontend (BFF) owned by an app team, a GraphQL server, or a federated graph assembled from many teams’ schemas.

Picture it first: one wedding planner, many vendors

Your cousin is getting married in Jaipur. Instead of phoning the caterer, decorator, photographer, band and hotel yourself, the family hires one planner. You give her one brief; she makes the calls.

• She calls independent vendors at the same time. The caterer doesn’t wait for the photographer.
• Some calls must wait. The decorator can’t quote until the venue is booked, so venue-then-decorator decides when you get your plan.
• When the band cancels, the wedding goes on with a playlist, and she tells you so. Only a lost venue stops the event.
• Your office offsite gets a different planner with a different brief, though some vendors are the same.
• For a huge wedding, each vendor fills in its own columns of one shared run sheet, keyed by event number, and the coordinator knows whom to ask for each column.

Map it back: the vendors are microservices, the planner is the aggregator, venue then decorator is the critical path, the playlist is a fallback for an optional part of the response, the offsite planner is a backend for a different frontend, and the shared run sheet keyed by event number is a federated graph whose entities are joined by key.

Why one screen needs many services

Open a product page in a Flipkart-style shopping app during a sale. One screen shows:

• title, images and specifications (Catalog),
• price and discount (Pricing),
• ratings and top reviews (Reviews),
• “delivery by Thursday” for your pincode (Delivery promise),
• seller name and rating (Sellers),
• bank and EMI offers (Offers),
• similar products (Recommendations).

That is seven services, often owned by seven teams; a Swiggy-style order-tracking screen is the same story. Letting the app call each one directly causes five problems:

• Round trips. Every level of dependency costs a full mobile round trip. What is an API? works through a chain of ten lookups: about 800 ms from the phone against 90 ms behind one endpoint.
• Leaked structure. The app’s code names your service boundaries. Split Pricing in two and every installed app version breaks, and old Android builds linger for months.
• Many front doors. Every service must be public, terminate TLS, check tokens and rate-limit, seven times over.
• Release coupling. Renaming one field needs an app release, a store review and users who actually update.
• Over-fetching. Each service returns its whole resource; the screen shows a handful of fields.

So composition moves to the server. Every option below gives the client one call; they differ in who owns the composing code and how failures and changes spread.

Gateway aggregation: the API composer

Chris Richardson’s microservices pattern catalogue calls this API Composition: an API composer invokes the services that own the data and joins the results in memory. Microsoft’s Azure Architecture Center describes the same move at the edge as Gateway Aggregation. Either way, the composer:

1. Accepts one request, already authenticated by the API gateway.
2. Knows the dependency graph: which calls need only the request’s inputs, and which need a field from another response.
3. Starts every independent call at once, and each dependent call the moment its own inputs arrive.
4. Gives each call a timeout carved from one overall deadline, and a fallback for optional parts.
5. Merges, trims to what the screen shows, and returns one response that says which parts are missing.

Azure adds three practical points: run the aggregator near the services, so its fan-out uses roughly 1 ms data-centre hops; don’t let it become a single point of failure or a bottleneck; and prefer an aggregation service behind the gateway to aggregation inside it, because fan-out needs different resources from routing. A sketch in TypeScript:

async function productPage(id: string, pincode: string) {
  const deadline = Date.now() + 250;                      // budget for the whole page
  const t = (cap: number) => Math.max(0, Math.min(cap, deadline - Date.now()));
  const degraded: string[] = [];
  const optional = <T>(p: Promise<T>, fallback: T, name: string) =>
    p.catch(() => { degraded.push(name); return fallback; });

  // Need only the product id: start at once.
  const productP = catalog.get(id, t(80));                // required
  const priceP   = pricing.get(id, t(80));                // required
  const reviewsP = optional(reviews.summary(id, t(120)), lastGoodReviews(id), "reviews");
  const etaP     = optional(delivery.promise(id, pincode, t(60)), null, "deliveryPromise");

  // Need fields from other answers: start the moment those answers land.
  const sellerP = productP.then(p => optional(sellers.get(p.sellerId, t(60)), null, "seller"));
  const recosP  = productP.then(p => optional(recos.similar(p.categoryId, t(150)), [], "recommendations"));
  const offersP = Promise.all([productP, priceP]).then(([p, price]) =>
    optional(offers.best(id, price, p.sellerId, t(60)), [], "offers"));

  const [product, price, review, eta, seller, recommendations, offerList] =
    await Promise.all([productP, priceP, reviewsP, etaP, sellerP, recosP, offersP]);
  return { product, price, review, eta, seller, recommendations, offers: offerList, degraded };
}

If Catalog or Pricing fails, the whole call fails and the app shows an error. Anything else degrades and is named in degraded.

The latency arithmetic

Give the seven calls illustrative times: Catalog 40 ms, Pricing 30, Reviews 60, Delivery promise 25, Seller 20, Offers 35 and Recommendations 90. Seller and Recommendations need IDs from Catalog; Offers needs Catalog and Pricing.

Three numbers describe any composition:

• Sequential sum, calling one after another: 40 + 30 + 60 + 25 + 20 + 35 + 90 = 300 ms.
• Parallel max, if nothing depended on anything: the slowest call alone, 90 ms.
• Critical path, the real answer: the longest chain of dependent calls. Catalog then Recommendations is 40 + 90 = 130 ms. The other chains are shorter: Catalog then Seller is 60 ms, Catalog then Offers is 75 ms, Reviews alone is 60 ms.

Speeding up anything off the critical path changes nothing: halve Reviews and the page still takes 130 ms. Shorten the chain instead. If Recommendations accepted the product ID and looked up the category itself, it would start at 0 ms, and the page would take max(90, 75, 60) = 90 ms.

Code structure matters too. Composers written in waves (“await group one, then start group two”) add a barrier: Seller, Offers and Recommendations wait for Reviews, so the page takes 60 + 90 = 150 ms instead of 130. That is why the sketch chains each dependent call on exactly the promises it needs.

Now add the phone. Through the composer, the app pays one mobile round trip, say 80 ms, plus 130 ms: about 210 ms. Composing in the app, Catalog lands at 80 + 40 = 120 ms, then Recommendations needs another 80 + 90, finishing at 290 ms, even with HTTP/2 multiplexing the parallel calls. Server-side composition saves one mobile round trip per level of dependency.

Fan-out and the latency tail

A page that waits for every call is slow whenever any call is slow. If each call is independently slow 1% of the time, the chance that at least one of N calls is slow is 1 − 0.99^N.

Jeffrey Dean and Luiz André Barroso use this example in “The Tail at Scale” (Communications of the ACM, 2013): servers that usually answer in 10 ms but take a second at the 99th percentile, with a request fanning out to 100 of them, leave 63% of user requests taking over a second. In a real Google service they measured, one leaf’s p99 was 10 ms but the p99 for all leaves was 140 ms, and the slowest 5% of leaves caused half of it.

Turn the formula round to set targets. For the seven-call page to be slow only 1% of the time, each call may be slow at most 1 − 0.99^(1/7) ≈ 0.14% of the time. Every service’s “p99” goal quietly becomes a p99.86 goal.

Ways to fight the tail:

• Call fewer things. Batch endpoints, merge tiny services that are always called together, and cache stable fragments.
• Hedge. If a read hasn’t answered by its 95th-percentile expected latency, send a copy to another replica and cancel the loser. Dean and Barroso say this limits extra load to about 5%; in a Google benchmark reading 1,000 keys from 100 servers, hedging after 10 ms cut the 99.9th-percentile latency from 1,800 ms to 74 ms for 2% more requests. Hedge only idempotent reads.
• Accept “good enough”. Their search systems skip non-essential subsystems such as ads or spelling correction when they are late. On a product page, that is Recommendations.

When one piece fails

Fan-out multiplies failures too. If seven services each answer 99.9% of the time and the page needs all seven, it works 0.999^7 ≈ 99.3% of the time: seven times the failure rate of any one service. So decide per fragment: required (no product, no page) or optional (no reviews, show the page anyway).

The toolkit, in the order a request meets it:

• One deadline, many timeouts. The page gets a budget, say 250 ms, and each call gets the smaller of its own cap and the time left. Pass the remaining time downstream: gRPC has no default deadline, so a call without one can wait forever; with one, it sends the time left as a timeout (clock skew doesn’t matter) and the server cancels work once it passes. Java and Go propagate deadlines by default; C++ must opt in. REST vs GraphQL vs gRPC draws the budget shrinking hop by hop.
• Fallbacks. Serve the last good value with a staleness flag, a generic default (bestsellers instead of personal picks), or nothing.
• Circuit breakers. After repeated failures, stop calling a sick service and go straight to the fallback instead of spending 120 ms per page rediscovering the outage. See circuit breaker.
• Bulkheads. Give each downstream its own connection pool or concurrency limit, so a slow Recommendations service can’t hold every connection and starve Pricing. See bulkheads.
• Retries, sparingly. A retry spends budget the page doesn’t have: at most one, idempotent reads only, if time remains. See timeouts, retries and backoff.
• Say what is missing. Newman stresses that the client must be able to interpret a partial response and render it correctly. A degraded list does that for JSON.

GraphQL builds the marker in. The specification says an execution error makes that response position null and adds an entry to errors whose path names the field, while sibling fields carry on:

{
  "data": {
    "product": { "title": "Steel water bottle, 1 L", "price": 34900, "rating": null }
  },
  "errors": [
    {
      "message": "Reviews did not answer within 120 ms",
      "path": ["product", "rating"],
      "extensions": { "code": "DOWNSTREAM_TIMEOUT" }
    }
  ]
}

One trap: if the failed field is declared non-null, the null propagates to the nearest nullable parent. Declare rating: Float! and one slow Reviews call wipes out the whole product. Make optional fragments nullable.

A consistency caveat: each service is read at a slightly different moment, so the page is a collage, not a snapshot. A lightning-deal price can change between the Pricing and Offers calls. That is fine for display if checkout re-validates price and stock.

Backends for frontends

A shared composer serving every client soon hits a people problem: the Android app wants small payloads, the website wants server-rendered pages, the seller app wants bulk tools, and every change queues behind one backend team. Sam Newman’s write-up of the Backends for Frontends pattern, drawn from SoundCloud and REA, gives the answer: one backend per user experience, owned by the team that builds that experience. (The name comes from SoundCloud; Newman cites ex-SoundCloud engineer Phil Calçado for it.) Think of the app as two parts: a client outside your perimeter and a BFF inside it.

What belongs in a BFF:

• Screen-shaped endpoints. GET /screens/product/p1 returns exactly the card the app draws, images sized for the device, composed and degraded as above.
• Client-specific concerns. A web BFF can keep the session in a cookie and hold access tokens server-side, away from browser JavaScript. A mobile BFF keeps endpoints alive for app versions still installed; a TV BFF sends bigger images and fewer words.
• Releases in step with its app. The team changes a screen and its endpoint in one sprint, with no ticket to another team, and can move logic server-side to skip an app-store review.

What stays out: cross-cutting edge work such as authentication, rate limits and TLS (Azure says to abstract these, typically into the gateway), and business rules. Prices, eligibility and stock reservations live in domain services, because a second client will need exactly the same answer.

How many BFFs? A guideline Newman likes, from Stewart Gleadow: one experience, one BFF. REA ran one per mobile platform; SoundCloud shared one between its iOS and Android listener apps but gave its creator app its own, and its engineers said they might split the shared one if choosing again. Team structure usually decides it.

You gain screen-perfect payloads, one round trip per screen, team autonomy and failure isolation: one client’s BFF failing need not affect another’s. You pay with duplication (Azure calls it a probable outcome), more services to run, an extra hop, and steady pressure to merge the BFFs back into one general-purpose edge API, which Newman says has “proven time and again” to bloat. When the same join shows up in a third BFF, extract it: a shared library for purely internal code, or a downstream service if it is a real domain concept (Newman pushes a wishlist join into the Wishlist service).

GraphQL as the composition layer

A single GraphQL server is another composer. Its schema is the contract, its resolvers are the composing code, and because each client names its fields, one schema serves many screens without an endpoint per screen. Azure’s BFF guidance notes that GraphQL with frontend-specific resolvers can make separate BFFs unnecessary.

The production basics are in REST vs GraphQL vs gRPC: per-request DataLoader batching that turns 1 + N resolver calls into 2, cost and depth limits, and persisted queries so a CDN can cache. For composition, GraphQL adds partial results with error paths for free.

The limit is organisational. Netflix’s studio engineers described the arc in a 2020 post. Teams first wrote duplicate aggregation code in BFFs or mid-tier services, or kept materialized views of other teams’ data, which lagged; inconsistent data across studio apps was the top support issue of 2018. A central graph, Studio API, fixed consistency, because every field resolves through one piece of code. But its small team became the bottleneck: cut off from domain expertise, wiring in each backend by hand, carrying a growing operational load. The fix was federation.

GraphQL federation

There are two ways to build one graph from many teams’ GraphQL services. In schema stitching, a gateway imports each service’s schema and the gateway’s owners write the glue linking types across services. In federation, each service declares its share of the common types and how to look them up, and a composition step merges the schemas mechanically. Apollo Federation names the parts:

• A subgraph is one team’s GraphQL service. The supergraph is the composed whole that clients see.
• The router is the single entry point. It plans each operation across subgraphs, runs the fetches and merges the results.
• An entity is a type that several subgraphs contribute fields to, identified by a key declared with @key, much like a candidate key in a database.
• A reference resolver (__resolveReference in Apollo Server) lets a subgraph load an entity from a representation: the key fields plus __typename, such as { "__typename": "Product", "id": "p1" }. The router sends representations through a generated _entities field.

Three teams extend one Product:

# Catalog subgraph: owns the root field
type Product @key(fields: "id") {
  id: ID!
  title: String!
  brand: String
}
type Query {
  product(id: ID!): Product
}

# Pricing subgraph: adds a field to the same entity
type Product @key(fields: "id") {
  id: ID!
  price: Int        # in paise
}

# Reviews subgraph
type Product @key(fields: "id") {
  id: ID!
  rating: Float     # nullable, so a Reviews failure nulls only this field
}

Netflix’s post walks through the same mechanics: the gateway turns a client query into a query plan of sub-queries, ordered into sequential and parallel steps, and planning plus (de)serialising responses added about 10 ms in the worst case. Each domain team runs a domain graph service, and a schema registry accepts a new subgraph schema only if it is valid, composes with every other schema and is backward compatible.

A reference resolver can be called many times for one query, say once per product in a list of 40. Apollo’s docs warn that this is the N+1 problem again, so batch it:

// Pricing subgraph (Apollo Server)
const resolvers = {
  Product: {
    // ref = { __typename: "Product", id: "p1" }
    __resolveReference(ref, { loaders }) {
      return loaders.priceById.load(ref.id); // 40 products → one Pricing lookup
    },
  },
};

Federation is not only Apollo’s: the GraphQL Foundation’s Composite Schemas working group is drafting a vendor-neutral federation specification, still at a preliminary stage as of this writing.

When to stop composing: precompute

Request-time composition shines when a request needs a few fresh items by key. It strains in three cases:

• Big joins and cross-service filters. “All orders over ₹5,000 from Pune customers with an open refund ticket” means pulling large sets from three services and joining them in memory. Richardson lists exactly this, inefficient in-memory joins of large data sets, as the pattern’s drawback, and names CQRS as the alternative.
• Hot read paths. At 50,000 product-page views per second during a sale, seven calls per view is 50,000 × 7 = 350,000 backend calls per second, each with its own tail.
• A critical path you can’t shorten, or a page that must be a consistent snapshot rather than a collage.

Then build a pre-joined read model: services publish change events, a projector keeps a denormalised “product page” document in a key-value store or search index, and the page becomes one read. You pay with lag (often seconds), storage, a pipeline to run, and rebuilds when the view’s shape changes; Netflix’s studio teams found materialized-view lag unacceptable for their most important workflows. A common middle path: precompute the stable part (title, images, specs, review summary) and compose the volatile bits (price, stock, delivery promise) live. See event sourcing and CQRS.

Choosing a composition style

These mix: a large app may run a federated graph for its apps, a thin web BFF for sessions, a read model for listing pages, and widgets that load on their own.

Going deeper

The god aggregator and where logic goes

Every composition layer drifts toward the general-purpose backend Newman warns about: one deployable every team must change, a release queue, and an on-call rotation paged for everyone’s outages. That is a distributed monolith, and Netflix’s Studio API hit the same wall. Split by experience (BFFs) or by domain (federation), and keep the layer thin. Composition code may fetch, join, trim, format and pick screen defaults; it must not decide prices, eligibility, refunds or stock. If two BFFs contain the same rule, it is domain logic missing a home. See monolith vs microservices.

Schema governance

With federation, the schema is shared infrastructure. Compose in CI rather than at router start-up, have the registry reject changes that don’t compose or would break operations clients actually send, and retire fields with @deprecated once usage reaches zero.

Observability

Start a trace at the edge and carry its ID through the composer to every call; Azure asks for correlation IDs on each one. Record latency and degradation per fragment, not just per page, and alert on the rate of errors entries and degraded markers, because a degraded GraphQL response is still an HTTP 200. See observability.

Security

A composer must call services as the user, forwarding the end user’s identity, not with a powerful service account that can read anyone’s data. In a federated graph, the router can enforce field-level rules: Apollo’s router filters out fields and types the request isn’t authorized for, using @authenticated, @requiresScopes and @policy. Object-level checks (“may this user see order 9876?”) still belong in the subgraph that loads the object. See service-to-service auth and mTLS.

Caching composed responses

Newman warns that a cached composed response must expire as soon as its most volatile piece does. Usually better: cache fragments separately, each with its own TTL (catalog details 10 minutes, a review summary 5, a price seconds), and never share personalised fragments between users. See caching 101.

When composition belongs in the client

Newman points out that desktop web, on fast connections, can often afford several calls. Independently loading widgets are a feature: if recommendations load in their own request below the fold, a slow Recommendations service never delays first paint. Compose server-side what must arrive together; let the client fetch what can arrive late.

Cheat sheet

How to say it in the interview

“The product page needs seven services, so the app makes one call and an aggregator near the services does the fan-out. Independent calls start together; Seller and Recommendations start when Catalog returns their IDs, so latency is the critical path, about 130 ms, not the 300 ms sum. The page gets a 250 ms deadline that every call inherits. Catalog and Pricing are required; reviews and recommendations fall back to cache or disappear, behind breakers and separate pools, and the response lists what’s degraded. Each app team owns a BFF; at our scale, domain teams would publish subgraphs to a federated graph. Listing and search pages read a precomputed view.”

Your product page calls 8 services and p99 is bad. What do you do? Trace a slow request and find the critical path; remove dependencies so calls start earlier; cut fan-out with batch endpoints and fragment caches. With eight calls each slow 1% of the time, 1 − 0.99^8 ≈ 7.7% of pages hit a slow call, so hedge idempotent reads after the p95 and drop optional fragments that miss their timeout.

BFF or GraphQL? Both give one round trip per screen. A BFF suits a few distinct experiences, each with its own team, and keeps plain HTTP caching. GraphQL suits many screens and clients changing weekly, at the price of cost limits and persisted queries. With many backend teams, federate; a thin BFF can still front the web for sessions.

How do you handle one slow service? Give it a timeout from the page’s deadline, isolate it in its own pool, wrap it in a circuit breaker, and serve a cached or default value marked as degraded. If it is required, fail fast with a clear error instead of hanging.

Where does business logic go? In domain services. The composition layer fetches, joins and shapes for a screen; if two BFFs need the same rule, it is a missing domain service.

When would you stop composing at request time and precompute? When a query joins or filters large sets across services, when sale-day views would multiply into hundreds of thousands of backend calls per second, or when the page must be a consistent snapshot. Then I’d build an event-fed read model and accept seconds of lag.

Where to go next

• API gateway: authentication, limits and routing before any composition.
• REST vs GraphQL vs gRPC: DataLoader, cost limits, persisted queries and deadline propagation.
• What is an API?: chatty versus chunky calls.
• API design for system design interviews: sketching screen-level endpoints.
• Timeouts, retries and backoff with jitter, circuit breaker and bulkheads: the partial-failure toolkit.
• Event sourcing and CQRS: building a pre-joined read model.
• Monolith vs microservices: why there are so many services to compose.
• Chatty I/O: the anti-pattern composition fixes.
• Observability: logs, metrics and traces: tracing one page across its services.

References and further reading

1. Pattern: API Composition (Chris Richardson, microservices.io)microservices.ioThe API composer that calls the services owning the data and joins in memory; inefficient joins of large data sets as its drawback, and CQRS as the alternative.
2. Gateway Aggregation pattern (Azure Architecture Center)learn.microsoft.comOne request fanned out to many back ends: keep it near the services, use bulkheads, breakers and timeouts, return partial or cached data, and consider an aggregation service behind the gateway.
3. Backends for Frontends pattern (Azure Architecture Center)learn.microsoft.comOne backend per frontend, owned by that frontend’s team; duplication as a probable cost, cross-cutting concerns kept out, and when GraphQL makes a BFF unnecessary.
4. Pattern: Backends For Frontends (Sam Newman)samnewman.ioThe original write-up from SoundCloud and REA practice: one experience, one BFF; parallel downstream calls; degrading on partial failure; duplication and the general-purpose edge API trap.
5. The Tail at Scale (Dean and Barroso, Communications of the ACM, 2013)barroso.orgHow fan-out amplifies rare slow responses (63% of requests slow at a fan-out of 100), measured Google fan-out latencies, hedged requests and “good enough” results.
6. GraphQL Specification (September 2025 edition)spec.graphql.orgExecution errors become null at a response position plus an entry in the errors list with a path; nulls propagate up through non-null fields.
7. Introduction to Entities (Apollo Federation documentation)apollographql.comEntities, the @key directive, representations carrying __typename, and reference resolvers that must handle N+1 with data loaders.
8. Authorization in the GraphOS Router (Apollo documentation)apollographql.comField- and type-level authorization in a federated router with @authenticated, @requiresScopes and @policy, which filter out unauthorized fields.
9. How Netflix Scales its API with GraphQL Federation, Part 1 (Netflix Tech Blog)netflixtechblog.comFrom BFFs and materialized views to one central graph to federation: domain graph services, a schema registry, query plans and their roughly 10 ms worst-case overhead.
10. gRPC guide: Deadlinesgrpc.ioNo default deadline, server-side cancellation, and propagation of the remaining time as a timeout so clock skew does not matter.