Distributed transactions: two-phase commit vs sagas

A distributed transaction is one business action whose writes land in several services or databases that must end up agreeing: a food order that charges the customer, gets the restaurant to accept and assigns a rider. Two-phase commit (2PC) makes the participants agree atomically, but it holds locks across the network and can block if its coordinator dies. A saga commits each step locally and repairs a failure with compensating transactions, trading isolation for availability.

Picture it first: a wedding planner and three vendors

Your cousin’s wedding needs the hall, the caterer and the band on the same date; one without the others is useless. The planner has three ways to work.

Hold, then confirm. The planner asks each vendor: “Can you do 14 February? If yes, keep the date for me until I call back.” All three say yes, so the planner writes “confirmed” in a diary and rings each one back. That is two-phase commit. If the planner falls ill after the three yeses, every vendor is stuck: they promised the date, so they can’t sell it, but they can’t assume the wedding is on.

Book for real, cancel if needed. The planner books the hall, then the caterer. The band is busy, so the planner cancels both and pays the fees. That is a saga: each booking is final, and failure is repaired by new actions that undo the effect in business terms.

Take holds that lapse. The hall holds the date for 48 hours unless the advance is paid. That is TCC (try-confirm-cancel): a reservation with an expiry, so a vanished planner can’t block anyone for long.

Map it back: the vendors are participants, the planner is the coordinator or orchestrator, “hold the date” is prepare, a cancellation is a compensating transaction, and the fee is a reminder that compensation is a semantic undo, not a rollback.

The problem: one action, many databases

Take a ₹450 food order. The app must create the order (order service, its own Postgres), collect ₹450 (payment service, via a UPI or card gateway), get the restaurant to accept (restaurant service) and assign a rider (delivery service). A trip booking has the same shape: a flight, a hotel room and a payment, each from a different system. A partial result is a real problem: money taken and no food coming, or a flight with nowhere to stay.

Inside one database this is easy. An ACID transaction is atomic because the database writes one commit record to one write-ahead log: before that record is durable nothing happened, after it everything did. Four services mean four logs, and some participants aren’t databases at all: a payment gateway or an airline gives you an HTTP API, not a BEGIN.

Time is the other problem. An order takes 30–45 minutes from tap to doorstep, so you can’t hold database locks for its whole life. Garcia-Molina and Salem’s 1987 sagas paper starts from the same complaint about long-lived transactions.

That leaves two families of answers: make the participants agree atomically (an atomic commit protocol such as 2PC), or let each step commit on its own and repair partial progress (sagas and their cousin, TCC). Calling each service in turn and hoping is not a third option; it is how these incidents start.

Two-phase commit, step by step

2PC has one coordinator (the transaction manager) and N participants, usually databases. The application does its work on every participant under one global transaction ID, then asks the coordinator to commit.

Phase 1: prepare and vote. The coordinator sends PREPARE. A participant that can commit forces a prepared record to its log (redo information plus the locks it holds), then votes YES. From then on it may not commit or abort on its own. One that can’t commit votes NO and aborts.

Phase 2: decide and tell. If every vote is YES, the coordinator forces a commit record. That flush is the commit point: the transaction is committed even though no participant knows yet, and the application can be told. One NO, or a missing vote, means abort. Each participant then forces its own commit record, releases its locks and ACKs; after the last ACK the coordinator writes an unforced end record.

Mohan, Lindsay and Obermarck’s 1986 paper on IBM’s R* system gives the exact bookkeeping: each participant forces two records (prepared, commit) and sends two messages (vote, ACK); the coordinator sends two messages per participant and forces one commit record. With N participants:

• messages: 4N, so 12 for a food order across 3 databases;
• forced log writes: 2N + 1, so 7 for N = 3;
• waits before the decision is durable: one round trip plus two flushes in sequence (the participants’ prepare, then the coordinator’s commit record).

Gray and Lamport count it more tightly (3N − 1 messages, N + 1 stable writes) by leaving out the ACKs and the participants’ commit records. Either way, no 2PC commit finishes faster than two flushes in a row.

Why 2PC hurts across services

The blocking problem

The danger starts when a participant votes YES. It is now in doubt: it can’t commit, because another participant may have voted NO, and it can’t abort, because the coordinator may already have decided COMMIT. So it waits, holding its locks.

If the coordinator crashes right after collecting the votes, every participant waits for it to restart and read its log. Asking a peer helps only if some peer already got the decision. Gray and Lamport show this is inherent: 2PC blocks whenever its single coordinator fails at the wrong moment.

The locks hurt more than one order: every transaction on those rows, such as the restaurant’s stock or the customer’s wallet, queues behind them.

Heuristic decisions

Operators can’t always wait. R* let an operator force a prepared transaction to commit or abort, warning that a wrong guess commits part of a transaction and aborts the rest. XA calls this a heuristic decision, and Java’s XAException lists the outcomes: heuristic commit, heuristic rollback, heuristic mixed (parts went each way) and heuristic hazard (parts may have). Mixed means atomicity is gone and a human must reconcile. In PostgreSQL it is a DBA running ROLLBACK PREPARED on a row from pg_prepared_xacts.

Latency, locks and availability

Without failures, 2PC is slow where it hurts: lock time. After its last statement a participant keeps its locks through its prepare flush, the slowest vote, the coordinator’s flush, the COMMIT message and its own commit flush: at least one round trip plus three flushes, against one flush for a local commit. With illustrative numbers of 0.5 ms per round trip and 1 ms per flush, that is 0.5 + 3 × 1 = 3.5 ms, capping a hot row such as one restaurant’s stock at 1000 / 3.5 ≈ 285 commits per second. Put one participant 60 ms away and the locks last about 63 ms: 1000 / 63 ≈ 16 per second.

Availability multiplies down. Every participant must vote, so four at 99.9% give 0.999⁴ ≈ 99.6% (availability and the nines). Over a 30-day month that is 43,200 × 0.004 ≈ 173 minutes of failed orders instead of 43, before counting the coordinator.

Why XA is rare between microservices

• Many participants can’t prepare. Payment gateways and partner APIs have no prepare step, and Kafka can’t join an external 2PC: a restarted producer aborts its open transaction, and brokers abort any transaction older than transaction.max.timeout.ms (15 minutes by default). KIP-939 would change that; it is accepted but, as of October 2026, unreleased.
• It couples runtimes. Every service’s latency and outages become every order’s, the opposite of why you split services.
• It needs a highly available coordinator and operators who can finish in-doubt transactions.

So the common database-plus-broker dual write uses a transactional outbox, and cross-service workflows use sagas.

Where 2PC works well: inside one database

2PC’s flaws come from a coordinator whose log lives on one machine and from participants owned by different teams. A distributed database owns every participant and replicates the coordinator’s log with consensus.

• Spanner runs 2PC across Paxos groups. Prepare and commit records are written through Paxos, so a machine failure doesn’t lose the decision, and single-group transactions, the common case, skip 2PC. The 2012 paper measured mean commit latency of 17.0 ms with one participant, 42.7 ms with 50 and 71.4 ms with 100.
• CockroachDB keeps a replicated transaction record (PENDING, STAGING, COMMITTED or ABORTED) that its write intents point to. If the coordinator’s heartbeats stop, the record moves to ABORTED, so nobody waits forever, and Parallel Commits cuts commit latency from two consensus rounds to one.
• Kafka transactions are 2PC too: the transaction coordinator writes PREPARE_COMMIT to its log, an internal topic replicated three ways by default, then pushes the decision to every partition.
• PostgreSQL provides the participant half. PREPARE TRANSACTION stores the transaction on disk, and any session can later COMMIT PREPARED or ROLLBACK PREPARED it. It is meant for external transaction managers and ships switched off: max_prepared_transactions defaults to 0. A forgotten prepared transaction holds its locks and blocks VACUUM; in the extreme, the server shuts down to avoid transaction ID wraparound.

-- On each participant, inside the global transaction:
BEGIN;
UPDATE wallets SET balance = balance - 450 WHERE user_id = 42;
PREPARE TRANSACTION 'order-9f3c:wallet';  -- vote YES: forced to disk, locks kept

-- Later, from any session, once the coordinator has decided:
COMMIT PREPARED 'order-9f3c:wallet';      -- or ROLLBACK PREPARED

-- Find stuck ones (these block VACUUM and hold locks):
SELECT gid, prepared, owner, database FROM pg_prepared_xacts ORDER BY prepared;

Two answers to blocking get one line each. Paxos Commit (Gray and Lamport) runs Paxos on each participant’s vote with 2F + 1 coordinators, surviving F failures with 2PC’s two flush delays but more messages (for five participants and F = 1, with the coordinators running on participant machines, 17 messages against 2PC’s 12); 2PC is its F = 0 case. Three-phase commit adds a pre-commit round so a backup coordinator can finish, but assumes timely messages and no partitions; Gray and Lamport knew of no version with a complete correctness proof.

Sagas: local commits plus compensations

A saga is a sequence of local transactions T1 … Tn, each committed in one service’s database, with a compensating transaction Ci for each step that may need undoing. The 1987 guarantee: the system runs either T1 … Tn, or T1 … Tj followed by Cj … C1.

Compensation is a semantic undo. Undoing “charge ₹450” means “refund ₹450”, a new ledger entry, not a deleted charge. In the paper’s airline example, cancelling a seat subtracts one from the count rather than restoring the old count, because other bookings happened in between. Some compensations only approximate: a cancellation fee or, in the paper’s words, a second letter explaining the first.

Partial results are visible. Until the saga ends, others can see its writes, and nobody is told they saw something later compensated. That is the price of holding no locks.

Compensatable, pivot and retriable steps

Not every step can be undone, so order them on purpose. Saga guides, Azure’s among them, use three kinds:

• Compensatable: can be semantically undone (create the order as PENDING, charge the customer).
• Pivot: the point of no return. If it succeeds the saga must finish; if it fails, the saga unwinds. Here it is the restaurant accepting: you can refund money, but you can’t uncook a biryani.
• Retriable: idempotent steps after the pivot that must eventually succeed (assign a rider, confirm, notify).

The paper calls the two directions backward recovery (compensate) and forward recovery (re-run the missing steps). So run the checks most likely to fail, such as stock and serviceability, before money moves; put the pivot as late as you can; and treat a retriable step that genuinely can’t finish (no rider for 40 minutes) as a business exception for a policy or a human, not an automatic rollback.

Choreography vs orchestration

Choreography has no driver. Each service runs its local transaction and publishes an event, and the next one reacts: OrderPlaced leads to a charge, PaymentCaptured to the restaurant, OrderAccepted to a rider. Failures are events too: OrderRejected makes payment refund. It is event-driven architecture applied to one flow.

Orchestration has one orchestrator that owns the saga’s state machine, sends each participant a command, waits for the reply and runs compensations on failure. It can be a state table in the order service or a workflow engine such as Temporal or AWS Step Functions, which persist progress and retry steps for you.

Choreography suits short flows: nothing new to deploy and no central component. As the flow grows, it exists only as the sum of handlers in several codebases: adding a step touches several services, and “where is order 9f3c stuck?” means correlating four services’ logs. Orchestration puts the flow in one state machine you can read, test and monitor, at the cost of one more component that must be durable and highly available, and the risk of a “god” orchestrator. Keep business rules in the participants; the orchestrator only sequences.

Rule of thumb: choreograph two or three linear steps; orchestrate beyond that, or once there are branches and timeouts. Either way, each step must update its database and emit its event atomically (a transactional outbox), every participant must be an idempotent consumer, and the API answers before the saga ends: return 202 Accepted with the order ID, then let the client poll or receive a push.

Sagas give up isolation

A saga is ACID without the I: with no locks across steps, others see half-finished work. Three anomalies recur: lost updates (a customer’s cancel overwritten by the saga’s later CONFIRMED, a classic lost update), dirty reads (reading a value that is later compensated) and fuzzy reads (two steps of one saga reading different values).

The dangerous one for money is the dirty read. Picture a wallet top-up saga that credits the wallet first and debits the bank second.

The countermeasures, as the Azure guide lists them, each buy back part of the missing I:

• Semantic lock. A step flags the record as in flight (PENDING, HELD) and everyone else respects the flag: available balance excludes pending credits, and a cancel on a PENDING order is refused or queued.
• Commutative updates. balance = balance + 500, not balance = 600: increments and decrements give the same result in any order.
• Pessimistic view. Reorder so the risky update is a retriable step after the pivot and is never compensated: debit the bank first, credit the wallet last.
• Reread value. Check that the record hasn’t changed since the saga read it, a version check as in optimistic locking, and restart if it has.
• Version file. Log the operations on a record so they apply in order, even a cancel that arrives before its create.
• Risk-based concurrency (by value). Choose by risk: a plain saga for a ₹200 order, something stricter for a large transfer.

Going deeper: compensations, retries and stuck sagas

Most saga bugs live in the failure paths. Five rules:

1. A timeout means “unknown”, not “failed”. The charge may have gone through. Look the step up by its idempotency key before deciding, and make compensations safe even if the forward step never happened; Temporal’s saga examples register each compensation before calling its step, which covers exactly this case.
2. Compensations are idempotent and retried until they succeed, with backoff and jitter. They must have no business reason to fail: a refund that a “refund window closed” rule can reject is not a compensation.
3. Retry forward first. Compensate only on a business failure or an exhausted retry budget.
4. Fence late arrivals. A timed-out step can land after its compensation. Record each step in a fence table keyed by saga ID and step, in the same local transaction, so a late step that finds “compensated” becomes a no-op.
5. Log every transition before acting, in a state table or a workflow engine’s history, so a crashed orchestrator resumes where it stopped.

Then watch for stuck sagas. Alert when one stays unfinished beyond a few times its p99 duration (stuck compensating is the most urgent), track the compensation rate per step, and tag every message, log line and span with the saga ID (observability). Steps that exhaust their retries go to a dead-letter queue with audited operator actions, and a nightly reconciliation catches the rest.

TCC: reserve, then confirm or cancel

Try-confirm-cancel sits between 2PC and sagas. Each participant exposes three idempotent operations. Try checks and reserves in a local transaction that commits: berth 23 is HELD for order 9f3c until 10:42. Confirm turns the hold into the real thing and shouldn’t fail for business reasons, because Try already validated. Cancel releases the hold.

A coordinator calls Try everywhere, then Confirm everywhere or Cancel everywhere: 2PC’s shape in business APIs, with two differences. No database locks are held between phases, because a hold is a committed row others can see, and holds expire, so a vanished coordinator can’t block anyone for long. Card payments work the same way: authorise, then capture or void.

The HELD state is a built-in semantic lock, so TCC isolates better than a plain saga. You pay with three endpoints per participant, a sweeper for expired holds, and two races Apache Seata documents: an empty rollback (Cancel arrives for a Try that never landed: record it and return success) and hanging (the Try arrives after its Cancel: refuse it). Confirm can also race the expiry, so make it conditional:

UPDATE seats
SET    status = 'BOOKED'
WHERE  seat_id = 23 AND hold_id = '9f3c'
  AND  status = 'HELD' AND expires_at > now();
-- 0 rows updated after the payment was captured? The hold expired: refund.

For the full seat-hold design, see contention and double booking.

Comparing the options and choosing

Money deserves one more sentence. Banks don’t lock each other’s databases: a UPI-style transfer is a chain of local postings tied together by one transaction ID that each side uses as an idempotency key; a lost reply leaves the payment pending until a status check settles it, and reconciliation fixes mismatches with reversing entries, never deletes.

Cheat sheet

How to say it in the interview

“An order touches payment, the restaurant and delivery, each with its own database, so no local transaction covers it. Two-phase commit could, but every service must be up, locks span two message rounds and several fsyncs, and if the coordinator dies after the votes, participants sit in doubt holding locks. So 2PC stays inside one database like Spanner. Here I’d run an orchestrated saga: create the order as PENDING, charge, then the restaurant accepts, which is the pivot, then retriable steps to assign a rider and confirm. If the restaurant rejects, the orchestrator refunds and cancels. Each step uses an outbox and idempotency keys, and I alert on sagas stuck past their p99.”

Why not just use 2PC between the services? Most can’t take part: payment gateways have no prepare step, and Kafka can’t join an external 2PC. Where they can, availability multiplies down (four at 99.9% give 99.6%) and a dead coordinator blocks everyone.

What if a compensation fails? Retry it with backoff; it must be idempotent and have no business reason to fail. If it still can’t finish, park the saga in a visible “compensation failed” state and alert a human. Reconciliation catches the rest.

Sagas have no isolation. How do you stop them interfering? Semantic locks (PENDING or HELD states that others respect), commutative updates, steps reordered so risky writes come after the pivot, and version checks before updates.

Choreography or orchestration? Choreography for two or three linear steps. Orchestration beyond that, because the flow lives in one state machine you can read, test and monitor, usually on a workflow engine.

How does Spanner get away with 2PC? Its coordinator and participants are Paxos groups, so the decision is replicated and survives a machine failure. Single-group transactions skip 2PC, and mean commit latency stayed under 45 ms up to 50 participants.

Where to go next

• Transactional outbox: how each saga step updates its database and publishes atomically.
• Idempotent consumers and the inbox pattern: surviving duplicate commands.
• Workflow engines (Temporal): durable orchestration you don’t write yourself.
• Multi-step workflows: the interview pattern for long-running flows.
• ACID transactions: what a local transaction guarantees.
• Consensus and leader election: what makes 2PC safe inside Spanner.
• Contention and double booking: seat holds and expiries.
• Design a payment system: sagas and reconciliation with real money.

References and further reading

1. Sagas (Garcia-Molina and Salem, SIGMOD 1987)cs.cornell.eduThe original saga: long-lived transactions split into local transactions with semantic compensations, the T1…Tj, Cj…C1 guarantee, visible partial results, and backward versus forward recovery.
2. Consensus on Transaction Commit (Gray and Lamport, ACM TODS 2006)microsoft.comWhy 2PC blocks when its coordinator fails, its normal-case message and stable-write counts, the trouble with three-phase commit, and Paxos Commit with 2F + 1 coordinators.
3. Transaction Management in the R* Distributed Database Management System (Mohan, Lindsay and Obermarck, ACM TODS 1986)cs.cmu.eduWhich 2PC log records are forced, the four messages per participant, the commit point, and the operator override for blocked prepared transactions.
4. Spanner: Google’s Globally-Distributed Database (Corbett et al., OSDI 2012)static.googleusercontent.comTwo-phase commit across Paxos groups, single-group transactions that skip it, and measured commit latency from 1 to 200 participants.
5. PostgreSQL documentation: PREPARE TRANSACTIONpostgresql.orgThe prepared state on disk, COMMIT or ROLLBACK PREPARED from any session, held locks, VACUUM and wraparound risks, and keeping the feature off without a transaction manager.
6. Pattern: Saga (microservices.io, Chris Richardson)microservices.ioSagas as coordinated local transactions, choreography versus orchestration, no automatic rollback and no isolation, and how a client learns the outcome.
7. Azure Architecture Center: Saga distributed transactions patternlearn.microsoft.comCompensable, pivot and retryable steps; choreography and orchestration trade-offs; lost updates, dirty and fuzzy reads; and the isolation countermeasures.
8. KIP-939: Support Participation in 2PC (Apache Kafka)cwiki.apache.orgHow Kafka’s transaction coordinator already runs a 2PC internally, the 15-minute transaction timeout, and why Kafka cannot yet join an external 2PC.
9. CockroachDB documentation: Transaction layercockroachlabs.comTransaction records and write intents, heartbeat-based aborts of abandoned transactions, and Parallel Commits cutting commit latency from two consensus rounds to one.
10. How Seata solves idempotence, hanging and empty rollback in TCC (Apache Seata blog)seata.apache.orgTry-confirm-cancel as reserve, then commit or release; why confirm and cancel must be idempotent; and the empty-rollback and hanging races with a fence table as the fix.