Serialization
0-day streakReview

Your progress

Level 1

0 XP100 XP to level 2
0day streak
0best streak
0topics started
0cards reviewed
    Review cards

    Progress is stored only in this browser. No account is required.

    Serialization and schema evolution: the 60-second story

    Serialization and schema evolution

    Tuesday’s deploy adds one field. Whose code breaks?

    Objects cross the wire and sit on disk as bytes. Mid-rollout, v1 and v2 read each other’s bytes, and stored data outlives both.

    One order: 48 bytes in JSON, 10 in Avro

    The values cost about the same in every format. The gap is how each one says which field is which: names, numbers or nothing.

    Is binary always worth it?

    Bad: Binary everywhere: 3.9× smaller

    After gzip, 1,000 orders in JSON and Avro were only 1.3× apart.

    Good: JSON at the edge

    Readable everywhere. Binary inside, where volume or CPU truly matters.

    One rollout needs both directions

    Nobody upgrades at once. Old consumers meet new records mid-rollout; upgraded ones replay old records. Backward: new reads old. Forward: old reads new.

    Backward compatible? Upgrade consumers first. Forward? Producers first. Then a bad release gets rolled back.

    The catch

    Rollbacks flip the order. For anything important, aim for full compatibility and deploy in any order.

    Reuse field 4 and a ₹50 discount becomes 5,000 points

    Bytes carry numbers, never names, so a field is its number. Old records still hold 4. reserved 4; makes the compiler refuse.

    A v1 service saves the order. The ₹20 tip vanishes.

    It decodes JSON into a class with no tip field, sets DELIVERED and saves it all back. No error. Protobuf binary keeps unknown fields.

    Rename amount without breaking anyone

    1. Expand: writers send both amount and amount_paise
    2. Migrate: readers use amount_paise, falling back to amount
    3. Backfill old records, or wait for retention
    4. Contract: stop writing amount once nothing reads it

    Remember

    JSON at the edge, Protobuf or Avro inside. Versions always overlap, so changes are additive with defaults, field numbers are never reused, and a registry checks compatibility in CI.