I still remember the Slack message. “hey is the profile API down?? names are gone on the dashboard.” It wasn’t down. It was doing exactly what I’d told it to do about twenty minutes earlier, which was kind of the problem.

Honestly, the PR looked boring. That was sort of the point. We had a bigger refactor coming and I wanted the profile endpoint cleaned up first. So one field got a new name (the old one had annoyed me for months). Another got deleted, since I couldn’t see who’d still read it. Oh, and the user ID went from an integer to a string. The database team was moving to UUIDs and asked if I could handle that bit while I was in there. Sure, why not? CI passed. My reviewer dropped a thumbs up. I merged and went to make tea.

Our React dashboard, the Android app and the internal admin panel all broke within the hour, each in its own creative way. This post is basically everything I wish I’d known before that afternoon about changing an API other people depend on.

What Actually Broke (and Why None of Our Tests Noticed)

The annoying part, looking back, is that the backend was fine. Every response matched the new code, and our unit tests checked exactly that. They had no reason to fail.

The dashboard was still asking for the old field name. No error, nothing red in the console. Just empty little boxes where customer names used to be. Android was messier. That app’s in Kotlin and it wanted a Long for the ID. It got a string, the parser bailed, and the profile screen crashed on open. Sentry lit up. And the admin panel had a search filter built on the field I deleted, so support suddenly couldn’t look anyone up.

None of those teams reviewed my PR. Why would they? Nobody thought of it as their change. My tests didn’t know those apps existed, and to be fair, I had mostly forgotten too.

Why Small API Changes Turn Into Breaking Changes

I’d been writing backend code for about six years at that point, and somewhere along the way I’d picked up a lazy rule: adding a field is safe, anything else, use your judgment. My judgment that day was bad. What I’d glossed over is that the moment a client parses your response, it forms an opinion about what that response looks like, and you don’t get a vote on that opinion after the fact.

Renaming or Removing Fields

Why rename it at all? Mostly because the old name lied about what it held, and I’d tripped over that more than once. Fair reason to want it gone. Terrible reason to rip it out in a cleanup PR. The dashboard didn’t care that the new name was nicer. It wanted the old key and got undefined. As for the field I deleted, I’d grepped the backend repo, found zero references and called it dead. Our frontend code lives in three other repos. I didn’t open any of them.

Quiet Data Type Changes

If you only remember one bit of this post, make it this one. Going from 42 to “42” sounds like nothing, and our schema validation shrugged at it too. But our Android app parses responses into Kotlin data classes, and a Long field receiving a string isn’t a warning there, it’s an exception. Our iOS developer told me later that Swift would have reacted the same way. We only dodged that one because the iOS app wasn’t calling that endpoint yet. Lucky, not smart.

Changed Defaults, Error Payloads, and Status Codes

Different incident, same family. This one wasn’t me, thankfully. Some months later a teammate touched the not-found case on a lookup endpoint. It used to be a 404. He made it a 200 with an empty object, and his reasoning in the PR thread honestly made sense. The problem? The mobile app’s retry code saw that 200, figured all was well, and cached the empty profile. So users opened the app to a blank account. No crash, no alert, nobody paged. A support ticket a week later was the first we heard of it, and I’d argue that one was worse than mine.

The Tests We Had vs. the Tests We Needed

During the incident review someone asked, pretty fairly, how a change like that got through CI. So I spent an evening going through what our tests actually checked. Plenty of unit tests, and they weren’t bad. A couple of integration tests that started a real Postgres in Docker. A Cypress suite for checkout that took around fourteen minutes and failed for no reason often enough that everyone just clicked rerun without reading the logs.

Every one of those tests had been written next to a new feature, to prove that feature worked. Not a single one went back after a change to check that the endpoints our existing apps called still returned what those apps expected. That’s the job regression testing in software testing is meant to do, and at the API layer we’d simply never set it up. Nobody decided against it. It just never made it onto a sprint.

I’d bet most teams are closer to where we were than they’d like to admit. Tests for the thing you’re building feel like progress. Tests for the thing you’re not touching feel optional, right up until a Slack message tells you otherwise.

Backward Compatibility Lessons That Stuck

Add First, Remove Later

These days a rename takes me a few weeks instead of one PR, and I’m weirdly fine with that. I ship the new field alongside the old one, so for a while the response carries both and looks a little redundant. The frontend teams move over whenever their next release goes out. I added a counter that logs every read of the old key, and when it’s sat flat at zero for two weeks or so, I open the PR that deletes it. Someone on our team calls this expand and contract. I think of it as the version of me that doesn’t get pinged during dinner.

Version Your API, but Don’t Hide Behind It

We’ve got a v2 for two endpoints, both cases where the original design was honestly beyond saving. I’m careful about adding more, though, because I’ve seen where that goes. At my last company one team spun up a new version any time they wanted to change something awkward, and by the time I left they had three live versions of the orders API, each with its own little bugs that someone had to remember. API versioning is a tool I still use. It’s just not my first move anymore.

Let Consumers Define the Contract

Consumer-driven contract testing is the piece I’d add first if I were starting over. In our setup, the dashboard and mobile teams each keep a small file describing what they read from our API, down to field names, types and the status codes they handle. CI runs the backend against those files on every build. Had that existed back then, my rename would have gone red on my branch before I even asked for a review. I’d have grumbled for twenty minutes and fixed it, and customers would never have known.

Test Against Real Traffic, Not What You Remember

Contracts have a blind spot, though. Somebody has to write them, and you can’t write down a dependency you don’t know you have. When I asked the Android team about the ID type, the honest answer was “we never thought about it, it was always a number.”

That’s what pushed us to start recording real API traffic and replaying it as tests. Keploy, for example, captures actual requests and responses, plus the outgoing calls to databases and other dependencies, and turns them into test cases and mocks without anyone writing them by hand. So now when I touch an endpoint, the new responses get checked against what real clients sent and got back, instead of against whatever I happen to remember about the API. What it flags is mostly unglamorous, a key with a new name or a number that’s suddenly a string. Exactly the kind of thing that broke three apps on us.

Deprecate Out Loud

When something is going away, we now do the slightly annoying thing and announce it in a few places: a deprecation header on the response, a line in the changelog, and a post in the frontend channel with an actual removal date. Probably more than needed. I’ll take a few eye-roll emojis over another surprise crash report.

What My Pre-Merge Routine Looks Like Now

It’s not a formal checklist, more of a pause. If a PR changes a response body, I stop for a minute before merging and go through it in my head. Did anything get renamed, dropped or change type? If so, are we still sending the old shape too? And did the replay tests actually run on this branch, or did I talk myself out of them because it was “just a small cleanup”?

That last one still catches me. Every change that’s broken our clients started out as a small cleanup.

Where I’ve Landed

If you’d asked me before all this, I’d have filed backward compatibility under API design, something you sort out once in a planning doc. I see it differently now. It’s closer to a reflex: you touch the JSON, you think about who’s reading it. Contract tests and traffic replay make that reflex cheaper to act on, but they didn’t give it to me. That afternoon did.

One small thing you could try this week. Take an endpoint your team changed recently and search every repo in your org for the field names it returns. There’s a decent chance you’ll find a consumer nobody mentioned. I found two.

The API Change That Broke Three Frontends: Lessons in Backward Compatibility was originally published in Coding Nexus on Medium, where people are continuing the conversation by highlighting and responding to this story.