Working with APIs & Integrations
Anyone can make an API call that works once. Building an integration that keeps working is a different discipline - and it's the one that pays the bills.
The first time you call a third-party API, it feels like magic. A dozen lines of Python, a key pasted into a header, and suddenly your little script is reading live data from a system on the other side of the planet. It works. You ship it. And then, somewhere between the demo and the real world, it stops working - quietly, intermittently, at the worst possible moment.
That gap between "it works" and "it keeps working" is the entire job. Making a single happy-path request is trivial. Building an integration you can trust is a craft, and it's worth naming the disciplines that separate the two.
You are a guest in someone else's house
The mental shift that changes everything is this: when you consume an API, you do not control it. You did not design the endpoints. You cannot fix the bugs. You will not be consulted before they deprecate a field, throttle your traffic, or have a bad afternoon and start returning 500s. Your code lives downstream of decisions made by a team you will never meet.
Internalising that humility is the start of reliability. Every assumption you make about the other system is a future incident waiting to happen. The provider's documentation says tokens last an hour - until the day it's fifty minutes. The endpoint returns a list - until the day it returns an empty object. Defensive integration means treating every remote response as a hypothesis to be verified, not a fact to be trusted.
The network is not your friend
The biggest lie in a tutorial is the absence of failure. Tutorials show the call succeeding. Production shows the call timing out, getting rate-limited, returning malformed JSON, or hanging forever because nobody set a timeout. The network sits between you and the API, and the network is hostile by nature: packets drop, connections reset, DNS hiccups, and latency spikes without warning.
So the question is never "what does this look like when it works?" It is "what does this look like when it fails?" A robust client answers that question on every line. It sets a timeout on every request, because a call with no timeout is a worker thread you will eventually lose. It retries transient failures - the 429s, the 503s - with exponential backoff and jitter, so a struggling provider gets room to breathe instead of a stampede. It refuses to retry the things that must not be retried, because charging a customer twice is worse than charging them zero times. And for writes that absolutely must survive a retry, it leans on idempotency keys so that "try again" never means "do it twice."
None of this is glamorous. All of it is the difference between an integration that pages you at 3 a.m. and one that quietly recovers on its own.
Secrets are a liability, not a convenience
Every integration carries credentials, and every credential is a liability you are now responsible for. The failure modes here are not subtle - a key committed to a public repo, a token printed in a log, a secret baked into a mobile app that anyone can decompile. These mistakes are common precisely because the lazy path is so easy: just paste the key in the code, it'll be fine.
It will not be fine. Credentials belong in environment variables or a secrets manager, scoped to the least privilege they need, separated per environment, and rotated on a schedule. They should never appear in source control, in logs, in error messages, or in URLs. Treat the question "could a leaked log file expose this secret?" as something you must be able to answer with a confident no - before you ship, not after the breach.
Conversations go both ways
For a long time, integration meant you calling them. But the modern integration is a two-way conversation: they call you, too. Webhooks flipped the model - instead of polling a payments API every thirty seconds to ask "anything new?", you give the provider a URL and they tell you the instant something happens.
That power comes with a new class of responsibility. Your webhook endpoint is exposed to the entire internet, which means anyone can forge an event unless you verify the signature on every request - over the raw bytes, with a constant-time comparison, no exceptions. Providers retry deliveries when they're unsure, which means you will receive duplicates and must make every handler idempotent. And they expect a fast acknowledgement, which means the real work belongs in a queue, not in the request handler. Get these wrong and you have either a security hole, a double-processing bug, or a provider that thinks you're down and floods you with retries.
Reliability is a habit, not a feature
There is no single line of code you add to make an integration reliable. Reliability is the sum of small, unglamorous habits: classify errors before reacting to them, translate the provider's data into your own vocabulary at a single boundary, test the sad paths as carefully as the happy one, log enough to diagnose an incident without leaking a secret, and watch the integration closely enough to know it's sick before a customer does.
The engineers who are trusted with the important integrations are not the ones who can make the first call. Everyone can make the first call. They're the ones who assumed the other system would fail, and built something that held anyway.
No comments:
Post a Comment