Begin with the failure model
A reliable client is not “a client with retries.” First identify what can fail: DNS lookup, connection establishment, TLS negotiation, response headers, response body streaming, or the downstream service itself.
Each phase consumes part of the caller's latency budget.
Prefer a typed boundary
A typed client keeps base address, headers, serialization, and failure handling close to one downstream dependency. Business services depend on the typed abstraction instead of constructing requests everywhere.
Set a total timeout budget
Suppose the user-facing request has a 2-second budget. Three attempts with a 1-second timeout can never fit once network and application overhead are included.
Choose an overall budget, reserve time for the caller, and divide the remaining time deliberately across attempts.
Retry only safe transient failures
Reasonable candidates include a connection reset, a short-lived 503, or a throttling response that supplies a usable delay. Avoid retrying:
- 400 validation errors.
- 401 or 403 authorization failures.
- A non-idempotent write without an idempotency key.
- Every exception indiscriminately.
Use exponential backoff with jitter so many clients do not retry together.
Circuit breakers protect the system
A circuit breaker temporarily stops calls when a dependency is consistently failing. This reduces wasted work and gives the dependency room to recover. It does not fix the dependency and should not hide an outage.
Observe every outbound dependency
Capture:
- Dependency name and operation.
- Duration and final status.
- Attempt count.
- Timeout versus caller cancellation.
- Trace or correlation identifier.
Avoid logging secrets, tokens, or complete sensitive payloads.
A strong interview answer
Describe the dependency, its latency budget, which failures are transient, whether the operation is idempotent, the retry/circuit-breaker policy, and how you would measure the result. That explanation demonstrates engineering judgement rather than framework memorisation.