Sherman Hayes Minneapolis, MN

Timeouts Are a Contract

reliability

Give every hop in a call chain its own one-second timeout and you have not built a one-second system. You've built a system where the front door gives up after a second while three services behind it keep working on a request nobody is waiting for anymore.

Per-hop timeouts don't compose. Service A calls B with a one-second timeout; B calls C with a one-second timeout. A second in, A gives up and returns an error to the user. B doesn't know that. C doesn't know that. Both keep going, holding a connection, a worker, and a database cursor, doing work whose only possible destination is a socket that has already closed.

Under load this is not a minor inefficiency. It's the thing that turns a slowdown into a collapse: the fraction of your capacity spent on abandoned work grows exactly when you can least afford it.

Pass a deadline, not a duration

The fix is to send an absolute deadline with the request and have every hop derive its own budget from what remains:

incoming deadline  = T+1000ms
now                = T+840ms
remaining          = 160ms      -- this is your budget, not 1000ms
                                -- if it's already negative, fail immediately

A hop with 40 ms left should refuse to start a call that it knows takes 200 ms on a good day. Failing instantly is strictly better than failing in 200 ms, and it's the cheapest capacity you will ever recover.

Cancellation is the other half

A deadline tells downstream how long it may take. Cancellation tells it to stop now, because the caller has left. Without it, deadline propagation still leaves work running to the end of a budget nobody is waiting on. Most RPC frameworks give you both and people wire up only the first one.

And the worst timeout is the one you didn't set

Every client library has a default and it is almost always either infinity or something absurd like two minutes. Those are the timeouts that hold a connection pool hostage. Grep your dependencies for the ones you never configured; that list is usually longer than you'd like and it's the cheapest afternoon of reliability work available.