also fail fast
A switch that stops calling a failing dependency for a while after repeated errors, so you fail fast instead of piling up waiting requests, then lets a few test calls through to see if it recovered.
You have done this if
You stopped sending requests to a model endpoint after a burst of 5xx errors and routed to a fallback for a minute.
Say it in a review
The gateway trips a breaker per provider; after five failures in thirty seconds we route around it.
On the AI Application map LLM Gateway, API Gateway