An HTTP retry is a feature of many HTTP clients/proxies that automatically resends failed client requests according to preconfigured conditions — until the request is successfully transferred or the client gets an error message. A number of factors such as an unstable network connection, latency, or security measures (such as a firewall) can prevent HTTP requests from successfully reaching their destination. Alternatively, the destination web server can encounter a transient problem such as its software being restarted.
HTTP retries are thus essential error-handling mechanisms. It's not always possible (or easy) to actively monitor a system and backend health. Accordingly, request retries give infrastructure teams a first line of defense against connection issues without having to actively intervene, unless deeper issues arise.
Clients and reverse proxies — including load balancers — began adding HTTP retry functions in response to transient failures. While request failure rates aren't very high across most services, the retry feature gives teams easy insurance against unexpected downtime.
How do HTTP retries work?
HTTP retries attempt to solve this using a specialized HTTP header that can instruct clients on the following:
How many times it should try to connect to the backend
How often or how much time should elapse before attempting a retry (including timeouts)
Under which conditions the client should retry the request, at all
Which alternative backend servers to connect to if the first server is unreachable
You should be cautious when configuring HTTP retries, however, since a large volume of simultaneous retries can congest the network. Some requests may repeat actions, such as billing a user's credit card, if they were partially completed but failed to return responses to the client. Establishing a circuit breaker is a good way to limit such impacts when configuring a retry strategy. Always consider current and potential server load while implementing retries.
Plus, retries are somewhat "expensive" since they demand more of the backend server's attention within that specific attempt window. A bottlenecked API may benefit from circuit breaking to temporarily block access instead of retrying infinitely. This gives the backend time to recover.
HTTP retries and status codes
Response status codes also clue clients into which requests should be retried and which shouldn't. Some failures are recoverable and others aren't — as a major issue might waste a retry and congest the network. Here are some status codes that may warrant an automatic HTTP retry:
408 (Request Timeout) – Retries may be possible when the server takes too long to respond.
429 (Too Many Requests) – Retries may be possible after being rate limited according to your
Retry-Afterheader and its directives.500 (Internal Server Error) – Retries may be possible when server issues resolve themselves.
502 (Bad Gateway) – Retries may be possible when temporary network interruptions or service issues arise unexpectedly, but then resolve themselves.
503 (Service Unavailable) – Retries may be possible when service outages are temporary.
504 (Gateway Timeout) – Retries may be possible when recoverable, downstream server errors cause failures.
Conversely, HTTP retries typically aren't recommended after sending invalid requests, when authentication fails, when client permissions are inadequate, when semantic errors exist within the request, or when the requested resource isn't found. These retries are likely to fail again while potentially impacting performance at scale.
Some cloud platforms can counteract these slowdowns by using a feature called "backoff," which forces the client to wait a certain amount of time before retrying instead of doing so immediately. There's also exponential backoff, which increases the waiting time between subsequent failures until the retry is cancelled according to the header directive. A combination of retries and backoff may be effective in some instances. However, choosing one option may be necessary to keep clients connected with minimal frustration. This is where maximum HTTP retry caps come in handy.
What are the benefits of HTTP retries?
HTTP retries offer the following advantages to teams needing extra backend resilience:
They contribute to high availability by ensuring more reliable connectivity between clients and services.
They're relatively easy to implement and are widely documented.
When encountering a small number of failures, they're computationally cheap and add minimal backend load.
They give teams an automatic and passive way to solve request failures.
They're loggable and can be analyzed while auditing client behavior, alongside system performance.
They're a reliable method for solving brief network faults.
As mentioned, HTTP retries aren't always a great solution. In large quantities, they can degrade network performance via congestion and demand too many resources from the server. Application performance may drop when using retries to solve long-running issues from which it's hard to recover.
You’ve mastered one topic, but why stop there?
Our blog delivers the expert insights, industry analysis, and helpful tips you need to build resilient, high-performance services.
Does HAProxy support HTTP retries?
Yes! HAProxy and HAProxy One enable users to set HTTP retry behaviors according to number of retry attempts, time elapsed between retries, healthy backup backends (when the primary isn't reachable), and predetermined conditions in your configuration.
To learn more about HTTP retries in HAProxy and supplemental strategies, check out our guide to implementing a circuit breaker.