Spring WebClient Retry and RetryWhen (with Examples)

When a service downstream doesn’t respond within a specific time limit or replies with a server error related to its momentary inability to process the request, you can configure your client to try again. This article explores the concept of retrying HTTP requests with Spring Boot WebClient to build more resilient and reliable applications.

We are starting with a quick reference to methods discussed in this article. In the next sections, we will discuss each method in detail.

1. What is a Retry? Why do we need it?

With the rise in the adoption of Microservices, there is an increasing need to make external API calls for various usecases. When we invoke remote APIs, failures may happen due to various reasons such as a network outage, server being down, network glitch, rate limit, etc. In such cases, we usually try to retry the operation a few times before sending an error response to the client.

Retries increase the chance of getting a response back from a remote service when it’s momentarily overloaded or unresponsive.

We need to carefully design the retry strategy in our application to make it efficient. Let’s list down some best practices for retries.

• Limit the number of retries: Do not retry indefinitely. Have a hard stop on the number of retries. If the API call is still failing after the set number of retries, it’s better to communicate the failure.
• Retry only for transient/server-side errors: Transient errors are temporary and are most likely resolved by retrying a few times. The remote server being unavailable is one such example. Do not retry for failures caused by invalid data or authentication errors since retries would not help here.
• Use exponential back-off for retry: Exponential back-off is to increase the delay between each retry. For example, if we have 3 retry attempts and the first retry is after 2 seconds, the second retry could be after 3 seconds, and the third after 5 seconds. We could also add a random jitter with exponential back-off.
• Watch out for time spent: We must remember to balance resilience and user experience. You don’t want users to wait too long while retrying the request behind the scenes. If you can’t avoid that, make sure you inform the users and give them feedback about the status of the request.

Also, consider combining timeout with the retry mechanism for a time-bound response to users. It has been discussed later in this article.

2. Initial Setup

We will be creating a simple Spring Boot web application which would call mocked external APIs using WebClient. For mocking the webserver, we will be using WireMock.

2.1. Maven

To use WebClient, we must have the spring-boot-starter-webflux module imported into our Spring Boot project.

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-webflux</artifactId>
</dependency>

2.2. Application Setup

Let’s create our Controller which would expose ‘http://localhost:8000/helloWorldResource‘. This would invoke the service which uses WebClient for external calls.

@RestController
public class WebClientController {

	@Autowired
	private HelloWorldService helloWorldService;

	@GetMapping(value = "/helloWorldResource")
	public Mono<String> getResource() {
		return helloWorldService.getResource();
	}
}

Let’s add the WebClient configuration which will help us configure the ‘baseURL‘ for the remote APIs.

@Configuration
public class WebClientConfig {

	@Bean
	public WebClient webClient() {
		return WebClient.builder().baseUrl("http://localhost:8080/resource").build();
	}
}

Here, we configure the baseUrl as ‘http://localhost:8080/resource‘ which we setup in the next section.

2.3. Simulating Service Failures using WireMock

Let’s simulate an API would can be invoked at http://localhost:8080/resource. The first 2 invocations of this API would fail, whereas the third invocation would return a successful response. To do the same we need to use WireMock’s stateful behavior.

public class MockServer {

	private static final String SECOND_FAILURE = "Second Failure";
	private static final String FIRST_FAILURE = "First Failure";
	private static final String SERVICE_UNAVAILABLE = "!!! Service Unavailable !!!";
	private static final String RETRY_SCENARIOS = "RetryScenarios";
	private static final String API_URL = "/resource";
	private static final String SUCCESS_RESPONSE = "Hello world!";

	public static void main(String[] args) {
		WireMockServer wireMockServer = new WireMockServer();
		wireMockServer.start();

		stubFor(get(urlEqualTo(API_URL)).inScenario(RETRY_SCENARIOS).whenScenarioStateIs(Scenario.STARTED)
				.willReturn(aResponse().withStatus(503).withBody(SERVICE_UNAVAILABLE))
				.willSetStateTo(FIRST_FAILURE));

		stubFor(get(urlEqualTo(API_URL)).inScenario(RETRY_SCENARIOS).whenScenarioStateIs(FIRST_FAILURE)
				.willReturn(aResponse().withStatus(503).withBody(SERVICE_UNAVAILABLE))
				.willSetStateTo(SECOND_FAILURE));

		stubFor(get(urlEqualTo(API_URL)).inScenario(RETRY_SCENARIOS).whenScenarioStateIs(SECOND_FAILURE)
				.willReturn(aResponse().withStatus(200).withBody(SUCCESS_RESPONSE))
				.willSetStateTo(Scenario.STARTED));
	}
}

3. Retry Mechanisms with WebClient

Spring WebClient provides several built-in mechanisms for handling retries for Mono and Flux APIs.

• retry() – Indefinite retries in case of errors. It reties the entire HTTP request, regardless of the response status code.
• retry(count) – Takes a long parameter to limit the number of retries.
• retryWhen(Retry retrySpec) – Retries based on the strategy defined in the Retry specification.

3.1. Default Behavior

By default, retry() would re-subscribe indefinitely. This is not ideal and can bring down the remote APIs. We must retry only a few times before failing.

Since our Mock server would fail twice before giving a successful response, let’s retry thrice before failing. In this case retry() would re-subscribe to the Mono three times before failing.

@Component
public class HelloWorldService {
	
	@Autowired
	private WebClient webClient;

	public Mono<String> getResource() {
		return webClient.get()
			.retrieve()
			.bodyToMono(String.class)
			.retry(3);
	}
}

The same behavior can be achieved using retryWhen() using the Retry spec. We are also adding logging to see if it’s actually retrying.

public Mono<String> getResource() {
	return webClient.get()
		.retrieve()
		.bodyToMono(String.class)
		.retryWhen(Retry.max(3)
			.doBeforeRetry(x -> logger.info("Retrying " + x.totalRetries())));
}

This produces the following output which indicates the third invocation gave a successful response.

2023-10-07T19:36:52.295+05:30  INFO 24892 --- [ctor-http-nio-3] c.h.caffeine.service.HelloWorldService   : Retrying 0
2023-10-07T19:36:52.303+05:30  INFO 24892 --- [ctor-http-nio-3] c.h.caffeine.service.HelloWorldService   : Retrying 1

Notice that the above retries are instant, which means the client does not wait before retrying. Starting a sequence of retry attempts, one after the other, risks making the system even more unstable. You don’t want to launch a DoS attack on your own applications!

Let’s see how we can overcome this.

3.2. Retry with Fixed Delay

Ideally, we may want to avoid the retry storm and give the server some time to recover before retrying again. In such instances, it’s a good practice to add some delay before trying again.

Let’s add a delay of 2 seconds before each retry using the Retry.fixedDelay() method.

public Mono<String> getResource() {

	return webClient.get()
	.retrieve()
	.bodyToMono(String.class)
	.retryWhen(Retry.fixedDelay(3, Duration.ofSeconds(2)));
}

3.3. Retry with Exponential Backoff

A better approach is using an exponential backoff strategy to perform each retry attempt with a growing delay. By waiting for more and more time between one attempt and the next, you’re more likely to give the backing service time to recover and become responsive again.

Exponential backoff is used to increase the delay between each retry. This is achieved using the Retry.backoff() method and gives the server an even better chance of recovery and hence greater chances of a successful response on retry.

public Mono<String> getResource() {

	return webClient.get()
		.retrieve()
		.bodyToMono(String.class)
		.retryWhen(Retry.backoff(3, Duration.ofSeconds(2)));
}

3.4. Adding Randomness with Jitter

To add randomness between the already delayed retries, we may consider adding the jitter to the delay interval. When you have multiple instances of API clients running, the jitter factor ensures that the replicas will not retry requests simultaneously.

In the following example, the code will retry with a jitter of at most 75% of the computed delay. By default, a jitter of at most 50% of the computed delay is used.

public Mono<String> getResource() {

	return webClient.get()
		.retrieve()
		.bodyToMono(String.class)
		.retryWhen(Retry.backoff(3, Duration.ofSeconds(2)).jitter(0.75));
}

4. Handling Exhausted Retries

There will be cases when retries may not help solve the problem. This would mean we exhaust all the retries available and the client would fail with a RetryExhaustedException. We can override this behavior to handle exhausted retries using the onRetryExhaustedThrow() method.

Let’s create a custom ServiceException which would be thrown once the retries are exhausted.

public class ServiceException extends RuntimeException {
	public ServiceException(String message) {
		super(message);
	}
}

We can now configure to throw the ServiceException once the retry limit is reached.

public Mono<String> getResource() {

	return webClient.get().retrieve()
	.bodyToMono(String.class)
	.retryWhen(Retry.backoff(3, Duration.ofSeconds(2))
		.onRetryExhaustedThrow((spec, signal) -> {
			throw new ServiceException(
					"Service call failed even after retrying " + signal.totalRetries() + " times");
		}));
}

This will ensure if the request fails after retries, a ServiceException is thrown. We can handle this exception using @ControllerAdvice and @ExceptionHandler annotation.

5. Retry Only for Specific Status Codes

WebClient also offers the possibility of retrying on specific HTTP response codes. For example, we can add retry logic only if the server returns a 5xx HTTP code in the response.

To achieve status-code-based retries, we throw ServiceException in case of 5xx response codes. For any other response codes, no exception is thrown. We can take advantage of this to filter out only ServiceException for retrying.

A typical flow would look like:

• Check the response status using onStatus()
• If the status is 5xx, throw ServiceException
• Filter for ServiceException in retryWhen()

public Mono<String> getResource() {

	return webClient.get()
		.retrieve()
		.onStatus(HttpStatusCode::is5xxServerError,
			resp -> Mono.error(new ServiceException(resp.statusCode().toString())))
		.bodyToMono(String.class)
		.retryWhen(Retry.backoff(3, Duration.ofSeconds(2))
			.filter(ex -> ex instanceof ServiceException));
}

Let’s run this and observe the output. Note that post 2 retries, we will be able to get a successful response.

2023-10-08T17:25:14.608+05:30  INFO 18084 --- [ctor-http-nio-3] c.h.caffeine.service.HelloWorldService   : Retrying 0
2023-10-08T17:25:17.124+05:30  INFO 18084 --- [ctor-http-nio-3] c.h.caffeine.service.HelloWorldService   : Retrying 1

6. Retry with Timeout

Be very careful when you combine the timeout() method with retry logic. Their order of appearance matters a lot and can change their meaning completely:

Placing the retryWhen() operator AFTER timeout() means that the timeout is applied to each retry attempt.

public Mono<String> getResource() {

	return webClient.get()
		.retrieve()
		.bodyToMono(String.class)
		.timeout(Duration.ofSeconds(2))
		.retryWhen(Retry.backoff(3, Duration.ofSeconds(2)));
}

Placing the retryWhen() operator BEFORE timeout() means that the timeout is applied to the overall operation. It means that the whole sequence of the initial request and retries has to happen within the given time limit

public Mono<String> getResource() {

	return webClient.get()
		.retrieve()
		.bodyToMono(String.class)
		.retryWhen(Retry.backoff(3, Duration.ofSeconds(2)))
		.timeout(Duration.ofSeconds(2));
}

7. Conclusion

In this tutorial, we learned about retries and how they can be configured in WebClient including the retry() and retryWhen() operators. We also listed best practices for retries and demonstrated various retry handling strategies such as fixed delay, backoff, error handling and logging.

Remember to adjust your retry strategies and logging based on the specific needs and characteristics of your application and the services it interacts with. Additionally, monitor the behavior of your retry mechanisms in production to ensure they perform as expected and provide the desired level of resilience.

Happy Learning !!

Source Code on Github