Spring Batch Skip and Retry — Fault-Tolerant Batch Processing (Complete Guide)
In real-world enterprise applications, batch jobs often operate on hundreds of thousands or even millions of records at a time. During these long-running processes, failures are not just a possibility; they are an absolute certainty. Network connections drop, cloud APIs experience transient latency, and input data from third-party vendors is rarely clean.
A production-ready Spring Batch job must handle these inevitable failures gracefully. If a job processing 50,000 records fails on record 49,999 because of a single malformed email address, stopping the entire job and forcing a manual restart is a catastrophic waste of computing resources.
This guide explores exactly how to implement Skip and Retry in Spring Batch 5 using a real, runnable example. We will go deep into how the framework evaluates these rules at runtime, how to configure Backoff Policies, and how to capture bad records using a SkipListener.
📺 Visual Learner? Watch the Video Walkthrough!
Watch me simulate API timeouts and construct this exact fault-tolerant Skip and Retry architecture live on YouTube.
1. Understanding the High-Level Execution Flow
Before diving into code, it is crucial to understand the order of operations within a chunk-oriented processing step. When an item is read and passed to the processor, Spring Batch wraps the execution in a transaction. If an exception occurs, the framework does not immediately skip the item.
Spring Batch always attempts retry first, provided the exception matches your retry configuration. Only when the maximum number of retry attempts is completely exhausted does the framework evaluate your skip rules.
Reader → Processor (Exception Thrown)
↓
Retry Evaluated (Loop until limit reached)
↓
Skip Evaluated (After retry exhausted or if explicitly non-retryable)
↓
Writer✔ Retry: Use for transient failures (timeouts, temporary network issues, database deadlocks).
✔ Skip: Use for permanent failures (bad input formatting, strict validation errors, 400 Bad Request).
2. Defining Custom Exceptions: Transient vs. Permanent
The most important architectural design decision you will make is deciding which failures are retryable and which are skippable.
A transient error is something that might succeed if you simply try again a few seconds later (e.g., HTTP 503 Service Unavailable). We define an ApiTimeoutException for this.
public class ApiTimeoutException extends RuntimeException {
public ApiTimeoutException(String message) {
super(message);
}
}A permanent error will never succeed, no matter how many times you retry it (e.g., HTTP 400 Bad Request due to a malformed email). Retrying this is a waste of CPU cycles. We define a BadRequestException which we will instruct Spring Batch to skip immediately.
public class BadRequestException extends RuntimeException {
public BadRequestException(String message) {
super(message);
}
}3. The ItemProcessor: Simulating API Failures
In most enterprise applications, data enrichment or external API calls happen inside the ItemProcessor. To make this code runnable locally without needing a separate mock server, we are simulating API behavior using a ConcurrentHashMap to track attempts.
import org.springframework.batch.item.ItemProcessor;
import org.springframework.stereotype.Component;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
@Component
public class PersonProcessor implements ItemProcessor<Person, Person> {
private final Map<String, Integer> attempts = new ConcurrentHashMap<>();
@Override
public Person process(Person person) {
String email = person.getEmail();
// 1. Simulating a transient network glitch that resolves on the 3rd try
if ("retry@yopmail.com".equalsIgnoreCase(email)) {
int count = attempts.merge(email, 1, Integer::sum);
System.out.println("Attempt " + count + " for email: " + email);
if (count < 3) {
throw new ApiTimeoutException("API timeout connecting to registration server");
}
}
// 2. Simulating a permanent validation error from the external API
if (email.contains("badrequest")) {
throw new BadRequestException("400 Bad Request: Invalid email format");
}
// Success path
return person;
}
}4. The Missing Piece: Implementing a SkipListener
When you tell Spring Batch to "skip" a record, it drops that record from the chunk and continues processing. But in a production environment, skipped records cannot just disappear into a black hole. You must audit them so business analysts or engineers can fix the data and reprocess it later.
We do this by implementing a SkipListener.
import lombok.extern.slf4j.Slf4j;
import org.springframework.batch.core.SkipListener;
import org.springframework.stereotype.Component;
@Slf4j
@Component
public class BadDataSkipListener implements SkipListener<Person, Person> {
@Override
public void onSkipInRead(Throwable t) {
log.error("Error occurred while reading data: {}", t.getMessage());
}
@Override
public void onSkipInProcess(Person person, Throwable t) {
// Here you would typically write the failed record to a "Dead Letter" Database Table
log.warn("SKIPPED PROCESS for Person [{}]. Reason: {}", person.getEmail(), t.getMessage());
}
@Override
public void onSkipInWrite(Person person, Throwable t) {
log.error("SKIPPED WRITE for Person [{}]. Reason: {}", person.getEmail(), t.getMessage());
}
}5. Step Configuration: Wiring Fault Tolerance (Spring Batch 5)
Now we tie the Processor, the Listener, and our rules together in a Spring Batch 5 @Configuration class. Notice the inclusion of a FixedBackOffPolicy. Without a backoff policy, Spring Batch would execute all 3 retries within milliseconds of each other, which defeats the purpose of waiting for a struggling downstream API to recover.
import org.springframework.batch.core.Step;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.batch.core.step.builder.StepBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.retry.backoff.FixedBackOffPolicy;
import org.springframework.transaction.PlatformTransactionManager;
@Configuration
public class FaultTolerantStepConfig {
@Bean
public Step processingStep(JobRepository jobRepository,
PlatformTransactionManager txManager,
PersonReader reader,
PersonProcessor processor,
PersonWriter writer,
BadDataSkipListener skipListener) {
// Wait exactly 2 seconds between retry attempts
FixedBackOffPolicy backOff = new FixedBackOffPolicy();
backOff.setBackOffPeriod(2000);
return new StepBuilder("fault-tolerant-step", jobRepository)
.<Person, Person>chunk(5, txManager)
.reader(reader)
.processor(processor)
.writer(writer)
// --- FAULT TOLERANCE CONFIGURATION ---
.faultTolerant()
.retry(ApiTimeoutException.class)
.retryLimit(3)
.backOffPolicy(backOff)
.skip(BadRequestException.class)
.skipLimit(10) // Fail the entire job if more than 10 bad records are found
.listener(skipListener) // Attach our audit listener
.build();
}
}6. Common Spring Batch Anti-Patterns
❌ Retrying Validation Errors
One of the most common mistakes is applying a blanket Exception.class to the retry logic. If a record fails because a required field is missing, retrying it 5 times will simply result in 5 identical failures. This wastes processing time and extends your batch window. Always strictly define which exceptions warrant a retry.
❌ Using Large Chunk Sizes for Unreliable APIs
While a chunk size of 1000 is great for writing to a local database, it is terrible for calling fragile third-party REST APIs. If the 999th API call in a chunk fails, the entire transaction rolls back, and Spring Batch has to manage complex state to figure out which items need to be re-processed. Keep chunk sizes small (1-50) when external network calls are involved.
❌ Forgetting the Backoff Policy
If an external service throws a Rate Limit Exceeded (HTTP 429) error, immediately slamming the service with another request a millisecond later will only guarantee another failure. You must configure a FixedBackOffPolicy or an ExponentialBackOffPolicy to give the struggling service time to recover.
⚙️ Master Spring Batch Fault Tolerance
To build truly resilient background jobs, you must understand how skip logic interacts with transactions, chunk sizes, and multithreading. Explore these related architecture guides.
Understand how ItemReader, ItemProcessor, and ItemWriter behave when a transaction rolls back during a retry.
Move beyond simple skip limits and learn how to implement dynamic Custom SkipPolicies for complex error handling.
Track the total number of skipped items at the Job level and send email alerts if the skip threshold gets dangerously high.
Learn how to route a batch job to a specific "Error Notification Step" if too many records trigger the SkipListener.
Understand the strict limitations and dangers of combining Skip/Retry logic with multithreaded concurrent steps.
Learn when to abandon Chunk processing entirely and use Tasklets for safer, single-pass fault-tolerant operations.