Spring Batch — Import CSV to Database (Complete Guide)
Importing large CSV files into a relational database is one of the most fundamental requirements in backend engineering. Whether you are migrating legacy systems, processing daily financial reports, or importing user data, relying on basic Java loops will eventually lead to OutOfMemory (OOM) errors and locked database tables.
To build a scalable, enterprise-grade solution, we use Spring Batch. This guide goes beyond a simple "Hello World" tutorial. We will construct a production-ready pipeline that utilizes Chunk Processing, robust data validation, fault-tolerant Skip Policies, and highly optimized JDBC batch writes.
📺 Visual Learner? Watch the Complete Implementation!
See exactly how the Reader, Processor, and Writer interact in real-time. Watch the full video tutorial where we build this exact Spring Batch CSV-to-DB pipeline from scratch.
1. Maven Dependencies
We begin by adding the core Spring Batch starter, the Spring JDBC starter (required for our high-performance writer), and an H2 in-memory database for testing purposes. In a production environment, you would simply replace the H2 driver with your preferred database (MySQL, PostgreSQL, or Oracle).
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-batch</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<scope>runtime</scope>
</dependency>2. Application Properties and Tuning
A common mistake when building batch jobs is leaving spring.batch.job.enabled=true. This causes the job to execute immediately upon application startup, which makes programmatic triggering (like via a REST API) difficult. We disable it here. Additionally, we must ensure our HikariCP connection pool is large enough to handle our batch writing threads.
# Database Configuration
spring.datasource.url=jdbc:h2:mem:testdb
spring.datasource.username=sa
spring.datasource.password=
spring.h2.console.enabled=true
# Spring Batch Execution Control
spring.batch.job.enabled=false
# Hikari Connection Pool Tuning
spring.datasource.hikari.maximum-pool-size=30
3. The Domain Model and Sample Data
For this example, we will import an employees.csv file. Notice that row #3 contains intentional bad data ("invalid-email" and a missing ID). We will use this to demonstrate how Spring Batch handles failures gracefully.
id,name,email,department,salary
1,John Doe,john@example.com,Engineering,72000
2,Jane Smith,jane@example.com,HR,50000
3,Broken,invalid-email,Sales,45000
4,Sam Blue,sam@example.com,Engineering,80000
public class Employee {
private Long id;
private String name;
private String email;
private String department;
private BigDecimal salary;
// Standard Getters & Setters omitted for brevity
}
4. The Reader: FlatFileItemReader
The FlatFileItemReader is the industry standard for parsing flat files. We configure it to skip the first row (the header) and map the comma-delimited fields directly into our Employee POJO.
Architectural Insight: Notice the @StepScope annotation. This is crucial. It tells Spring Batch to instantiate this reader only when the Step begins execution. This allows us to dynamically inject the file path via JobParameters at runtime, meaning this single reader can process thousands of different files without restarting the application.
@Bean
@StepScope
public FlatFileItemReader<Employee> reader(@Value("#{jobParameters['file']}") String file) {
return new FlatFileItemReaderBuilder<Employee>()
.name("employeeReader")
.resource(new FileSystemResource(file))
.linesToSkip(1) // Skip CSV Header
.delimited()
.names("id","name","email","department","salary")
.fieldSetMapper(fieldSet -> {
Employee e = new Employee();
e.setId(fieldSet.readLong("id"));
e.setName(fieldSet.readString("name"));
e.setEmail(fieldSet.readString("email"));
e.setDepartment(fieldSet.readString("department"));
e.setSalary(fieldSet.readBigDecimal("salary"));
return e;
})
.build();
}
5. The Processor: Applying Business Validation
The ItemProcessor acts as the middleman between reading and writing. This is where you enforce business rules, clean up strings, or fetch supplementary data from external APIs. If a record fails validation, we intentionally throw an IllegalArgumentException. Our fault-tolerance configuration (configured later) will catch this exception, discard the bad row, and seamlessly continue processing the rest of the file.
public class EmployeeProcessor implements ItemProcessor<Employee, Employee> {
@Override
public Employee process(Employee emp) throws Exception {
if (emp.getEmail() == null || !emp.getEmail().contains("@")) {
throw new IllegalArgumentException("Invalid email format for user: " + emp.getName());
}
if (emp.getSalary() == null || emp.getSalary().doubleValue() < 0) {
throw new IllegalArgumentException("Invalid salary detected.");
}
// Normalize data: Strip trailing whitespaces
emp.setName(emp.getName().trim());
return emp;
}
}
6. The Writer: JdbcBatchItemWriter
When importing massive CSV files, you should avoid using JpaItemWriter if possible. JPA introduces significant overhead because it must manage entity state within a persistence context (the Hibernate L1 Cache).
Instead, we use JdbcBatchItemWriter. It bypasses the ORM entirely and uses raw JDBC batching, executing a single PreparedStatement for the entire chunk, which drastically reduces database round-trips and speeds up processing times.
@Bean
public JdbcBatchItemWriter<Employee> writer(DataSource ds) {
return new JdbcBatchItemWriterBuilder<Employee>()
.dataSource(ds)
.sql("INSERT INTO employee (id, name, email, department, salary) VALUES (:id, :name, :email, :department, :salary)")
.beanMapped()
.build();
}
7. Job Configuration and Fault Tolerance
Here we bring the Reader, Processor, and Writer together inside a Step. We define our chunk size as 100. This means Spring Batch will read 100 lines from the CSV, process them one by one, and then execute a single database commit.
We also implement Fault Tolerance. By declaring .skip(IllegalArgumentException.class), we guarantee that the bad data row in our sample CSV won't crash the entire job.
@Bean
public Job importJob(JobRepository jobRepo, Step importStep) {
return new JobBuilder("importJob", jobRepo)
.start(importStep)
.build();
}
@Bean
public Step importStep(JobRepository jobRepo, PlatformTransactionManager txManager,
FlatFileItemReader<Employee> reader,
EmployeeProcessor processor,
JdbcBatchItemWriter<Employee> writer) {
return new StepBuilder("importStep", jobRepo)
.<Employee, Employee>chunk(100, txManager)
.reader(reader)
.processor(processor)
.writer(writer)
.faultTolerant()
.skip(IllegalArgumentException.class) // Catch our processor validation exceptions
.skipLimit(50) // Fail the job if more than 50 rows are corrupted
.listener(new LoggingSkipListener())
.build();
}
8. Triggering the Job via REST API
Finally, we expose a REST endpoint to trigger this job on demand. Because we used @StepScope in our reader, we can pass the exact file path we want to process as a URL parameter. We also append the current system timestamp to ensure that Spring Batch registers this as a unique JobInstance every time the endpoint is hit.
@RestController
@RequestMapping("/api/jobs")
public class JobLauncherController {
@Autowired
private JobLauncher jobLauncher;
@Autowired
private Job importJob;
@GetMapping("/import-csv")
public String importCsv(@RequestParam String filePath) throws Exception {
JobParameters params = new JobParametersBuilder()
.addString("file", filePath)
.addLong("executionTime", System.currentTimeMillis())
.toJobParameters();
jobLauncher.run(importJob, params);
return "Batch Job Launched Successfully for file: " + filePath;
}
}
Conclusion
By combining the memory-efficient FlatFileItemReader, the raw speed of JdbcBatchItemWriter, and the transactional safety of chunk processing, you now have a highly resilient data pipeline. This pattern handles malformed data gracefully, prevents database locks, and can easily scale to process CSV files containing millions of records.
📥 Related Spring Batch CSV Processing Guides
Build robust CSV-to-database batch pipelines by exploring related Spring Batch concepts such as file handling, processing logic, error management, and performance tuning.
🧱 Spring Batch Core Components
Understand how ItemReader, ItemProcessor, and ItemWriter work together in a complete CSV-to-DB batch job.
📂 Read Multiple CSV Files
Extend CSV-to-DB jobs to support multiple input files using resource-aware ItemReaders.
🔄 Spring Batch ItemProcessor Example
Apply validation, transformation, and enrichment logic before writing CSV records into the database.
🚫 Skip Policy & Error Handling
Handle malformed CSV rows and database write failures gracefully using skip and fault-tolerant configurations.
🔀 Conditional Flow in Spring Batch Jobs
Control job execution paths based on CSV validation or database write outcomes.
🧵 Multithreaded Step in Spring Batch
Improve throughput when importing large CSV files into the database using parallel processing threads.