Spring Batch — Upload Files to SFTP Server (Complete Guide)
Uploading files to an external SFTP server is one of the most common requirements in enterprise batch processing. Whether you are sending daily financial settlement reports to a bank, exchanging inventory files with a third-party partner, or archiving internal data logs, secure file transfers are a critical component of backend infrastructure.
In this comprehensive guide, we will walk through how to implement a production-ready SFTP file upload job by combining the scheduling and state management of Spring Batch with the powerful connectivity of Spring Integration SFTP. We will cover everything from connection configuration to handling edge cases in production.
📺 Video walkthrough:
If you prefer learning visually, watch the full implementation explained step by step on YouTube:
👉
Spring Batch SFTP File Upload – Complete Tutorial
Why Use a Tasklet for SFTP Uploads?
When developers first learn Spring Batch, they are usually introduced to the chunk-oriented processing model (ItemReader, ItemProcessor, ItemWriter). However, uploading an entire file to an SFTP server does not fit cleanly into a record-by-record processing model.
SFTP uploads are fundamentally file-oriented operations. Using a Spring Batch Tasklet is the correct architectural choice here because it provides a single, cohesive block of execution. A Tasklet gives you full control over checking if the local file exists, validating the remote directory structure, and executing the transfer logic all at once. It also vastly simplifies restart semantics; if the upload fails halfway through, restarting the job simply re-triggers the Tasklet to attempt the file upload again.
Setting Up Maven Dependencies
To get started, we need to bring in Spring Batch for our job orchestration and Spring Integration for our SFTP capabilities. The spring-integration-sftp module handles the heavy lifting of establishing SSH connections and managing the underlying JSch (Java Secure Channel) library.
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-batch</artifactId> </dependency> <dependency> <groupId>org.springframework.integration</groupId> <artifactId>spring-integration-sftp</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency>
Application Properties Configuration
Next, we define our database connections for Spring Batch's internal metadata tables, alongside our SFTP connection credentials. In a real production environment, you should never hardcode passwords; these values should be injected via environment variables or a secrets manager.
spring.application.name=spring-batch-sftp-upload spring.datasource.url=jdbc:oracle:thin:@localhost:1521/XEPDB1 spring.datasource.username=dbadmin spring.datasource.password=dbadmin1 spring.datasource.driver-class-name=oracle.jdbc.OracleDriver spring.batch.job.enabled=false sftp.host=eu-central-1.sftpcloud.io sftp.port=22 sftp.username=your-username sftp.password=your-password sftp.remote-dir=/pub/example/
Configuring the SFTP Session Factory
Spring Integration relies on a DefaultSftpSessionFactory to create and manage connections to the remote server. We will configure this factory using the properties we defined above. We then wrap this factory in an SftpRemoteFileTemplate, which provides a clean, high-level API (similar to `JdbcTemplate` or `RestTemplate`) for executing remote commands.
@Configuration
public class SftpConfig {
@Value("${sftp.host}")
private String host;
@Value("${sftp.port}")
private int port;
@Value("${sftp.username}")
private String username;
@Value("${sftp.password}")
private String password;
@Bean
public DefaultSftpSessionFactory defaultSftpSessionFactory() {
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory();
factory.setHost(host);
factory.setPort(port);
factory.setUser(username);
factory.setPassword(password);
factory.setAllowUnknownKeys(true);
return factory;
}
@Bean
public SftpRemoteFileTemplate sftpRemoteFileTemplate() {
return new SftpRemoteFileTemplate(defaultSftpSessionFactory());
}
}
Defining the Spring Batch Job
Our job configuration is highly streamlined. We define a single step, `sftpUploadStep`, and attach our custom Tasklet to it. We then build a Job that starts with this step.
@Configuration
public class SpringBatchConfig {
@Bean
public Step sftpUploadStep(JobRepository jobRepository,
PlatformTransactionManager transactionManager,
SftpUploadTasklet tasklet) {
return new StepBuilder("sftpUploadStep", jobRepository)
.tasklet(tasklet, transactionManager)
.build();
}
@Bean
public Job dailyFileJob(JobRepository jobRepository,
Step sftpUploadStep) {
return new JobBuilder("dailyFileJob", jobRepository)
.start(sftpUploadStep)
.build();
}
}
Implementing the SFTP Upload Tasklet
This is where the actual business logic resides. The tasklet first validates that the local file exists to prevent unexpected runtime exceptions. Then, it uses the sftpRemoteFileTemplate.execute() method to open a session.
Inside the session lambda, we perform a crucial safety check: verifying if the remote target directory exists, and creating it if it does not. Finally, we open an InputStream for our local file and stream it directly to the remote server.
@Component
public class SftpUploadTasklet implements Tasklet {
@Autowired
private SftpRemoteFileTemplate sftpRemoteFileTemplate;
@Override
public RepeatStatus execute(StepContribution contribution,
ChunkContext chunkContext) throws Exception {
File file = new File("F:/youtube-videos/LikeSubscribe.jpg");
if (!file.exists()) {
throw new RuntimeException("File does not exist at specified path");
}
sftpRemoteFileTemplate.execute(session -> {
String remoteDir = "/pub/example";
// Ensure the destination directory exists before attempting upload
if (!session.exists(remoteDir)) {
session.mkdir(remoteDir);
}
// Try-with-resources ensures the InputStream is safely closed
try (InputStream is = new FileInputStream(file)) {
session.write(is, remoteDir + "/" + file.getName());
}
return null;
});
return RepeatStatus.FINISHED;
}
}
Triggering the Job via REST API
While many batch jobs run on a fixed schedule (e.g., using `@Scheduled`), file uploads often need to be triggered on-demand by external events or downstream microservices. Here, we expose a simple REST endpoint using JobLauncher to manually kick off the SFTP upload process. We pass the current system time as a job parameter to ensure Spring Batch treats every trigger as a unique job instance.
@RestController
@RequestMapping("/api/batch")
public class JobController {
private final JobLauncher jobLauncher;
private final Job dailyFileJob;
public JobController(JobLauncher jobLauncher, Job dailyFileJob) {
this.jobLauncher = jobLauncher;
this.dailyFileJob = dailyFileJob;
}
@GetMapping("/trigger-sftp")
public ResponseEntity<String> triggerSftpJob() {
try {
JobParameters params = new JobParametersBuilder()
.addLong("startAt", System.currentTimeMillis())
.toJobParameters();
jobLauncher.run(dailyFileJob, params);
return ResponseEntity.ok("SFTP Upload Job triggered successfully.");
} catch (Exception e) {
return ResponseEntity.internalServerError()
.body("Job failed: " + e.getMessage());
}
}
}
Production Best Practices for SFTP Uploads
Writing a basic upload script is easy, but making it robust for enterprise production environments requires extra care. Here are the core best practices you should implement:
1. Use Temporary Filenames During Transfer
If you are uploading a 5GB file, the transfer will take time. If a downstream system starts reading that file while your Spring Batch job is only halfway done uploading it, the downstream system will process incomplete, corrupted data. Always upload the file with a temporary extension (like `data.csv.tmp`). Once the upload is 100% complete, use a remote command to rename the file to its final `data.csv` name.
2. Key-Based Authentication
Avoid using plain-text passwords for SSH/SFTP connections. Instead, configure your DefaultSftpSessionFactory to use an RSA or Ed25519 private key. This is significantly more secure and prevents unauthorized access if your configuration properties are ever leaked.
3. Managing Idempotency and Restarts
If an upload fails due to a network timeout, Spring Batch marks the step as FAILED. When you restart the job, the Tasklet will execute again. Ensure your Tasklet logic is idempotent—meaning it can safely run multiple times without causing data duplication or errors on the remote server.
Frequently Asked Questions
Why not use an ItemWriter for SFTP?
An ItemWriter is designed to write chunks of records (like rows to a database or lines to a CSV). If you already have a complete file resting on your local disk, using a Tasklet is a much simpler, more intuitive way to handle the network transfer.
Can I upload multiple files dynamically?
Yes! Instead of hardcoding a single file path, you can use a Tasklet to read a directory, loop through all files matching a specific regex (like `*.csv`), and upload them sequentially.
Is SftpRemoteFileTemplate thread-safe?
Yes, SftpRemoteFileTemplate is thread-safe. However, if you are using a multithreaded step to upload dozens of files concurrently, ensure your connection pool on the SFTP server is configured to handle the simultaneous SSH sessions.
Conclusion
Using Spring Batch coupled with Spring Integration SFTP provides a clean, robust, and highly observable way to manage file transfers. By utilizing Tasklets, you gain exact control over the execution flow, making your enterprise integrations resilient to network hiccups and downstream server issues.
🎥 Don’t forget to watch the full YouTube walkthrough:
Spring Batch SFTP File Upload – Step-by-Step Video Guide
🧱 Spring Batch Core Components
Understand how ItemReader, ItemProcessor, and ItemWriter work together when exporting data to CSV files.
🔄 Spring Batch ItemProcessor Example
Apply transformation and formatting logic before writing records into CSV output files.
🔁 CSV to Database with Spring Batch
Compare inbound (CSV → DB) and outbound (DB → CSV) batch processing patterns.
🚫 Skip Policy & Error Handling
Handle write failures and formatting errors gracefully while exporting large datasets.
🔀 Conditional Flow in Spring Batch Jobs
Control job execution paths based on CSV generation success or failure.
🧵 Multithreaded Step in Spring Batch
Improve export performance by parallelizing data processing and CSV writing steps.