Spring Boot Batch and Quartz Configuration Example

Learn to configure Quartz scheduler to run Spring batch jobs configured using Spring boot Java configuration. Although Spring’s default scheduler is also good, quartz does the scheduling and invocation of tasks much better and in a more configurable way. This leaves the Spring batch to focus on creating batch jobs only, and let Quartz execute them.

In this tutorial, to keep things focused, we will create a simple batch Job with a single Step that prints a message in the console. Then we will schedule this Job with Quartz to execute at every 10-second interval.

At a very high level, we follow these steps to configure the Quartz triggers with Spring Batch Job execution:

• Add Spring Batch and Quartz dependencies to the project.
• Configure the Spring Batch Job and Steps.
• Create a Quartz JobDetail that references the Spring Batch Job in the QuartzJobBean.
• Create a Quartz Trigger to specify when the Job should be triggered.
• Connect the Trigger and JobDetail in the SchedulerFactoryBean.
• Run your Spring Boot application and observe the scheduled execution of your Spring Batch job based on the Quartz trigger.

1. Maven

Spring boot has built-in support for quartz, so all we need is to import dependencies such as spring-boot-starter-quartz and spring-boot-starter-batch.

Please note that Spring Batch needs at least one database available to store job execution details. In this example, I am using H2 database that Spring Boot supports out of the box. So include 'h2' dependency as well.

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-batch</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-quartz</artifactId>
</dependency>
<dependency>
  <groupId>com.h2database</groupId>
  <artifactId>h2</artifactId>
  <scope>runtime</scope>
</dependency>

2. Create Spring Batch Job and Step

We are creating a very simple Job with a single step to print a message in the console. The Job is registered in the ApplicationContext with the name “customJob“.

import com.howtodoinjava.demo.batch.quartz.tasklets.CustomTasklet;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.job.builder.JobBuilder;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.batch.core.step.builder.StepBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.transaction.PlatformTransactionManager;

@Configuration
@EnableBatchProcessing
public class BatchConfig {

  JobRepository jobRepository;
  PlatformTransactionManager txManager;

  public BatchConfig(JobRepository jobRepository, PlatformTransactionManager txManager) {
    this.jobRepository = jobRepository;
    this.txManager = txManager;
  }

  @Bean
  public Step step1(CustomTasklet customTasklet) {

    var name = "step1";
    var builder = new StepBuilder(name, jobRepository);
    return builder.tasklet(customTasklet, txManager).build();
  }

  @Bean
  public Job customJob(Step step1) {

    var name = "customJob";
    var builder = new JobBuilder(name, jobRepository);
    return builder.start(step1).build();
  }
}

The custom tasklet that does the actual work is:

import org.springframework.batch.core.StepContribution;
import org.springframework.batch.core.scope.context.ChunkContext;
import org.springframework.batch.core.step.tasklet.Tasklet;
import org.springframework.batch.repeat.RepeatStatus;
import org.springframework.stereotype.Component;

@Component
public class CustomTasklet implements Tasklet {

  public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) {
    System.out.println("CustomTasklet start..");

    // ... your code

    System.out.println("CustomTasklet done..");
    return RepeatStatus.FINISHED;
  }
}

3. Spring Batch Schema Creation

Additionally, Spring batch persists its execution data into the database so we need to configure a database and initialize it with the batch schema SQL file.

• The @QuartzDataSource specifies a DataSource used by Quartz other than the application’s main DataSource.
• The @QuartzTransactionManager specifies a TransactionManager used by Quartz other than the application’s main TransactionManager.

import org.springframework.boot.autoconfigure.quartz.QuartzDataSource;
import org.springframework.boot.autoconfigure.quartz.QuartzTransactionManager;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.ClassPathResource;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import org.springframework.jdbc.datasource.init.DataSourceInitializer;
import org.springframework.jdbc.datasource.init.ResourceDatabasePopulator;

import javax.sql.DataSource;

@Configuration
public class DataSourceConfig {

  @QuartzDataSource
  DataSource dataSource;

  public DataSourceConfig(DataSource dataSource){
    this.dataSource = dataSource;
  }

  @Bean
  @QuartzTransactionManager
  public DataSourceTransactionManager transactionManager() {
    return new DataSourceTransactionManager(dataSource);
  }

  @Bean
  public DataSourceInitializer databasePopulator() {
    ResourceDatabasePopulator populator = new ResourceDatabasePopulator();
    populator.addScript(new ClassPathResource("org/springframework/batch/core/schema-h2.sql"));
    populator.setContinueOnError(false);
    populator.setIgnoreFailedDrops(false);
    DataSourceInitializer dataSourceInitializer = new DataSourceInitializer();
    dataSourceInitializer.setDataSource(dataSource);
    dataSourceInitializer.setDatabasePopulator(populator);
    return dataSourceInitializer;
  }
}

4. Creating QuartzJobBean

The QuartzJobBean is a simple implementation of the Quartz Job interface so it can be executed from the Quartz triggers. when executed, it invokes the executeInternal() method which has reference to the Spring Batch JobLauncher, JobLocator and Job beans. In the executeInternal(), we use the Spring batch beans and launch the batch Job.

import lombok.Getter;
import lombok.Setter;
import org.quartz.JobExecutionContext;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.configuration.JobLocator;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.scheduling.quartz.QuartzJobBean;
import org.springframework.stereotype.Component;

@Component
public class CustomQuartzJob extends QuartzJobBean {

  @Getter @Setter
  String jobName;

  JobLauncher jobLauncher;
  JobLocator jobLocator;

  public CustomQuartzJob(JobLauncher jobLauncher, JobLocator jobLocator) {
    this.jobLauncher = jobLauncher;
    this.jobLocator = jobLocator;
  }

  @Override
  protected void executeInternal(JobExecutionContext context) {
    try {
      Job job = jobLocator.getJob(jobName);

      JobParameters params = new JobParametersBuilder()
          .addString("JobID", String.valueOf(System.currentTimeMillis()))
          .toJobParameters();

      jobLauncher.run(job, params);
    } catch (Exception e) {
      e.printStackTrace();
    }
  }
}

5. Quartz JobDetail and Trigger

Next, we need to create the Trigger and JobDetail beans that we have registered with the SchedulerFactoryBean in the above configuration. In these beans, we have to define the Job trigger initial delay and schedule information.

Spring boot automatically detects these beans and configures a SchedulerFactoryBean for us.

import com.howtodoinjava.demo.batch.quartz.quartzJobs.CustomQuartzJob;
import org.quartz.*;
import org.springframework.batch.core.configuration.JobLocator;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class QuartzConfig {

  @Autowired
  private JobLauncher jobLauncher;

  @Autowired
  private JobLocator jobLocator;

  @Bean
  public JobDetail jobDetail() {
    //Set Job data map
    JobDataMap jobDataMap = new JobDataMap();
    jobDataMap.put("jobName", "customJob");

    return JobBuilder.newJob(CustomQuartzJob.class)
        .withIdentity("customJob")
        .setJobData(jobDataMap)
        .storeDurably()
        .build();
  }

  @Bean
  public Trigger jobTrigger() {
    SimpleScheduleBuilder scheduleBuilder = SimpleScheduleBuilder
        .simpleSchedule()
        .withIntervalInSeconds(10)
        .repeatForever();

    return TriggerBuilder
        .newTrigger()
        .forJob(jobDetail())
        .withIdentity("jobTrigger")
        .withSchedule(scheduleBuilder)
        .build();
  }
}

6. Demo

Before running the application, make sure you have disabled the batch jobs auto-start feature in the application.properties file.

spring.datasource.url=jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1
spring.datasource.username=sa
spring.datasource.password=

spring.batch.job.enabled=false

Now run the application as the Spring batch application and check the logs.

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.ComponentScan;

@SpringBootApplication
public class BatchJobByQuartzApplication {
  public static void main(String[] args) {
    SpringApplication.run(BatchJobByQuartzApplication.class, args);
  }
}

The program output. We can see that the Job is repeated after every 10 seconds.

2023-11-29T02:10:13.155+05:30  INFO 23800 --- [eduler_Worker-1] o.s.b.c.l.support.SimpleJobLauncher      : Job: [SimpleJob: [name=customJob]] launched with the following parameters: [{'JobID':'{value=1701204013149, type=class java.lang.String, identifying=true}'}]
2023-11-29T02:10:13.160+05:30  INFO 23800 --- [eduler_Worker-1] o.s.batch.core.job.SimpleStepHandler     : Executing step: [step1]
CustomTasklet start..
CustomTasklet done..
2023-11-29T02:10:13.163+05:30  INFO 23800 --- [eduler_Worker-1] o.s.batch.core.step.AbstractStep         : Step: [step1] executed in 2ms
2023-11-29T02:10:13.166+05:30  INFO 23800 --- [eduler_Worker-1] o.s.b.c.l.support.SimpleJobLauncher      : Job: [SimpleJob: [name=customJob]] completed with the following parameters: [{'JobID':'{value=1701204013149, type=class java.lang.String, identifying=true}'}] and the following status: [COMPLETED] in 9ms

2023-11-29T02:10:23.164+05:30  INFO 23800 --- [eduler_Worker-2] o.s.b.c.l.support.SimpleJobLauncher      : Job: [SimpleJob: [name=customJob]] launched with the following parameters: [{'JobID':'{value=1701204023161, type=class java.lang.String, identifying=true}'}]
2023-11-29T02:10:23.164+05:30  INFO 23800 --- [eduler_Worker-2] o.s.batch.core.job.SimpleStepHandler     : Executing step: [step1]
CustomTasklet start..
CustomTasklet done..
2023-11-29T02:10:23.177+05:30  INFO 23800 --- [eduler_Worker-2] o.s.batch.core.step.AbstractStep         : Step: [step1] executed in 13ms

Clearly both Spring batch jobs are executing based on their schedule configured in quartz triggers.

7. Storing Quartz Jobs

7.1. Using RAMJobStore

When using Spring Boot, by default, an in-memory JobStore (RAMJobStore) is used to store and run the quartz Jobs. We can even declare it explicitly using the property ‘spring.quartz.job-store-type‘.

spring.quartz.job-store-type=memory

7.2. Using JDBC-based JobStoreCMT

To replace the in-memory storage with a JDBC-based store we need to set the ‘spring.quartz.job-store-type‘ to ‘jdbc‘. This automatically causes the database schema to be initialized on startup by using the standard scripts provided by the Quartz library.

spring.quartz.job-store-type=jdbc
spring.quartz.properties.org.quartz.jobStore.tablePrefix=QRTZ_
spring.quartz.properties.org.quartz.jobStore.isClustered=false
spring.quartz.properties.org.quartz.scheduler.instanceName=MyScheduler
spring.quartz.properties.org.quartz.threadPool.threadCount=5

7.3. Using Custom Schema

The default scripts drop existing tables, deleting all triggers on every restart. It is also possible to provide a custom script by setting the ‘spring.quartz.jdbc.schema‘ property.

This may be helpful in cases where the default schema is not compatible with the database in use. We can modify the schema file and use it as the custom schema to overcome similar issues.

spring.quartz.jdbc.schema=classpath:/custom_quartz_schema.sql

8. Conclusion

In this Spring batch tutorial, we walked through the steps to configure and set up the integration, emphasizing the use of Quartz as a scheduler for Spring Batch jobs. We learned to create Spring batch jobs and tasklets, and then execute then using QuartzJobBean.

Finally, we learned to use either in-memory storage or JDBC-based persistent datasource for storing the Quartz jobs.

Drop me your questions in the comments section.

Happy Learning !!

Source Code on Github