Learn to configure Quartz
scheduler to run Spring batch jobs configured using Spring boot Java configuration. Although Spring’s default scheduler is also good, quartz does the scheduling and invocation of tasks much better and in a more configurable way. This leaves the Spring batch to focus on creating batch jobs only, and let Quartz execute them.
In this tutorial, to keep things focused, we will create a simple batch Job with a single Step that prints a message in the console. Then we will schedule this Job with Quartz to execute at every 10-second interval.
At a very high level, we follow these steps to configure the Quartz triggers with Spring Batch Job execution:
- Add Spring Batch and Quartz dependencies to the project.
- Configure the Spring Batch Job and Steps.
- Create a Quartz JobDetail that references the Spring Batch Job in the QuartzJobBean.
- Create a Quartz Trigger to specify when the Job should be triggered.
- Connect the Trigger and JobDetail in the SchedulerFactoryBean.
- Run your Spring Boot application and observe the scheduled execution of your Spring Batch job based on the Quartz trigger.
1. Maven
Spring boot has built-in support for quartz, so all we need is to import dependencies such as spring-boot-starter-quartz and spring-boot-starter-batch.
Please note that Spring Batch needs at least one database available to store job execution details. In this example, I am using H2
database that Spring Boot supports out of the box. So include 'h2'
dependency as well.
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-batch</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-quartz</artifactId>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<scope>runtime</scope>
</dependency>
2. Create Spring Batch Job and Step
We are creating a very simple Job with a single step to print a message in the console. The Job is registered in the ApplicationContext with the name “customJob“.
import com.howtodoinjava.demo.batch.quartz.tasklets.CustomTasklet;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.job.builder.JobBuilder;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.batch.core.step.builder.StepBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.transaction.PlatformTransactionManager;
@Configuration
@EnableBatchProcessing
public class BatchConfig {
JobRepository jobRepository;
PlatformTransactionManager txManager;
public BatchConfig(JobRepository jobRepository, PlatformTransactionManager txManager) {
this.jobRepository = jobRepository;
this.txManager = txManager;
}
@Bean
public Step step1(CustomTasklet customTasklet) {
var name = "step1";
var builder = new StepBuilder(name, jobRepository);
return builder.tasklet(customTasklet, txManager).build();
}
@Bean
public Job customJob(Step step1) {
var name = "customJob";
var builder = new JobBuilder(name, jobRepository);
return builder.start(step1).build();
}
}
The custom tasklet that does the actual work is:
import org.springframework.batch.core.StepContribution;
import org.springframework.batch.core.scope.context.ChunkContext;
import org.springframework.batch.core.step.tasklet.Tasklet;
import org.springframework.batch.repeat.RepeatStatus;
import org.springframework.stereotype.Component;
@Component
public class CustomTasklet implements Tasklet {
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) {
System.out.println("CustomTasklet start..");
// ... your code
System.out.println("CustomTasklet done..");
return RepeatStatus.FINISHED;
}
}
3. Spring Batch Schema Creation
Additionally, Spring batch persists its execution data into the database so we need to configure a database and initialize it with the batch schema SQL file.
- The @QuartzDataSource specifies a DataSource used by Quartz other than the application’s main DataSource.
- The @QuartzTransactionManager specifies a TransactionManager used by Quartz other than the application’s main TransactionManager.
import org.springframework.boot.autoconfigure.quartz.QuartzDataSource;
import org.springframework.boot.autoconfigure.quartz.QuartzTransactionManager;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.ClassPathResource;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import org.springframework.jdbc.datasource.init.DataSourceInitializer;
import org.springframework.jdbc.datasource.init.ResourceDatabasePopulator;
import javax.sql.DataSource;
@Configuration
public class DataSourceConfig {
@QuartzDataSource
DataSource dataSource;
public DataSourceConfig(DataSource dataSource){
this.dataSource = dataSource;
}
@Bean
@QuartzTransactionManager
public DataSourceTransactionManager transactionManager() {
return new DataSourceTransactionManager(dataSource);
}
@Bean
public DataSourceInitializer databasePopulator() {
ResourceDatabasePopulator populator = new ResourceDatabasePopulator();
populator.addScript(new ClassPathResource("org/springframework/batch/core/schema-h2.sql"));
populator.setContinueOnError(false);
populator.setIgnoreFailedDrops(false);
DataSourceInitializer dataSourceInitializer = new DataSourceInitializer();
dataSourceInitializer.setDataSource(dataSource);
dataSourceInitializer.setDatabasePopulator(populator);
return dataSourceInitializer;
}
}
4. Creating QuartzJobBean
The QuartzJobBean is a simple implementation of the Quartz Job interface so it can be executed from the Quartz triggers. when executed, it invokes the executeInternal() method which has reference to the Spring Batch JobLauncher, JobLocator and Job beans. In the executeInternal(), we use the Spring batch beans and launch the batch Job.
import lombok.Getter;
import lombok.Setter;
import org.quartz.JobExecutionContext;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.configuration.JobLocator;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.scheduling.quartz.QuartzJobBean;
import org.springframework.stereotype.Component;
@Component
public class CustomQuartzJob extends QuartzJobBean {
@Getter @Setter
String jobName;
JobLauncher jobLauncher;
JobLocator jobLocator;
public CustomQuartzJob(JobLauncher jobLauncher, JobLocator jobLocator) {
this.jobLauncher = jobLauncher;
this.jobLocator = jobLocator;
}
@Override
protected void executeInternal(JobExecutionContext context) {
try {
Job job = jobLocator.getJob(jobName);
JobParameters params = new JobParametersBuilder()
.addString("JobID", String.valueOf(System.currentTimeMillis()))
.toJobParameters();
jobLauncher.run(job, params);
} catch (Exception e) {
e.printStackTrace();
}
}
}
5. Quartz JobDetail and Trigger
Next, we need to create the Trigger and JobDetail beans that we have registered with the SchedulerFactoryBean in the above configuration. In these beans, we have to define the Job trigger initial delay and schedule information.
Spring boot automatically detects these beans and configures a SchedulerFactoryBean for us.
import com.howtodoinjava.demo.batch.quartz.quartzJobs.CustomQuartzJob;
import org.quartz.*;
import org.springframework.batch.core.configuration.JobLocator;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
public class QuartzConfig {
@Autowired
private JobLauncher jobLauncher;
@Autowired
private JobLocator jobLocator;
@Bean
public JobDetail jobDetail() {
//Set Job data map
JobDataMap jobDataMap = new JobDataMap();
jobDataMap.put("jobName", "customJob");
return JobBuilder.newJob(CustomQuartzJob.class)
.withIdentity("customJob")
.setJobData(jobDataMap)
.storeDurably()
.build();
}
@Bean
public Trigger jobTrigger() {
SimpleScheduleBuilder scheduleBuilder = SimpleScheduleBuilder
.simpleSchedule()
.withIntervalInSeconds(10)
.repeatForever();
return TriggerBuilder
.newTrigger()
.forJob(jobDetail())
.withIdentity("jobTrigger")
.withSchedule(scheduleBuilder)
.build();
}
}
6. Demo
Before running the application, make sure you have disabled the batch jobs auto-start feature in the application.properties file.
spring.datasource.url=jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1
spring.datasource.username=sa
spring.datasource.password=
spring.batch.job.enabled=false
Now run the application as the Spring batch application and check the logs.
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.ComponentScan;
@SpringBootApplication
public class BatchJobByQuartzApplication {
public static void main(String[] args) {
SpringApplication.run(BatchJobByQuartzApplication.class, args);
}
}
The program output. We can see that the Job is repeated after every 10 seconds.
2023-11-29T02:10:13.155+05:30 INFO 23800 --- [eduler_Worker-1] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=customJob]] launched with the following parameters: [{'JobID':'{value=1701204013149, type=class java.lang.String, identifying=true}'}]
2023-11-29T02:10:13.160+05:30 INFO 23800 --- [eduler_Worker-1] o.s.batch.core.job.SimpleStepHandler : Executing step: [step1]
CustomTasklet start..
CustomTasklet done..
2023-11-29T02:10:13.163+05:30 INFO 23800 --- [eduler_Worker-1] o.s.batch.core.step.AbstractStep : Step: [step1] executed in 2ms
2023-11-29T02:10:13.166+05:30 INFO 23800 --- [eduler_Worker-1] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=customJob]] completed with the following parameters: [{'JobID':'{value=1701204013149, type=class java.lang.String, identifying=true}'}] and the following status: [COMPLETED] in 9ms
2023-11-29T02:10:23.164+05:30 INFO 23800 --- [eduler_Worker-2] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=customJob]] launched with the following parameters: [{'JobID':'{value=1701204023161, type=class java.lang.String, identifying=true}'}]
2023-11-29T02:10:23.164+05:30 INFO 23800 --- [eduler_Worker-2] o.s.batch.core.job.SimpleStepHandler : Executing step: [step1]
CustomTasklet start..
CustomTasklet done..
2023-11-29T02:10:23.177+05:30 INFO 23800 --- [eduler_Worker-2] o.s.batch.core.step.AbstractStep : Step: [step1] executed in 13ms
Clearly both Spring batch jobs are executing based on their schedule configured in quartz triggers.
7. Storing Quartz Jobs
7.1. Using RAMJobStore
When using Spring Boot, by default, an in-memory JobStore (RAMJobStore) is used to store and run the quartz Jobs. We can even declare it explicitly using the property ‘spring.quartz.job-store-type‘.
spring.quartz.job-store-type=memory
7.2. Using JDBC-based JobStoreCMT
To replace the in-memory storage with a JDBC-based store we need to set the ‘spring.quartz.job-store-type‘ to ‘jdbc‘. This automatically causes the database schema to be initialized on startup by using the standard scripts provided by the Quartz library.
spring.quartz.job-store-type=jdbc
spring.quartz.properties.org.quartz.jobStore.tablePrefix=QRTZ_
spring.quartz.properties.org.quartz.jobStore.isClustered=false
spring.quartz.properties.org.quartz.scheduler.instanceName=MyScheduler
spring.quartz.properties.org.quartz.threadPool.threadCount=5
7.3. Using Custom Schema
The default scripts drop existing tables, deleting all triggers on every restart. It is also possible to provide a custom script by setting the ‘spring.quartz.jdbc.schema‘ property.
This may be helpful in cases where the default schema is not compatible with the database in use. We can modify the schema file and use it as the custom schema to overcome similar issues.
spring.quartz.jdbc.schema=classpath:/custom_quartz_schema.sql
8. Conclusion
In this Spring batch tutorial, we walked through the steps to configure and set up the integration, emphasizing the use of Quartz as a scheduler for Spring Batch jobs. We learned to create Spring batch jobs and tasklets, and then execute then using QuartzJobBean.
Finally, we learned to use either in-memory storage or JDBC-based persistent datasource for storing the Quartz jobs.
Drop me your questions in the comments section.
Happy Learning !!
Comments