Batch processing in Java with Spring


In times of business process engines, ESBs and SOAs you might think that the good old batch processing is forgotten. But insurance companies and banks have large amounts of data to be moved and here the batch is still the first option to choose. For some customers we have implemented Java-based batch processes and have made good experiences. In many projects we found complex and individual implementations. So, let’s have a look at some standard technology. It’s worth it.

Our partner springsource offers an Open Source Batch Framework in addition to the already very popular components such as the springframework, Spring MVC and Spring WebFlow. Unfortunately, there are many outdated web articles from the early days of the framework.

Technically, the framework is well documented. At this point I would like to show a practical example and not mentioning individual details. Using Spring Batch makes it possible to implement a working solution in a short time and only a little effort.

Example from a Car-Insurance-Company

A customer switches to another insurance company and tells his current claim categories to the new insurer. To check this, the new insurer contacts the GDV (association of german insurers). The GDV provides the so called VWB-service, the system to verify this specified categories. The basis for the communication are text files with a fixed-length record structure.

Basic Configuration

The configuration for the processing of incoming VWB-messages with Spring Batch Version 1.x looks like this:

<bean id="vwbIncomingJob" parent="simpleJob">
    <property name="name" value="vwbIncoming" />
    <property name="restartable" value="true" />
    <property name="steps">
            <bean parent="skipLimitStep">
                <property name="streams">
                        <ref bean="fileItemReader" />
                <property name="itemReader" ref="itemReader" />
                <property name="itemWriter" ref="itemWriter" />
                <property name="skippableExceptionClasses" value="java.lang.Exception" />
                <property name="fatalExceptionClasses">
                <property name="skipLimit" value="${job.vwbIncoming.skipLimit}" />
                <property name="commitInterval" value="${job.vwbIncoming.commitInterval}" />
                <property name="listeners">
                        <ref bean="inputFile"/>                             
                        <ref bean="logFileFail" />
                        <ref bean="logFileComplete" />
                        <ref bean="itemLoggerListener"/>                             
            <bean parent="taskletStep">
                <property name="tasklet" ref="mailTasklet" />

A Spring Batch job consists in most cases of 1-n steps. In this example a special SkipLimitStep is used, where you can configure exactly which types of exceptions will be accepted or will cancel the job directly. This is usually very helpful because not all records can be interpreted correctly and a restart is not necessary, when only a few items are wrong.

The configuration also shows that the individual resources (e.g. input file, log files) are injected in the step as a listener. The purpose here is to use a Spring Batch component, which is responsible for clean creation and processing of files. In addition it is also possible to use wildcards for file names.


<bean id="inputFile" class="org.springframework.batch.core.resource.StepExecutionResourceProxy">
     <property name="filePattern" value="file:${}${job.vwbIncoming.incoming.path}//"/>

The job is divided into the following tasks:

1. ItemReader: reading the file and transform the records into an XML document


<bean id="itemReader" class="org.springframework.batch.item.file.FlatFileItemReader" >    
    <property name="comments">
    <property name="lineTokenizer" ref="flatFileTokenizer"/>    
    <property name="resource" ref="inputFile"/>        
    <property name="fieldSetMapper" ref="vwbDokumentFieldSetMapper"/>                         
<bean id="flatFileTokenizer"   class="org.springframework.batch.item.file.transform.PrefixMatchingCompositeLineTokenizer">
    <property name="tokenizers">
            <entry key="10" value-ref="recordType10" />
            <entry key="20" value-ref="recordType20" />
            <entry key="21" value-ref="recordType21" />
<bean id="recordType10" class="org.springframework.batch.item.file.transform.FixedLengthTokenizer">
    <property name="names" value="recordType, operationNumber, date, vuGstNr, vsnr,      requestType, vehicleIdentificationNumber, [...]" />
    <property name="columns" value="1-2, 3-20, 7-14, 21-28, 29-48, 49-50, 51-67, [...]"/>               

This process can be configured exclusively in the XML file. The FlatFileItemReader receives a reference to the input file and delivers each line to a LineTokenizer. The standard implementation of PrefixMatchingCompositeLineTokenizer transforms the data into FieldSets, comparable to an array or a database ResultSet, where each field can be accessed by an index. The GDV provides each record with a record type as a prefix, so that the LineTokenizer always knows exactly which fields to be mapped. Different implementations for example using dynamic record lengths are available. The FieldSetMapper is the only place where you have to implement some code. The implementation of the method public Object mapLine(FieldSet fieldSet) creates from a FieldSet the target object. This example is using a generic implementation that creates a Java object, which is later transformed using XStream into an XML document.

2. ItemWriter: Processing and persistence of the items in the target system

From the perspective of Spring Batch not much is happening here and is not supposed to! The goal should always be to delegate to a business service which is responsible for the processing. This design rule leads to the advantage of better testability and reusability by other components – for example in online processing. In the first stage the document is attached only to the target contract. So manual processing is required after the batch.

3. Tasklet: E-Mail the log files

Of course, as with any clean implemented software component there must be a monitoring. Conceivable are different approaches. Spring Batch offers a listener interface for almost any place in the job. The VWB sample log entries are written per item, which provides information about the success/failure type of processing. In the last step the MailTasklet sends the appropriate log files to the responsible persons.

<bean id="vwbIncomingTasklet" class="com.codecentric.example.batch.tasklet.MailTasklet">
    <property name="mailTo">
    <property name="mailSubject" value="${job.vwbIncoming.betreff}" />
    <property name="mailText" value="${job.vwbIncoming.body}" />
    <property name="mailFrom" value="${jobs.mailtemplate.sender}" />    
    <property name="attachments">
            <entry key="vwbIncomingSuccesful" value-ref="logFileComplete" />
            <entry key="vwbIncomingFailure" value-ref="logFileFail" />


As expected from Spring the testability of components is very easy. The job configuration can be tested with all needed dependencies using the well-known testing framework components of Spring. Here is an example providing a basis for a test:

public class VwbIncomingJobITest extends AbstractJUnit4SpringContextTests {
    /** The Job-Executor */
    private JobLauncher jobLauncher;
    /** The job */
    private Job job;
    /** The Service for the processing of incoming documents */
    private BusinessService businessServiceMock;
    /** The inputfile */
    private static final String INPUT_FILE = "src/test/resources/vwbIncoming/vwbTest.txt";
    private JobParametersBuilder builder;
    public void setUp() {
        businessServiceMock= (BusinessService ) applicationContext.getBean("businessServiceMock");
        builder = new JobParametersBuilder();
        Resource inputFile = new FileSystemResource(INPUT_FILE);
        builder.addString("", inputFile.getFilename());
    public void testLaunchVwbIncomingJob() throws Exception {
        JobExecution jobExecution =, builder.toJobParameters());


The example shown is mostly based on the old version of the framework. Currently 2.1 is released and offers useful innovations, including a simplification of the configuration. In one of my next blog entries I will discuss the differences in detail. Another interesting topic in this context is the use of Spring Integration, where we would be back in the ESB world 😉 I would be grateful for any feedback and suggestions for topics relating to Spring Batch :-)


Dennis Schulte

Dennis Schulte

Dennis Schulte ist seit 2009 als Senior IT Consultant bei der codecentric AG tätig. Er unterstützt seine Kunden insbesondere im Bereich Enterprise-Architekturen und der Optimierung von IT-Prozessen. Aufgrund seiner langjährigen Erfahrung als Architekt und Entwickler verfügt er über ein umfassendes Wissen in den Bereichen Java, Open-Source, Microservices und Continuous Delivery. Seine Projektschwerpunkte liegen in der Architekturberatung und der Durchführung von Projekten im Versicherungsumfeld.

Share on FacebookGoogle+Share on LinkedInTweet about this on TwitterShare on RedditDigg thisShare on StumbleUpon


  • September 6, 2010 von DaDa15

    Thank youfor this good example it would be pleasant if i can have this example…

  • How about using pattern matching.



Your email address will not be published. Required fields are marked *