Spring batch

Spring Batch

Domain

• Benefit Management Application– Ledger information

Goal• Input CSV• The header System ID,Account ID,Plan

Number,Payee ID,Check Type,Check Date,Transaction Create Date,Check Number,Benefit Type,Pay Code,Pay Source 1,Pay Source 2,Pay Source 3,Pay Source 4,Pay Source 5,Pay Source 6,Pay Source 7,Pay Source 8,Pay Source 9,Deduction 1,Deduction 2,Deduction 3,Deduction 4,Deduction 5,Deduction 6,Deduction 7,Deduction 8,Deduction 9,Deduction 10,Payable ID,Customer ID,Category of Distribution,Customer Last Name,Customer First Name,Customer Middle Initial,…etc…

Output• 600+ bytes to send to Paying Agent• 'payCode' => '%-1.1s',• 'paySource5' => '%011d',• 'taxWithholdingElection' => '%-1.1s',• 'taxableMaritalStatusElectionCode' => '%-1.1s',• 'deduction10' => '%011d',• 'systemID' => '%-1.1s',• 'paySource1' => '%011d',• 'checkType' => '%-1.1s',• 'taxableAllowanceElection' => '%-2.2s',• 'financialInstitutionCity' => '%-16.16s',• 'checkCity' => '%-16.16s',• 'paySource9' => '%011d',• 'planNumber' => '%-6.6s',• 'payeeDateofDeath' => '%-8.8s',• 'aCHRoutingNumber' => '%-9.9s',• 'check3rdStreetAddress' => '%-32.32s',• 'deduction1' => '%011d',• 'transactionCreateDate' => '%-8.8s',• 'deduction2' => '%011d',• 'deduction5' => '%011d',• 'checkNumber' => '%-10.10s',• 'checkDate' => '%-8.8s',• 'financialInstitutionPostalCode' => '%-9.9s',• 'deduction8' => '%011d',• 'categoryofDistribution' => '%-2.2s',• 'customerFirstName' => '%-12.12s',• 'check1stStreetAddress' => '%-32.32s',

Tools

• Spring Source Tools IDE /Eclipse• Apache Maven• Perl• Bash Cygwin (Windows)• Spring Batch legacy code that currently

supports 1.6 spec but needs to upgrade to 1.9 spec

Spring Batch Project

• Start with the simplest hello-world project• Build more complexity on top

Spring Batch Application Context• <?xml version="1.0" encoding="UTF-8"?>• <beans xmlns="http://www.springframework.org/schema/beans"• xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"• xsi:schemaLocation="http://www.springframework.org/schema/beans • http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">

• <bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />

• <bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">• <property name="transactionManager" ref="transactionManager"/>• </bean>

• <bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">• <property name="jobRepository" ref="jobRepository" />• </bean>

• • • </beans>

Maven POM• <name>Spring Batch Hello World</name>• <properties>• <spring.version>2.5.4</spring.version>• <spring-batch.version>2.0.3.RELEASE</spring-batch.version>• </properties>• <dependencies>• <dependency>• <groupId>org.springframework.batch</groupId>• <artifactId>spring-batch-core</artifactId>• <version>${spring-batch.version}</version>• </dependency>• <dependency>• <groupId>org.springframework</groupId>• <artifactId>spring-aop</artifactId>• <version>${spring.version}</version>• </dependency>• <dependency>• <groupId>org.springframework</groupId>• <artifactId>spring-tx</artifactId>• <version>${spring.version}</version>• </dependency>• <dependency>• <groupId>org.springframework</groupId>• <artifactId>spring-core</artifactId>• <version>${spring.version}</version>• </dependency>• <dependency>• <groupId>org.springframework</groupId>• <artifactId>spring-beans</artifactId>• <version>${spring.version}</version>• </dependency>• <dependency>• <groupId>org.springframework</groupId>• <artifactId>spring-context</artifactId>• <version>${spring.version}</version>• </dependency>• <dependency>• <groupId>org.springframework</groupId>• <artifactId>org.springframework.test</artifactId>• <version>3.0.4.RELEASE</version>• </dependency>

Domain Class

Getter and setter(s)…

Tokenizer

• 

• <bean name="defaultTokenizer" class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer"/>

Field Set Mapper

• <bean name="pbtItemFieldSetMapper" class="aspire.mapper.PBTItemFieldSetMapper" />

Line Mapper

• <bean name="pbtItemLineMapper" class="org.springframework.batch.item.file.mapping.DefaultLineMapper">

• <property name="lineTokenizer" ref="defaultTokenizer"/>

• <property name="fieldSetMapper" ref="pbtItemFieldSetMapper"/>

• </bean>

Flat File Item Reader• <bean name="pbtItemReader"

class="org.springframework.batch.item.file.FlatFileItemReader">

• <property name="lineMapper" ref="pbtItemLineMapper"/>

• 

• <property name="resource" value="classpath:pbt-input-4.txt"/>

• </bean>

Input File(s)

Item Processor

• <bean name="pbtItemProcessor" class="aspire.processor.PBTItemProcessor">

• </bean>

Item Writer• <bean id="pbtItemWriter"• class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step">• <property name="resource" value="file:target/output-pbt-4.det"/>

• <property name="lineAggregator">• <bean• class="org.springframework.batch.item.file.transform.FormatterLineAggregator">• <property name="fieldExtractor">• <bean• class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">• <property name="names"• value="systemID,accountID,planNumber,payeeID,checkType,checkDate,transactionCreateDate,checkNumber,benefitType,payCode,paySource1,paySource2,paySource3,paySource4,paySo

urce5,paySource6,paySource7,paySource8,paySource9,deduction1,deduction2,deduction3,deduction4,deduction5,deduction6,deduction7,deduction8,……,financialInstitutionfinancialInstitutionAccountNumber,financialInstitutionAccountType,financialInstitutionStreet1,financialInstitutionCity,financialInstitutionStateCode,financialInstitutionProvince,financialInstitutionPostalCode,financialInstitutionIRSCountryCode,taxWithholdingElection,taxableMaritalStatusElectionCode,taxableAllowanceElection" />

• </bean>• </property>• • <property name="format" value="%-1.1s%-5.5s%-6.6s%-9.9s%-1.1s%-8.8s%-8.8s%-10.10s%-3.3s%-1.1s%011d%011d%011d%011d%011d%011d%011d%011d%011d%011d%011d%011d

%011d%011d%011d%011d%011d%011d%011d%-10.10s%-30.30s%-2.2s%-20.20s%-12.12s%-1.1s%-8.8s%-8.8s%-32.32s%-32.32s%-32.32s%-16.16s%-2.2s%-9.9s%-2.2s%-16.16s%-1.1s%-9.9s%-17.17s%-32.32s%-32.32s%-2.2s%-32.32s%-16.16s%-2.2s%-16.16s%-9.9s%-2.2s%-1.1s%-1.1s%-2.2s" />

• </bean>• </property>• <property name="footerCallback" ref="pbtfootercallback" />• </bean>

• <bean id="pbtfootercallback" class="aspire.core.PBTFooterCallback">• <property name="delegate" ref="pbtItemWriter"></property>• </bean>

Field Set Mapper…

Batch Job• <batch:job id="pbgcPBTIncomingJob">• • <batch:step id="step4">• <batch:tasklet>• <batch:chunk reader="pbtItemReader" processor="pbtItemProcessor"

writer="pbtfootercallback" commit-interval="1">• <batch:streams>• <batch:stream ref="pbtItemReader" />• <batch:stream ref="pbtItemWriter"/>• </batch:streams> • </batch:chunk> • <batch:listeners>• <batch:listener ref="pbtfootercallback"/>• </batch:listeners> • </batch:tasklet>• </batch:step>• </batch:job>

Footer Callback

Footer callback …(2)

How to test/run

• Command line– mvn exec….– jUnit test case

Junit Test Case• import org.springframework.batch.core.repository.JobRestartException;• import org.springframework.beans.factory.annotation.Autowired;• import org.springframework.beans.factory.annotation.Qualifier;• import org.springframework.test.context.ContextConfiguration;• import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;• import org.springframework.util.StopWatch;

• @ContextConfiguration("classpath:simpleJob.xml")• @RunWith(SpringJUnit4ClassRunner.class)• public class HelloTestCase {

• private static Log logger = LogFactory.getLog(HelloTestCase.class);

• @Autowired• @Qualifier("pbgcPBTIncomingJob")• private Job job;

• @Autowired• private JobLauncher jobLauncher;

• @Test• public void test() {• //fail("Not yet implemented");• JobExecution execution = null;• try {• logger.debug("hello world");• logger.debug("Job name: " + job.getName());• execution = jobLauncher.run(job, new JobParameters());

• //execution = jobLauncher.run(job, • //new JobParametersBuilder()• //.addString("inputFile", "ack-test-new.xlsx")• //.addString("outputFile", "1002").toJobParameters());• } catch (JobExecutionAlreadyRunningException e)

jUnit Eclipse Plug-in

Sample output…• Abbreviated Input• 12:47:43,019 DEBUG main PBTItemFieldSetMapper:19 - [1, AGENCY1, ….., …….., 61, RA3, R, 45677,

7, 6, 5, 4, 3, 2, 1, 0, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1,…….., 7, SMITH, JOE, F, 123456, *, 742 ANY STREET,, *, *, ANY CITY, OH, ……, US, *, A, ………….., *, *, *, *, *, *, *, *, *, N, *, 0]

• Abbreviated Output• 12:47:43,019 DEBUG main PBTItemProcessor:21 - after processing [SystemID = 1 AccountID =

AGENCY1 PlanNumber ……… BenefitType = RA3 PayCode = R PaySource1 = 45677 PaySource2 = 7 PaySource3 = 6 PaySource4 = 5 PaySource5 = 4 PaySource6 = 3 PaySource7 = 2 PaySource8 = 1 PaySource9 = 0 Deduction1 = 10 Deduction2 = 9 Deduction3 = 8 Deduction4 = 7 Deduction5 = 6 Deduction6 = 5 Deduction7 = 4 Deduction8 = 3 Deduction9 = 2 Deduction10 = 1 ……….Check2ndStreetAddress = * Check3rdStreetAddress = ……CheckIRSCountryCode = US CheckProvince = * PaymentDestinationType = A ………… FinancialInstitutionPostalCode = * FinancialInstitutionIRSCountryCode = * TaxWithholdingElection = N TaxableMaritalStatusElectionCode = * TaxableAllowanceElection = 0]

• 12:47:43,019 DEBUG main PBTFooterCallback:44 - write...callback...• 12:47:43,019 DEBUG main PBTFooterCallback:65 - 8• ….• 12:47:43,035 DEBUG main PBTFooterCallback:31 - FooterCallback....• 12:47:43,035 INFO main SimpleJobLauncher:111 - Job: [FlowJob: [name=pbgcPBTIncomingJob]]

completed with the following parameters: [{}] and the following status: [COMPLETED]

Counting the bytes

• Fires up cygwin• user@PC123456> /cygdrive/c/dev/spring-

batch/batch-dir/Spring-Batch-Hello-Wor ld/target– $ head -n1 output-pbt-4.det | wc -c– 669

Be aware of newlines char

• Bash/gnu wc utility• userID@PC123456 /cygdrive/c/dev/spring-

batch/batch-dir/Spring-Batch-Hello-World/target– $ head -n1 output-pbt-4.det | perl -ne 'print

substr($_,667,2);' | od -c– 0000000 \r \n– 0000002

• So technically we only 667 bytes instead of 669 as reported by wc

Spring batch

Documents

Transcript of Spring batch