Discussing Batch Interfacefor ConnectCICTM

CommSys Inc is presenting information to our ConnectCICTM Partners on the Batch Interface offered as an additional module to our ConnectCICTM middleware solution

Thursday, November 8, 2012

1pm EST

Introduction

Our goal is to provide an overview education and insight to the Batch Interface available for the ConnectCICTM Middleware solution

Presenter:Robert Turner – President at CommSys, Incturner@commsys.com937-425-0402

What does “batch” mean?

• It is an old mainframe term in this context

• In many applications the “processing” might only happen once a day– Or each hour– Program or job executes at a set time, read

data, and produces output

Batch Advantages

• In certain applications, batch architecture is an advantage

– Simplifies the programming model and code development

– Creates a more robust application with clearer workflows

– Removes certain security issues

Why batch in a CJIS Environment?

• The CJIS environment is a asynchronous, message oriented environment– In English:

• Communications with CJIS systems are not like interacting with a database system

• Messages are sent but there is no guarantee of timing or order in the responses

– This is difficult – especially when there is no user interface or user involvement needed

Why batch in a CJIS Environment?

• Does not require partner application to support real-time transaction flow to the state

– allows many applications not requiring interactive access to CJIS data to utilize state and federal information in a mode which is more architecturally compatible

– Issues of complex communication “real-time” communication are avoided. Programmatically you are writing to and reading from a file.

What type of workload is batch oriented?

• There are situations when submitting large quantities or “batches” of transactions is needed

– Such as: vehicle registration lookups for traffic citations or pawned stolen article serial number checks

What type of workload is batch oriented?

• These activities are “non-real-time”, as there is no need or an immediate response to the inquiry, or completely separate from the originating action

– Many older applications are designed for “overnight” processing

ConnectCICTM Batch Interface

• The ConnectCIC™ Batch Interface program stands alone and is designed to function without operator interaction: – processes the requests in each file and

submits to the state message switch via a normal ConnectCIC configuration

– Responses are returned via a file as well

ConnectCICTM Batch Interface Features

• Runs as a Windows Service

• Responses from state are returned in a separate response text file again in the same ConnectCIC API XML format

• Provides the ability to throttle transactions to the state based on configuration – the number of messages per minute which

will be sent

Wiring Diagram

Relationship toConnectCICTM

• Uses a separate ConnectCIC™ connection from interactive application users

– Coexists with a normal interactive connection from CAD/RMS/Mobile

Workflow Overview

• Batch Processor Overview– Requests

• monitors a configured input folder for files • processes the requests in each file and submits to

the state message switch • when processing has completed, the file is moved to

a configured processed folder

– Responses• responses received from the state message switch

are stored in another configured response folder• any errors that occur during processing are stored in

a configured errors folder

Application Interfacing

• Batch Processor and Application program can be on separate servers

– Application has to know how to write and read files in ConnectCICTM API

– You need to define a method of file transfer

File Transfer Mechanism

• ConnectCICTM Batch Interface doesn’t care what you use– We recommend that the folder is local

(direct drive or SAN)

• If you are moving confidential data an encrypted method should be used– Like SFTP

• You have a lot of latitude in what you use

End-to-End Workflow

• Court system creates batch file for transmission • Batch file is encrypted for transport and then renamed • Batch inquiries are passed to ConnectCIC™ (CCIC™) for

processing as a state or NCIC transaction• Inquiries are passed to state message switch for completion • Inquiry returns are transported back to CCIC™ • CCIC™ hands responses back to CCIC™ Batch processor • Batch processor creates response file

– sizing the file first against the number of responses received – then by the time interval between response receptions.

• Court system retrieves the response file(s) Encrypts the file(s) and brings them back under Courts system control for further usage

Example Applicationsfor Batch Processor

Citation Update System

• An example application that batch processor is used for a Citation status update system

– We have a state that has a regional citation system accessible via the state system (therefore ConnectCICTM)

– Citation system is not on Police LAN– It produces and transfers a file to system with

Batch Processor and ConnectCICTM to update citations (paid etc.)

Pawn Query

• A Police Agency is responsible for supervision and processing of pawn tickets

– Online system for pawn shops to submit their intake– The pawn system produces a file to inquire against

NCIC stolen article file– Any hits are processed back to detectives to

evaluate hit and if they wish to contact pawn shop

Selling ConnectCICTM

Batch Interface

Legacy ApplicationMigration

• Many municipalities and counties have legacy applications which use the state system

• The state interface on these systems tend to be hard to maintain “one offs”

• ConnectCICTM Batch Processor can make internal support of these applications easier– Especially when the state changes things

Tool to “soften”CJIS Security Policy

• CJIS Security Policy is very difficult on systems no owned by law enforcement– Courts– Non-LE Government

• Batch Processor allows separation of application from state connection

• Control of users, transactions and CJI is still necessary

Licensing

• We license the ConnectCICTM Batch Processor in two ways:

– First for the processor itself

– Second on the number of applications using the software

• Utilizes all the options that have been purchased for ConnectCICTM (i.e. Data Mining)

Multiple Applications

• An application in Batch Processor terms is the number of directories we look at

• Most users want separate processing queues for separate applications– Allows separate programs to use the batch

processor simultaneously– Essentially partitioning of the file directories

Batch ProcessorTechnical Discussion

Configurations

• There are a variety of configuration options for the ConnectCIC™ Batch Interface application

• All configuration options are stored in an XML configuration file stored in the installation folder

• The following slides provide a list of each configuration option supported by the ConnectCIC™ Batch Interface application

ConnectCICTM Workflow

• The ConnectCIC™ Batch Interface process polls the request folder for files, finding the new file put in place by the client application

• The ConnectCIC™ Batch Interface opens the file and begins submitting each request to the state switch

• After the request file has been successfully processed, it is deleted by the ConnectCIC™ Batch Interface– The ConnectCIC™ Batch Interface then continues on to the

next request file and the process is repeated until there are no more request files to process

Request File Format

• The request file consists of two pieces of information per record:– an optional length and the content of the request

to be processed*

– if multiple requests are to be stored per file, the length is required

• otherwise, the length of the request is assumed to be the length of the file

* The length of the request is to be written as a 6 character, zero filled, right justified ASCII number.

Request Workflow

• The client application generates a request file – store multiples requests within a single file or a single request per file

• The client application copies, moves, or uploads the request file to a temporary file name in the request folder monitored by the ConnectCIC™ Batch Interface– this can be done with a file copy operation, FTP upload, or any

other means of transferring files

• The client application renames the temporary file to a file name expected by the ConnectCIC™ Batch Interface process

Batch Interface Diagram

Response Workflow

• When the ConnectCIC™ Batch Interface receives a response and it is stored in a response file– the ConnectCIC™ Batch Interface continues writing responses to the

response file based on the configuration options

• After the ConnectCIC™ Batch Interface determines that a new response file is to be generated, the old file is renamed appropriately and a new file is started

• At some point, polled or as a preconfigured time, the client application should check the response, processed, and error folders for files created by the ConnectCIC™ Batch Interface– these files should be downloaded and deleted from the ConnectCIC™

Batch Interface folders

Response File Format

• The response file format is similar to that of the request file format– However – the six byte ASCII length and more than

one response [may or may not] be written to the file depending on the configuration settings

* The length of the request is to be written as a 6 character, zero filled, right justified ASCII number.

File Naming Conventions

• All files must start with the configured client prefix followed by a ten character, zero filled, right justified ASCII sequence number should be unique and persist across application restarts

• Note that while a file is being generated, copied, moved, or uploaded into the configured folder, it should be given a temporary file extension

• The sequence number of a request file is not related to the sequence number of a response file

Thank You

Presenter:Robert Turner – President at CommSys, Inc.

turner@commsys.com937-425-0402

Coordinator:Kelli Adkins – Marketing Director at CommSys, Inc.

marketing@commsys.com937-425-0411

Q & A

www.commsys.com

800-842-5225

info@commsys.com