Etl Application

7/31/2019 Etl Application

1/65

ETL APPLICATION

A.SRADHANAND

DEVELOPED

BY

M.Sc. (Comp. Sc.)

4602-525


2/65

CONTENTS

ABSTRACT

SYSTEM ANALYSIS

SYSTEM DESIGN

SCREENS

TESTING

IMPLEMENTATIONFEATURES

CONCLUSION

BIBLIOGRAPHY

INTRODUCTION

ORGANIZATION PROFILE


3/65

INTRODUCTION

Now a days there are so many databases are availablelike ORACLE, SQL SERVER, MS-ACCESS, DBASE etc.,

Some organizations are using some type of database

and some are using another.

The different databases are provides different

types of securities. Some provides high security

(ORACLE, SQL SERVER), some provides low security

(MS-ACCESS etc.,).


4/65

ABSTRACT

The project entitles Extractor Transformer Loader

deals with automation of all departments. Presently the

transactions at Head of the department of SPIL

Technologies are carried in different database applications

(ORACLE, MS-ACCESS, DBASE).

Departments that play a major role at SPIL

Technologies are:

Extractor

Transformer

Loader


5/65

In simple, using ETL Application any developer with

minimal knowledge can easily do the following things

Connect to multiple databases easily

Create databases online

Modify databases online

Execute SQL statements online Extract the database, Transform the required

database and load that database into destination

user


6/65

ORGANIZATION PROFILE

SPIL Software Limited provides a wide range of value-added

Information Technology (IT) services. We specialize in

providing Contract Programming services, On Site Software

Development services and Permanent Staffing in the

following areas:

Internet / Web

Client / Server

Mainframe / Middleware

UNIX / Databases Telecommunication / Networking And other

platforms


7/65

EXISTING SYSTEM

In the source and destination different databases

are present. In that department (at destination), the data

are collected from the different databases manually from

different departments and later are fed into differentdatabases respectively. From the corresponding databases

create reports and make hardcopies. These processes are

consuming more time.


8/65

PROPOSED SYSTEM

To avoid this problem, they are proposed todevelop one application in such a way that

It connect the databases automatically according

to user requirements, converting the source database intodestination database (ORACLE) and creates softcopies

in destination user.


9/65

FEASIBILITY ANALYSIS

In this, I analyzing the feasibility according to

the following points

Operational Feasibility

Technical Feasibility

Economical Feasibility


10/65

Data stores in their respective database

Generating the soft copies Transformation is completed in less amount of

time

System will not slow down the performance in any

area

Operational Feasibility


11/65

Technical Feasibility

Any no. of users can use the system in any time

There are technical guarantees like accuracy,

reliability and data security


12/65

Economical Feasibility

The proposed system is developed using

Java2.0(JDK) Software, which is available free of costat www.java.sun.com.
http://www.java.sun.com/http://www.java.sun.com/http://www.java.sun.com/http://www.java.sun.com/http://www.java.sun.com/http://www.java.sun.com/http://www.java.sun.com/http://www.java.sun.com/


13/65

ENVIRONMENT SPECIFICATION

Hardware:

Processor : Pentium III

Clock Speed : 500 MHz

RAM : 64 MB

Hard Disk : 4.3 GB


14/65

Software:

Operating System : Windows NT/Windows 98/Windows 95/ XP

Back End : 1.Source : DBASE, MS-Access, Oracle

2.Destination : OracleCompiler :Javac

Language :Java

GUI :Swing

Data Connectivity :JDBCODBC Driver


15/65

SYSTEM DESIGN

Tables

Data Flow Diagrams

Flow Charts


16/65

TABLES

Table name: Branch_info

Field Name Data Type Description

BRANCH_CODE VARCHAR2 Branch codeBRANCH_NAME VARCHAR2 Branch name

ADD1 VARCHAR2 Address1

ADD2 VARCHAR2 Address2

CITY VARCHAR2 City name

STATE VARCHAR2 State

PINCODE NUMBER Pin code numberPHONE NUMBER Phone number

FAX NUMBER Fax number

EMAIL VARCHAR2 Email id


17/65

Table name: Branch_db


BRANCH_CODE VARCHAR2 Branch code

DSN_NAME VARCHAR2 Data Source Name

DB_TYPE VARCHAR2 Database Type

DB_NAME VARCHAR2 Table names

Table name: Zonal_db


D_BRANCH_CODE VARCHAR2 Branch code

D_DSN_NAME VARCHAR2 Data Source Name

D_DB_TYPE VARCHAR2 Database Type

D_DB_NAME VARCHAR2 Table names

NO_OF_FIELDS NUMBER Number of Fields in Table

NO_OF_RECORDS NUMBER Number of Records


18/65

DATA FLOW DIAGRAMS


19/65


20/65

FIRST LEVEL DFD:

1) Request Branch for Retrieving/removing/adding

Branch information

2) Retrieving/removing/adding Branch information

3) Retrieving/removing/adding Database information

4) Receiving respected manipulation information

5) Request ETL Wizard for transformation

6) Retrieving the Branch information7) Retrieving the Branch database

8) Retrieving/storing branch and table information after

transformation/loader is occurring.

9) Storing structure and records of transformed database

10) Receiving ETL information11) Request Help for more information about application

12) Received required response

13) Request to close application

14) Response for closing application


21/65


22/65

SECOND LEVEL DFD1 :

1) Requesting for add branch

2) Adding branch information

3) Adding branch database

4) Response for add branch

5) Requesting for remove branch

6) Removing requested branch information

7) Removing requested branch database8) Response for remove branch

9) Requesting for view branch

10) Retrieving requested branch information

11) Retrieving requested branch database12) Response for view branch

13) Requesting for view all branches

14) Retrieving all branches information

15) Retrieving all branches database

16) Response for view all branches


23/65


24/65

SECOND LEVEL DFD2 :

1) Requesting for ETL Wizard2) Extracting the branch information

3) Extracting the branch database

4) Extracting information passed to transformer

5) Storing branch and table information6) Storing table structure and records

7) Informing loader that transform is completed

and invoking the load module

8) Retrieving table records

9) Shows the request response to user


25/65


26/65

SECOND LEVEL DFD3 :

1) Requesting author information

2) Displays author information

3) Requesting shortcut information

4) Displays shortcut information


27/65

FLOW CHARTS


28/65


29/65


30/65


31/65


32/65


33/65


34/65


35/65


36/65


37/65


38/65


39/65

FUNCTIONAL DESCRIPTOIN

WITH SCREENS

General Functions:

Branch

ETL Wizard Queries

Help

Exit


40/65

The various activities of the ETL Application are dealt as

different functions in the system. The main function is

performing ETL Wizard.


41/65

I) Branch:This function is related to branches manipulation.

In this function (Menu) we are having again different

functionalities (Menu items). They are

1)Add Branch

2)Remove Branch

3)View Branch

4)View Branches


42/65


43/65

a. Add Branch:

For the department if any new branch is added then

by using this function we can add the database of new branch

into their respective databases.


44/65

b. Remove Branch:

If any branch is removed from the department then by

using the function Remove Branch we can remove the

particular branch database.


45/65

c. View Branch:

If any body (department related persons) wants to see

the particular database of the branch then we can see the

database of the particular branch by using the function ViewBranch.


46/65

d. View All Branches:

If any body (department related persons) wants to see the all

databases of the branches then we can see the databases of

the all branches by using the function View All Branches.


47/65

II) ETL Wizard

1) Start Wizard:

Whenever we are choosing this function, it will display

the screen called Select Branch, in which user has to

choose the branch code and depending on the branch code

it will automatically displays the branch name, DSN name,

all databases which are in the particular branch and type

of the database. After choosing the branch code user has to

select the database then click on Submit then it will

displays the screen called Transaction Center. If userselecting or clicking the button called Abort then the

process is aborted and goes to the main menu.


48/65


49/65

In the Transaction Center, it will display the

database structure and the related database recordssimultaneously. In this there are two buttons are there. They

are Transformation required and Transformation not

required. If the user clicking on the button Transformation

required then it will transform the data of selecting databaseinto the destination database (Oracle) and display the message

Transaction is completed. If the user selecting or clicking the

button Transaction is not required then it will aborting the

process and goes to the main menu.


50/65


51/65

After transformation is completed then another screen is

displays. The screen name is Loading. Through this screen

user can store the transformed data into destination user

(Oracle user) and also it will show the transformed data. In this

also there are two buttons. If we clicking on the View then the

database is loading into the destination user and shows theTransformation Loading records on different screen. Whenever

user is closing the widow it will automatically goes to the main

menu. If we select Abort then it will abort the process and

goes to the main menu.


52/65


53/65

III) .Queries:

Whenever user wants to create a query then he can

choose the Queries. In this there is one function is available.That function (Menu item) is Query.

Whenever the user choosing or selecting the Query then

it will display the separate window called Query Form.Through this screen user can check the databases that are in the

destination with the attribute names and related query response

information at a time.

Whenever the user is closing the window then it will

automatically goes to the main menu.


54/65


55/65

IV. Help:

In this function (Menu) we are having two different

functions (Menu item). They are

1) Shortcuts

2) About Me


56/65

a. Shortcuts:

This will display the shortcuts for different menu items,

which are using in this application.


57/65

b. About Me:

This will display the programmer and designer name


58/65

V. Exit:

In this function (Menu) we are having one function

(Menu item) called Exit. Whenever we are choosing thisfunction the application is closed with the message called

Thankyou to using this Application.


59/65

TESTING OF SYSTEM DEVELOPED FOR

SPIL TECHONOLOGIES

The system developed for SPIL Technologies was

thoroughly tested for any errors all through the development

phases. First, after the development of each screen the relevant

test data is used as input to test the screen. This way all the

screens were tested and at the same time cross transactions

(data transformation without losing any data or not) and also

checked for valid postings.

After the screen wise testing is completed the system istested with some sample data as a finally run of the free (the test

data is used as input is enclose in the report format).

Finally, the system was cleared for implementation.


60/65

Implementations of Developed System at SPIL

Technologies

Present system was developed for and at SPIL

Technologies, Ameerpet, Hyderabad where the actual

implementation of the system is performed. In the view above,

the system developed is delivered to SPIL Technologies, after

thoroughly testing with the sample data. The system is checkedfor coding errors, runtime errors, data validations and postings

The equipment installation and hardware and software

check out are properly taken care. As SPIL Technologies is

already using the computers no new equipment need be

installed. Only the system developed has to be installed on the

available equipment. The client already fulfills the hardware

and software requirements by the time the order placement was

finalized.


61/65

The end users, management and user personnel who will

be directly or indirectly involved in the usage of the system at

SPIL Technologies are trained for a short duration whichconcludes the implementation.


62/65

FEATURES:

Manipulating the branches

Conversion wizard

Shortcuts

Saving Output results

Multiple database connections

Displays database schema and its tuples


63/65

CONCLUSION

The main aim of this project is to automaticallyextracting source databases (Oracle, MS-Access, SQL Server)

and transformed those databases to Oracle database and

loading into destination user.

I am in the process of enhancing this project to

generalization. i.e., Converting any source database to any other

destination database.


64/65

BIBLIOGRAPHY

1. Modern System Analysis And Design : Jeffrey A. Hoffer

2. Software Engineering Concepts : Richard Fairley

3. Complete Reference Java 2.0 : Patrick Naughton

4. Thinking in Java : Bruce Eckel


65/65

THANK YOU

Etl Application

Documents

Transcript of Etl Application