Development of the Information Analysis System of the ... · Development of the Information...

1

Development of the

Information Analysis System

of the Ministry of Finance of

Belarus

ASFR organizational and technical structure

DBs

Local area network (LAN)

Server group

Workplaces of MoF specialists

MoF

DBs

DBs DBs

Server group

Server group Server group

Local area network (LAN)

Local area network (LAN) Local area network (LAN)

DBs

DBs

DBs

LAN 100 Mb

LAN 100 Mb

LAN 100 Mb

Workplaces of developers

Workplaces of Oblast Fin. Dept staff Workplaces of the Main Dept staff

Workplaces of the staff of units of Main Directorates

Workplaces of the staff of units of Main Directorate

Workplaces of Rayon Fin. Div. staff Workplaces of Rayon Fin. Div. staff

Data Processing Center (of the MoF)

Oblast Fin. Dept. Main Treas. Dept of MoF for

the oblast

Secure data exchange environment

VPN (Beltelecom) VPN (Beltelecom) VPN (Beltelecom)

Server group

Server group

Server group

Rayon Fin. Division

Units of the MoF Main Directorate for oblast

Units of the MoF Main Directorate for oblast and Rayon Fin. Dept.

Budget organizations

(spending units)

ADSL modem

ADSL modem

HDSL modem HDSL modem

2

Main Objectives of Creating the PTC

(program-technical complex)

to create a data warehouse (DW) providing fast access to

the information generated at different tiers of budget

execution; the DW shall provide reliable data;

to generate reports based on the detailed and summarized

data;

to reduce the time to generate customized reports by

eliminating the programming phase in the report generating

process;

to reduce labor intensity and the time to generate non-

customized reports by replacing the programming phase by

generating reports in the visual design environment;

to provide a consistent and intuitive interface for report

generation.

Solution structure Users get predesigned reports. Analysts can get customized reports

applying the above business terms.

PTC layer

Business intelligence (BI) tools enable users to work with business

terms that are understandable to the users when hiding the backend of the data warehouse model. Generating predesigned reports.

Based on the data, the information of the semantic layer is generated in the data warehouse database.

Cleaned data is uploaded from the staging area directly to the data warehouse database.

The received data is uploaded to the staging area according to the schedule based on the developed loading scenarios. Data cleaning,

aggregating, normalizing.

Source systems layer

After the staff of the unit finishes work, the source systems upload data based on the schedule.

MoF staff (users)

MoF staff (analysts)

MoF telecommunication network

MoF telecommunication network

User layer

BI server

DWH server

ETL server

DB: Analiz_new DB: Common DB: GGK DB: Mtreasury

PTC “Data vault and reporting system”

Business intelligence (BI) tools

ETL

Data warehouse (DW)

Semantic layer

DB Data Warehouse (DWH)

DB Staging Area (SA)

3

Implementation of the PTC solution

“Data warehouse and reporting system”

MoF employee

MOF ASFR Data Center

Internet Explorer (operative)

E-mail:

Task scheduler

DB: Analiz_new

DB: GGK

DB: Common

Data files

DB: MTreasury

Data packets

“Extract Data from Source Systems” SW

FTP server Flat files DB: SA DB: DWH DB: Semantic layer SAP BusinessObjects 4

Teradata 13.0 DBMS

SAP metadata (universe objects,

reports)

Web-based report generating (Intranet)

Sending e-mails in Microsoft Outlook (scheduled)

Access via Internet Explorer (operative)

“Processing of received packets” software system

Task Scheduler 2

Informatica PC



MoF employee



E-mail:

Task scheduler

DB: Analiz_new

DB: GGK

DB: Common

Data files

DB: MTreasury

Data packets



Teradata 13.0 DBMS


reports)





Task Scheduler 2

Informatica PC

4 databases as data source Common - reference data

Mtreasury – Expenditure execution of the republican budget

GGK – Revenue execution of the republican budget; reference data

Analiz_new - reference data

4



MoF employee



E-mail:

Task scheduler

DB: Analiz_new

DB: GGK

DB: Common

Data files

DB: MTreasury

Data packets



Teradata 13.0 DBMS


reports)





Task Scheduler 2

Informatica PC

1. Downloading information from the database “Extract Data from Source Systems” software

116 scripts for data download



MoF employee



E-mail:

Task scheduler

DB: Analiz_new

DB: GGK

DB: Common

Data files

DB: MTreasury

Data packets



Teradata 13.0 DBMS


reports)





Task Scheduler 2

Informatica PC

2. Increments formation “Extract Data from Source Systems” software

5



MoF employee



E-mail:

Task scheduler

DB: Analiz_new

DB: GGK

DB: Common

Data files

DB: MTreasury

Data packets



Teradata 13.0 DBMS


reports)





Task Scheduler 2

Informatica PC

3. Generating data packages and loading them to the FTP server “Extract Data from Source Systems” software



MoF employee



E-mail:

Task scheduler

DB: Analiz_new

DB: GGK

DB: Common

Data files

DB: MTreasury

Data packets



Teradata 13.0 DBMS


reports)





Task Scheduler 2

Informatica PC

4. Preparing packages for loading “Processing Received Packets” software

6



MoF employee



E-mail:

Task scheduler

DB: Analiz_new

DB: GGK

DB: Common

Data files

DB: MTreasury

Data packets



Teradata 13.0 DBMS


reports)





Task Scheduler 2

Informatica PC

5. Uploading data to the data warehouse Informatica PowerCenter (116 scenarios for data uploading into SA

with subsequent uploading to DW (including 1946 SQL-scripts)



MoF employee



E-mail:

Task scheduler

DB: Analiz_new

DB: GGK

DB: Common

Data files

DB: MTreasury

Data packets



Teradata 13.0 DBMS


reports)





Task Scheduler 2

Informatica PC

Data warehouse: Teradata 13.0 314 tables

272 links between tables

302 dampers in the semantic layer

tables with 90 Gb of data

tables with 929 165 095 rows

7



MoF employee



E-mail:

Task scheduler

DB: Analiz_new

DB: GGK

DB: Common

Data files

DB: MTreasury

Data packets



Teradata 13.0 DBMS


reports)





Task Scheduler 2

Informatica PC

6. Report generation SAP BusinessObjects 4

241 representations for generating the Summary Report



MoF employee



E-mail:

Task scheduler

DB: Analiz_new

DB: GGK

DB: Common

Data files

DB: MTreasury

Data packets



Teradata 13.0 DBMS


reports)





Task Scheduler 2

Informatica PC

7. Generating statutory reports SAP BusinessObjects 4

8



MoF employee



E-mail:

Task scheduler

DB: Analiz_new

DB: GGK

DB: Common

Data files

DB: MTreasury

Data packets



Teradata 13.0 DBMS


reports)





Task Scheduler 2

Informatica PC

8. Forwarding reports to users

Implementation Technology

1. Based on the actual report we decide what we extract and

from where 2. Software development for data extraction

3. Development of DW structure

4. Uploading data to DW (cleaning, aggregating)

5. Universe creation

6. Report generating

9

System Software in Use

• DBMS: Teradata v.13.0

• ETL — Informatica PowerCenter

• BI - SAP BusinessObjects 4

Justification of Choice

• Reference: “The best in class”

• Scalability

• Popularity

• Support

10

Selection of the Data Warehouse.

Technical Criteria.

Criterion Requirements Teradata Oracle

Exadata IBM DB2

Large amount of data / Data scalability (Data Scalability)

Supporting concurrent request processing (Query Concurrency)

Ad hoc queries (Query of Freedom)

Data scanning rate (Scan Rate)

Backing-up & restoring features (Backup & Restore)

Data status (Data Freshness)

Usability (Ease of Use)

Excellent (5) Too bad (1) Satisfactory (3) Good (4) Bad (2)

Technical support (Customer Service)

Teradata Advantages

• The database management system (DBMS) has been originally designed as an analytical tool (OLAP)

• Ease of development (if compared to competitors)

• Scalability up to a petabyte (PB)

• Multiple customers in the CIS countries (Russian government agencies, Sberbank (Russia), VTB 24, etc.)

• High level of Russian-language vendor support, including the phase of solutions development

11

Selection of BI

• We analyzed and tested:

– Microstrategy

– Cognos

– Prognoz

– SAP BusinessObjects

– Galaktika-BI

Key Indicators for Comparison

BI function

Preparing object metadata (dimensions and indicators) for reporting

Report debugging

Generating a customized standardized report

Generating a customized non-standardized report

Generating a non-customized standardized report

Generating a non-customized non-standardized report

Data analysis

Real-time data access

Scheduled data access (generation and distribution of reports, data cube generation after data upload)

Working speed

Ease of operation

Availability of documentation

12

Thank you for your attention

Development of the Information Analysis System of the ... · Development of the Information...

Documents

Transcript of Development of the Information Analysis System of the ... · Development of the Information...