Post on 19-May-2019
1
Development of the
Information Analysis System
of the Ministry of Finance of
Belarus
ASFR organizational and technical structure
DBs
Local area network (LAN)
Server group
Workplaces of MoF specialists
MoF
DBs
DBs DBs
Server group
Server group Server group
Local area network (LAN)
Local area network (LAN) Local area network (LAN)
DBs
DBs
DBs
LAN 100 Mb
LAN 100 Mb
LAN 100 Mb
Workplaces of developers
Workplaces of Oblast Fin. Dept staff Workplaces of the Main Dept staff
Workplaces of the staff of units of Main Directorates
Workplaces of the staff of units of Main Directorate
Workplaces of Rayon Fin. Div. staff Workplaces of Rayon Fin. Div. staff
Data Processing Center (of the MoF)
Oblast Fin. Dept. Main Treas. Dept of MoF for
the oblast
Secure data exchange environment
VPN (Beltelecom) VPN (Beltelecom) VPN (Beltelecom)
Server group
Server group
Server group
Rayon Fin. Division
Units of the MoF Main Directorate for oblast
Units of the MoF Main Directorate for oblast and Rayon Fin. Dept.
Budget organizations
(spending units)
ADSL modem
ADSL modem
HDSL modem HDSL modem
2
Main Objectives of Creating the PTC
(program-technical complex)
to create a data warehouse (DW) providing fast access to
the information generated at different tiers of budget
execution; the DW shall provide reliable data;
to generate reports based on the detailed and summarized
data;
to reduce the time to generate customized reports by
eliminating the programming phase in the report generating
process;
to reduce labor intensity and the time to generate non-
customized reports by replacing the programming phase by
generating reports in the visual design environment;
to provide a consistent and intuitive interface for report
generation.
Solution structure Users get predesigned reports. Analysts can get customized reports
applying the above business terms.
PTC layer
Business intelligence (BI) tools enable users to work with business
terms that are understandable to the users when hiding the backend of the data warehouse model. Generating predesigned reports.
Based on the data, the information of the semantic layer is generated in the data warehouse database.
Cleaned data is uploaded from the staging area directly to the data warehouse database.
The received data is uploaded to the staging area according to the schedule based on the developed loading scenarios. Data cleaning,
aggregating, normalizing.
Source systems layer
After the staff of the unit finishes work, the source systems upload data based on the schedule.
MoF staff (users)
MoF staff (analysts)
MoF telecommunication network
MoF telecommunication network
User layer
BI server
DWH server
ETL server
DB: Analiz_new DB: Common DB: GGK DB: Mtreasury
PTC “Data vault and reporting system”
Business intelligence (BI) tools
ETL
Data warehouse (DW)
Semantic layer
DB Data Warehouse (DWH)
DB Staging Area (SA)
3
Implementation of the PTC solution
“Data warehouse and reporting system”
MoF employee
MOF ASFR Data Center
Internet Explorer (operative)
E-mail:
Task scheduler
DB: Analiz_new
DB: GGK
DB: Common
Data files
DB: MTreasury
Data packets
“Extract Data from Source Systems” SW
FTP server Flat files DB: SA DB: DWH DB: Semantic layer SAP BusinessObjects 4
Teradata 13.0 DBMS
SAP metadata (universe objects,
reports)
Web-based report generating (Intranet)
Sending e-mails in Microsoft Outlook (scheduled)
Access via Internet Explorer (operative)
“Processing of received packets” software system
Task Scheduler 2
Informatica PC
Implementation of the PTC solution
“Data warehouse and reporting system”
MoF employee
MOF ASFR Data Center
Internet Explorer (operative)
E-mail:
Task scheduler
DB: Analiz_new
DB: GGK
DB: Common
Data files
DB: MTreasury
Data packets
“Extract Data from Source Systems” SW
FTP server Flat files DB: SA DB: DWH DB: Semantic layer SAP BusinessObjects 4
Teradata 13.0 DBMS
SAP metadata (universe objects,
reports)
Web-based report generating (Intranet)
Sending e-mails in Microsoft Outlook (scheduled)
Access via Internet Explorer (operative)
“Processing of received packets” software system
Task Scheduler 2
Informatica PC
4 databases as data source Common - reference data
Mtreasury – Expenditure execution of the republican budget
GGK – Revenue execution of the republican budget; reference data
Analiz_new - reference data
4
Implementation of the PTC solution
“Data warehouse and reporting system”
MoF employee
MOF ASFR Data Center
Internet Explorer (operative)
E-mail:
Task scheduler
DB: Analiz_new
DB: GGK
DB: Common
Data files
DB: MTreasury
Data packets
“Extract Data from Source Systems” SW
FTP server Flat files DB: SA DB: DWH DB: Semantic layer SAP BusinessObjects 4
Teradata 13.0 DBMS
SAP metadata (universe objects,
reports)
Web-based report generating (Intranet)
Sending e-mails in Microsoft Outlook (scheduled)
Access via Internet Explorer (operative)
“Processing of received packets” software system
Task Scheduler 2
Informatica PC
1. Downloading information from the database “Extract Data from Source Systems” software
116 scripts for data download
Implementation of the PTC solution
“Data warehouse and reporting system”
MoF employee
MOF ASFR Data Center
Internet Explorer (operative)
E-mail:
Task scheduler
DB: Analiz_new
DB: GGK
DB: Common
Data files
DB: MTreasury
Data packets
“Extract Data from Source Systems” SW
FTP server Flat files DB: SA DB: DWH DB: Semantic layer SAP BusinessObjects 4
Teradata 13.0 DBMS
SAP metadata (universe objects,
reports)
Web-based report generating (Intranet)
Sending e-mails in Microsoft Outlook (scheduled)
Access via Internet Explorer (operative)
“Processing of received packets” software system
Task Scheduler 2
Informatica PC
2. Increments formation “Extract Data from Source Systems” software
5
Implementation of the PTC solution
“Data warehouse and reporting system”
MoF employee
MOF ASFR Data Center
Internet Explorer (operative)
E-mail:
Task scheduler
DB: Analiz_new
DB: GGK
DB: Common
Data files
DB: MTreasury
Data packets
“Extract Data from Source Systems” SW
FTP server Flat files DB: SA DB: DWH DB: Semantic layer SAP BusinessObjects 4
Teradata 13.0 DBMS
SAP metadata (universe objects,
reports)
Web-based report generating (Intranet)
Sending e-mails in Microsoft Outlook (scheduled)
Access via Internet Explorer (operative)
“Processing of received packets” software system
Task Scheduler 2
Informatica PC
3. Generating data packages and loading them to the FTP server “Extract Data from Source Systems” software
Implementation of the PTC solution
“Data warehouse and reporting system”
MoF employee
MOF ASFR Data Center
Internet Explorer (operative)
E-mail:
Task scheduler
DB: Analiz_new
DB: GGK
DB: Common
Data files
DB: MTreasury
Data packets
“Extract Data from Source Systems” SW
FTP server Flat files DB: SA DB: DWH DB: Semantic layer SAP BusinessObjects 4
Teradata 13.0 DBMS
SAP metadata (universe objects,
reports)
Web-based report generating (Intranet)
Sending e-mails in Microsoft Outlook (scheduled)
Access via Internet Explorer (operative)
“Processing of received packets” software system
Task Scheduler 2
Informatica PC
4. Preparing packages for loading “Processing Received Packets” software
6
Implementation of the PTC solution
“Data warehouse and reporting system”
MoF employee
MOF ASFR Data Center
Internet Explorer (operative)
E-mail:
Task scheduler
DB: Analiz_new
DB: GGK
DB: Common
Data files
DB: MTreasury
Data packets
“Extract Data from Source Systems” SW
FTP server Flat files DB: SA DB: DWH DB: Semantic layer SAP BusinessObjects 4
Teradata 13.0 DBMS
SAP metadata (universe objects,
reports)
Web-based report generating (Intranet)
Sending e-mails in Microsoft Outlook (scheduled)
Access via Internet Explorer (operative)
“Processing of received packets” software system
Task Scheduler 2
Informatica PC
5. Uploading data to the data warehouse Informatica PowerCenter (116 scenarios for data uploading into SA
with subsequent uploading to DW (including 1946 SQL-scripts)
Implementation of the PTC solution
“Data warehouse and reporting system”
MoF employee
MOF ASFR Data Center
Internet Explorer (operative)
E-mail:
Task scheduler
DB: Analiz_new
DB: GGK
DB: Common
Data files
DB: MTreasury
Data packets
“Extract Data from Source Systems” SW
FTP server Flat files DB: SA DB: DWH DB: Semantic layer SAP BusinessObjects 4
Teradata 13.0 DBMS
SAP metadata (universe objects,
reports)
Web-based report generating (Intranet)
Sending e-mails in Microsoft Outlook (scheduled)
Access via Internet Explorer (operative)
“Processing of received packets” software system
Task Scheduler 2
Informatica PC
Data warehouse: Teradata 13.0 314 tables
272 links between tables
302 dampers in the semantic layer
tables with 90 Gb of data
tables with 929 165 095 rows
7
Implementation of the PTC solution
“Data warehouse and reporting system”
MoF employee
MOF ASFR Data Center
Internet Explorer (operative)
E-mail:
Task scheduler
DB: Analiz_new
DB: GGK
DB: Common
Data files
DB: MTreasury
Data packets
“Extract Data from Source Systems” SW
FTP server Flat files DB: SA DB: DWH DB: Semantic layer SAP BusinessObjects 4
Teradata 13.0 DBMS
SAP metadata (universe objects,
reports)
Web-based report generating (Intranet)
Sending e-mails in Microsoft Outlook (scheduled)
Access via Internet Explorer (operative)
“Processing of received packets” software system
Task Scheduler 2
Informatica PC
6. Report generation SAP BusinessObjects 4
241 representations for generating the Summary Report
Implementation of the PTC solution
“Data warehouse and reporting system”
MoF employee
MOF ASFR Data Center
Internet Explorer (operative)
E-mail:
Task scheduler
DB: Analiz_new
DB: GGK
DB: Common
Data files
DB: MTreasury
Data packets
“Extract Data from Source Systems” SW
FTP server Flat files DB: SA DB: DWH DB: Semantic layer SAP BusinessObjects 4
Teradata 13.0 DBMS
SAP metadata (universe objects,
reports)
Web-based report generating (Intranet)
Sending e-mails in Microsoft Outlook (scheduled)
Access via Internet Explorer (operative)
“Processing of received packets” software system
Task Scheduler 2
Informatica PC
7. Generating statutory reports SAP BusinessObjects 4
8
Implementation of the PTC solution
“Data warehouse and reporting system”
MoF employee
MOF ASFR Data Center
Internet Explorer (operative)
E-mail:
Task scheduler
DB: Analiz_new
DB: GGK
DB: Common
Data files
DB: MTreasury
Data packets
“Extract Data from Source Systems” SW
FTP server Flat files DB: SA DB: DWH DB: Semantic layer SAP BusinessObjects 4
Teradata 13.0 DBMS
SAP metadata (universe objects,
reports)
Web-based report generating (Intranet)
Sending e-mails in Microsoft Outlook (scheduled)
Access via Internet Explorer (operative)
“Processing of received packets” software system
Task Scheduler 2
Informatica PC
8. Forwarding reports to users
Implementation Technology
1. Based on the actual report we decide what we extract and
from where 2. Software development for data extraction
3. Development of DW structure
4. Uploading data to DW (cleaning, aggregating)
5. Universe creation
6. Report generating
9
System Software in Use
• DBMS: Teradata v.13.0
• ETL — Informatica PowerCenter
• BI - SAP BusinessObjects 4
Justification of Choice
• Reference: “The best in class”
• Scalability
• Popularity
• Support
10
Selection of the Data Warehouse.
Technical Criteria.
Criterion Requirements Teradata Oracle
Exadata IBM DB2
Large amount of data / Data scalability (Data Scalability)
Supporting concurrent request processing (Query Concurrency)
Ad hoc queries (Query of Freedom)
Data scanning rate (Scan Rate)
Backing-up & restoring features (Backup & Restore)
Data status (Data Freshness)
Usability (Ease of Use)
Excellent (5) Too bad (1) Satisfactory (3) Good (4) Bad (2)
Technical support (Customer Service)
Teradata Advantages
• The database management system (DBMS) has been originally designed as an analytical tool (OLAP)
• Ease of development (if compared to competitors)
• Scalability up to a petabyte (PB)
• Multiple customers in the CIS countries (Russian government agencies, Sberbank (Russia), VTB 24, etc.)
• High level of Russian-language vendor support, including the phase of solutions development
11
Selection of BI
• We analyzed and tested:
– Microstrategy
– Cognos
– Prognoz
– SAP BusinessObjects
– Galaktika-BI
Key Indicators for Comparison
BI function
Preparing object metadata (dimensions and indicators) for reporting
Report debugging
Generating a customized standardized report
Generating a customized non-standardized report
Generating a non-customized standardized report
Generating a non-customized non-standardized report
Data analysis
Real-time data access
Scheduled data access (generation and distribution of reports, data cube generation after data upload)
Working speed
Ease of operation
Availability of documentation
12
Thank you for your attention