Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.
-
Upload
samuel-dorsey -
Category
Documents
-
view
215 -
download
0
Transcript of Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.
![Page 1: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/1.jpg)
Asuri Saranathan
![Page 2: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/2.jpg)
Agenda
Introduction Best Practices – Over View Deep Dive Conclusion Q & A
![Page 3: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/3.jpg)
Introduction
![Page 4: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/4.jpg)
Speaker
Holds Bachelor degree in Physics and Electrical and Electronics Engineering.
Over 26 years of Experience in Power System and Information Technology field.
Has built several large scale applications including online CRM for multinationals.
Has managed several Data Warehousing and BI projects for Direct Marketing, Manufacturing and Auto finance verticals.
Functioned as Solution Architect for Data warehousing and reporting projects.
ISO auditor Certified Bullet Proof Manager from CrestComm USA.
![Page 5: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/5.jpg)
Best Practices- Overview
![Page 6: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/6.jpg)
What are Best Practices?
Is it a technology? Is it application of a set of best tools
available in the market? Is it a Framework?
![Page 7: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/7.jpg)
Best Practices Definition A framework of a set of processes
or method that exhibits achievement of specific results in a specific manner over a sustained period of time.
The framework should have certain characteristics in that they should be repeatable.
![Page 8: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/8.jpg)
Do Best Practices Evolve? Yes they do.
Because of Innovation Changes in Technology Changes in Law or Governance
Structure. Expectations, Values , Knowledge or
other that makes the practice outdated or inappropriate.
![Page 9: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/9.jpg)
Where can it be Applied?
Practically in all fields.
![Page 10: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/10.jpg)
How do we apply Best Practices to Data Warehousing and Business Intelligence?
![Page 11: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/11.jpg)
Data Warehouse - Definition In an elementary form , it is the
collection of key information that can be used by the business users to become more profitable.
But Is this definition sufficient ? We need much more precise
definition of what a data ware house is .
![Page 12: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/12.jpg)
What is a Data Warehouse? A Data warehouse is the Data ( Meta / Fact / Dimension/ Aggregation)
and The Process Managers ( Load / Warehouse /
Query)
That make information available , enabling the user to make informed decisions.
![Page 13: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/13.jpg)
Deep Dive
![Page 14: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/14.jpg)
What is the Challenge?
Business is never Static. And so is Data Warehouse.
In order to respond to today’s requirement for instant access to corporate information , the data warehouse should be designed to respond to this need in a optimal way.
Business itself probably not aware of what information is required in the future. This requires a fundamentally different
approach than the traditional waterfall method of software development for the Data warehouse.
![Page 15: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/15.jpg)
Experience so far…
Most Enterprise Data Warehousing projects tend to have development cycle of between 18 – 24 months from start to finish.
Justification of this investment is substantial. Businesses would prefer a better approach to
justify the investment.
![Page 16: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/16.jpg)
What should be done?
Focus on Business Requirements A clear understanding of what is short
term and long term requirement of the data warehouse.
An Architecture design that would evolve.
Identification of quick win that delivers business benefit in the first build.
![Page 17: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/17.jpg)
Scalability for Growth
Scalability means ability of the underlying Hardware and Software to support increasing demands over a period of time.
![Page 18: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/18.jpg)
Horizontal Scalability
CPU
CPU
CPU
CPU
RAM
RAM
RAM
RAM
DB DB DB DB
CPU
CPU
CPU
CPU
RAM
RAM
RAM
RAM
DBDBDB DB
High Speed Network
Multiple servers are connected thru a network and use the data partitioning feature of the Database to tie the CPUs together.
![Page 19: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/19.jpg)
Data Warehouse Environment
Staging Area
Data warehouse(System of Record)
Full History in 3rd Normal Form
No User Access
Summary Area Full HistoryUser Access
Analytical Area
User Access
Source Systems
Data Mart
![Page 20: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/20.jpg)
Data Governance
Metadata Mgmt•Architecture•Integration•Control•Delivery
Data Architecture•Entp. DM
• Value Chain
Data Quality• Spec•Analysis
•Measurement•Improvement
Document / Content Mgmt
•Acquisition & Storage•Backup & Recovery
•Content •Retrieval•Retention
DWH / BI•Architecture•Implementatio
n•Training and
Support•Tuning
Data Security•Standards
•Classification•Administration•Authentication
•Auditing
Reference and MDM
•External & Internal Code•Customer Data•Product data
Data Operations•Acquisition•Recovery•Tuning•Purging
Data Development•Analysis
•Data Modeling•DB Design
•Implementation
Strategy
![Page 21: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/21.jpg)
Environment
Goals & Objective
s
Technology Activities
Organization & Culture
Roles & Responsibilitie
s
Practices & Techniques
Deliverables
![Page 22: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/22.jpg)
Architecture Requirements
Must recognize change as a constant Take incremental development approach Existing applications must continue to work Need to allow more data and new types of
data to be added
![Page 23: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/23.jpg)
High Level
Remember the different “worlds” On-line transaction processing (OLTP) Business intelligence systems (BIS)
Users are different Data content is different Data structures are different Architecture & methodology must be
different
![Page 24: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/24.jpg)
April 18, 2023DW Architecture Best Practices 24
Best Practice #1
Use a Data model that is optimized for Information retrieval dimensional model denormalized hybrid approach
![Page 25: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/25.jpg)
April 18, 2023DW Architecture Best Practices 25
Best Practice #2
Carefully design the data acquisition and cleansing processes for your DW Ensure the data is processed efficiently
and accurately Consider acquiring ETL and Data
Cleansing tools Use them well!
![Page 26: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/26.jpg)
April 18, 2023DW Architecture Best Practices 26
Best Practice #3
Design a metadata architecture that allows sharing of metadata between components of your DW consider metadata standards such as
OMG’s Common Warehouse Metamodel (CWM)
![Page 27: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/27.jpg)
April 18, 2023DW Architecture Best Practices 27
Best Practice #4
Take an approach that consolidates data into ‘a single version of the truth’ Data Warehouse Bus
conformed dimensions & facts OR?
![Page 28: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/28.jpg)
April 18, 2023DW Architecture Best Practices 28
Best Practice #5
Consider implementing an ODS only when information retrieval requirements are near the bottom of the data abstraction pyramid and/or when there are multiple operational sources that need to be accessed Must ensure that the data model is
integrated, not just consolidated May consider 3NF data model Avoid at all costs a ‘data dumping ground’
![Page 29: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/29.jpg)
Pitfalls to be Avoided
Engagement of Non-BI Manger in a BI delivery Project. Trying to please the client and the user community. Expecting the Service Provider to own the Project
completely. Bringing the Solution Architect half way into the project. Allowing the Business Users to drive the Data Model. Not having the right people with right skills in tool selection
process. Expecting the contractor to deliver all that they presented. Over dependency on the Service provider or contractor in
execution. Assuming that the Data quality will be handled somehow. Assuming that the Data warehouse project is over once it is
deployed.
![Page 30: Asuri Saranathan. Agenda Introduction Best Practices – Over View Deep Dive Conclusion Q & A.](https://reader035.fdocuments.in/reader035/viewer/2022081519/56649d215503460f949f5cc0/html5/thumbnails/30.jpg)
Data Warehouse Data Warehouse Architecture Best PracticesArchitecture Best Practices
Thank You