Defining Data Warehouse Concepts and Terminology

Chapter 3

Definition of a Data Warehouse

“ An enterprise structured repository of subject-oriented, time-variant, historical data used for information retrieval and decision support. The data warehouse stores atomic and summary data.”

Oracle Data Warehouse Method

Data Warehouse Properties

DataWarehouse

Integrated

Time VariantNon Volatile

SubjectOriented

Subject-Oriented

Data is categorized and stored by business subjectrather than by application

EquityPlans Shares Customer

financialinformation

Savings

Insurance

Loans

OLTP Applications Data Warehouse Subject

Integrated

OLTP Applications

SavingsSavings

Currentaccounts

Currentaccounts

LoansLoans

Data Warehouse

Data on a given subject is defined and stored once.

Customer

Time-Variant

Data is stored as a series of snapshots, each representing a period of time

Time DataJan-97 JanuaryFeb-97 FebruaryMar-97 March

Nonvolatile

Typically data in the data warehouse is not updated or delelted.

Insert UpdateDelete

Read Read

Operational Warehouse

Load

Changing Data

Warehouse Database

First time load

Refresh

Refresh

Refresh

Operational Database

Data Warehouse Versus OLTP

PropertyResponseTime

Operations

Nature of Data

Data Organization

Size

Data Source

Activities

OperationalSub seconds to seconds

DML

30-60 days

Applications

Small to large

Operational, Internal

Processes

Data Warehouse

Seconds to hours

Snapshots over time

Subject, time

Large to very large

Operational, Internal,External

Analysis

Primarily read only

Usage Curves

Operational system is predictableData warehouse - Variable - Random

User Expectations

Control expectationsSet achievable targets for query

responseSet SLAsEducateGrowth and use is exponential

Enterprisewide Warehouse

Large scale implementationScope the entire businessData from all subject areasDeveloped incrementallySingle source of enterprisewide dataSingle distribution point to

dependent data marts

Data Warehouses Versus Data Marts

Property Data Warehouse Data MartScope Enterprise DepartmentSubject Multiple Single-subject, LOBData Source Many FewSize(typical) 100 GB to>1 TB <100 GBImplementation time Months to years Months

DataWarehouse

DataWarehouse

DataMartDataMart

Dependent Data Mart

MarketingSales

FinanceHuman Resources

MarketingSales

FinanceHuman Resources

MarketingMarketing

MarketingMarketing

MarketingMarketing

External Data

DataWarehouse

OperationalSystems

Flat Files

Data Marts

Independent Data Mart

OperationalSystems

External Data

Sale or Marketing

Flat Files

Data Warehouse TerminologyOperational data store (ODS) Stores tactical data from production

systems that are subject-oriented and integrated to address operational needs

Metadata

MetadataMetadata

Data Warehouse Terminology

DataIntegration

Enterprise data warehouse

Business areawarehouse

Source data

Architecture

Methodolgy

Ensures a successful data warehouseEncourages incremental developmentProvides a staged approach to an

enterprisewide warehouse - Safe - Manageable - Proven - Recommended

Modeling

Warehouses differ from operational structures: - Analytical requirements - Subject orientationData must map to subject oriented information: - Identify business subjects - Define relationships between subjects - Name the attributes of each subjectModeling is iterativeModeling tools are available

Extraction, Transformation, and Transportation

Purchase specialist tools, or develop programsExtraction-- select data using different methodsTransformation--validate, clean, integrate, and

time stamp dataTransportation--move data into the warehouse

OLTP Databases Staging File Warehouse Database

Data Management

Efficient database server and management tools for all aspects of data management

Imperatives - Productive - Flexible - Robust - EfficientHardware, operating system and network

management

Data Access and Reporting

Tools that retrieve data for business analysis Imperatives - Ease of use - Intuitive - Metadata - Training More than one tool may be required

WarehouseDatabase

Simple Queries

Forecasting

Drill-down

Oracle Warehouse Components

Relational / Multidimensional

Text, image Spatial

Web Audio video

Externaldata

Operational data

Relational tools

OLAP tools

Applications/Web

Any DataAny Source Any Access

Oracle Data Mart Suite

Data ModelingOracle Data Mart Designer

OLTPEngines

OLTPDatabases

DataExtraction

Oracle Data MartBuilder

Ware-housingEngines

Data MartDatabase

SQL*Plus

DataManagement

Oracle EnterpriseManager

Data Access& Analysis

Discoverer &Oracle Reports

Data Mart Implementation with the Oracle Data Mart Suite

Oracle Enterprise ServerOracle Enterprise ManagerOracle Data Mart BuilderOracle Data Mart DesignerOracle DiscovererOracle Web Application ServerOracle Reports

Oracle Warehouse Builder Architecture

Sources

ExtractionFacilities• Loader• Remotes SQL• Gateways - OLE-DB/ODBC - Mainframe - Specialized• ERP Data - SAP - Peoplesoft - Oracle

PL/SQL, JavaTransforms

TransformDriver

PL/SQL, JavaWrapper

ExternalFunctions

TargetTablesFilter

Transform

Oracle 8i

Oracle Business Intelligence Tools

Current Tactical Strategic

IS developsuser’s Views Business users Analysis

Oracle Reports Oracle Discover Oracle Express

The Tool for Each Task

Tool

OracleReports

OracleDiscover

OracleExpress

Productionreporting

Ad hocquery and analysis

Advancedanalysis

Question

What were sales byregion last quarter?

What is driving the increase in NorthAmerican sales?

Given the rapid increasein Web sales, what willtotal sales be for the restof the year?

Task

Oracle Warehouse Services

OracleEducation

OracleConsulting

Oracle Support Services

Customers

SummaryThis lesson covered the following topics:Identifying a common, broadly accepted definition

of the data warehouseDistinguishing the differences between OLTP

systems and analytical systemsDefining some of the common data warehouse

terminologyIdentifying some of the elements and processes in

a data warehouseIdentifying and positioning the Oracle Warehouse

vision, products, and services