INFORMATION WAREHOUSE IN THE RETAIL INDUSTRY

INFORMATION WAREHOUSE INTHE RETAIL INDUSTRY

Document Number GG24-4342-00

August 1994

International Technical Support OrganizationSan Jose

Take Note!

Before using this information and the product it supports, be sure to readthe general information under “Special Notices” on page xiii.

First Edition (August 1994)

This edition applies to the following products:

• DataPropagator Relational Version 1 Release 1, Program Number5622-244

• DataHub/2 Version 1 Release 1, Program Number 5667-134• DataGuide/2 Version 1 Release 1, Program Numbers 5622-487 and

5622-488• FlowMark Version 1 Release 2, Program Number 5621-290• DataPropagator NonRelational Version 1 Release 2, Program Number

5696-705• DataRefresher Version 3 Release 1, Program Number 5696-703• Visualizer Query Version 1 Release 0, Program Number 5871-BBB• S/390 Parallel Query Server

Order publications through your IBM representative or the IBM branch officeserving your locality. Publications are not stocked at the address givenbelow.

An ITSO Technical Bulletin Evaluation Form for readers′ feedback appearsfacing Chapter 1. If the form has been removed, comments may beaddressed to:

IBM Corporation, International Technical Support OrganizationDept. 471, Building 070B5600 Cottle RoadSan Jose, California 95193-0001

When you send information to IBM, you grant IBM a non-exclusive right touse or distribute the information in any way it believes appropriate withoutincurring any obligation to you.

Copyright International Business Machines Corporation 1994. All rightsreserved.Note to U.S. Government Users — Documentation related to restricted rights— Use, duplication or disclosure is subject to restrictions set forth in GSAADP Schedule Contract with IBM Corp.

Abstract

This publication is one of three publications that relate Information Ware-house architecture and products to industry applications and requirements.These three publications are:

• Information Warehouse in the Finance Industry• Information Warehouse in the Insurance Industry• Information Warehouse in the Retail Industry.

The publications describe the Information Warehouse Architecture I andemphasize the following products:

• DataPropagator Relational• DataHub• DataGuide• FlowMark• DataPropagator NonRelational• DataRefresher• Visualizer.

These products provide a variety of functions defined in Information Ware-house Architecture I.

This publication is intended for business analysts acting as consultants to anInformation Warehouse implementation project and technical professionalswho are designing Information Warehouse solutions in the Retail industry. Aknowledge of the Information Warehouse framework is assumed.

DS (137 pages)

Abstract iii

iv The Retail Industry IW

Contents

PART 1. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Chapter 1. Industry Library Introduction . . . . . . . . . . . . . . . . . . . . . . 31.1 Library at a Glance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.3 Introduction to Solution Threads . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

PART 2. THE BUSINESS VIEW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Chapter 2. Retail Industry Perspective . . . . . . . . . . . . . . . . . . . . . . . 112.1 Retail Industry Trends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.2 Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.3 Retail Industry Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.3.1 The Store Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.4 Corporate Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.5 Retail Enterprise Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.5.1 Target Business Units for Solution . . . . . . . . . . . . . . . . . . . . . . . 222.5.2 Data Processing Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.6 Key Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302.6.1 Store Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302.6.2 Corporate Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

Chapter 3. Retail Industry Business Requirements . . . . . . . . . . . . . . 333.1 Value of Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.1.1 Precision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343.1.2 Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.1.3 Business Process Reengineering . . . . . . . . . . . . . . . . . . . . . . . . 37

3.2 Solution Thread Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383.2.1 View of Data Everywhere . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393.2.2 Access to Data Everywhere . . . . . . . . . . . . . . . . . . . . . . . . . . . 403.2.3 Access to Local In-Store Processor Data . . . . . . . . . . . . . . . . . . . 413.2.4 Access Summary and Detail Data . . . . . . . . . . . . . . . . . . . . . . . 413.2.5 Four Years of Trend Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423.2.6 Dissemination of Analysis Results . . . . . . . . . . . . . . . . . . . . . . . 42

3.3 Origin of Business Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . 423.3.1 Inhibitors to Business Growth . . . . . . . . . . . . . . . . . . . . . . . . . . 433.3.2 Qualifying an Information Warehouse Approach . . . . . . . . . . . . . . . 45

Chapter 4. The Retail Solution Thread . . . . . . . . . . . . . . . . . . . . . . . 474.1.1 Information Warehouse Framework in Retail . . . . . . . . . . . . . . . . . 484.1.2 New Scenarios for Profit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494.1.3 Requirements Mapping: Summarized→Detailed . . . . . . . . . . . . . . . 52

PART 3. THE TECHNOLOGY VIEW . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Contents v

Chapter 5. Retail Industry Architecture . . . . . . . . . . . . . . . . . . . . . . 575.1 Retail Application Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575.2 The Store Logical Data Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

5.2.1 The IBM In-Store Processing Strategy . . . . . . . . . . . . . . . . . . . . . 645.2.2 The Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 655.2.3 SLDM Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

Chapter 6. Information Warehouse Framework . . . . . . . . . . . . . . . . . 676.1 Value of the Information Warehouse Framework . . . . . . . . . . . . . . . . . 686.2 Why Data Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

6.2.1 Operational Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 696.2.2 Database Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 706.2.3 Cost of Data Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 706.2.4 Historical Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 716.2.5 Ownership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 716.2.6 Point-in-Time Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 716.2.7 Reconciliation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

6.3 The Information Warehouse Architecture . . . . . . . . . . . . . . . . . . . . . 716.4 Using the Information Warehouse Architecture . . . . . . . . . . . . . . . . . . 736.5 Access Enablers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

6.5.1 Embedded SQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 766.5.2 SQL Call Level Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 766.5.3 Distributed Relational Database Architecture . . . . . . . . . . . . . . . . . 77

6.6 The Retail Industry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 776.7 Information Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

6.7.1 Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 806.7.2 Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

6.8 Information Warehouse Architecture Products . . . . . . . . . . . . . . . . . . 816.8.1 The DataGuide Family . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 816.8.2 S/390 Parallel Query Server . . . . . . . . . . . . . . . . . . . . . . . . . . . 826.8.3 DataPropagator Relational . . . . . . . . . . . . . . . . . . . . . . . . . . . . 836.8.4 Personal AS/2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

6.9 Why Use the Information Warehouse Architecture . . . . . . . . . . . . . . . . 85

Chapter 7. Organization Asset Data . . . . . . . . . . . . . . . . . . . . . . . . 877.1 The Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 907.2 S/390 Parallel Query Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

7.2.1 Software Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 947.2.2 Information Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 947.2.3 Retail Enterprise Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

7.3 Technical Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 957.3.1 Types of Parallelism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 957.3.2 S/390 Parallel Query Server Design . . . . . . . . . . . . . . . . . . . . . . 977.3.3 Query Splitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 987.3.4 Front-End MVS System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

Chapter 8. Information Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . 1018.1 Information Catalog Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1028.2 DataGuide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

8.2.1 Basic Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1058.2.2 Knowledge Worker Functions . . . . . . . . . . . . . . . . . . . . . . . . . 1068.2.3 Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1078.2.4 Launch Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1138.2.5 Create Collections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1138.2.6 Display Contact Information . . . . . . . . . . . . . . . . . . . . . . . . . . 1148.2.7 View Current News . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1148.2.8 View Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1158.2.9 Administrator Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1158.2.10 Extending DataGuide/2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

vi The Retail Industry IW

8.2.11 Meta-data Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1208.2.12 DataGuide Data Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1218.2.13 Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

Chapter 9. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

Appendix A. Models and Modeling . . . . . . . . . . . . . . . . . . . . . . . . 129A.1 The Construction Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

A.1.1 Entity: Things . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130A.1.2 Entity: Agreements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

A.2 The Annual Report As a Model . . . . . . . . . . . . . . . . . . . . . . . . . . 131A.3 Information Warehouse and Modeling . . . . . . . . . . . . . . . . . . . . . . 131

List of Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

Contents vii

viii The Retail Industry IW

Figures

1. Basic Set of Business Objects . . . . . . . . . . . . . . . . . . . . . . . . . . 62. Current Business Environment of the Retail Enterprise . . . . . . . . . 153. EFT Transaction in the Retail Enterprise . . . . . . . . . . . . . . . . . . . 194. Generic Department Store Network . . . . . . . . . . . . . . . . . . . . . . 215. Retail Enterprise Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226. Retail Enterprise Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267. Business Modeling of Key Retail Entities . . . . . . . . . . . . . . . . . . 278. GSA′s Item File Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279. File Redirection Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

10. Retail Industry Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4811. Selective Marketing Campaign . . . . . . . . . . . . . . . . . . . . . . . . . 5012. RAA Business Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5813. The Information Warehouse Architecture and Information Processing

Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6214. In-Store Processing Application Development Strategy . . . . . . . . . 6415. SLDM Model and Database . . . . . . . . . . . . . . . . . . . . . . . . . . . 6616. Information Warehouse Architecture . . . . . . . . . . . . . . . . . . . . . 7217. Information Warehouse Framework . . . . . . . . . . . . . . . . . . . . . . 7418. Access Enablers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7519. Future Retail Enterprise Network: Connectivity and Operating

Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8420. Organization Asset Data in the Retail Industry . . . . . . . . . . . . . . . 8821. Query Cost as a Function of Query Complexity and Data Volume . . 9022. Special and General-Purpose Solutions . . . . . . . . . . . . . . . . . . . 9123. Databases and Files in Retail Enterprise Network . . . . . . . . . . . . 9224. Connectivity between Personal AS/2 and S/390 Parallel Query

Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9325. Parallel CPU Processing Environments . . . . . . . . . . . . . . . . . . . 9626. Information Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10127. A Card Catalog Information Catalog . . . . . . . . . . . . . . . . . . . . . 10228. DataGuide/2 Initial Panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10729. Search Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10830. Search Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10931. Save Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11032. New Initial Work Area Panel . . . . . . . . . . . . . . . . . . . . . . . . . . 11133. Navigation Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11234. A Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11435. Initial and DataGuide Administrator Panels . . . . . . . . . . . . . . . . 11736. DataGuide Administrator Panel . . . . . . . . . . . . . . . . . . . . . . . . 11737. Object Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11838. Create Object Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11939. Add Object Type Property . . . . . . . . . . . . . . . . . . . . . . . . . . . 12040. DataGuide Categories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

Figures ix

x The Retail Industry IW

Tables

1. Library of Information Warehouse: Products Covered . . . . . . . . . . . 42. Store Responsibilities and Functions . . . . . . . . . . . . . . . . . . . . . 163. Corporate Responsibilities and Functions . . . . . . . . . . . . . . . . . . 174. Relationship between Scenarios and RAA Constructs . . . . . . . . . . 615. Logical Design Points for an Information Warehouse . . . . . . . . . . 776. Physical Design Points for Information Warehouse Implementation . 797. DataGuide Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . 1058. DataGuide Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

Tables xi

xii The Retail Industry IW

Special Notices

This publication is intended to help:

• Business analysts understand Information Warehouse (IW) architectureconcepts

• IBM technical professionals understand industry environments• Customer data processing personnel understand industry environments.

The information in this publication is not intended as the specification of anyprogramming interfaces that are provided by a variety of products thatperform functions described in the Information Warehouse architecture. Seethe PUBLICATIONS section of the IBM Programming Announcement for theseproducts for more information about what publications are considered to beproduct documentation.

References in this publication to IBM products, programs or services do notimply that IBM intends to make these available in all countries in which IBMoperates. Any reference to an IBM product, program, or service is notintended to state or imply that only IBM′s product, program, or service maybe used. Any functionally equivalent program that does not infringe any ofIBM ′s intellectual property rights may be used instead of the IBM product,program or service.

Information in this book was developed in conjunction with use of the equip-ment specified, and is limited in application to those specific hardware andsoftware products and levels.

IBM may have patents or pending patent applications covering subject matterin this document. The furnishing of this document does not give you anylicense to these patents. You can send license inquiries, in writing, to theIBM Director of Licensing, IBM Corporation, 500 Columbus Avenue,Thornwood, NY 10594 USA.

The information contained in this document has not been submitted to anyformal IBM test and is distributed AS IS. The information about non-IBM(VENDOR) products in this manual has been supplied by the vendor and IBMassumes no responsibility for its accuracy or completeness. The use of thisinformation or the implementation of any of these techniques is a customerresponsibility and depends on the customer′s ability to evaluate and inte-grate them into the customer′s operational environment. While each itemmay have been reviewed by IBM for accuracy in a specific situation, there isno guarantee that the same or similar results will be obtained elsewhere.Customers attempting to adapt these techniques to their own environmentsdo so at their own risk.

Special Notices xiii

The following terms, which are denoted by an asterisk (*) in this publication,are trademarks of the International Business Machines Corporation in theUnited States and/or other countries:

The following terms, which are denoted by a double asterisk (**) in this publi-cation, are trademarks of other companies:

Other trademarks are trademarks of their respective companies.

BookManager CICS/ESACommon User Access CUADataGuide DataHubDB2 DB2/2DB2/6000 IBMImagePlus IMS/ESAMVS/ESA PS/2QMF RISC System/6000

Apple Apple Computer, Inc.BRIDGE/FASTLOAD Bridge Technology, Inc.KnowledgeWare Application Devel-opment Workbench

KnowledgeWare, Inc.

MacIntosh Apple Computer CompanyMicrosoft, Windows Microsoft CorporationMotif Open Software FoundationOSF Open Software Foundation, Inc.ObjectStore Database Object Design, Inc.1-2-3, Lotus, Freelance, FreelanceGraphics

Lotus Development Corporation.

OMegamon Candle CorporationFast Load PLATINUM Technology Inc.Fast Unload PLATINUM Technology Inc.Rapid Reorg PLATINUM Technology Inc.Quick Copy PLATINUM Technology Inc.In2itive LEGENT

xiv The Retail Industry IW

Preface

This document is intended to merge industry analysis, industry architecture,the Information Warehouse architecture, new product discussion, and spe-cific solutions to industry requirements. It contains discussion of specificindustry issues, industry architecture for data processing, Information Ware-house architecture, and solutions.

This document is intended for business analysts and data processing profes-sionals.

How This Document Is Organized

The document is organized as follows:

• Introduction

This part introduces the library within which this particular book isincluded.

• The Business View

This part establishes the business-oriented level set from which businessrequirements and Information Warehouse solutions are developed. Thefirst chapter presents a perspective on the Retail industry that includestrends and challenges, key systems, and information technology′s posi-tion in the Retail industry.

• The Technology View

This part presents the technology solutions for the business require-ments established in the business view. It includes an overview of theindustry application architecture and the Information Warehouse architec-ture. It then discusses the individual components of the InformationWarehouse architecture and the solutions to those components,according to the needs of the industry.

Preface xv

Related Publications

The following publications are considered particularly suitable for a moredetailed discussion of the topics covered in this document:

• “An Architecture for a Business and Information System,” B.A. Devlinand P.T. Murphy, IBM Systems Journal, Vol. 27, No. 1 (1988)

• “Building Business and Application Systems with the Retail ApplicationArchitecture,” P. Stecher, IBM Systems Journal, Vol. 32, No. 2 (1993)

• Client-Server Computing: The Design and Coding of a Business Applica-tion, GG24-3899-00.

• DataGuide/2 V1: Using DataGuide/2, SC26-3365

• Delivering Data to the Information Warehouse, Rob Goldring, InfoDBSummer 1992

• Financial Application Architecture: FAA Concepts of Application andSystem Architectures, LY38-4402-0

• Information Technology and the Management Difference: A Fusion Map,IBM Systems Journal, Vol. 32, No 1 (1993)

• Financial Application Architecture Introduction, GC31-3932-0

• “The Future of Health Care Information Systems,” Michael Carrigan, Hos-pital Materiel Management Quarterly, August 1993

• Information Warehouse Architecture I, SC26-3244

• Insurance Industry Futures: Directions for the 21st Century, AndersonConsulting and LOMA 1993

• “Loaning Banks Some Courage,” Information Week, August 12, 1993

• “The Model Business,” IW Today, August 12, 1993

• Principles of Life and Health Insurance, G. Morton, 1988 LOMA

• “The Spectrum of Data Delivery for Business Information Systems,” RobGoldring, DB/Expo92

xvi The Retail Industry IW

International Technical Support OrganizationPublications

• Information Warehouse Architecture and Info. Catalog, GG24-4019

• Information Warehouse Storage Management Guidelines and Consider-ations, GG24-4336

• Information Warehouse in the Finance Industry, GG24-4340

• Information Warehouse in the Insurance Industry, GG24-4341

• Library for System Solutions: Data Reference, GG24-4103

A complete list of International Technical Support Organization publications,with a brief description of each, may be found in:

Bibliography of International Technical Support Organization TechnicalBulletins, GG24-3070.

To get a catalog of ITSO technical publications (known as “redbooks”) online,VNET users may type:

TOOLS SENDTO WTSCPOK TOOLS REDBOOKS GET REDBOOKS CATALOG

How to Order ITSO Technical Publications

IBM employees in the USA may order ITSO books and CD-ROMs usingPUBORDER. Customers in the USA may order by calling 1-800-879-2755

or by faxing 1-800-284-4721. Visa and Master Cards are accepted.Outside the USA, customers should contact their local IBM office.

Customers may order hardcopy ITSO books individually or in customizedsets, called GBOFs, which relate to specific functions of interest. IBMemployees and customers may also order ITSO books in online format onCD-ROM collections, which contain books on a variety of products.

Preface xvii

Acknowledgments

The advisor for this project was:

Steve SchafferInternational Technical Support Organization, San Jose

The authors of this document were:

Normand BrinIBM Canada

Wojeich ZagalaIBM Australia

Steve SchafferInternational Technical Support Organization, San Jose

This publication is the result of a residency conducted at the InternationalTechnical Support Organization, San Jose.

Thanks to the following people for the invaluable advice and guidance pro-vided in the production of this document:

Paul Englefield, IBM WarwickRob Goldring, IBM SWS, Santa TeresaEileen Hiltbrand, IBM USJacques Labrie, IBM SWS, Santa TeresaBill Martin, IBM USMark Mauriello, IBM Charlotte Finance IndustryRita Neuberg, IBM CharlotteBill Payne, IBM Charlotte Insurance Industry

Thanks to the following people for reviewing this document:

Thomas Bilfinger, IBM ITSO, San JoseDon Cameron, IBM ITSO, San JoseDon Murray, IBM USRalph Naegeli, IBM SwitzerlandTom Romeo, IBM USMichele Schwartz, IBM SWS, Santa Teresa

Special thanks to Ueli Wahli for developing the tool to generate margin com-ments.

xviii The Retail Industry IW

Part 1. Introduction

Part 1. Introduction 1

2 The Retail Industry IW

Chapter 1. Industry Library Introduction

This volume is one of three that look at Information Warehouse architectureand products in the finance, insurance, and retail industries. The threestudies yielded somewhat different results but are presented in a standardstructure. This introductory chapter describes the library, so that it may beused easily and most effectively by the individual.

1.1 Library at a Glance

The study of the finance, insurance, and retail industries has been madeavailable as a library of three books. Each book takes the same approach todiscussing the respective industry, though there is some variation in theaspects of the Information Warehouse architecture covered in each industry.Table 1 on page 4 gives an overview perspective of this library. Thelibrary′s structure is based on a common set of topics that are addressedconsistently across the industry studies, and different aspects of the Informa-tion Warehouse architecture being addressed in each study. This structureminimizes redundancy. Therefore, a complete review of Information Ware-house architecture function and products may require reading of all threeindustry studies.

The common set of topics presents the study flow from a high-level perspec-tive of the industry down to the discussion of the Information Warehouseproduct technology. These topics are broken down into the business viewand technical view as follows:

• Business view− Industry perspective− Business requirements

• Technology view− Industry application architecture− Information Warehouse architecture− Information Warehouse framework components.

Chapter 1. Industry Library Introduction 3

1.2 Terminology

The finance, insurance, and retail industry studies address general require-ments of the respective industry. They are not studies of a real-world or acontrived enterprise. Rather, they are studies of industry issues and require-ments put in the context of an industry enterprise. For this reason, we usethe term financial enterprise, insurance enterprise, and retail enterprise,respectively, to reflect this approach.

Table 1. Library of Information Warehouse: Products Covered

Industry Product

Finance DataPropagator Relational

Insurance FlowMarkVisualizer

Retail DataGuide*S/390 Parallel Query Server

We begin our study by identifying the knowledge worker as the primary focusof Information Warehouse technology. Knowledge workers are the individ-uals in an enterprise who make decisions. They exist at every organizationallevel and have one thing in common: they all need information to make deci-sions. They get the information through informational applications, alsocalled executive information systems, decision support software, and deci-sion support tools. Informational applications are used to display informationprovided by the data replication products in the Information Warehouse sol-ution. Therefore, the objective of the studies is to describe knowledge work-er ′s business need for information and the addressing of that need throughthe Information Warehouse architecture and products.

1.3 Introduction to Solution Threads

The know-ledge workeris the focus

A solution thread is a vehicle for applying architectures and products to ageneric requirement of the industry. It is a generalized approach, in that theInformation Warehouse architecture it uses is generalized for a given dataprocessing function (for example, decision support) across industries, andthe industry architecture it uses is generalized for enterprises within a givenindustry. The products used by the solution thread are generalized for thatdata processing function, rather than for a business requirement or hardwareplatform. The solution thread meets the business requirement by custom-izing the product to the needs of the business.

Solutionthread:applying tech-nology to aproblem


The Information Warehouse architecture is a generalized architecturalapproach to managing data and information in a complex business environ-ment. The industry architectures—financial, insurance, and retail—discussedin the three books of this library represent structured approaches to ana-lyzing specific business environments. This analysis leads to specification ofrequirements, expressed in business terms, which data processing technolo-gies must address. This library presents the Information Warehouse archi-tecture as an architected approach to data processing functions requiredacross the industries, and across the lines of business within each industry.The value of using both the industry-specific architectures and the general-ized data processing architecture (Information Warehouse architecture) is toleverage the Information Warehouse architecture functions across the linesof business and to leverage the resources already invested in the industry-specific architectures.

Not all features of the Information Warehouse architecture nor every productis considered in each of the three industry studies chosen for this library.Review of the three studies as a set will provide information on most of theInformation Warehouse architecture features and Information Warehouseframework products. We discuss IBM′s published Information Warehousearchitecture as a template for connecting business requirements to actualtechnical implementations, including products plugged into an InformationWarehouse framework.

The following example taken from the insurance industry illustrates the lev-erage of the Insurance Application Architecture (IAA) together with the Infor-mation Warehouse architecture. Figure 1 on page 6 shows a set of businessobjects for the insurance industry. These business objects are used asexamples of modeling entities—OBJECT, AGREEMENT, andDEMAND_FOR_DELIVERY—found in the IAA model. IAA defines a modelingentity called an OBJECT. This entity is used to symbolically represent any-thing that can be insured, such as a life. IAA defines another modeling entitycalled Agreement. We could use this modeling entity to represent the phys-ical insurance entity called Policy in either personal or property and casualtyinsurance. This is an example of the generality of the IAA being leveragedacross lines of business. We could further use another entity, calledDEMAND_FOR_DELIVERY, to represent Claims and PREMIUMS for Premiums.

The Informa-tion Ware-housearchitectureis a general-ized approach


Figure 1. Basic Set of Business Objects. Dashed arrows indicate data flow.

These basic entities correspond to the real-world objects. The Claims entitycorresponds to the many claims made by insurees. The Premiums entitycorresponds to the many premium payments made by insurees. Theseclaims and payments are implemented as records in an operational data-base, say, IMS/DB, DB2*, or VSAM. The prime concern of the insurancecompany is profitability. In this simple exercise, profitability is defined astotal premiums minus total claims.

Assuming claims records and premiums records are kept in separate data-bases, there is a need to bring those two sets of data together. There is anadditional need to reconcile this data as it is brought together. The Informa-tion Warehouse architecture defines the process by which this can be donein an architected manner. This architected approach to extracting data iscalled data replication. Data replication is generalized, so that it can be asolution for bringing together Claims and Premiums data for Life insurance,Property and Casualty, or any other insurance product that takes in pre-miums and pays out claims.

The Information Warehouse architecture provides guidance for data accessas well as data replication functions. Data access applies to operational dataor informational data copies of the operational data, generated through datareplication. Access to operational data (direct access) or informational datashares certain requirements for implementation. These requirements can bebest understood in terms of the business requirements for data access. Thelines of business are assumed to have their own data stores containingrecords of insurees, sometimes called client files. The Life insurance line ofbusiness has an interest in using the client file owned by the property andcasualty line of business for prospecting purposes, and vice versa. Thisrequirement brings up two issues relevant to the Information Warehouse


framework: what data is available, and how to access it. The first require-ment is resolved through the Information Catalog, while the second isresolved through access enablers or data replication. The point here is thatboth lines of business have the same needs to access data, and both needscan be resolved by solutions based on the Information Warehouse architec-ture.


Part 2. The Business View

Chapter 2. Retail Industry Perspective . . . . . . . . . . . . . . . . . . . . . . . 112.1 Retail Industry Trends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.2 Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.3 Retail Industry Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.3.1 The Store Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.4 Corporate Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.5 Retail Enterprise Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.5.1 Target Business Units for Solution . . . . . . . . . . . . . . . . . . . . . . . 222.5.2 Data Processing Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.5.2.1 Merchandising and Operations Roles . . . . . . . . . . . . . . . . . . . 232.5.2.2 Retail Enterprise Data Flows . . . . . . . . . . . . . . . . . . . . . . . . 26

2.6 Key Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302.6.1 Store Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302.6.2 Corporate Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

Chapter 3. Retail Industry Business Requirements . . . . . . . . . . . . . . 333.1 Value of Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.1.1 Precision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343.1.2 Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.1.3 Business Process Reengineering . . . . . . . . . . . . . . . . . . . . . . . . 37

3.2 Solution Thread Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383.2.1 View of Data Everywhere . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393.2.2 Access to Data Everywhere . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

3.2.2.1 Ease of Use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413.2.2.2 Ad Hoc Query Capability . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.2.3 Access to Local In-Store Processor Data . . . . . . . . . . . . . . . . . . . 413.2.4 Access Summary and Detail Data . . . . . . . . . . . . . . . . . . . . . . . 413.2.5 Four Years of Trend Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423.2.6 Dissemination of Analysis Results . . . . . . . . . . . . . . . . . . . . . . . 42

3.3 Origin of Business Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . 423.3.1 Inhibitors to Business Growth . . . . . . . . . . . . . . . . . . . . . . . . . . 43

3.3.1.1 Faster Market Analysis and Reaction . . . . . . . . . . . . . . . . . . . 443.3.1.2 Rapidly changing criteria for POS data as information . . . . . . . . . 443.3.1.3 Increased Volume of Valuable Data . . . . . . . . . . . . . . . . . . . . 44

3.3.2 Qualifying an Information Warehouse Approach . . . . . . . . . . . . . . . 45

Chapter 4. The Retail Solution Thread . . . . . . . . . . . . . . . . . . . . . . . 474.1.1 Information Warehouse Framework in Retail . . . . . . . . . . . . . . . . . 484.1.2 New Scenarios for Profit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

4.1.2.1 Scenario 1: Differential Marketing by Preference Group . . . . . . . . 494.1.2.2 Scenario 2: Quick Market Response and Follow-on Adjustments . . 504.1.2.3 Scenario 3: Sales Analysis for Promotion Negotiations . . . . . . . . 514.1.2.4 Scenario 4: Buying on Key Product Attributes . . . . . . . . . . . . . . 51

4.1.3 Requirements Mapping: Summarized→Detailed . . . . . . . . . . . . . . . 52

Part 2. The Business View 9

Chapter 2. Retail Industry Perspective

Retailing is arguably the oldest profession in the world. From its inception inbartering and community trading to neighborhood corner stores, retailing hasevolved into one of the largest, most competitive businesses across theglobe. Retailing has traditionally offered fewer barriers to entry than indus-tries such as manufacturing, publishing, or transportation. This ease of entryhas led to new formats, concepts, and competitive forces in retailing thatconstantly challenge existing retailers.

Retailers are moving toward an unprecedented level of sophistication. Theact of buying was even recently considered an art form, a “feel,” and an intu-ition. The best merchants were measured by their instinct to know or abilityto predict what the customer would want in the future. The financial side ofretail, however, was more scientific in nature; not concerned with trends,demographics, or merchandising. It focused instead on gross margin returnon assets (GMROA), return on investments (ROI), and dollar sales per squarefoot of space as the measure of success.

The changes in retail in the 1980s brought about greater transformation in theindustry than in the entire previous century. The retail segments havebecome increasingly blurred, making it difficult to uniquely identify a full-service provider, discounter, or specialty store. Retailers now exist whooffer multiple services and formats, and they have evolved from regional

Chapter 2. Retail Industry Perspective 11

players to national and international powerhouses. Thus, the intuition uponwhich merchants and operational and financial executives have relied tomake their business and investment decisions has often proven insufficientto compete effectively in the new global retail marketplace.

Operational and merchandising executives need to understand and embracemore detailed information to compete. Retail information systems were for-merly deemed sufficient to capture transactional levels of data, such as whatsold yesterday, where, and at what price. This transactional level of data,while critical, is no longer sufficient to make informed retail investment deci-sions.

Today′s market—a buyer′s market—forces wholesalers and retailers tocompete intensely for customers. In the retail industry, this means devel-oping customer loyalty and influencing customer purchase decisions concur-rently with expense reduction. Retailers today must be able to take historicaldata—what sold, where, and at what price—and infer from that data what willsell tomorrow:

• Did the blouse that sold in Boston sell because it was blue?• Did it sell because it was silk?• Did it sell because of its ruffled collar?

Alternatively, if the blue silk blouse with ruffles sold, does that mean theretailer could exploit this emerging trend by allocating more open-to-buy tothe silk dress category? Also, if it sold in Boston yesterday, how, based ontrends, will that blouse sell in Columbus tomorrow?

These kinds of questions demand, for their resolution, a tremendous amountof data and information. More importantly, they demand access to andmanipulation of that data throughout and between all levels of the retailorganization. The role of providing information has traditionally rested withthe information systems department. The demands that are being placed onretail information systems departments for information, and the ability toprovide that information in such a manner as to let the user manipulate andquery it presents a formidable challenge to information systems profes-sionals. These information demands require new approaches to the capture,storage, and access to data.

This document offers insights into those new approaches. The InformationWarehouse framework supplies the tools that enable the retailer to provideconsistent, clean data and information to all of its users throughout theorganization. The Information Warehouse implementation is built on widelyaccepted standards for the storage and transmission of and access to infor-mation. Furthermore, the Information Warehouse architecture is not proprie-tary. It is open to any and all users and information technology providers, sothat a retailer is not bound to a specific information technology vendor fordatabase, application, operating system, or hardware solutions.

The Information Warehouse architecture is built on the use of relationaldatabases—the evolving standard for informational data storage—and Struc-tured Query Language (SQL) for accessing informational data across theenterprise. The ability to establish an accepted standard for an architectureallows all vendors and users to participate in the delivery of solutions thatcan benefit the entire industry. The Information Warehouse allows the retail-er ′s current “legacy” systems, such as payroll and accounts receivable, to


participate in the solution, so an entire rewrite of established mission criticalsystems is unnecessary.

The ability of a retailer to develop information systems that not only allow itto know what happened yesterday, but also provide insights into what willhappen tomorrow, is an unparalleled competitive advantage. This bookdescribes how the retailer can develop this capability through the InformationWarehouse architecture.

A great deal of the initial historical effort to streamline retail operations hasfocused on the product resource. Today, in many retail businesses, point-of-sale (POS) systems are the primary feeder of data into the retail enterprise.Credit card purchases and electronic funds transfer (EFT) transactions arefinancial processes that complement the POS system and generate a greatdeal of detailed data on the client resource. Retail operations capable ofexploiting credit card information can be found in many market sectors (forexample, food, automotive, and pharmacy).

The retailindustryfocus: theproductresource

The POS, credit card, and EFT systems feed data to software packages thatmanage merchandising, inventory, labor, price, and store productivity. Soft-ware packages—fed by the POS systems—execute on the POS systems, anin-store processor, and the large server. All of these packages contribute toproduct management or controlling the infrastructure of the retail business.They also keep costs low and thereby enable effective marketplace competi-tion. Specialized program development, geared to product and enterpriseinfrastructure, goes hand-in-hand with customer support systems.

The POS,credit card,and EFTsystems feedother systems

With POS data added to markets resource data, the collective informationrepresents a large and continuously growing asset. Trend analysis is seenas the methodology for understanding client and market direction, hidden inthe large volumes of data. Therefore, large volumes of data need to bemanaged.

The organization of this retail industry study follows that of the end-to-enddesign of a robust business system:

• General business issues are discussed.• A brief modeling of enterprise business is undertaken.• Required business systems are identified.• Detailed requirements are gathered for those business units most in

need of a solution.• Requirements are refined down to a level that can be directly translated

into technology.• Maintenance issues are resolved by automation and procedures.

The difference between the development of a single business application andthe development of an approach to developing business applications lies inthe end result—the business system. In the traditional approach, the endresult is an application program or multiple programs delivering end-userfunction and selective data access. In the case of the retail solution threadused in this book, we deliver an architecture-based system to support deci-sion making. The recipient of the Information Warehouse solution, who wewill sometimes call a knowledge worker or analyst, determines function andrequired data access.

Trend anal-ysis discoversthe business


Enterprises from different segments of the retail marketplace face commonchallenges. This book uses an Information Warehouse solution thread toaddress selected challenges of a representative retail business.

2.1 Retail Industry Trends

Today′s retail enterprises are subject to many of the forces other industrialsectors face:

• The drive to improve quality of service• The drive to improve profitability• Competition• Government regulation• Managing mergers with other companies• The desire to exploit cheaper technology.

IBM has involved representative enterprises from many retail market sectorsin an effort to respond to these forces. The result is the development of theRetail Application Architecture as a beginning point for the solution. Thatsolution is an overall information systems strategy for managing the busi-ness environment.

IBM discovered during its work with retail enterprises that retailers identifiedthree basic business resources as key to their business. Furthermore, theyattached a specific hierarchy of importance to these resources for their dailyoperations, as follows:

• Product• Client

The client resource refers to specific individual customers.• Market

The market resource is made up of customers treated as one or moregroups. Suppliers to the retail industry are a subset of this resource.

2.2 Challenges

The retail enterprise′s corporate mission is to provide quality service to sat-isfied customers and maintain corporate profit. The objectives in the currentmarket environment are to:

• Reduce the cost of doing business• Attract and retain customers• Improve business processes• Reduce cycle times.

Typical strategies are to:

• Increase profitability from within by:− Leveraging external resources− Focusing on costs− Emphasizing creative uses of technology (for example, database

mining)− Reengineering business processes.


• Differentiate customer service from the competition• Improve merchandise flow management through:

− Quick response: reacting to trends and getting merchandise to thefloor quickly

− Electronic data interchange− Effective inventory management− Profit contribution analysis: focus on specific product profit, not

overall revenue.

2.3 Retail Industry Organization

Figure 2 shows the positioning of business roles (units) within the retailenterprise. Technology that addresses the business strategies listed abovemust have the potential to integrate these major business units. The busi-ness roles are typical of many market segments in the retail industry. Theneed to connect these business units to the same source of information willaffect an Information Warehouse implementation.

Figure 2. Current Business Environment of the Retail Enterprise. Arrows indicatecurrent business unit relationships. Not all relationships are implementedby electronic connections.


Some business roles exist only at the corporate level (for example, finance);while others exist at the store level (for example, department manager).Specific business units or departments within the retail enterprise (forexample, the purchasing department) may include tasks that cross theselevels.

Roles can bewithin oracross units

Within the retail enterprise, there is a division of business responsibilitybetween store and corporate headquarters personnel. That distinction,however, is becoming more blurred as management initiative seeks toempower workers. Today, store personnel are primarily responsible for theday-to-day operations of their assigned store. Table 2 shows the responsi-bilities at the store level and the functions performed within the scope ofthose responsibilities.

Table 3 on page 17 shows the responsibilities of corporate personnel andthe functions performed within the scope of those responsibilities. Theseresponsibilities tend to be concerned with tasks that are common to allstores.

Empoweringworkerschanges rolesin the organ-ization

Table 2. Store Responsibilities and Functions

Responsibility Function

Product • Controlling store inventory• Local pricing initiatives• Replenishment ordering

Administration • Labor management (primarily shift sched-uling)

• Time and attendance reporting

Customer • Customer credit management• Customer service

Store operations • Checkout• Loss prevention• Receiving• Financial records• Training


Table 3. Corporate Responsibilities and Functions

Responsibility Function

Market • Advertising and promotions• Marketing campaigns• Site selection

Product • Distribution• Merchandising• Volume purchasing

Operations • Employee statistics• Employee welfare and career planning• Training not covered in the store

Administration • Corporate finances, taxes, and legal matters• Fixed asset management: buildings, equipment,

insurance• Payroll• Translating mission, objectives, and strategy into

company initiatives and measurement systems

A particular business unit may be responsible for several functions. Morethan one business unit may share responsibility for a given task. Forexample, the merchandising business unit in Figure 2 on page 15 is prima-rily responsible for managing marketing and product range. It must createmarketing campaigns based on trends in consumption, the fashion of theday, or competitor activity. A campaign requires the involvement of stafffrom the operations business unit who create the advertising copy, informthe stores of price changes and promotion terms and conditions, and ensurethat stock levels are adequate at stores affected by the campaign. Finally,operations must coordinate with the distribution business unit to ensuretimely delivery of campaign items.

This marketing campaign scenario illustrates the process-oriented relation-ships among business units. Figure 2 on page 15 depicts these relation-ships with arrows. Given the sheer number of relationships among businessunits, it is easier to represent the relationships by connecting each unit to itslevel of business processing (for example, corporate or store level). Thescenario also implies a high degree of information sharing. It implies thatthe merchandising unit has access to the trend information held by corporatethat was derived from sales information held by stores. The store and corpo-rate network is the vehicle by which data and information flow through theenterprise to support information sharing.

Roles canspan organ-ization struc-tures


2.3.1 The Store Environment

The store depicted in Figure 5 on page 22 represents a standard configura-tion found in all other stores of the network. The POS system is the mainsource of data within the retail enterprise. Approximately 85% of all data inthe company is derived, in some form, from POS data. POS systems use ascanner to read in the universal product code (UPC) from the merchandise,support dial-in credit card purchase verification, and handle EFT transactionsfor purchases made with a bank debit card. POS machines are connected tothe store LAN shown in Figure 5 on page 22. One particular POS machine,the POS controller, performs some intermediate processing of entered data,such as financial summarization by day and week, and functions as a backupserver in the event that the in-store processor fails.

The in-store processor is the main POS server. It stores the item file usedfor all price lookups. It also runs software to support operations activities inthe store, as follows:

Labor management Labor management includes scheduling of shifts,breaks and workload assignment.

Inventory management Inventory management includes setting of orderpoints for all items, creating replenishment orders,transmitting orders where electronic data inter-change has been enabled, receiving merchandise,and scheduling returns.

Time and attendance Time and attendance includes clock-in, clock-out,and related functions.

Inventory checks Inventory checks uses a hand-held terminal (HHT)(see Figure 4 on page 21) to scan and transmitcurrent quantities from the shelf or storeroom areato the in-store processor and compare them withexpected quantities.

POS systemsbring in 85%of the data

Some of these software packages (for example, inventory management)execute automatically at the end of the shopping day. They generatereports for store personnel to read and file the next day and transmit data tothe large server to be used for further processing and reports. The start ofin-store processing is controlled remotely by large server operations staff. Inthe event of an in-store processor problem, some or all stages of in-storeprocessing can be completed at the host by uploading data already proc-essed and raw POS data.

Softwareturns today′sdata intotomorrow ′sreports

The store LAN configuration does more than support POS communications: itenables communication between programmable workstations (PWSs) in thestore and in-store processor and large server programs. These PWSscontain spreadsheet and word processing software. Communications soft-ware is also installed on each PWS, enabling sessions to be established withthe large server. A variety of software is available on the large serversystem. Common spreadsheet files and backlevel copies of the item filefrom the in-store processor are stored on the LAN file server. Some of thesespreadsheets are downloaded and maintained by the corporate large server.

The storeLAN is a com-municationsvehicle


EFT is a complex transaction that involves many external organizations. Itdelivers convenience to customers, improved cash flow to the store, anddocumentation of individual client preferences and buying patterns for busi-ness analysts. Figure 3 shows the communications required between thestore and the many outside agencies involved in an EFT transaction. Thesecommunications support the following major processes in an EFT trans-action:

• Credit and debit card validation• Withdrawal of funds from client account• Crediting of funds to retailer account• Imprinting transaction information (franking) of POS documents• Validation of third-party checks.

EFT is avehicle fordata creationand move-ment

Figure 3. EFT Tra nsaction in the Retail Enterprise


In summary, the in-store processor produces the reports, forms, and ordersrequired to support the store′s immediate business function. It is also theprimary file server for POS systems in the store network. Large server proc-essing adds similar information objects and data stores to the large serverenvironment. The only difference is that large server information supportsthe additional business responsibilities of corporate staff. Some of this hostinformation is sent back to the store when it reflects regional trends ofinterest to store managers for strategic decision making.

2.4 Corporate Environment

The in-storeprocessorserves mul-tiple purposes

The large server provides additional processing of transmitted store data tosupport tasks that are common to all stores. As part of that processing,some data is filtered back to the in-store processor (for example, pricechanges), from which it may be passed on to the POS machines. Additionalreports such as upcoming promotion announcements, names of customershaving written bad checks, and interstore transfer requests are also sent tothe store for processing and printing by the in-store processor.

Value of Summary Data

Summary data helps control the business while detailed information helpsmanagement analyze how to improve it.

Informationflows in twodirections

Workstations on the corporate LAN also contain the same basic decisionsupport applications as the stores for support of spreadsheets, word proc-essing, and communications. In addition, some staff have specialized soft-ware to support their tasks. For example, the operations business unit hassoftware that enables them to design advertising copy layout. Software sup-porting graphics creation (for example, presentation charts and graphs) ismaintained on the corporate LAN.

2.5 Retail Enterprise Network

Figure 4 on page 21 shows a common configuration in retail departmentstore networks. A variety of communications protocols moves data withinthe store network and to other participants in the stores business: corporate,financial institutes, distributors, and suppliers.

Workstationshave commonand special-ized software


Figure 4. Generic Department Store Network

We limit the scope of this study to three distinct levels of data processing, asfollows:

• Retail application processing• In-store processing• Large server processing.

The network shown in Figure 5 on page 22 indicates where some of the per-sonnel roles execute their data processing tasks.


Figure 5. Retail Enterprise Network

This chapter discusses the general placement and flow of information in thisnetwork environment. For a more detailed description of specific data flowsand the products used, see 3.3, “Origin of Business Requirements” onpage 42.

2.5.1 Target Business Units for Solution

The Merchandising and Operations business units were selected to pilot testthe recipients of the first phase of the retail solution thread. Selection wasbased on interviews with key corporate executives and managers from eachbusiness unit. Data from the interviews was incorporated into a businessmodeling tool. The tool recommends business systems with a certain datausage profile. This profile was then mapped against the capabilities ofsystems currently in existence. Selection of target business units was thenbased on the following considerations, in highest to lowest priority sequence:

• Corporate versus business unit urgency• Absence of data processing function in current business systems• Volume of data required• Nature of data analysis demanded by the solution• Sharing potential of Information Warehouse-generated data among busi-

ness units.

The driving business urgency for both business units was support of thereports described in section 4.1.2, “New Scenarios for Profit” on page 49.


The objectives of the pilot test and the strategies established to meet thoseobjectives are as follows:

Objective Strategy

Attract and retain customers Focus on costs

Improve business processes Use technology creatively

Reduce cycle times Reengineer business processes through newuses of data.

We assume that both business units lack business systems capable of eco-nomically meeting these objectives.

2.5.2 Data Processing Functions

In this section, we expand the overview of data processing at the retail enter-prise. Specifically, we examine personnel roles in the Merchandising andOperations business units and position them with respect to the network andbusiness systems. We also go into more detail on what happens to data as itis handled by the in-store processor and the large server.

2.5.2.1 Merchandising and Operations Roles

In general, corporate personnel make better use of their LAN systems thando store personnel. This is as much a function of job responsibilities as anyother factor. Corporate personnel are closer to information systems depart-ment assistance and generally have more time allocated to Information Tech-nology tools usage. The analysis they perform and the decisions andrecommendations they make are tied to the data they access through Infor-mation Technology tools.

Most of the data of interest to corporate personnel is stored on the largeserver where it is initially accumulated from the stores. Traditionally, reportsagainst this data are generated using server-based software. Analysts in theMerchandising and Operations units typically use Personal System/2 desktopcomputers. Access to server applications, such as the report generator, isthrough terminal emulation products, with the data flows passing over thetoken-ring LAN. The interface is one that can be easily displayed on a non-programmable terminal.

The more user-friendly graphical user interface (GUI) is deployed on thespreadsheet, word processor, graphics, and publishing software on the LANfile server or personal workstations.

Marketing Business Unit: One of the more common positions in the mar-keting business unit is that of the buyer, the position responsible for productlines. We focus on a buyer responsible for purchasing blouses. The retailenterprise carries 200 different blouse lines. Weekly summary reports arerun on sales across all outlets. One section of this report summarizes salesby product type. The buyer can select options from the software program tocreate a hardcopy version of blouse product type sales. This report showsmany variations of aggregated information related to blouses:

• Week-ending totals for sales by product line code by region


A brief description accompanies each product line code; for example:

Brand X: Rayon, Polo style

• Week-ending totals for sales by product line code by store number• Week-ending totals for sales by product line code by region for the last

12 weeks• The same 12-week summary but by store number• The top 10 product lines by store number and summarized by region.

Also included with the key information on each product line is quantity sold,average price, number of returns, and amount left in stock. The buyer canobtain more descriptive detail on a particular blouse in the manufacturer′scatalog.

The buyer′s most pressing informational need for detailed sales informationat a store level is unfulfilled. Some information about daily totals is availablefrom a server database used by the Distribution and Operations businessunits. This information is used to monitor on-hand quantities and, hence, ismore amenable to inventory management than management informationsystems functions. Management information systems, or decision support,analysis requires additional massaging of the data to determine detailedinformation to answer questions such as:

What color is selling well?

Some of the problems that arise in answering this question include:

• Decoding

Color is indicated by a code for which a lookup table is scanned to estab-lish a text description of the color. This lookup and translation require-ment is addressed in the Information Warehouse architecture as datareconciliation.

• Lack of ad hoc access

The traditional file and database systems used in the store environmentare awkward, at best, for ad hoc access. Informational analysis of thisoperationally oriented data would require data processing expertise forapplication programming and a business analyst′s knowledge to navigatethrough the required tables and produce valid results from the correctsource data and the correct analysis.

• Busy analyst

Traditional approaches to informational analysis depended on an infor-mation specialists. The information specialists were responsible foracquiring operational data and transforming it into informational datathrough extraction and enhancement programs. They were then respon-sible for making the informational data available to business analystssuch as the buyer. This approach was embodied in the informationcenter. The analysts are a limited resource and are not always availablebecause of normal business cycles and the peak demands they create.


• Lack of information

The sales volume of a particular item must take returns into account.The net success of the item is the difference between the volume soldand the volume returned. The buyer must go to other sources to getdaily details on why certain blouses were returned and who returnedthem.

Trend information beyond the 12 week summary report is difficult to build.The knowledge worker could manually transfer data from various sourcesinto a single spreadsheet, but the time constraint makes this approach unde-sirable. Major merchandising decisions, however, occasionally make suchexercises mandatory.

When the buyers try to identify the product lines to reorder, cancel, or targetfor marketing campaigns, they normally have as little as one or two weeks′warning. Therefore, they make decisions using aggregated information thatis readily available. As the deadline approaches, the possibility of using thedetailed information to verify any assumptions vanishes, and the buyer mustoften go with instinctive choices.

Analysts mustmake deci-sions quickly

The buyer would prefer to make decisions based on dynamically definedsummarization and aggregation reports. Each report defined may in turnlead to other summarization and aggregation reports. The retail enterpriserequires specialized software, hardware and the access to detailed data tosupport effective decision making.

Operations Business Unit: One of the more common positions in theoperations business unit is an inventory specialist. An inventory specialist′smain responsibility is to maintain cost-effective levels of store merchandiseby working in concert with department management at individual stores andthe distribution unit. A large part of an inventory specialist′s workload is thecoordination of marketing campaigns and of major shifts in the merchandisemix at particular stores. An inventory specialist typically has direct responsi-bility for some number of stores and coordinates with other inventory spe-cialists on national campaigns.

Most of the informational data is gleaned using standard reports from theIMS inventory database. Occasionally, when the need arises, additionalqueries can be created. However, each of these requirements implies a dis-tinct coding effort to build the extract, reconciliation, and aggregation logic todeliver the information. The cost associated with this effort is an inhibitor;business analysts must go through a justification process to get this informa-tion. Analysts with experience in creating queries perform more and morework for the marketing business unit. This results in a shift of work withinthe work group.

Analysts needspecial soft-ware toanalyze data


2.5.2.2 Retail Enterprise Data Flows

Data flows documented in this section take place at various points in theretail enterprise network configuration seen in Figure 5 on page 22. Weexamine the movements and processing of data that occur at the three levelsof the network: POS network, in-store processor, and the large server.

Figure 6. Retail Enterprise Network. Network elements and operating systems

Although data is present in many different forms at different network levels,many of the applications executing at these levels use a consistent model ofthe data, depicted in Figure 7 on page 27. This model is important to thedefinition of informational data within the Information Warehouse.


Figure 7. Business Modeling of Key Retail Entities

POS Network Data Activity: Data is either read or changed in two keydata objects: the Item file (for a representation of the Item file structure usedin the General Store Application (GSA), see Figure 8) and the transactionlog, which records data related to a purchase as a single record in a flat file.Relational technology is gaining acceptance as a database for these twoobjects.

Figure 8. GSA ′s Item File Format


4690 POS terminals scan the item file for prices and register purchases inthe transaction log throughout the day, under the control of GSA. The in-store processor contains the two files; the 4690 POS devices access themusing file redirection services provided by the 4690 operating system.Figure 9 outlines what services are used in enabling this file redirection.Both the 4690 operating system and GSA provide safeguards to ensure thattransaction data is not lost in the event of a failure.

Figure 9. File Redirection Services

The key point in the 4690 POS operational environment is the efficientstorage of data in a minimal number of files on the in-store processor. Datafrom 4690 POS devices passes through 4690 controllers. These devices canbe configured to provide reports on cashier till balances by the day or week.

In-Store Processor Data Activity: The in-store processor at the retailenterprise runs software packages that assist store personnel with theirtasks, including labor and inventory management and product audit. Labormanagement and product audit are not mentioned further in the retail sol-ution thread, because the data they produce is not immediately relevant tosolving the solution thread scenarios.

The inventory management software supports the Order and Shipment enti-ties, as follows:


Entity Definition

Order The software matches current inventory against pre-determined order points for each product and generatesorders. These orders are transmitted to the large server aspart of end-of-day in-store processing. These orders may begrouped by item supplier. However, store or corporate per-sonnel may decide to source particular items from one of theirwarehouse depots or from a different store.

Shipment This entity is used to account for received merchandisethrough adjustments applied to overall item inventory at boththe corporate and store levels. Therefore, all shipments sincethe last processing period must also be transmitted to thelarge server.

In addition to these two entities, information from both the current item fileand the transaction log must be sent to the large server. Because both thestore and corporate staff are interested in changes to these two files from theprevious processing period, some programs have been written to extractchanged information. Some of these programs run on the in-store processorand create reports for store personnel.

Large Server Data Activity: Most of the processing that occurs at night onthe large server is vital to the enterprise′s survival. Programs that helpmaintain cash flow management perform the following functions:

• Transmit checks and debit card information to the bank clearing housefor nightly reconciliation

• Process information from bank clearing houses such as payments fromcustomers using a department store credit card

• Transmit credit card purchase information to credit agencies• Transmit coupon data to coupon agencies to obtain reimbursement• Process department store credit card transactions to keep the individual

statements for customers up to date• Transmit payments to suppliers based on processed invoices• Transmit purchase orders to suppliers.

Most of these programs require data from the POS transaction logs origi-nating in many stores. The large amount of data that must be transmitted forjust one day′s processing precludes sending the entire transaction log anditem file. Extract programs run on the in-store processor send a subset ofboth files to the large server. These large server processes require addi-tional processing of both files, as follows:

Time stamping The data may typically be stored in IMS and VSAM files.An index is used to preserve the uniqueness and integ-rity of data. A time stamp is a key attribute of the index.

Reconciliation Interstore transfers, done under the control of personnelfrom both stores, must be checked against the inventoryof both stores (their respective item files). Also, trans-actions must be checked to ensure that they are notbeing double processed; that is, a transaction transmittedfor the current processing period should not have beenprocessed in a previous processing period.


Figure 7 on page 27 includes data entities that correspond to the productand invoice business entities. These entities and their processing requirecreation of additional tables, as follows:

Product To simplify the sales reporting process, many items may be cate-gorized under a single product name. For example, a shirt withlong sleeves of a given brand and style may have color and sizevariations, all of which are grouped under one product name andnumber.

Invoice This table is used to keep track of purchases made by customersusing a department store credit card. Data in this table is usedduring month-end processing to generate and mail a statement toeach customer. Customers pay this statement at their bank or bymail. Invoice is not used to track invoices from suppliers.

The information systems department recognizes specific difficulties of ad hocaccess to IMS data. It decided to add some additional processing at the endof the batch window to extract the day′s records to a sequential file. Thissequential file serves as input to spreadsheet programs. The extracts arereplicated to the corporate LAN file server where a process is initiated toimport the records into a new spreadsheet file. Users select the variousspreadsheets based on a file naming convention. Periodically, olderspreadsheets must be purged to avoid filling up the server′s disk space.Knowledge workers wanting data from the purged files must return to theIMS source. A selected subset of the spreadsheet files is also transmitted tosome store LANs using replicator services of the LAN management software.

2.6 Key Systems

Move data tothe know-ledge worker

The retail industry information systems evolved from the bottom up and thetop down. That is, the bottom of the technology network has POS registers.POS registers are descendents of mechanical cash registers, which havebeen a fundamental part of the retail industry since their inception. The topof the technology network in the retail industry contains large servers, whichwere used for traditional administrative functions. Large server processingwas performed independent of evolving POS systems. Because of thisdichotomous evolution, the key systems in the retail industry are discussedin terms of store and corporate systems.

2.6.1 Store Systems

POS andlarge serverconverged inthe retailindustry

The POS system is the key system in the store environment. The POSsystem originally managed the traditional functions of product management:recording what was sold, when, and at what price, along with other aspectsof direct selling. It is a transactional system, and the results of its activitiesare simple transaction records. The retail industry at large needed toanalyze this information for strategic planning purposes. Therefore, thetransaction data was uploaded to the large server for reconciliation and anal-ysis. It should be noted that recent technology and product developmentsmake it possible for the POS systems to perform some of the functions tradi-tionally performed only on the large server.

Data origi-nates in thePOS system


2.6.2 Corporate Systems

The key corporate systems include the following:

• Sales analysis• Merchandise analysis• Inventory management.

Input data for key systems at the corporate level is generated at the storelevel. The retail enterprise′s infrastructure must provide a vehicle for repli-cating data from the POS system to the large server running key system soft-ware. The replication requirement has both historical and technologicalimplications.

The historical implication recognizes the POS machines as specialized; theywere manufactured for the sole purpose of recording merchandise sales. Atthe time, there was no vision of a client-server function, so that POS systemsdid not even have a communications capability. The economies of scale andthe state of (micro) processor technology dictated that these systemsperform a minimum function. Client-server technology introduced the com-munications capability to the POS system to ease movement of data to thelarge server. Large server processing of POS data is easier with electronicdata replication than with manual copying of data.

History haslimited thePOS role

Advances in microprocessor technology and costs have positioned the POSsystems to perform some of the functions reserved for the large server. The4690 supports the 4690, DOS, and OS/2 operating systems. The result is thatthe 4690 hardware can run informational applications executable on the DOSand OS/2 operating systems in the store. This flexibility makes the 4690 aplatform alternative for some applications in a large retail enterprise andmakes it an all-in-one system for small retail enterprises.

Data volume remains a technology issue. A large retail enterprise willalways have data volumes that surpass the capacity of store-level systems.Volumes associated with historical data in even the small retail enterprisemay exceed that capacity. For these reasons, the large server will continueto have a role in support of key systems. That role is particularly significantfor historical data and trend analysis against historical data or any very largedata volume environment. S/390 Parallel Query Server is a solution to thisdata volume issue.

Technologyadvanceshave widenedthe POS role

Sales analysis incorporates reconciliation of data for the purposes of salestracking and history. The primary objective is the analysis of sales data indifferent dimensions. A typical use of sales data is support of sales cam-paigns, wherein the marketing organization defines a specific set of productsto be emphasized in a given time period. The emphasis can come in theform of discounts, rebates, or highlighting in a catalog. Analysis of salesdata helps the business understand the success of the promotion.

Sales analysisshows pastsuccess andfuturestrategy


Merchandise analysis focuses on the sale and physical movement of product,the volume of product sold, and the price at which the product sold. It usesas input the raw data representing what was sold and feeds analyses that“hypothesize” on why product was sold. It is historical in nature and thefundamental source of data for trend analysis. Retail enterprises benefit fromunderstanding how products sell over a long period of time as well as inshort-term campaigns; the range of historical trend analysis is limited only bythe data available.

Merchandiseanalysis tellswhat′s goneout the door

Inventory management functions include distribution, warehouse manage-ment, stocking levels, reorder points, and quantities to order. This keysystem clearly sits in a grey area between the operational systems neededto run the day-to-day business and the informational systems that supportlong-term strategic decisions. The function may be performed at the storelevel or at the corporate level for chain-wide inventory decisions. This func-tion is an example of the distribution of a traditional retail industry functionacross the platforms in the enterprise.

Inventorymanagementis tactical andstrategic inscope


Chapter 3. Retail Industry BusinessRequirements

In this chapter, we use business requirements to describe the needs for dataprocessing solutions to the challenges in the retail industry. These chal-lenges derive from the trends, directions, and pressures of the industry. Weassume that the pressures will persist over a relatively long period of timeand that they are general in nature, rather than specific to an immediatenarrow-focused need. Applications or application systems have been theusual response to specific, line-of-business requirements.

An architec-ture helpssolve stra-tegic prob-lems

In this study, we stipulate that the solution to a strategic pressure must beexecuted in an architected context. This context assumes that multiple appli-cations will be needed over the lifetime of the business pressure. The archi-tecture in this context presents a standard approach for all applicationsdeveloped to meet different aspects of the business pressures. The architec-tures we use are the Retail Application Architecture (RAA) to addressindustry-specific requirements of the retail enterprise, and the InformationWarehouse architecture to address the cross-industry requirements tomanage informational data and applications. These architectures are usedas a guide for developing or acquiring software solutions to the respectiverequirements.

3.1 Value of Information

Businesspressures arestrategic, asare architec-tures

Retail enterprises have a wealth of operational data that documents theirbusiness past. Hidden in these large volumes of data are trends and cyclesof consumer buying habits. The resource requirements to uncover thesefacets of the retail enterprise′s business are a challenge to the informationsystems department. The retail environment demands that the enterpriseuse this data to compete. Trend analysis—the complex analysis of largevolumes of data—is a key to understanding fad-oriented customers andtuning the product supply process to meet their demands.

Trend anal-ysis is key tocompeting

Chapter 3. Retail Industry Business Requirements 33

The successful implementation of trend analysis systems, often involvinglarge detail-oriented relational databases and powerful complex query proc-essing capability, yields value in three areas:

• Precision• Discovery• Business process reengineering.

Precision and discovery result from the analysis of large volumes of data.They help knowledge workers better understand their business and makebetter strategic decisions for the business. This analysis feeds the reengi-neering of retail business processes. This, in turn, leads to new operationaldata for the cycle of data analysis, process evaluation and change, andfurther data analysis.

3.1.1 Precision

Trend anal-ysis yieldsprecision, dis-cover, andbusinessimprovement

The operational segment of the business performs its day-to-day activitiesusing transaction-based data. In contrast, the strategic knowledge-basedsegment of the organization operates on a more intuitive basis. The Informa-tion Warehouse architecture and decision support strategies define ways forsupporting this intuitive approach, using informational rather than operationaldata. Large informational data volumes and high-performance query toolsbring a higher quality of business precision to the intuitive decision makingprocess.

Trend anal-ysis supportsintuitive deci-sion making

Target marketing in the mail-order catalog business offers a good example ofprecision in the retail industry. We assume that the retail enterprise wants tointroduce a new “boutique” catalog of tailored fashions for women. Logictells management that women who are interested in this more expensive lineof products represent less than 20% of their customer base. Lacking theknowledge of who those customers are, the company would send the catalogto their entire mailing list.

With some summary knowledge from a modest decision support database,the retail enterprise can refine the mailing to customers interested in specificprice range products. A more robust database might identify customers whohave previously mail-ordered specific items, improving the likelihood ofsuccess even further. This target marketing minimizes the expense ofsending the catalog to unqualified customers. In this example, that could beup to 80% of the printing and mailing costs of distributing the catalog to theentire customer base.

Trend anal-ysis provesout intuition

We now assume that the same precision mechanism is available to thedesigners of the fashions and the company′s inventory buyers. They coulduse the information to assist them in determining most-favored colors andpatterns, demographic tendencies, and seasonal habits. All of these factorscan influence both the fashions offered as well as the catalog pictorial layout.The net result is customizing the appeal to a very focused market niche. Thevalue of this precision is increased order performance within the micro-market. The more pleasing the catalog is to the ladies′ tastes and habits, themore likely they are to buy from the catalog and recommend the catalog toothers.

Trend anal-ysis can helpthe manufac-ture andsupply steps


Finally, the business unit executive could use the information for evaluatingthe potential of the strategy before it is executed. This is clearly better thanrelying on gut feel, staff intuition, or consumer-research sampling. The preci-sion of knowing the actuals, in terms of realistic performance expectations,aids the executive′s management effectiveness throughout the subsequentcampaign.

Trend anal-ysis helpspredict futuresuccess

These retail industry examples of precision value depend on informationaldata derived from operational detail and historical data. These data assetshave been sitting idle in a variety of inaccessible storage media. Data repli-cation products take this source data and replicate it to a platform which sup-ports large data volumes and complex query—for example, S/390 ParallelQuery Server. The data asset is now available for use by the business ana-lysts to improve business processes. The preceding examples illustrate howthe S/390 Parallel Query Server can bring new value to the business byimproving bottom-line margins. The S/390 Parallel Query Server solutionalso contributes soft-dollar results such as happier customers and moreeffective leadership. This all leads to a more proactive business philosophyand operation.

3.1.2 Discovery

Data repli-cationdelivers datafor the benefitof the deci-sion maker

Some business events are unplanned and unexpected; often, management isunaware that the events occurred. Such business phenomena can have apositive or negative impact on the business. In all cases, business analystsdesire earlier awareness of their occurrence. Earlier awareness of any busi-ness event gives management the opportunity to maximize the benefit fromthe positive event and minimize or eliminate the impact of the negative.

Analysis oflarge datavolumesreveal theunpredicted

An airline cash-flow phenomenon is a good example of the use of data todiscover events. The airline industry accounts for cash when a reservationis booked and paid for by credit card or equivalent means. It recognizesrevenue once the reservation converts to a boarding pass and the customeractually takes the flights. In our example, we assume that the cash-flowbalance starts to increase beyond the traditional ratio of cash-to-revenue pat-terns of previous quarters or years. There appears to be about 10% morecash than what would normally be expected.

Discoveryhelps even incash-flowmanagement

At first, management looks at this phenomenon with a surprised, but indif-ferent attitude. As the 10% ratio increase grows to 20% and then 30%, man-agement becomes curious about the business phenomenon that iscontradicting their assumptions of how their business works. At this point,the business analysts are only mildly concerned, the answer is not conven-iently at hand, and the interest is not at the level to motivate investigation.When the ratio exceeds 40%, the situation is elevated to an all-out audit inorder to determine the cause.

Business ana-lysts changetheir viewbased on theinformation


The first inclination might be to suspect some sort of error in the financialsystem. An expensive and disruptive examination of all cash accounting pro-cedures and programs is carried out. However, the audit shows no irregular-ities in the financial system. The knowledge workers′ interest level is nowelevated as they sense the change in their business operations may becaused by change in customer behavior.

Knowledgeworkers gofrom curiosityto denial toconfusion

Now the situation escalates to a management-level crisis. A passengerservice agent detects an interesting new pattern by analyzing a large detail-level database that tracks all frequent flyer travel activities and itineraries.The agent discovers that more customers are booking their reservationsfurther in advance. Drilling down deeper into the detail, the analyst deter-mines that, on average, frequent flyers are now making reservations 7 to 10days sooner than they had in previous years. The reason behind this trendis still a mystery.

Detailed datahelps know-ledgeworkersunderstandhidden trends

Further exploration reveals that the increased ratio is related to a newprogram run by the marketing organization. Several months before the cash-flow picture changed, the marketing organization had announced a new freeupgrade policy for frequent flyers. In response, customers were bookingtheir itineraries further in advance of their flight, hoping to secure betterpositions in a space available list. This was a passenger action pattern thatmarketing had not anticipated.

Drill down ofdetailed datareveals thesecret

The growing awareness of the upgrade program and subsequent advancedreservation actions cause a swelling of cash on hand. Once executive man-agement understands the cause of this new business circumstance, it feelscomfortable capitalizing on it. In this case, management could invest theextra cash in higher-yield financial instruments. However, without the assur-ance of the reason for the cash increase, it would undoubtedly be muchmore reserved. If the reason for the increase in cash-to-revenue ratio weredifferent—for example, an increase in double-booking in anticipation of laborunrest, the airline′s actions would be different.

Awareness ofcause sup-ports action

The example illustrates the value of gaining an understanding of theunknown. Obviously, the sooner the discovery can be made, the soonerprofit-oriented actions can be taken. The accounting department can rallythe cash and the marketing department can encourage even more advancedcustomer reservations—all for improved business gain.

Awarenessmust betimely to bevaluable

Discovery often leads to precision; that is, an S/390 Parallel Query Serveruser might discover a new customer need and then propose a new productor service with targeted promotional materials. For example, retailers withPOS transaction capture can quickly spot new regional consumer habits andfads and then target appropriate promotions. A telephone company can dis-cover new calling patterns and offer packaged services to take advantage ofthem.

Discoveryleads to pre-cision


3.1.3 Business Process Reengineering

Historically, computer automation has focused on the start-to-finish functionsof the business process rather than the individual steps in the process. Abusiness process may encompass multiple activities, each of which has beenautomated independently. One of the principal goals of business processreengineering is to rethink the process and apply appropriate informationtechnologies in a cross-functional manner. Data and quick information flowbecome key integration factors.

Use data tochange theway businessis done

Enterprises intent on business process reengineering are designing architec-tures as a foundation for the effort. Large Information Warehouse implemen-tations containing the operational and informational detail of their corebusiness form the centerpiece of the system. This resource is surroundedby a network that services a variety of client processors. The combination ofMVS, DB2, and S/390 Parallel Query Server provides these emerging data-centered architectures with invaluable support.

Architectureimproves thelikelihood ofsuccess

The reengineering effort of a retail enterprise is an excellent example. At thecenter of the system is a DBMS with terabyte volumes of item-movementdetail. The system is fed each night with the POS transaction files from allstores in the enterprise, with two to three months of history maintained.Every item sold in every store, along with time of day, price, and referencesto other items in the shopping basket, is recorded.

Item detail isthe startingpoint

Initially, this information offers the value of precision to planners and mer-chandise managers. Yet, by also allowing their suppliers to access the infor-mation, executives are redefining the manner in which their business isconducted. With access to this detailed item-movement database, consumer-goods companies can determine the regional inventory levels and offer auto-matic resupply, eliminating the old paper-based and burdensome functions oforder-writing and order-filling.

Merchandisemanagersbenefit first

The consumer goods supplier now has an accurate view of the movement ofparticular products sold in quantity, price, and location terms. The retailerscan then adjust their payment agreement with the supplier, accordingly.Rather than processing brand or order level invoices, the retailer and sup-plier can quickly determine the volume of the supplier′s goods sold during atime period, multiply the totals by the wholesale prices, and calculate appro-priate payment. This reduces massive and costly accounting functions,including invoicing and Accounts Payable, for both parties.

Informationaccesschanges sup-plier andretailer proc-esses

Both partners of the business equation benefit from reduced overhead andshortened times to complete the transactions, a win-win situation. The sup-plier can also use the information to leverage greater precision in item man-ufacturing forecasts, as well as in competitive and promotional analysis. Theretailer is relieved financially from inventory management—it is now the sup-plier ′s responsibility—and can therefore apply greater focus to shelf activityand customer service, another win-win situation.

Both partiesbenefit


The strategic enabler for this new way of doing business is a large, detail-oriented relational database. Broad value in terms of improved efficienciesand effectiveness is observed throughout the selling chain. Recognizing thevalue of their information and data resource, visionary business leaders areusing information today to change the way they do business and the waythey manage. This includes making strategic decisions about their market-place, pricing, quality programs, and resources. These decisions aredependent on the availability of timely and accurate data. The ability toaccess information and act on it quickly will become more and more criticalto an enterprise′s success in the 1990s.

3.2 Solution Thread Requirements

The Informa-tion Ware-houseenablesprocessimprovement

The following business requirements are the major drivers for selecting tech-nology used in the solution thread:

• View of data everywhere (by store and corporate personnel)

• Access to data anywhere (by corporate personnel)

The access must be easy for business analysts who may not be comfort-able with data processing systems. The solution must support ad hocquery.

• Access to data on the in-store processor by store personnel

• Need to access both summary and detail data

• Must keep four years of trend data

• Rapid dissemination of the products of knowledge worker analysis.

Each business requirement is explained in more detail in the sections thatfollow. This linkage will be reinforced as each feature of the solution threadis introduced by associating the original, summarized business requirementwith the proposed feature. The requirements are listed in priority sequencewith respect to their impact on profitability. The priority also relates to thedegree of cost justification anticipated from implementing each requirementat the retail enterprise.

Requirementsdrive tech-nology sol-utions

The solution chosen to meet the requirements must also address the needsof the information systems department. These needs are more oriented todata processing and operational—in support of retail operations—concernsthan the pure business needs identified above. These constraints were asfollows:

• Solution must interoperate across vendors.

• Very large data volumes are involved in current store operations(terabytes (10• bytes) of data).

• The current batch window is 60% loaded.

The batch shift produces the highest processor demand of any shift onthe corporate processing system. As a result of this constraint, discrete

Business anddata proc-essingrequirementsapply


(nightly) full refreshes are unacceptable, and any additional workloadwould have to be offloaded to nonbatch shifts to defer a CPU upgrade.

• Limits on support staff hiring have been imposed across the company.This translates to:

− No store batch operations

Remote operation is essential to manage the workload in the storenetwork. Operators in the corporate data processing center need toremotely manage the in-store processor and control the flow of addi-tional data among the corporate host, in-store processors, and POSterminals.

− Information systems staff is not available for new query development

Staff in the business unit requiring the queries must absorb thisworkload. It may be possible to devote a partial information systemsheadcount who would function as a helpdesk for development ofsuch queries.

− Limited information systems resource to define new informationextracts

It is also expected that business unit staff will be responsible foridentifying new sources of data to incorporate into the retail enter-prise Information Warehouse implementation. The informationsystems staff will then make the necessary changes to share the newinformation and promote its use.

The requirements for the solution thread are discussed below.

3.2.1 View of Data Everywhere

A view of data does not necessarily translate into accessing that data. Thisis the rationale for separating the awareness of data′s existence from theaccessing of data. Accordingly, the knowledge worker works with specificattributes of informational data, as follows:

Business term Terms follow the business terminology adopted for theretail business. IBM ′s Retail Application Architectureis a source for building a business model of the enter-prise, for the terms used in the retail business, and forthose terms added by refinement of the RAA model.

Business description Presents more detail about the business term. Forexample, a business term of Transaction_Type mighthave the following business description:“Identif ies the type of POS transaction. Possiblevalues are cash purchase, credit card purchase, tilladjustment, credit card refund, cash refund, and audittotals.”

Type of information The object is either a column, table, query, report,image, or subject area.

Informationawarenessand informa-tion accessare separateissues


Last Refresh Date Documents the data′s currency, when it was lastupdated.

Where Used Identifies the queries, reports, tables, or other objectsin which the data of interest is used.

Source Identifies the location—for example, a corporate infor-mational database, or an in-store processor at locationX—from which the object originates.

Derivation Describes the enhancements—for example, aggre-gation, subsetting, and calculation of newvalues—applied to the data. The data could be anunmodified copy of the original operational data.

Steward of the data Identifies a person to be contacted to gain access tothe data.

Query/report/chart run timeRelates specifically to queries on detailed data where itwould help to know how long a particular query, report,or other “query object” would take to run.

Note that the decision to allow a query to be started from such a view ofenterprise information is a matter of what can be feasibly implemented andsupported using current technology. There is still a large benefit to the retailenterprise in being able to know that an information item exists even if itcannot be accessed using conventional means. More discussion of thisissue can be found in Chapter 8, “Information Catalog” on page 101.

3.2.2 Access to Data Everywhere

Just beingaware ofinformationaldata hasvalue

Knowledge workers would like to access certain information regardless ofwhere it is stored in the enterprise. The current sources of this data may bestored on different platforms and in different formats. Knowledge administra-tors must make a decision about how and where the new information is to belocated. The location, however, need not be an inhibitor to the knowledgeworkers ′ accessing of the information.

Knowledgeworkersdemand infor-mation,regardless oflocation

Performance is a major criterion in the data location decision. This criterionis influenced by the data volume; the detailed data—real-time data—sourcefor trend analysis might involve gigabytes or terabytes of operational data.The information must first be isolated from access by mission-critical trans-actions supporting the day-to-day business. That is, there should be nodirect access to point-of-sale data.

Data locationdepends onresourcedemand andcapacity

In a corporate-centric retail enterprise, it is advantageous to reuse existingnetwork and systems capability to transform operational data into informa-tional data on the large server. Centrally created informational data could becopied to regional locations (store networks), depending on the volume ofthe informational data created, its expected refresh activity, and the antic-ipated frequency of access.

Data repli-cationdepends ondata volume,refresh, andaccess fre-quency


3.2.2.1 Ease of Use

Details of data access on different platforms should be masked by the dataaccess tool. Merging of data from different platforms should also not be acomplex task for the knowledge worker. The lowest level of detail requiredof the knowledge worker should be SQL syntax. When possible, even thislevel of detail should be hidden using easy-to-use tools.

3.2.2.2 Ad Hoc Query Capability

Minimize thetechnologyknowledgeneeded

Knowledge workers should be able to take an existing query, identifiedthrough the view of all enterprise information, and modify it to suit their anal-ysis needs. They should then be able to execute the new query. Finally,there should be some mechanism for storing useful queries and allowingthese queries to be incorporated into the enterprise view of information.Storing the queries is also desirable for follow-on analytical results of theinitial query (for example, reports, charts, and graphs).

3.2.3 Access to Local In-Store Processor Data

Manage theAd Hoc Queryenvironment

Store personnel concerned with improving daily operations can use ad hocquery of in-store processor data sources to do so. They rarely need toaccess enterprisewide information because that level of information tends tohave strategic value, rather than the tactical value they require. For thosecases in which enterprise-wide information is of interest, store personnelrequire specialized software tools to access it. The tools supplying theirview of all enterprise information should support the use of existing or cre-ation of new queries.

3.2.4 Access Summary and Detail Data

Considerawarenessand access oflocal ANDenterprisedata

Summary data is derived or aggregated from detailed informational data,usually by time period (for example, day, month, or quarter) for trend anal-ysis. Detailed data includes data from individual point-of-sale transactionsand text descriptions of the products sold. This version of data is called rec-onciled data in the Information Warehouse architecture. Easy access tothese two categories of informational data in a single query increases thevalue of the data.

Make derivedand recon-ciled dataavailable

Reconciled data can be transformed from existing operational data with avariety of software. The software can be purchased and used as is, pur-chased with exits used to perform reconciliation, or custom built. The recon-ciled data often is the source for deriving the summary data. Thissummarized version of information is called derived data in the InformationWarehouse architecture.

Reconcileddata is thesource forderived data


3.2.5 Four Years of Trend Data

Four years is used in the solution thread as the minimum history period forderived data to ensure statistically meaningful trend analysis. The sameguideline for reconciled data has not yet been determined. Reconciled datais therefore being kept for a two-year period; a decision about its value willbe made after evaluating its frequency of access and its perceived useful-ness by the knowledge worker community.

3.2.6 Dissemination of Analysis Results

Establish aguideline forhistorical datakept

The dissemination of informational analysis results is the next key issue inthe overall process of information utilization. This dissemination must berapid to take advantage of the dynamic nature of decision support itself. Forexample, a decision to run a local campaign must be transmitted from thecorporate merchandising business unit using the quickest communicationsvehicle available, usually electronic mail (E-mail). The campaign blueprintwould need to be accompanied by documentation of database changes nec-essary for the item involved in the campaign. Changes to the promotionalitem in the database at the store level may also be needed if the campaigninvolves special discounts based on purchase volume, product brand, orother item attribute. This aspect of the solution thread is not specifically partof the Information Warehouse architecture but must be taken into accountwhen integrating the Information Warehouse solution into the retail cycle.

3.3 Origin of Business Requirements

The following summary of urgent business needs is taken from 2.2,“Challenges” on page 14:

• Reduce the cost of doing business• Attract and retain customers• Improve business processes• Reduce cycle times• Increase profitability from within by:

− Leveraging external resources− Focusing on costs− Emphasizing creative uses of technology− Business process reengineering

• Improve merchandise flow management through:− Quick response: reacting to trends and getting merchandise to the

floor quickly− Electronic data interchange− Effective inventory management− Profit contribution analysis: focus on profit not overall revenue.

The Merchandising and Operations business unit has been targeted for anInformation Warehouse solution (see section 2.5.1, “Target Business Units forSolution” on page 22) to focus on the following business needs:

• Attract and retain customers• Improve business processes

Rapid dissem-ination ofreports ismandatory


• Improve merchandise flow management through the implementation of aquick response merchandising system.

An Information Warehouse solution thread could meet many of the businessneeds listed at the beginning of this section. We chose to focus on three ofthose requirements for several reasons. The value of an Information Ware-house implementation is the ability to take raw operational data and presentit to the knowledge worker in finished form. This suggests the need for anend-to-end methodology, and the requirements chosen are simple enough topresent such a methodology.

End-to-endmethodologyis essentialfor dataaccess

The retail industry is in a saturated and overdeveloped market state.Retailer capacity clearly surpasses the customer demand, so the customerdrives the business. A variety of strong retail channels exist and are verystrong today, including catalog and sales club channels. For these reasons,competing effectively for customers is an urgent need.

Focus on themore urgentrequirements

Business processes will inherently improve as knowledge workers use infor-mation from the Information Warehouse implementation in new and creativeways. We also show what activities may need to be added to the InformationWarehouse solution to develop business processes that cover the entireretail cycle from supply to sales.

The need to attract and retain customers focuses on uses of the informationderived from credit card and EFT transactions. The need to implement aquick response system focuses on the analysis of customer preferences in amore general way. As an additional function under the solution of this need,we show how Information Warehouse technology improves product line deci-sions.

We apply Information Warehouse technology to a fundamental problem in theretail industry: managing the volume of raw data that flows in through thePOS outlets and using it for competitive advantage. Information Warehousetechnology is well suited to this task but provides enough generality to beapplicable in most environments.

3.3.1 Inhibitors to Business Growth

Businessprocess wil limproveacross theboard

The traditional approach to applying data processing technology to businessrequirements is becoming less and less adequate. Typically, requirementsfor data and function were identified and used to construct applications thatmet broad business needs. In this way, technology is being used only toincrease the retail enterprise′s operational efficiency. New market pressuressuch as the following are forcing us to reexamine this approach:

• Faster market analysis and reaction• Rapidly changing criteria for POS data as information• The increased volume of valuable data.

A newapproach toapplying DPtechnology isneeded


3.3.1.1 Faster Market Analysis and Reaction

Competitors who are fast to reach the marketplace with new concept mer-chandise that meets consumer demand realize larger profit margins thanthose who follow. While decreasing product cycle time requires speeding upprocesses at many different points in the retail cycle, analysis of trends andconsumer buying patterns has historically been one of the slower processes.

3.3.1.2 Rapidly changing criteria for POS data as information

Speed meansprofit

Trend analysis has shown its merit in increasing profit. This has driven aneed for gathering more POS data and aggregating that data in unlimitedways—as information—for informational analysis. Different aggregations aredefined by the different fields over which the aggregation is applied (forexample, item number, color, or size) and to which level each is aggregated(for example, department, store, region, or country).

Trend anal-ysis drivesthe gatheringand aggre-gation of data

Static applications were designed around static data models, clearly deficientfor an informational environment. Keeping these applications synchronizedwith the new POS file structures requires extensive program maintenance.In cases where the cost of such maintenance is perceived to outweigh thebenefits, the maintenance is not performed or the additional POS data col-lection is not done. Maintenance already consumes up to 80% of most retailinformation systems department resource, so additional demands on thisresource are difficult to service.

Static applica-tion designand develop-ment is unac-ceptable

The preferred approach is to develop an information delivery system that canproduce different aggregations of the POS data quickly. This approachfocuses on the aggregation process. The objective is to make the processas dynamic and reusable as possible. To the extent possible, the system isa dynamic function that takes the aggregation specifications as an inputparameter and generates the aggregation without requiring recoding orrecompiling. The key is a detailed informational data layer that can serviceknown and unknown aggregation specifications.

3.3.1.3 Increased Volume of Valuable Data

Make theinformationdeliverysystemdynamic

The volume of valuable data resident in the data processing organization andavailable from outside sources is increasing dramatically. A vast amount ofmarket information is being generated by the use of credit cards and bankdebit cards to pay for purchases. Knowledge workers see no limit to thevariety of ways to slice the information, the only limit being the knowledgeworkers ′ imagination. The retail enterprise needs to integrate informationfrom public databases and electronic newsclipping services. This latter cate-gory offers information that is filtered and categorized from worldwidesources. The impact of the volume of data is exacerbated by the speed withwhich data enters the retail environment.

There is nolimit to infor-mation orinformationanalysis


These three inhibitors to growth place a stronger emphasis on deliveringinformation to the desktop than on filtering it through the lens of a businessapplication. The data delivery approach reinforces the evolving role of theinformation systems department as a provider of information. The informa-tion systems department must deliver the information in a flexible manner, indynamically changing forms to meet the changing demands and analyses ofthe knowledge workers.

The key isbeing flexiblein deliveringinformation

Ad hoc analysis is available in a limited form. It depends on the informationsystems department for providing specific extracts of the source data. Addi-tional assistance may be required to create the IMS reports or spreadsheet.The existing maintenance workload to maintain the operational applicationsand other essential services creates a conflict for the information systemsdepartment. The information systems department is forced to create a dataprocessing environment that can support ad hoc analysis without beinginhibited by this resource crunch. Without many of the time constraintsimposed by application development, knowledge workers can spend moretime on validating analysis results and on creating new analyticalapproaches.

3.3.2 Qualifying an Information Warehouse Approach

This section positions the Information Warehouse framework in context withthe larger view of the enterprise′s information technology. The positioningcan be used to apply the Information Warehouse framework concepts acrossindustry sectors.

Ad hoc anal-ysis dependson informa-tion systemsservices

Existing retail information systems meet established business requirements(for example, payroll, inventory, and accounts receivable). These systemsgenerate the operational data that is the source of informational data used ininformational analysis. Many enterprises apply workstation-based decisionsupport tools to extracted POS data to create spreadsheets, reports, andvarious graphical presentation style objects (for example, piecharts and linegraphs).

Operationaldata feedsinformationalanalysis

In focusing on the value of an Information Warehouse implementation to yourbusiness, one of the first tasks is to evaluate the perceived effectiveness ofcurrent decision support. The evaluation should be directed principally atthe users of those systems, with an eye toward soliciting their input onwhere the systems could be improved. The evaluation should also considerhow much time is spent gathering and providing the information and howmuch time is spent utilizing it through informational applications. In somecases, the tools for analysis may be good, but the underlying data is suspector too sparse. Survey information of this nature would then become a part ofthe overall requirements list.

Assesscurrent deci-sion supportvalue


The requirements are demanding and diverse. The methodologies for pro-viding dynamically changing information and informational analysis can beapplication systems developed iteratively to extract, enhance, and load infor-mation into informational data stores. This approach, based on traditionalapplication development, is clearly unsatisfactory for even this relativelysmall set of requirements, let alone the requirements of the enterprise atlarge. The alternative is to adopt an Information Warehouse strategy. Thisstrategy identifies common functions (for example, load) and tools that canperform the function across the scenarios. The strategy also limits the workinvolved in implementing the information delivery process to minorcustomization of a common tool for a particular requirement.

Take a stra-tegicapproach


Chapter 4. The Retail Solution Thread

The retail solution thread focuses on a retail enterprise—a large departmentchain in the retail industry—modeled on generic characteristics of the retailindustry. The examples are designed to be valid for the industry in general,so that the concepts can be applied to other retail market sectors. Smallerenterprises can apply subsets of the concepts as they fit an organizationwithin the large retail enterprise.

The largeenterprise isa model forall retailenterprises

The key objective of the solution thread is to present business concepts thatare applicable to segments throughout the retail industry. This objective isconsistent with, and makes use of, RAA constructs, which are customized tothe individual retail enterprise. This customization is normally performed bysystems or data analysts within the enterprise, or by outside consultants.RAA furnishes the data entities required to populate an Information Ware-house implementation. The same entities can be used as a template forreengineering business processes. Reengineering of business processes,however, is beyond the scope of this solution thread. The department chainnetwork presented is based on common elements found in many departmentstore chains today.

The retail industry solution thread addresses a subset of the issues thatpertain to clients and markets. We have chosen representative examples(see 2.1, “Retail Industry Trends” on page 14) that reflect broad industrytrends. This study begins with business requirements that are relevant tothe retail industry in general. It then proceeds, in progressive steps, to showthe relationship of these business requirements to an actual InformationWarehouse implementation.

RAA appealsacross theretail industry

Figure 10 on page 48 shows a progressive approach to applying informationsystems to retail industry requirements. Business analysts use the top levelof the model as a communications vehicle. The top level provides businessterms for data or process entities easily understood by the analyst anddirectly translatable to business things and events. This top level is directlymapped to a middle layer, which is more detailed than the top layer and mayactually be multiple layers, but it is not as detailed as the bottom layer. Thebottom layer is understood by information systems staff just as the top layeris understood by business analysts and executives. The middle layer, then,is the translation between the business-speak of the top layer and the infor-mation systems-speak of the bottom layer. Against this background of cre-ating a business system, we discuss the role of the DataGuide/2* and S/390Parallel Query Server products.

The RAAmodel is acommuni-cationsvehicle

Chapter 4. The Retail Solution Thread 47

Of equal importance to understanding the roles of these new products is adiscussion of issues found in all phases of creating a retail Information Ware-house implementation.

Figure 10. Retail Industry Study

4.1.1 Information Warehouse Framework in Retail

Analysis software capability must be as dynamic as current events, the eco-nomic climate, fashion trends, and the competition are. That is, the dataprocessing industry has recognized the need for decision support capabilityto support dynamic analysis. The same dynamics apply to the informationpresented to analysts, information that is derived from POS operational data.Accordingly, the process of generating information from operational datamust be dynamic, necessitating a methodology that provides this capability.The ultimate goal is using data for competitive advantage.

Dynamic busi-ness meansdynamic anal-ysis anddynamicinformation

The information managed in the Information Warehouse implementation isnot necessarily for the sole use of the owning enterprise. Just as knowledgeworkers in an enterprise want to make use of information from bulletinboards or electronic data services, it is likely that the enterprise will make itsinformation available to outsiders. Outside suppliers tapping into the sameInformation Warehouse implementation as the owning company increase thevalue of the information and the implementation. The existence of the databegins to drive business process reengineering on its own.

Informationhas valuebeyond theenterprise


These factors (data volumes, dynamic analysis, changing data gathering)dictate a departure from traditional application development to use POS dataeffectively. The new approach is an architecture-driven strategy based onthe Information Warehouse architecture. This strategy economically and effi-ciently supports dynamically changing demands for information and analysesof information.

4.1.2 New Scenarios for Profit

InformationWarehouseframework isthe rightapproach

The industry trends discussed in 2.1, “Retail Industry Trends” on page 14lead to a focus on improving the retail enterprise′s relationship with its cus-tomers. To effect this improvement, we need information about the buyinghabits of shoppers. One source of such information are the transactionrecords created by credit card purchases and EFT. Shoppers′ buying habitscan be analyzed by correlation with their credit card or checking account.This analysis enables both the chain and individual stores to channel mar-keting activities selectively and more efficiently.

The trend:focus on thecustomer

In the remainder of this section, we outline analysis scenarios whoseenablement would cost-justify an Information Warehouse implementation.

The following scenarios are numbered for reference in the discussion thatfollows and later chapters:

1. Differential marketing by preference group

2. Quick market response and follow-up adjustments

3. Sales analysis for promotion negotiations

4. Buying on key product attributes.

These scenarios result in reports that address business requirements whichin turn help the enterprise address the trends in the retail industry.

4.1.2.1 Scenario 1: Differential Marketing by Preference Group

The retail enterprise wants stronger loyalty among its clientele. Oneapproach is to use differential marketing campaigns based on customer pref-erences. Market research has shown that customers are more receptive tomailings personalized to their tastes. Rather than blanketing a particulargeographic area with mailings, selected customers found through analysis ofcredit card and EFT transactions are divided into preference groups. Eachpreference group would then receive coupons and special promotional litera-ture. The groups would vary in terms of the promotions and coupons theyreceive. Afterwards, sales to these particular customers would be analyzedto determine the effectiveness of the campaign and chart new campaigns.Figure 11 on page 50 shows an example of a targeted marketing campaign.

Better infor-mation anal-ysis justifiesthe cost


Figure 11. Selective Marketing Campaign

4.1.2.2 Scenario 2: Quick Market Response and Follow-onAdjustments

Enterprise analysts and managers need to analyze the product mix in the dif-ferent stores, comparing private labels for a certain product type such aswomen ′s hats. Products and labels that do not sell must be quickly identi-fied. Drops in sales for certain products can be noticed earlier and correc-tive measures taken. Possible measures might include a sales campaignwith special discounts or cutbacks on reorders. Such actions resulting fromthe analysis of information can cut warehousing costs.

The same analysts also need to be aware of developing trends in sales ofcertain products. This trend observance might be related to activity at com-petitive stores or current events. Noting trends and correlating them withcausal events must lead to quick supplier orders and distribution to retailoutlets. Subsequent analysis must also prove the reliability of the observedtrend so that corrective measures can be taken (for example, increase insupplier order points, volume purchases, returns to suppliers, discounts, andother special promotions).


4.1.2.3 Scenario 3: Sales Analysis for Promotion Negotiations

Knowledge gained from the analysis of sales before, during, and after pro-motions can be used to gain better leverage in negotiating promotional pro-grams with manufacturers.

4.1.2.4 Scenario 4: Buying on Key Product Attributes

Buyers of women′s blouses, for example, need analysis of a wide range ofdata to assist in buying decisions. At a summary level, they need to seewhich lines are the best sellers and which lines are the poor performers.They also want to know which lines are the high and low performers at aregional level. However, product lines sometimes disappear or are trans-formed from year to year; the buyers thus need to understand product per-formance in terms of product attributes. For example, they may want tounderstand if the good or bad performers for a region have a common attri-bute (for example, natural cotton material or a particular style). This aware-ness contributes to better buying decisions for next year′s lines, even if apopular line was superseded by a new line with unknown characteristics.

They may also want to look more closely at seasonal issues: was thereheavier buying activity for one particular line at a certain time of the year?Was this burst of buying activity out of line with other top performers in theblouse lines? Finally, they might like to see what the blouse looks like to getan overall esthetic feel for a particular line. Perhaps a picture of the blousehas been stored by an image scanner or there is an example in a catalogstored at a particular locale; they would like to know where to look.

A common requirement for the blouse buyer′s analysis is the need to look atsummary data and then drill down to more detailed data for a particularchronological period. Another requirement is to see information on theblouse product lines that turns out to be inaccessible through normal deci-sion support software. For example, nontextual information such as graphicimages of the blouse could be of use to the buyer. The normal decisionsupport software tool might not support presentation of the image, and theenterprise might choose not to invest in the data processing technology tomake presentation of that data possible. In this case, the requirement is toknow where to find the image data, since online access has been ruled out.The solution is described by the Information Warehouse architecture as theInformation Catalog and is implemented by the DataGuide family of products.

Scenario 4 is a prime example of the need for dynamic information, arequirement that holds for all of the scenarios. The dynamic nature of infor-mational data needs is characterized by a reiterative process made up of thefollowing steps:

1. Informational analysis2. New information requirement3. Information generation process.


The underlying data must be capable of supporting many different analyses.Many of the decision support analyses and the information required toperform them cannot be predicted at the time the Information Warehouseimplementaion is initially built. Because informational analysis itself tends togenerate new requirements for information, the processes that deliver dataas information must be flexible.

4.1.3 Requirements Mapping: Summarized →Detailed

In this section, we take the business descriptions of the scenarios for profitand express them in detailed terms, suitable for Information Warehouseimplementation. The detailed requirements for the scenarios are as follows:

• Scenario 1: Differential marketing by preference group

The detailed requirements for Scenario 1 are as follows:

− Access to customer purchase information

Must include all purchases by purchase line item for the credit cardor debit card used. Also, include payments by check based onchecking account number.

− Flexibility to sort purchases by any combination of attributes

− Summarization of information at varying levels

The solution must support summarization of information by user-specified ranges over various attributes. In addition to summariza-tion by month and year, the solution should allow aggregation byquarter, season, and start and end date. It should also support sum-marization by store, geographic region, or chain name if the retailenterprise includes multiple chains.

− Ability to show historical trend for four years

Historical data analysis is necessary to identify changes in buyingpreferences.

− Facility for sharing interim and final analysis results between Mer-chandising and Operations units

Sharing analyses increases synergy between the two groups.

− Online access to public domain information

Public domain information includes competitive informational data-bases, news services, and census data for buyer demographics. Theenterprise uses information from these sources in analysis of theretail enterprise′s own information.

• Scenario 2: Quick market response and follow-on adjustments


− Need to arbitrarily group items in different ways

Grouping may not always follow the static product categorization.

Informationdelivery mustbe flexible


− Need to look at item stock levels and purchase activity at the sametime

Item stock levels at both stores and warehouses are needed.

− Need a summary view of purchase activity

A standard summary by product and item will be appropriate most ofthe time, though analysts occasionally need to adjust summarizationcriteria. An example of different criteria is examining summaries byarbitrary item groupings.

− Analysis must be capable of generating required stock level changesacross all stores

This information is needed for Operations personnel to synchronizequickly with the Distribution unit.

− Need to perform a great deal of what-if analysis at all levels (bothdetailed and summary information)

− Need to perform financial analysis in different forms

The different forms focus on particular items or item groups in a pro-posed campaign to improve sales.

− Need to share derived logistics for a sales campaign with store man-agers

− Need to understand the origins of the baseline data

Of particular interest is knowing whether adjustments have beenapplied to historical information and the nature of those adjustments.

• Scenario 3: Sales analysis for promotion negotiations


− Must be able to analyze sales quickly by arbitrarily selected timeperiods

− Need to eliminate seasonal purchases background for a product oritem from the purchases that can be directly attributed to the salescampaign

− Must be able to share sales analysis information with store man-agers

− Need to perform multiple what-if financial analysis using previoussales as a model

Selection of type of financial analysis should be open-ended. In thiscase, the Operations personnel need to determine promotion condi-tions that will guarantee profit for the company, the manufacturer,and the supplier.

− Capability to relate analysis results to follow-on actions

Examples of actions might include orders to suppliers, returns tosuppliers, redistribution of goods between store and warehouse,price changes made to store item files, and finally, addition of termsand conditions to store promotion databases.


• Scenario 4: Buying on key product attributes


− Exception reporting

Analysts need to have exception reporting by product lines or otherattributes. This could be implemented as highlighting extraordinaryresults in existing reports or generating separate reports for thoseresults.

− Summarization at user-specified levels

For example, the buyers would like to see the detailed sales by size,color, and style for a given region, once they have identified thatregion as their most profitable one. They are interested in the sixhottest selling product lines in March and April over the past threeyears.

− Location of nontextual information

This might be as simple as knowing that the manufacturer′s catalogfor a certain blouse style is in the local catalog library or the generalcatalog library. It may be as complicated as giving the executioninstructions for a graphical program to view the scanned image ofthe blouse at the buyer′s workstation.

− Reporting on product line status

Analysts need to know which product lines have been withdrawn,replaced, obsoleted, or undergone a name or description change.

− Access to four years of information

Analysts need access to at least four years of historical informationto get a statistically accurate perspective of trends.

− Access to item and purchase information in a single report

Would like to join various attributes from each information source; forexample, item, purchase transactions, and summary information.


Part 3. The Technology View

Chapter 5. Retail Industry Architecture . . . . . . . . . . . . . . . . . . . . . . 575.1 Retail Application Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575.2 The Store Logical Data Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

5.2.1 The IBM In-Store Processing Strategy . . . . . . . . . . . . . . . . . . . . . 645.2.2 The Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 655.2.3 SLDM Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

Chapter 6. Information Warehouse Framework . . . . . . . . . . . . . . . . . 676.1 Value of the Information Warehouse Framework . . . . . . . . . . . . . . . . . 686.2 Why Data Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

6.2.1 Operational Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 696.2.2 Database Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 706.2.3 Cost of Data Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 706.2.4 Historical Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 716.2.5 Ownership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 716.2.6 Point-in-Time Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 716.2.7 Reconciliation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

6.3 The Information Warehouse Architecture . . . . . . . . . . . . . . . . . . . . . 716.4 Using the Information Warehouse Architecture . . . . . . . . . . . . . . . . . . 736.5 Access Enablers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

6.5.1 Embedded SQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 766.5.2 SQL Call Level Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 766.5.3 Distributed Relational Database Architecture . . . . . . . . . . . . . . . . . 77

6.6 The Retail Industry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 776.7 Information Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

6.7.1 Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 806.7.2 Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

6.8 Information Warehouse Architecture Products . . . . . . . . . . . . . . . . . . 816.8.1 The DataGuide Family . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 816.8.2 S/390 Parallel Query Server . . . . . . . . . . . . . . . . . . . . . . . . . . . 826.8.3 DataPropagator Relational . . . . . . . . . . . . . . . . . . . . . . . . . . . . 836.8.4 Personal AS/2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

6.9 Why Use the Information Warehouse Architecture . . . . . . . . . . . . . . . . 85

Chapter 7. Organization Asset Data . . . . . . . . . . . . . . . . . . . . . . . . 877.1 The Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 907.2 S/390 Parallel Query Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

7.2.1 Software Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 947.2.2 Information Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 947.2.3 Retail Enterprise Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

7.3 Technical Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 957.3.1 Types of Parallelism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 957.3.2 S/390 Parallel Query Server Design . . . . . . . . . . . . . . . . . . . . . . 977.3.3 Query Splitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 987.3.4 Front-End MVS System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

Part 3. The Technology View 55

Chapter 8. Information Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . 1018.1 Information Catalog Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1028.2 DataGuide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

8.2.1 Basic Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1058.2.2 Knowledge Worker Functions . . . . . . . . . . . . . . . . . . . . . . . . . 1068.2.3 Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1078.2.4 Launch Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1138.2.5 Create Collections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1138.2.6 Display Contact Information . . . . . . . . . . . . . . . . . . . . . . . . . . 1148.2.7 View Current News . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1148.2.8 View Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1158.2.9 Administrator Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1158.2.10 Extending DataGuide/2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1168.2.11 Meta-data Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1208.2.12 DataGuide Data Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1218.2.13 Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

Chapter 9. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127


Chapter 5. Retail Industry Architecture

In Chapter 3, “Retail Industry Business Requirements” on page 33, we pre-sented a justification for architectures to address strategic industry pres-sures. In this chapter, we investigate the Retail Application Architecture(RAA) as an example of such an architecture. We also discuss the StoreLogical Data Model as an underlying model to the architecture. The modelserves as a basis for managing the people, places, things, resources, andactivities in the retail industry business that need to be administered by dataprocessing systems. The overall goal of the architecture is to meet the pres-sures of the industry and minimize the cost of the information systems usedto meet those pressures.

RAA is a stra-tegicapproach tosolving stra-tegic prob-lems

It is doubtful whether the retail industry would ever be able to reduce thecost of its information systems unless a broad set of information models andstandards are accepted, by organizations and software vendors alike, as abasis for industry applications. IBM has developed the Retail ApplicationArchitecture as a common ground for the retail industry. RAA consists of aset of software architecture guidelines, interfaces, methods, and tools forretail applications.

5.1 Retail Application Architecture

An architec-ture helpsmeet busi-ness pres-sures atminimal cost

A retail enterprise′s organization structure (see Figure 2 on page 15) doesnot give a complete understanding of the retail business. Rather, it is thenormal day-to-day activities that define the company. IBM ′s RAA has identi-fied some major activities that are common across many sectors of the retailbusiness. Figure 12 on page 58 shows these business activities. We useconstructs from RAA to describe the business activities and common entitiesused by the retail enterprise. We expect the Information Warehouse solutionto address management of the following major business factors:

• Marketing• Product range• Customers.

RAA′s data model is very useful to the Information Warehouse solutionplanner. The model gives the planner the confidence to proceed with thefirst phase of the Information Warehouse implementation without being con-cerned about major rework for follow-on phases. Specifically, the plannertargets the Merchandising and Operations business units as an initial phase.In the second phase, the planners could target the distribution and human

RAA identifiescommon busi-ness proc-esses

Chapter 5. Retail Industry Architecture 57

resources units. The units in the second phase would introduce new busi-ness data entities. The RAA model, which is unified across the retail enter-prise, contributes to an easier integration of new entities withoutreengineering the Information Warehouse implementation data entitiesdefined during the initial effort.

The RAA can serve as a guide for reengineering business processes. Anenterprise can modify its business processes based on informational anal-ysis facilitated by the Information Warehouse implementation. The Informa-tion Warehouse implementation is not a solution to one specificline-of-business problem. Rather, it is a generic data processinginfrastructure that can be applied across lines of business. The InformationWarehouse implementation takes advantage of the definitions in RAA tosupport the analysis of business operations. The analysis leads to reengi-neering of the business processes that generated the informational dataunderlying that analysis. The reengineering generates new informationaldata for Information Warehouse analysis and the cycle of analysis, reengi-neering, and informational data generation repeats. Adopting the RAA datamodel today in conjunction with the Information Warehouse infrastructurepositions the enterprise for ongoing process improvement.

Use RAA forreengineeringbusinessprocesses

Figure 12. RAA Business Activities

The tasks of the RAA business activities, shown in Figure 13 on page 62 areas follows:


Manage marketing This process covers marketing analysis and cam-paign formulation.

Manage product range This process covers the selection of range of pro-ducts to be carried, inventory control, buying andcontracting, ordering and pricing.

Manage customers Addresses all aspects of building a long-termrelationship with customers, for example, creatingan interest in the products that the company offersby direct mail and managing credit available to cus-tomers.

Manage sales Covers all aspects of selling goods and services tothe customer, from assisting the customer in findingthe goods sought to taking payment.

Manage corporate goals and plansGeneral management; long- and medium-term plan-ning, reporting, analysis, organization, and strategyof the enterprise. Defines instructions that becomeinput to other processes, for example, setting objec-tives. Receives performance reports and resourcerequirements from other processes.

Handle products All aspects of handling goods and preparing themfor sale, from initial receipt to time of sale.

Manage personnel Human resource management including benefits,career, training, and personnel statistics.

Manage finance and legal mattersAll aspects of financial, accounting, tax, and legalmatters, including physical handling of cash.

Manage fixed assets Procurement and maintenance of assets owned orused by the enterprise, such as buildings, equip-ment, and insurance.

P. Stecher (see Building Business and Application Systems with the RetailApplication Architecture) groups entities used by each major business activ-ities into a construct called a resource. Resources in the RAA model arespecial kinds of entities that are “consumed by the enterprise in the fulfill-ment of its objectives.” The major retail resources are:

• Market• Product• Client• Personnel• Finance and legal matters• Fixed assets.


The retail cycle is divided into the supply and sales phases. The supplyphase consists of selection of vendors and product range, pricing, vendornegotiations, ordering products, and receiving and distributing products. Theretail supply phase is covered in RAA by the Manage Product Range andHandle Products business processes. The sales phase is covered in RAA bythe Manage Sales and Manage Customers business processes. ManageMarketing relates to both phases.

In the course of drawing on expertise in the retail industry to create RAA, itwas found that there was a definite order of importance of resources:

1. Product2. Client3. Market.

The Market resource includes suppliers from whom a retailer buys andclients to whom they sell. In the context of the Market resource, clients aretreated as an amorphous whole; in the Client resource, you are dealing witha known person.

The retailcycle has aSupply and aSales phase

The scenarios outlined in Chapter 4, “The Retail Solution Thread” onpage 47 were derived primarily from business requirements driven by retailindustry trends. We map those scenarios to RAA so that we can understandthe business entities, activities, and processes involved. The scenarios usea subset of the business entities in the enterprise, so this is a focusing exer-cise with respect to the RAA model. They view the enterprise with respectto business activities rather than business units to emphasize what happensrather than where it happens. Finally, the scenarios identify the overlapbetween the targeted business processes and the entities required to popu-late our informational system—the Information Warehouse implementation.Table 4 on page 61 shows the relationship between the scenarios, businessactivities, and resources used.

RAA helps usunderstandour business


Table 4. Relationship between Scenarios and RAA Constructs

Scen. Scenario Description RAA BusinessActivity

RAA ResourceUsed

1

Differential marketing bypreference group

• Managecustomers

• Managemarketing

• Client• Product

2

Quick market response andfollow-on adjustments

• Manageproductrange

• Managemarketing

• Market(both clientand sup-pliersubsets)

• Product

3

Sales analysis for pro-motion negotiations


• Managemarketing

• Market(both clientand sup-pliersubsets)

• Product

4

Buying on key product attri-butes


• Market(clientsubsetonly)

• Product

The scenarios are directed at understanding the business systems and Infor-mation Warehouse resources that are involved in populating the informa-tional system. The Information Warehouse architecture provides a long-termplan for defining the implementation. The Information Catalog facilitates end-user awareness of Information Warehouse resources. Its workflow manage-ment component manages the complexity of business processes as they areimplemented in data processing activities.

Figure 13 on page 62 reinforces this concept relationship. The top level ofthe diagram depicts business systems defined in business terminology; RAAprovides the definitions. The bottom of the diagram depicts the businesssystems in data processing terminology. The Information catalog bridgesthese two domains by maintaining a relationship between businessterms—meta-data—and their data processing equivalents.


Figure 13. The Information Warehouse Architecture and Information ProcessingArchitecture

RAA defines business systems in business terms. It therefore serves as auseful discussion vehicle for managers to understand the business they arein. Eventually, a business system documented in business terminology mustundergo translation to a more technical language that will define how theunderlying data processing applications will be built. The Information Ware-house architecture provides some key mapping of business terms to tech-nology terms in an open approach that is oriented to addressing keyInformation Warehouse challenges.

RAA definesbusinesssystems inbusinessterms


5.2 The Store Logical Data Model

Today′s in-store processing environments are complex, wherein POS data isstored in both keyed files and sequential files on an IBM 4690. Employeeand customer data is stored in a relational database on an in-store processorsuch as DB2/6000* on the IBM RISC System/6000. Applications such asinventory management reside on a Personal System/2 running DOS with datastored in ASCII flat files.

One problem with today′s in-store processing environment is data dupli-cation. In the example, a store may have multiple applications that requirefiles containing similar data. For example, there may be a POS applicationfrom source A and an Electronic Shelf Label application from source B. Eachrequires an item price file. This creates a problem when the item price datamust be modified. Updates to two different files are required to change theprice of a single item. Executive decision making is affected if the files arenot synchronized. Decisions may be made on incorrect, out-of-date data.Ideally, in this situation, a single database in the store would be shared bythe different applications. This eliminates the need to maintain the samedata in different places and formats. Executives are thus assured that theyare making decisions based on up-to-date, accurate data.

Adding new applications that require additional data can impact existingapplications. This is generally true when applications depend on a particularfile structure and a new application needs to modify that structure. Thisproblem can be resolved if application designers are given the ability to addnew data fields to a common data format without impacting applications thatdo not utilize the new data field.

Another problem inherent in the in-store processing environment is the lackof consistent business rules used across applications. Default reports gener-ated by the existing systems ignore this inconsistency and produce incorrectinformation. In many cases, the applications must be modified to meet therequirements of the organization as well as to provide additional data.Knowledge workers need a more flexible way of analyzing the data withoutimpacting the applications that gather it.

Data storage and access is also a problem for some applications. Generally,applications use physical file access methods to store and retrieve data.This presents a problem when differences exist between operating systemsunder which the applications execute. For example, the 4680 operatingsystem has a keyed file access method that is not available on some otheroperating systems. This problem could be reduced if a standard method forstoring and accessing data were available.

The Store Logical Data Model (SLDM) is a collection of common data objectsthat support retail applications in a store environment. You can use theSLDM to create a customized database design that will be tailored to theunique data needs of the store information processing environment. TheSLDM supports in-store processing applications by helping to:

• Identify the data objects needed by in-store processing applications

• Customize the data types

• Incorporate information that is unique to your applications.


These three steps result in a physical database definition tailored for theenvironment.

5.2.1 The IBM In-Store Processing Strategy

Figure 14 illustrates the IBM In-Store Processing strategy for applicationdevelopment. SLDM, represented in the diagram, is a tool used to defineand identify common data definitions based on a set of using applications.The resulting database implementation will enable sharing of this commondata. To integrate a new application into this environment, the external datais compared to the existing entities in SLDM and the application externalinterface is modified to use the common data. The sections that followprovide more information on SLDM.

Figure 14. In-Store Processing Application Development Strategy


5.2.2 The Model

SLDM is a collection of common data objects useful in developing storeapplications. Disparate applications can share data through the use of thesedata objects in the SLDM. SLDM defines real world objects, the character-istics of those objects, and the relationships between the objects. The SLDMallows application designers to select those objects relevant to applicationdevelopment in their unique store environment. Through use of SLDM andthe KnowledgeWare Application Development Workbench** (ADW) tool, aunique database definition is generated based on specific customer-definedapplication data requirements.

In the past, attempts have been made to create a single physical databasedefinition for use in application development in the retail environment. Thisapproach provided little flexibility in the development of future applications.Also, a physical database design does not take into account the data require-ments necessary for two different enterprises. For example, one enterprisemay use a social security number to uniquely identify each employee, whileanother may use a corporate-defined serial number. In addition, anychanges necessary to develop new applications require all previously devel-oped applications to change.

These problems are solved with the SLDM. Additional applications areadded by identifying new information requirements and incorporating theminto the existing definition through the use of SLDM and the ADW tool.Figure 15 on page 66 illustrates the relationship between data modeling anddatabase design.


Figure 15. SLDM Model and Dat abase

In addition, SLDM is used as an implementation methodology by IBM andIBM business partners to assist with the development of a suite of in-storeapplications that share data. SLDM can be used in conjunction with, or inde-pendent of, RAA.

5.2.3 SLDM Benefits

The implementation of the SLDM provides a means of consolidating data.Duplicate and inconsistent data is eliminated. As a result, executive deci-sions may be made based on consistent data. It provides a common startingpoint and communications mechanism for present as well as future applica-tion development. It helps retail business professionals and informationsystems professionals increase their understanding of each others′ needs.This leads to a more productive application development process and helpsensure that the applications developed by the information systems profes-sionals meet the needs of the business professionals who require data tomake executive decisions. The SLDM provides a stable development envi-ronment, within which applications share consistent, compatible data andmay be reused across market segments and heterogeneous hardware plat-forms.

The SLDM provides input into a user-defined database design. Data is cus-tomized and tuned to meet the needs of your business both now and in thefuture. IBM and its business partners are committed to developing applica-tions based on SLDM along with the RAA enterprise model.


Chapter 6. Information Warehouse Framework

Enterprises have long recognized the opportunities that would be available ifthey could make better use of their data. Data is typically stored in manylocations, in different formats, and managed by products from many differentvendors. It is usually difficult to access and use the data across locationsand vendor products. The Information Warehouse framework is a solution tothis long-standing problem. IBM and other vendors are working to makeaccess to data across vendor products and geographic locations easier byenabling their products to work together. The products and rules by whichthe products can work together constitute a framework. The InformationWarehouse framework is designed to provide open access to data acrossvendor products and hardware platforms.

The frame-work: betteruse of enter-prise data

Chapter 6. Information Warehouse Framework 67

IBM and its business partners have been delivering database, data access,and decision support products which fit into the framework and make it pos-sible for our customers to build effective integrated Information Warehousesolutions.

IBM andvendor sol-utions fit intothe frame-work

Enterprises benefit from the framework because they can increase the valueof the investment that they have in current databases and files. Enterprisescan get to the data that they need to effectively manage their businesses.

The frame-work enablesaccess to allenterprisedata

The Information Warehouse framework of products includes decision supportproducts. These applications can be used by end users to analyze andreport business data from many parts of the enterprise. The InformationWarehouse framework of products gives the end user access to data fromother workstations, LANs, and host databases. The end users spend lesstime gathering and accessing the data, and more time in analysis andreporting of data.

6.1 Value of the Information WarehouseFramework

The Information Warehouse framework is a comprehensive solution havingvalue beyond the simple collection of software products. The following char-acteristics differentiate the Information Warehouse framework from otherapproaches:

• Published architecture

Products that implement the published Information Warehouse architec-ture can work together with consistent user interfaces for easier opera-tion.

• Cross-platform coverage

Database products reside on multiple platforms, and access is enabled todatabase products from a variety of software vendors on platforms from avariety of hardware vendors.

• Architected connectivity

Distributed relational database connectivity is architecture-based. IBMhas products which support this architecture today.

• Vendor support

The Information Warehouse framework has the support of other vendorsin the industry. These vendors are working with IBM to bring a widevariety of software solutions to market.

• Architected systems management

DataHub* is a cornerstone of the Information Warehouse framework. It isan architected solution for systems management, with the intent to coop-erate with systems management products from other vendors.

End usersspend moretime usingdata, lesstime finding it


• Integrated database tools

Tools for database design work with tools for application development,saving customers time in application modeling and development.

We can address the data delivery requirement to generate informational datafrom operational data by either iteratively writing applications to extract,enhance, and load information, or by using the architecture-based Informa-tion Warehouse approach.

6.2 Why Data Replication

There are two approaches to accessing data for informational analysis:accessing the operational data directly and accessing a replication of thedata created by extraction, enhancement, and loading into the informationaldatabase. The consensus favors accessing informational copies of opera-tional data rather than direct access of operational data. Some of the consid-erations leading to this preference are as follows:

• Operational systems• Database technology• Cost of data access• Historical data• Ownership• Point-in-time data• Reconciliation.

6.2.1 Operational Systems

Operational systems manage the day-to-day business activities. As such,they are critical to the ongoing viability of the enterprise. These systemsoften perform at the limit of the hardware and software with which they areimplemented. They are often a key part of customer service, a major factorin the success of an individual retail enterprise in the very competitive retailindustry. Therefore, the ongoing operation of the operational systems is thehighest priority. The reasons for using copy-based informational systemswith respect to protecting existing operational applications fall primarily intothe areas of data accountability and application performance.

Operationalsystemsoperate attechnology′slimit

Data accountability includes security and audit considerations. Operationalsystems are designed from the beginning with a specific user community inmind. The data created and manipulated by those applications are carefullymanaged with respect to who has access to the data. The management isaccomplished through operational policies and security function in the soft-ware. It includes allowing access of specific sets of data by specific users orby certain user group identifiers and associating specific users with thosegroup identifiers. Specifying every combination of user and data is a consid-erable effort, so the group identifier approach is more practical. However,this may result in defining group identifiers for broad groups of users withdiverse profiles to access operational data. At the very least, allowing infor-mational access by knowledge workers would be an incremental burden onthe security administrator. It could possibly be a major burden on the secu-rity software and a complication for the security policies of the enterprise. It

An isolatedcopy is lesscomplicatedto secure


is easier to have a copy of the data in an isolated informational environmentwhere the access security can be handled independent of existing opera-tional systems policies.

Data placement is also an issue in security. Copying data could increase thesecurity exposure to the enterprise; once the copy is made, it is up to thepossessor to manage security. However, controlled copying is more securethan allowing broad groups of users with diverse access profiles to accessoperational data. At the very least, fresh copies of data are controlled.

Data locationimpacts secu-rity

Decision support activity is difficult to predict, but certain characteristics ofthis class of applications are well understood. Decision support queries tendto access large volumes of data; they tend to apply complicated, long-running manipulations against that data; and they may retain claims on thatdata for extended periods of human think time. The queries, then, can beexpected to interfere with transaction systems because of extensive locking,heavy I/O activity, and high demands on CPU and buffer pool resources.Informational analysis against copies of the operational data rather than theoperational data itself prevents this interference.

6.2.2 Database Technology

Performanceof operationalsystems mustbe protected

Mainframe database technology supports a wide range of operational func-tion in terms of concurrent access and data transfer rate. The InformationWarehouse architecture is platform-independent and is compatible withLAN-based operational databases and application strategies. The flexibilityof the mainframe platform makes it an ideal location for data copies. Themainframe platform can support a wide range of data volume and user popu-lations. It also leverages data by being a central location for data to beaccessed across work groups or lines of business.

Database soft-ware andhardwaretechnologyfavors copies

The LAN lags behind the mainframe in I/O interface technology and ability toprocess data in memory. Recoverability, availability, and security functionson the mainframe tend to have fuller function. This has historical reasons aswell as reasons based on the sheer volume of data inherently found on thelarger systems. There are, however, many situations where specific subsetsof data can be processed effectively in the LAN environment.

6.2.3 Cost of Data Access

The LAN plat-form hasvalue forcertain infor-mational uses

Communication costs are reduced and response time improved if a copy iscreated in one or more locations. The general rule is to bring copies of dataas close to the knowledge worker as possible. The store structure in retail isan example of this strategy. The network topology includes LAN-basedbranches and central host databases. Frequently accessed data is accessedin a most cost-effective way on a LAN server rather than running queries onthe host. Factors that influence placement of data copies on the LAN or thelarge server include the cost of data processing (host versus LAN server)and of sending the answer set from the host (LAN communication cost islower).

Bring thedata copy tothe know-ledge worker


6.2.4 Historical Data

Operational systems usually do not allow for historical data analysis, yet thisis a major concern to business analysts.

6.2.5 Ownership

Legacy system history along with security and availability requirements haveplaced data remotely from business activity. Copying data allows for dataplacement closer to knowledge workers responsible for making decisions.Ownership implies identifying an individual who is responsible for the qualityand currency of the informational object.

6.2.6 Point-in-Time Data

Operational data tends to change over time. A point-in-time picture of infor-mation may be necessary for comparisons and understanding trends. Point-in-time information is part of a historical database strategy.

6.2.7 Reconciliation

Reconciliation of operational data is impractical as a real-time operation.Staging of data and data replication techniques are necessary to perform thereconciliation required for informational analysis without impacting the oper-ational environment.

6.3 The Information Warehouse Architecture

The primary goal of the Information Warehouse architecture is to define abasis giving end users and applications easy access to data. The Informa-tion Warehouse architecture (see Figure 16 on page 72) defines a structure,formats, protocols, and interfaces as the basis for implementing InformationWarehouse solutions. This architected approach creates an environmentwherein solutions are leveraged. The leveraging is realized by reusing indi-vidual component solutions across implementations and by integrating off-the-shelf components in those implementations. This is not to say that anInformation Warehouse solution cannot be implemented without the architec-ture. Rather, the architecture contributes to the leveraging of effort andresources.


Figure 16. Information Warehouse Architecture

The long-term goal of the Information Warehouse framework is to provideaccess to data of all types in all stores in any environment, and the architec-ture is designed to accommodate the goal. The Information Warehousearchitecture is open in that the interfaces are published and extensible inthat software tools and data volume can be added without regressing theexisting implementation.

The Information Warehouse architecture defines interfaces, protocols, andformats for accessing information in an Information Warehouse implementa-tion. These interfaces are, as follows, grouped by focus area:

• Informational applications− End-user interface to informational applications

• Information Catalog and its access− Information Catalog API− Import/export interface to the Information Catalog

• Access to data− Embedded SQL API− Callable SQL API− Distributed Relational Database Architecture

• Data replication− Interface to the object handler meta-data− Tool invocation to tool interface− Interface to workflow management− Data staging interface.

The interfaces are identified and described in Information Warehouse Archi-tecture I for the public domain. The Information Warehouse architecture


approach, using a component structure with interfaces defined between thecomponents, and its openness regarding system platforms makes it easierfor an enterprise to implement an Information Warehouse solution on its ownor with the help of software vendors and service providers.

6.4 Using the Information Warehouse Architecture

Figure 17 on page 74 shows the three fundamental components of the Infor-mation Warehouse framework: the architecture, products, and services. Thethree components work together to build a foundation of an extensible, flex-ible, and scalable Information Warehouse implementation. The products andservices are requirements, whereas the architecture is a recommended par-ticipant in the Information Warehouse framework strategy.

InformationWarehouseframework foraccess to data

The products are considered requirements because the Information Ware-house framework is a software solution. The products component encom-passes any software solution to the Information Warehouse frameworkfunction requirement, not just purchased software. Off-the-shelf software hasits advantages in speed of implementation but costs real money and mayrequire some investment in resources to customize and integrate into theenterprise′s environment.

Off-the-shelfsoftware forproductivity

The services component refers to resources expended by people, be theyenterprise knowledge administrators or personnel hired from outside theenterprise. In either case, people must do the work of designing, devel-oping, and implementing software solutions.

Services:implementingthe solution

The Information Warehouse architecture is a structured approach for buildinga solution. Information Warehouse implementations built without the archi-tecture will solve a problem, but they may not be easily extended. Thearchitecture-based, leveraged approach allows for reuse of software forsimilar functions across lines of business or specific requests. It allows theincremental addition of new function without disruption to the existing imple-mentation. It also allows the growth in usage and data volumes without dis-ruption of the existing implementation. The Information Warehousearchitecture-based implementation is the recommended approach, though itis not the only approach.

The architec-ture for a flex-ible solution


Figure 17. Information Warehouse Framework

6.5 Access Enablers

Access enablers is the layer in the Information Warehouse architecturebetween the application and the data (see Figure 18 on page 75). The dataincludes meta-data as well as real-time, changed, reconciled, and deriveddata. In the Information Warehouse Architecture I, the focus is on informa-tional applications using SQL. These applications access relational data-bases locally with SQL or remotely with SQL and for example, DRDA. Thevalue in using SQL and DRDA is that the application uses the same dataaccess language to access local or remote relational databases. Nonrela-tional databases can be accessed by using SQL mappers in the implementa-tion of the access enablers layer.

Accessenablersconnect appli-cations anddata


Figure 18. Access Enablers

The four focus areas of Information Warehouse Architecture I limit the dis-cussion of the access enablers to the SQL application program interface andthe interface to the Information Catalog. The SQL API is discussed in thissection; the interface to the Information Catalog is covered in section 8.2.13,“Interfaces” on page 123. The concepts of enabling products and deployingproducts are key to understanding the use of the access enabler layer APIsand interfaces:

Enabling An enabling product is a software program that accepts thecommands defined in the API or interface and executes theirdefined function against a resource. DB2 is an enabling productfor the SQL API, and the DataGuide products are enabling pro-ducts for the interface to the Information Catalog.

Deploying A deploying product is a software program that submitsrequests for data resource in the form of commands defined inthe API or interface. The commands are submitted to the ena-bling product for execution. Visualizer is a deploying product ofthe SQL API, and the DataGuide knowledge worker end-userinterface is a deploying product for the interface to the Informa-tion Catalog.

An informational application could include commands defined in the interfaceto the Information Catalog and would become a deployer of both the interfaceto the Information Catalog and the SQL API. The advantage of this approachis that the knowledge worker continues to use the familiar environment ofthe informational application. The informational application would beenhanced by using business terminology stored in DataGuide.

Interfaces areenabled, thendeployed


SQL is used to access the informational and operational data categories andthe interface to the Information Catalog is used to access meta-data. SQL isan industry standard data access language for performing relational oper-ations, normally against data in a relational database. The access enablerslayer also includes the definition of SQL mappers that allow SQL operationsto be executed against nonrelational databases. Three key interfaces areincluded in the Access to Data focus area in Information Warehouse Architec-ture I and are related to the SQL data access language:

• Embedded SQL• Callable SQL (commonly referred to as SQL call level interface)• Distributed Relational Database Architecture.

The embedded SQL interface is an example of an interface taken from thepublic domain and is recognized by several standards bodies. The SQL calllevel interface is not as yet a standard; its prime focus is to enable softwarevendors to market shrink-wrap informational applications. The DistributedRelational Database Architecture was developed by IBM and is gainingrecognition and support from a range of software vendors.

6.5.1 Embedded SQL

Embedded SQL refers to commands included in informational applicationssource code. The informational application must undergo special processingprior to normal compilation. It is this special preprocessing requirementthat has fostered the development of the SQL call level interface.

6.5.2 SQL Call Level Interface

The SQL call level interface (CLI) is an alternative mechanism for invokingSQL from programs. The objective of CLI is to provide additional languagecommands (verbs) to extend the function of SQL. The most desired exten-sion to SQL function is support of “shrink-wrap” applications. Softwarevendors would like to market applications utilizing the SQL but have experi-enced difficulty with the preprocessing and BIND requirements of embeddedSQL. The informational applications are targeted for knowledge workers withlittle data processing skill. Requiring knowledge workers to go through theprecompile, compile, and bind steps would diminish the acceptance of theinformational applications by these users. The CLI introduces extensions tothe embedded SQL command set which allow run-time precompile and BIND.The informational application can be used out of the box and does notrequire systems or database administration resource for the SQL portion ofthe application.


6.5.3 Distributed Relational Database Architecture

Distributed Relational Database Architecture (DRDA) is a communicationsvehicle for issuing SQL statements to a remote relational database andreturning the results. The statements are executed against a remote rela-tional database rather than a local database. Though there may be differ-ences in the relational databases, such as the form and content of catalogs,the SQL statements themself normally should not have to be changed toreflect the new location. It is an evolving architecture: DRDA level 2 intro-duces distributed two-phase commit protocols. IBM has developed and pub-lished the DRDA for use by software developers throughout the industry.

The overall objective of the access enablers component is to insulate theapplication from the enterprise data format and location. SQL mappersmanage the mapping from SQL in the applications to nonrelational enterprisedata when necessary. DRDA allows for specification of the location of therelational enterprise data. That is, an application using SQL can executeagainst a local database on that programmable workstation′s DB2/2* data-base. That same application can execute against a relational database onthe LAN server running DB2/2 by moving the database and causing the appli-cation to connect to DB2/2 on the LAN server. That same application canexecute against a relational database on a remote server running any DB2family member by moving the database, using DRDA and causing the appli-cation to connect to the DB2 database on the server.

6.6 The Retail Industry

We have seen all of the requirements that, if met, will make the retail enter-prise more competitive and responsive in its business environment. Thissection uses specific Information Warehouse architecture components andconcepts to address business requirements in the retail industry.

Table 5 presents the logical design points for an Information Warehouseimplementation in the retail industry. The rationale for each design choice isnot explored in great detail within this book, as our intent is to show theapplicability of specific Information Warehouse products. However, adequatetime and resources should be allowed for considering these choices,because the purchase of products and services must be cost-justifiedthrough association with the original business requirements.

Table 5 (Page 1 of 2). Logical Design Points for an Information Warehouse

Business orInformation

SystemsRequirement

Logical Design Point

View of data any-where (store andcorporate)

Use a LAN- or large-server-based dictionaryproduct accessed from the workstation.


Table 5 (Page 2 of 2). Logical Design Points for an Information Warehouse


SystemsRequirement

Logical Design Point

Access to dataanywhere (corpo-rate only)

Use a decision support product that can operate inthe workstation or LAN environment. This productwill need to be able to trigger large queries on thehost or on the LAN database server transparently.

Access to datalocally (store only)

Use PWS-based decision support software that cantransfer data between dictionary product and itself.

View bothsummary anddetail

See Information Warehouse Architecture I for con-figuration possibilities.

Trend data for fouryears

Use a large volume database server (host).

Interoperable sol-ution

Use the open and extensible Information Ware-house architecture.

Terabyte datavolumes

Use specialized hardware and software to maintainacceptable response time.

No full refreshes Propagate source table updates automatically intoinformational tables.

Spread batch workacross the day

Trickle the batch work, reports, and snapshot copy.Use small data volumes, run throughout the day, assystem resources permit.

No store batch Use workflows designed to run under centralcontrol. Also ensure that there is remote operatorcapability so the work on in-store processors canbe controlled by operators on the large server.

No support fornew queries orextracts

• Use business unit staff to create new queries.• Use information systems staff to catalog and

propagate query definitions.• Use business unit staff, working with informa-

tion systems staff, to define new informationsources.

• Use information systems staff to justify, imple-ment, and promote the use of new informationsources.

The logical design points map to the following approach specifications forour Retail Solution Thread (see Table 6 on page 79). Figure 19 on page 84shows the new data processing environment created by the addition of anInformation Warehouse implementation in the retail enterprise. This diagramis an extension of the diagram in Figure 5 on page 22.


Table 6. Physical Design Points for Information Warehouse Implementation


SystemsRequirement

Physical Design Point

View of data any-where (store andcorporate)

Use DataGuide.

Access to dataanywhere (corpo-rate)

Use Personal AS/2* (PAS) with appropriate accessenablers to access LAN databases or S/390 ParallelQuery Server on the large server.

Access to datalocally (store)

Use Query/6000 to access databases on the AIXin-store processor and Visualizer to access data-bases on the store LAN database server.

View summaryand detail

Use DB2 and expand existing batch work to loadthe tables required for query work. Create subsetsof enhanced information from the DB2 databaseand use them to refresh databases on the LAN plat-forms, DB2/2 and DB2/6000.

Trend data for fouryears

Incorporate S/390 Parallel Query Server into theDB2 for MVS environment.

Terabyte datavolumes

Incorporate S/390 Parallel Query Server into theDB2 for MVS environment.

Interoperable sol-ution

Use the open and extensible Information Ware-house architecture.

No full refreshes Use DataPropagator Relational on DB2 and DB2/2platforms. Use DataHub/2 as the base for theDataPropagator Relational product and to supportoccasional full refreshes. Use Fastload** to loaddata into DB2/6000.

Spread batch workacross the day

Because trickle snapshots are medium-sized filetransfers with a small CPU consumption profile,some of this work may be moved outside the off-shift batch window.

No store batch Maintain the workflow logic centrally and providemechanisms and procedures that allow the In-StoreProcessor workload to be moved to host as abackup contingency.

No support fornew extract

Establish dictionary maintenance and data move-ment procedures that define roles and responsibil-ities between business unit and informationsystems staff.


6.7 Information Catalog

The role of the Information Catalog is to help knowledge workers (end users)find out what informational objects are available to them and what that datameans, in terms they understand. Once the informational object has beenidentified, the Information Catalog facilitates the use of informational applica-tions to retrieve and analyze it. It is oriented toward search and display ofmeta-data and the informational object to which the meta-data refers.

6.7.1 Function

InformationCatalog:awareness ofand access toinformation

To facilitate the access to informational objects through the use of meta-data,the Information Catalog provides a mechanism for performing the followingactivities:

• Accessing user-oriented meta-data• Interacting with informational applications• Distributing meta-data over multiple Information Catalogs• Collecting meta-data.

The meta-data collection function accommodates definitional sources dis-persed across heterogeneous meta-data stores and stored in variousformats.

It accessesmeta-dataand interactswith DSStools

Knowledge workers have a broad range of expertise in the use of computersystems. At one end of the spectrum are the users who will simply use theInformation Catalog to locate predefined informational objects (for example,queries and charts). At the other end of the spectrum are users who willbuild their own queries for data analysis and presentation; these users wantto locate and understand the data elements they can use as raw material.The Information Catalog supports this range of user requirements.

6.7.2 Interfaces

It supports aspectrum ofend-user skilllevels

Information Warehouse Architecture I describes two interfaces that aredirectly relevant to Information Catalog requirements: the interface to theInformation Catalog itself, and the export/import interface. Both are openinterfaces, in that their specifications, syntax, formats, and protocols will bepublished for use by software developers. The most immediate need is forthe interface to the meta-data. The Information Catalog′s greatest value is inmaking knowledge workers aware of meta-data in the enterprise. Use of theInformation Catalog interface requires a program on the workstation whichexecutes commands to retrieve meta-data from the Information Catalog. Thisrequirement is addressed as the Information Catalog Browser in the Informa-tion Warehouse architecture and is met by the DataGuide end-user interface.Therefore, IBM has provided the tool that uses the interface and meets therequirement.

Informationcatalog inter-faces helpintegrate thesolution


The export/import interface addresses the issue of moving meta-data fromCASE tools or relational database catalogs. The DataGuide family includesextractors for taking meta-data from DB2 for MVS and DB2/2. A requirementexists for tools utilizing the export/import interface for extracting meta-datafrom other sources. These tools might be written by the information systemsdepartment, by tool vendors, or other services providers. The DataGuidefamily products provide sample C programs for developing extractors forother meta-data sources. The interface is an intermediate data format:DataGuide has a facility to import from this format. The tools for othersources would extract the meta-data and write it to a file in this format.

6.8 Information Warehouse Architecture Products

This section presents products of the Information Warehouse framework. Ineach subsection, a brief description of the product is given with therequirement(s) that it addresses. A reference is then made to the chapterthat describes, in more detail, the product and its implementation within ourInformation Warehouse solution. Before focusing on these products, it wouldbe wise to remember that the configuration of information into detailed andsummary databases is equally as important as the products used to accessthat information.

6.8.1 The DataGuide Family

Export/importinterface forexisting meta-data

DataGuide provides an easy-to-use facility for finding enterprise data usingsearches based on business terminology. It also provides a vehicle forlaunching queries, examining existing reports, and launching any number ofrelated applications using located objects as input. It supports the followingbusiness requirements:

• View of data (store and corporate)

• Access to data (corporate only)

Access is limited to queries and modification of queries stored in deci-sion support tools that can be invoked through a command-line interface.A decision support product such as PAS can be invoked for detailed dataaccess and analysis.

• Easy to use for novice data processing staff

DataGuide provides a GUI; knowledge of SQL is unnecessary.

• Need to access both summary and detail data

DataGuide can invoke various decision support tools and the informa-tional objects defined to those decision support tools. Knowledge admin-istrators are responsible for creating a portfolio of decision support toolsand informational objects—charts and reports—at varying levels of detailand summarization. They are also responsible for organizing the infor-mational objects in a hierarchy for the end user, using DataGuide′sgrouping capability.

For more information on DataGuide, see 8.2, “DataGuide” on page 103.

DataGuidehelps findenterprisedata


6.8.2 S/390 Parallel Query Server

The S/390 Parallel Query Server solution is a package of hardware, software,and services that provides a specialized query engine for rapid resolution ofmedium to complex queries working on large amounts of data. S/390 Par-allel Query Server also solves a data access problem within performanceguidelines favorable to the knowledge worker. The S/390 Parallel QueryServer solution provides an excellent growth path with its wide range of con-figurations, even considering the fast rate of growth expected for detailedhistorical information. S/390 Parallel Query Server′s read-only nature is inperfect harmony with the read-only access envisioned for Information Ware-house requirements.

S/390 Parallel Query Server supports the following business requirements:

• Access to both summary and detail data• Keep four years of trend data• Ability to perform ad hoc query.

S/390 ParallelQuery Serverfor analysis oflarge datavolumes

S/390 Parallel Query Server is a solution for enterprises wanting to interro-gate large amounts of data (for example, an ad hoc query retrieving informa-tion from a large database) in an unpredictable manner. Such interrogationshelp the customer run their business smarter as well as improve their effi-ciency and effectiveness. The information that is gathered as a result of aninquiry would help knowledge workers realize the following business goals:

• Increase market share• Generate additional revenue• Create new demand for existing products and services• Control operating costs• Improve customer satisfaction.

When combined, achievement of the above goals should lead to improvedprofitability for your enterprise.

S/390 ParallelQuery Serveris a special-ized solutionfor largequeries

S/390 Parallel Query Server is a parallel database server that is dedicated toprocessing queries against large amounts of data (for example, historical) inrelational databases. S/390 Parallel Query Server accepts dynamic read-onlySQL queries from decision support tools, such as QMF* or Data Interpreta-tion System, or an enterprise′s own applications. S/390 Parallel QueryServer uses parallel processing to reduce query time and cost, making itpractical to extract data from tables with millions of rows of data, such assales records, medical records, detailed inventory records, or demographicinformation. S/390 Parallel Query Server can solve a new dimension of prob-lems and thereby provide new business value.

S/390 Parallel Query Server plus informational applications provide a com-plete Information Warehouse framework solution. That is, S/390 ParallelQuery Server provides support for the Organization Asset Data component ofthe Information Warehouse architecture. For more information on S/390 Par-allel Query Server, see Chapter 7, “Organization Asset Data” on page 87.

It is a paralleldatabaseserver


6.8.3 DataPropagator Relational

DataPropagator Relational is a data replication product that allows changesmade in selected database sources to be applied to selected databasetargets. The database sources of most interest in the retail enterprise areDB2 for MVS and DB2/2. The database targets of interest are DB2/2 data-bases on the store and corporate LANs.

DataPropagator Relational indirectly supports rapid dissemination of informa-tional objects by making views of data anywhere easier to implement. Itdecreases the load on the batch window by better managing the propagationof operational data changes to corresponding informational databases.DataPropagator Relational supports data currency through update propa-gation, so that full refreshes are avoided. It allows data propagation activityto be executed during online hours that otherwise would be performedduring the batch shift. DataHub, a prerequisite to the DataPropagator Rela-tional product, also has its own tool for refresh propagation between DB2 forMVS and DB2/2 tables. For more information on DataHub andDataPropagator Relational, see Information Warehouse in the FinanceIndustry and appropriate product documentation.

6.8.4 Personal AS/2

Personal AS/2 (PAS) is an informational application for the LAN and work-station environments. The key features for which PAS was chosen by theretail enterprise are its analysis tools and graphical representation abilities.

As such it supports the following business requirements:

• Ability to perform ad hoc query, particularly in the area of more compli-cated analysis.

• View of data everywhere

PAS can be launched by DataGuide/2.

• Access to data everywhere (corporate personnel)

PAS can access remote relational data using DRDA.

• Access to both summary and detailed data.

DataPropagatorRelationalhelps managedistributeddata


Figure 19. Future Retail Enterprise Network: Connectivity and Operating Systems. Also included is placement of relational data stores


6.9 Why Use the Information WarehouseArchitecture

An Information Warehouse solution maps well against business requirementspossessing the following key characteristics:

• Large volumes of data• Dynamic requirements for informational data.

The Information Warehouse implementation must be open and extensible tomeet these requirements. Operational data and software that are not bynature open and extensible must be accommodated by the Information Ware-house solution. The Information Warehouse architecture provides an openand extensible structure that can be used in an Information Warehouseimplementation today.

The IW mustbe open andextensible


Chapter 7. Organization Asset Data

Enterprises consider their data to be a vital asset. For historical and organ-izational reasons, very few enterprises have a master plan for managing theirdata. Data stored in databases and files is identified as data objects by theirdata processing technology name. There is little categorization of dataobjects and no enterprise-level directory documenting the ownership, busi-ness meaning, or relationship to applications of informational objects. Thislack of a data plan has contributed to the difficulty in making the data acces-sible to the business analysts who need it. The data categories of the Infor-mation Warehouse architecture provide a foundation for building a masterplan for the organization asset data.

IW architec-ture bringsorder to thedata chaos

The Information Warehouse architecture defines five categories of data withrespect to how they are used by applications and specifies four configura-tions, or collections of the five categories. Categorization and configurationof enterprise data are the first steps toward developing an enterprise man-agement system for data. DataGuide is a catalog for documenting what datameans from a business point of view. Information Warehouse architecture′sOrganization Asset Data component incorporates the categorization and con-figuration methodology. Figure 20 on page 88 highlights organization assetdata in the Information Warehouse architecture.

The IW archi-tecture is aplan forenterprisedata

Chapter 7. Organization Asset Data 87

Figure 20. Organization Asset Data in the Retail Industry

The organization asset data component consists of two elements: data andmeta-data. The business view of the data is achieved through the modelingprocess, whether it is informal or tool-based. The discussion in Appendix A,“Models and Modeling” on page 129 lays the groundwork for understandingthe meta-data to data relationship. The business analyst understands theenterprise′s business and the business objects and activities that contributeto that business. The model is a translation of that view to the data proc-essing view of the data processing objects and processes that mirror thebusiness objects and activities.

An industrymodel pre-sents thebusiness view

The model is a way of managing the categories of data defined in InformationWarehouse Architecture I and creating a channel of communication betweenbusiness analyst and information systems department staff. The model startsfrom a business perspective and progresses toward the implementation of anequivalent data processing perspective. The data categories in the Informa-tion Warehouse architecture are more geared toward how the data is createdand used by applications. The Information Warehouse architecture helps tomake informational objects available to informational applications by guidingthe extraction and enhancement of operational data to create the informa-tional objects. The business model view provides the business semantics ofthe data. You need both approaches to maintain an understanding of thedata and the management control of the data. The Information Warehousearchitecture data categories are as follows:

• Real-time• Changed

The data cat-egories com-plement theindustrymodel


• Reconciled• Derived• Meta-data.

Real-time data is created and manipulated by operational applications to runthe day-to-day business. Changed data represents the changes that trans-actions make to the operational data. Reconciled data is an informationalcopy of the operational data with basic conversions of codes into meaningfuldescriptions. Reconciled data also carries the resolution of inconsistenciesbetween data stored for different lines of business in the enterprise. Deriveddata contains records that are summarized, aggregated, and transformedfrom (detailed) reconciled data or from the real-time data. Meta-data isdescriptive data about the data in the other categories. It is used by know-ledge workers searching for and trying to understand the data in those othercategories. The meta-data is maintained in the Information Catalog.

We know from our business requirements and information supplied by theinformation systems function that we would need to:

• Access trend data for a four span• Access both summary and detail information• Access large data volumes• Support a large population of knowledge workers working concurrently• Avoid impact on the batch window• Isolate operational data from knowledge workers′ access• Limit informational data access to be read-only.

The volume of trend data necessary to satisfy the new scenarios in 4.1.2,“New Scenarios for Profit” on page 49 is expected to grow over time. Theexisting and new reports need both detailed and summary informational data(reconciled and derived data). The solution would need to accommodate thisgrowth with existing technology. These divergent requirements underscorethe new role of the information systems department, wherein informationsystems is a provider of information. information systems must be in a posi-tion to provide access to detailed, reconciled data or informational dataaggregated at any conceivable level at the request of the knowledge workergroups. It is further responsible for maintaining data currency and ensuringthe correlation between information and its business meaning. Choosing asuite of informational applications to present the informational data is left tothe business group. The business group is free to select any decisionsupport tool that provides the desired function and that utilizes the SQL APIfor data access.

The newinformationsystems role:provider ofinformation

A review of the potential queries indicated that most of the queries were ofmedium complexity, with a significant number of queries fitting into thehighly complex category. The majority of medium to complex queries areexpected to be unstructured; that is, ad hoc in nature and not tuned for per-formance. The lack of structure is caused by decision support products thatremoved the knowledge worker from the formulation of the SQL itself. Thedecision support GUI makes data access easier for the knowledge worker,but it inhibits fine tuning of the queries. If some means could be found tocontrol query structure, the new performance features of DB2 Version 3release 1 running on an ES/9000 platform may be an adequate solution. Therelational database tables accessed by the retail queries have data in themillions-of-rows range.

Consider thevariety ofinformationvolume andquery com-plexity


Both query complexity and data volumes will affect the amount of CPUresources expended to deliver the query result and the response time. Oneof the primary criteria for choosing a dedicated query engine lies in under-standing the complexity of the anticipated queries and the volume of datathat will be processed by the average and maximum query. Figure 21 indi-cates conceptually the interaction of both factors when coupled with theexpected number of concurrent users.

Figure 21. Query Cost as a Function of Query Complexity and Data Volume

7.1 The Solution

All of the above requirements point to a dedicated query solution. The S/390Parallel Query Server query data solution matched up well against theserequirements on the following points:

• Response time

S/390 Parallel Query Server is designed to deliver optimal performancefor queries against large data volumes. The design accommodatesincreases in the population of concurrent users.

• Scalable solution

S/390 Parallel Query Server provides room for growth. Data storage andprocessor capacity can be added as new query workloads are added.

• Packaged solution


The S/390 Parallel Query Server is delivered as a total solution of hard-ware, software, and associated services.

• Complementary technology

The S/390 Parallel Query Server solution complements existingESCON-capable processors at retail enterprises. There is no need forextensive training and revisions to operating procedures. Support canbe provided by established operating system and database specialists.Moreover, most of the system maintenance is packaged into the S/390Parallel Query Server solution.

• Fast batch operations

Loading of the query data is faster when using the high speed ESCONchannels and DB2 3.1 features that support I/O parallelism, more efficientutilization of memory, and data compression.

Figure 22 illustrates the tradeoffs between query complexity and data volumeversus general and special purpose solutions.

Figure 22. Special and General-Purpose Solutions


7.2 S/390 Parallel Query Server

S/390 Parallel Query Server is a specialized hardware and software solutionto the large data volume, complex query data access requirement. It pro-vides access to large volumes of informational data while optimizingresponse time. The initial appearance and subsequent reworking of thatinformation is managed by the informational application used by the know-ledge worker.

S/390 ParallelQuery Serveris a special-ized solution

S/390 Parallel Query Server would be used for storing reconciled and deriveddata. The S/390 Parallel Query Server system is chosen when the datavolume is large enough to require partitioning of the DB2 tables. Responsetime requirements are a key justification for environments where datavolume and query complexity would make general purpose solutions inade-quate. Query splitting is the major facilitator of the specialized solution, andit is assumed that at least one of the largest tables in the query is a parti-tioned table.

The common database and files found at the level of the in-store processorand POS systems are shown in Figure 23. This figure is included here as areminder of the source of the operational data that is the source of informa-tional data copied into S/390 Parallel Query Server. Increasingly, relationaldatabases are used for critical data such as ITEM, facilitating the incorpo-ration of this data into an Information Warehouse implementation.

S/390 ParallelQuery Serveris ideal forderived andreconcileddata

Figure 23. Databases and Files in Retail Enterprise Network


The initial configuration of S/390 Parallel Query Server provides access frommainframe informational application products. We need a mechanism toinvoke queries from the programmable workstation and LAN environments.Note that the LAN environment is the hardware platform for informationalapplications, DataGuide, and administering and launching data replicationtools. Today, we can provide such connectivity using Personal AS/2 Version3 on the programmable workstation, which calls Application System Version3 on the large server to execute a query. Alternatively, Personal AS/2 canuse DDCS/2 and DRDA protocol to access the large server database withoutthe need for host Application System Version 3. The product configurationrequired to support the Application System Version 3 approach is shown inFigure 24. For the remainder of this section, we elaborate on the configura-tion of the specialized query solution using S/390 Parallel Query Server.

Figure 24. Connectivity between Personal AS/2 and S/390 Parallel Query Server


7.2.1 Software Configuration

Access to S/390 Parallel Query Server data is supported through a front endMVS system running DB2 2.3 or 3.1 and access enabler software. Theaccess enabler software for DB2 access must be made known to the Applica-tion System Version 3 code for Application System Version 3 to pass its SQLrequests to S/390 Parallel Query Server. SQL initiated from the large serverApplication System Version 3 product is passed to the S/390 Parallel QueryServer complex depending on the table names used in the query. Dataresults would be passed directly back to the address space that containsApplication System Version 3.

At the workstation, knowledge workers using Personal AS/2 on the program-mable workstation receive resultant data rows through the Server-RequesterProgramming Interface (SRPI). The SRPI session between Personal AS/2and host AS is configured using Communications Manager/2. The answerset for a given query is input to Personal AS/2, then used by the knowledgeworker for further analysis. The final result of the SQL request and the Per-sonal AS/2 manipulation of the data is graphical display of the answer set.

7.2.2 Information Maintenance

To keep the information in the S/390 Parallel Query Server data store current,the S/390 Parallel Query Server system is quiesced at night after all the nec-essary extract and data replication programs have been run to gather data inthe form required to load the informational tables. Quiescing allows existingqueries to complete and prevent any new queries from being started. AllS/390 Parallel Query Server DB2 subsystems then deactivate their access tothe DB2 tables in S/390 Parallel Query Server. ESCON channels from thefront-end MVS system to the database DASD are varied online and the DB2subsystem on this front-end would perform the necessary data loads, reor-ganizations, backups, and updating of statistics. At the end of this mainte-nance cycle, the environment is reversed and DB2 subsystems in the S/390Parallel Query Server complex would regain read-only access to the DB2data.

7.2.3 Retail Enterprise Operations

This mode of maintenance is particularly appropriate to the retail enterprise′sdata processing environment. There is a necessary wait period at the end ofthe business day prior to availability of the POS data for processing. Theretail enterprise desires the S/390 Parallel Query Server system to be avail-able to business analysts until the last possible moment before informationalrefresh. The information systems department devises a method of gatheringupdated POS data, enhancing it, and cleansing it using staged data. Theseprocesses will be performed independent of the S/390 Parallel Query Serverduring a maintenance period, but before the quiesce and load period. Wecannot wait for the S/390 Parallel Query Server to be quiesced before begin-ning all the various extracts. The output of these various extracts must beavailable for immediate loading of the S/390 Parallel Query Server databases,to save time in the batch window.


7.3 Technical Issues

S/390 Parallel Query Server is a rack-mounted query processor that usesspecialized software to split SQL queries into separate job streams. Thequery splitter operates above the SQL API level and is designed to introduceparallelism into queries that must scan large amounts of data. The querysplitter takes the original query and splits it into component queries. Each ofthese component queries executes the original query on a subset of the data.This is accomplished by manipulating range predicates for each componentquery. Answer sets from each component query are then merged into a finalanswer set.

The objective is to convert a large tablespace scan chosen for the originalquery into multiple keyed index scans. We use the word keyed becausethere are rules that include splitting the query if it created full index scans inthe original query. S/390 Parallel Query Server uses a form of parallel proc-essing to enable concurrency. We review the basic concepts of parallelprocessing as a preparation for discussing components of this specializedsolution.

7.3.1 Types of Parallelism

S/390 ParallelQuery Serveris rack-mountedCMOS tech-nology

We discuss two forms of parallelism; I/O parallelism and processingparallelism. Prior to DB2 Version 3.1, most databases performed I/O oper-ations for a single query in a serial mode. Multiple DASD volumes werescanned, in sequence, to satisfy a particular query. This often left theprocessor with idle time while it waited for the longer-running I/O to com-plete. DB2 3.1 introduced query I/O parallelism for queries executing againstdata in partitioned tables and for certain joins. This allowed I/O to be runconcurrently against many different parts of the same table. This results inbetter utilization of the processor, because the data flow rate to theprocessor is increased. This can improve elapsed times by a factor of two orthree. I/O parallelism in DB2 Version 3.1 offers one approach to effectivelyhandle queries running against large volumes of data.

Parallelismcan beapplied to I/Oand theprocessor

The CPU parallelism introduced by S/390 Parallel Query Server goes onestep further and allows concurrent CPU processing to take place for a singlequery. This, coupled with I/O parallelism, allows effective handling ofcomplex queries run against large volumes of data. In Figure 22 onpage 91, the dotted curve shows the positioning of the S/390 Parallel QueryServer solution as a function of the mix of complexity and data volume in theretail environment.

We consider two approaches to implementing CPU parallelism: the parti-tioned approach and the shared approach (as shown in Figure 25 onpage 96). In the partitioned approach, the data is divided across multipleDASD. Each processor in the configuration has active channel connectionsto only one subset of this DASD. In executing a single query, the dispatchingtask will need to understand this hard-wired affinity between processor andDASD in allocating work.

CPUparallelismhelps addressquery com-plexity


Figure 25. Parallel CPU Processing Environments

The partitioned approach has the potential to create bottlenecks if severalqueries wait for work scheduled on a particular subset of DASD. The parti-tioned approach also requires a large amount of maintenance when addi-tional capacity is added. Let us assume the additional capacity is mandatedby growth in the size of the stored tables. To maintain a similar responsetime, typically more DASD must be added when additional processors areadded. This results in a redistribution of the data across the new comple-ment of DASD. Affinities between CPU and DASD must be redefined which inturn may impose application changes.

One variation of this partitioned parallel approach is to spread data from alltables—including the system catalog—randomly across all DASD. Thislargely solves the DASD bottleneck problem, partially solves the redistrib-ution problem, but causes other problems. For example, increasedparallelism is almost always at the expense of greatly reduced concurrency,and a failure of a single DASD or processor may impact the entire system.

If more than one index has been defined on the data, processing of such asecondary index can often result in an uneven distribution of work among thestorage devices. The secondary index is not aligned with the physical orderof the data, and I/O accesses may jump about the DASD volume rather thanproceeding stepwise in a single direction. The partitioned parallel architec-ture is very sensitive to the distribution of workload among the storage


devices. The shared parallel architecture is much less sensitive to this andtherefore handles secondary indices better.

In the shared approach to CPU parallelism, a unique sharing technologyallows all CPU′s to access any DASD volume. There is therefore less poten-tial of a bottleneck on a particular CPU. Adding capacity is also easier,because there is no hard-wired affinity between CPU and DASD. The dis-patcher software in S/390 Parallel Query Server can dynamically adjust itsallocation of workload to use the new CPU. We now look more closely at theS/390 Parallel Query Server solution.

7.3.2 S/390 Parallel Query Server Design

There are several models of the S/390 Parallel Query Server solution. Thebase model has a central electronic complex (CEC) of six central processors(CPs) running a single image of the MVS operating system. DASD isincluded in the complete hardware configuration. Subsequent models addmore CECs to a maximum of eight. The S/390 Parallel Query Server solutionis a package that includes hardware, software, and services. The base soft-ware functions in a S/390 Parallel Query Server CEC are as follows:

Query scheduler The query scheduler is linked to management of thechannel-to-channel (CTC) ESCON connections to thefront-end MVS system. It runs on only one of two CPs ineach CEC complex. The second CP is backup in theevent of primary CP failure. These two CPs have morememory allocated to each.

Query splitter The query splitter decides whether a query should besplit or sent as is to a single CEC-server combination. Italso has primary responsibility for merging answer sets.It can run on only one of the two lead CPs in eachcomplex. However, any complex can run splitter orserver tasks.

Server The server passes on the query to a DB2 subsystem andpasses the individual answer set for split queries back tothe splitter.

Figure 24 on page 93 shows multiple instances of the server code assysplex servers. Together, all the CECs in a S/390 Parallel Query Serverconfiguration function as a sysplex, using the MVS global resource serializa-tion services (GRS) feature to coordinate work among the CECs. Thefront-end MVS system is not part of the GRS ring used for task scheduling.All DASD is shared in read-only mode by every CEC in the sysplex. This isimplemented at the DB2 data level using the Shared Read-Only Data feature.Automation code ships with the product and runs from the front-end DB2subsystem to perform database updates. The automation code is based onNetView and Automated Operations Control (AOC/MVS).


7.3.3 Query Splitting

Query splitting is the process of taking a query whose predicates selectsome answer set, breaking it into multiple queries whose correspondinganswer sets are all disjoint subsets of the original, and constructing that ori-ginal set when UNIONed. Splitting is based on a partitioning key andrequires that original query can be resolved into mutually exclusive subsetqueries. These subset queries access either the constituent physical parti-tions of a partitioned table or the constituent tables that make up a concat-enated table. The concatenated table concept is enabled through accessenabler, not DB2 code. It represents logical partitioning of data, rather thanphysical partitioning of data.

The query splitter code contains logic to optimize the splitting process. Itmakes use of DB2 Explain data and is particularly sensitive to table parti-tioning and JOINs applied to partitioning indices.

7.3.4 Front-End MVS System

Query split-ting is the keyto S/390 Par-allel QueryServer ′s par-allel proc-essing

The front-end MVS system to the S/390 Parallel Query Server solution (seeFigure 24 on page 93) functions as both the sole access point for queries toS/390 Parallel Query Server and the maintenance engine for updating DB2tables in S/390 Parallel Query Server. We have chosen to use Personal AS/2communicating with the front-end system as our means for accessing S/390Parallel Query Server data. LAN decision support that uses TCP/IP con-nections to the front-end MVS will also work. In this case, VTAM must becorrectly configured to translate TCP/IP flows to SNA LU 6.2 flows. LU 6.2data flows over the ESCON CTC adaptor are the only means by which thefront-end system passes data to S/390 Parallel Query Server. Host tools andapplications that today submit SQL queries through the Call Attach, TSOAttach or CICS Attach facilities can link edit or load the S/390 Parallel QueryServer query receiver to send queries to S/390 Parallel Query Server instead.The query receiver sends dynamic, read-only SQL queries to the S/390 Par-allel Query Server for processing. Insert, update, and delete statements andstatic SQL are sent to the front-end DB2 system.

The front-end DB2 system can be either version 2.3 or 3.1. The choice ofversion will impact the efficiency of S/390 Parallel Query Server, whichexploits DB2 3.1 function, as follows:

• I/O parallelism for partitioned table queries

This promises a two- to three-fold reduction in elapsed time for selectedread-only queries.

• Hiperpool exploitation

This promises a 25% reduction in elapsed time for queries.

• Data compression

Data compression could potentially improve query response by reducingthe I/O required to produce the result set.

The front-endMVS is usedfor accessand mainte-nance


• Improved utility operation

Much of the improvements are for utilities directed against partitionedtables (LOAD, REORG, COPY) that will now use I/O parallelism.

Data compression cannot be used by S/390 Parallel Query Server if thefront-end DB2 is Version 2.3 because the front-end system must maintain thedatabases and therefore be able to read and write compressed data. I/Oaccess times for uncompressed data in DB2 2.3 will be longer than the sameaccesses against compressed data under 3.1. Additionally, many of the I/Oparallelism improvements in the standard DB2 utilities are lost if thefront-end system uses DB2 2.3.

The DB2 Version 3.1 data compression feature helps in managing a con-strained batch window. Use of data compression allows a larger amount ofinformational data to be stored on the same physical DASD. For example, inthe maximum S/390 Parallel Query Server Version 1 configuration, 960GB ofDASD can actually store 1.5TB of data. To support data compression, theexisting ES/9000 system would be upgraded to an ES/9000 model 511- or711-based processor.

After database maintenance, DB2 catalogs in each CEC, as well the accessenabler software directory used by the dispatcher and splitter logic must besynchronized. This directory can be changed when the associated accessenabler software is not executing. Changes to the DB2 catalogs are a biteasier and can be handled dynamically by automation tasks without takingdown any DB2 subsystems.


Chapter 8. Information Catalog

The Information Catalog is the centerpiece of the Information Warehouse sol-ution at the retail enterprise. It is the knowledge worker′s entry point to theInformation Warehouse implementation; that is, the ultimate goal of theknowledge worker is to access informational data for informational analysis.The Information Catalog is the first software tool used by the knowledgeworkers to access the informational data. Figure 26 highlights the pieces ofthe Information Catalog as they appear in the Information Warehouse archi-tecture.

The Informa-tion catalog isthe entrypoint to theIW solution

Figure 26. Information Catalog

Chapter 8. Information Catalog 101

8.1 Information Catalog Function

Figure 27 shows a popular analogy for understanding the functions of theInformation Catalog. The card catalog of a library serves as a vehicle fordiscussing those functions. Let′s assume the knowledge workers enter alibrary in search of a specific book. Typically, they do not know the title, buthave a specific or general idea of the subject of the book, or they might knowthe author′s name, or possibly even a keyword from the title. This otherinformation the knowledge worker brings to the card catalog is called meta-data in the context of the Information Warehouse framework. Our knowledgeworkers use this meta-data to find the book they are seeking The cardcatalog might have two different physical sets of drawers: one has cardssorted alphabetically by author, the other by subject keywords. The know-ledge workers find the card having the author′s name or the subject, note anumber for the book by which they can find the book on a shelf, and thenproceed to the shelf to get the book. Today′s libraries contain electronic ver-sions of the old card catalogs, but the search procedure is essentially thesame.

A library cardcatalog is akind of Infor-mationCatalog

Figure 27. A Card Catalog Information Catalog

The Information Catalog works in essentially the same way as a library cardcatalog. Knowledge workers enter business terms as keywords into theInformation Catalog. The Information Catalog uses those business terms to


find entries representing informational objects (for example, reports, queries,and pie charts). The Information Catalog also contains instructions on how toaccess these objects. In some cases, the objects are directly accessiblethrough informational applications invoked by the Information Catalog. Inother cases, the objects are not directly accessible, and the instructionsmight simply be text on how to get the object. An example might be anindustry report purchased as a paper document from an outside vendor. Theentry might have the name and phone number of the librarian at the corpo-rate library; it is up to the knowledge worker to go to the librarian to get thedocument.

8.2 DataGuide

Many of the most important business requirements established in previouschapters are met by IBM′s Information Catalog solution, DataGuide. In thischapter, we discuss some of the new capabilities of the DataGuide family ofproducts. The implementation of DataGuide/2, the programmable work-station member of the DataGuide family, with S/390 Parallel Query Serverextends query availability to multiple platforms with a smaller maintenancecost than with currently available products.

DataGuide

The DataGuide family of products is the IBM solution to the InformationCatalog requirements specified in the Information Warehouse architec-ture.


Knowledge workers desiring access to enterprise data must find it first;DataGuide is the first step in gaining that access. This locate and accesssequence makes the search capability DataGuide′s most important function.DataGuide′s searching function is easy-to-use, powerful, and flexible to moveknowledge workers past the curiosity stage.

Searching isthe mostimportantDataGuide/2function

Knowledge workers can specify search criteria for multiple properties of asingle object type. They can also specify a search criterion for the Nameproperty to search across object types. The requirement for searchingacross object types is derived from the fact that the Name property is theonly property known to the knowledge worker that is common to all objecttypes. If a value is not specified for a selected property, then DataGuidereturns a list of all object instances for the object type. Unqualified searchesreturn all object instances for the object type selected.

DataGuide/2′ssearch capa-bility is pow-erful

DataGuide recognizes that the search request itself is as much an asset asthe informational objects it returns; that is, knowledge workers invest time inconstructing search requests. They often start with broad search criteria andrefine them until they specify the exact informational object of interest. Thissearch criterion may be of value on an ongoing basis to the knowledgeworker who built it. Knowledge workers can save search requests inDataGuide, thereby leveraging the knowledge worker′s effort.

The searchrequest itselfis an asset

Finding informational objects is just a step in the process of decision making.Informational objects can be either physical things, such as a filed paperreport or an electronic report that can be displayed on the workstation.DataGuide offers Contact information for filed paper reports and the launchcapability for electronic reports. Knowledge workers obtain Contact informa-tion for an informational object or launch an informational application throughthe pop-up action window activated for the informational object they havelocated. DataGuide supports the launching of informational applications todisplay informational objects as a function of the knowledge worker end-userinterface.

Launchingapplications isthe goal

We first introduce the structure of DataGuide/2 and some key concepts.These concepts relate directly to the Information Warehouse architecture andthe new way of viewing meta-data defined by this architecture. We thenwork through three specific scenarios from the retail enterprise in the areasof searching for information, administration of DataGuide/2′s structure, andextending DataGuide/2 to support some functions specific to a retail enter-prise. Finally, we look at meta-data maintenance using DataGuide/2 and con-siderations for managing multiple copies of DataGuide/2.

DataGuide:finding infor-mation


8.2.1 Basic Structure

DataGuide is flexible in platform configuration. Table 7 shows the possibleconfigurations for DataGuide.

DataGuide isflexible inconfiguration

Table 7. DataGuide Configurations

Knowledgeworker(DataGuide)

Meta-dataServer

Meta-dataStore

Stand-alone Programmableworkstation(DataGuide/2)

DataGuide/2 DB2/2

LAN 1 Programmableworkstation(DataGuide/2)

DataGuide/2 DB2/2 on LANserver

LAN 2 Programmableworkstation onLAN 1(DataGuide/2)

DataGuide/2 DB2/2 throughLAN bridge

Large server 1 Programmableworkstation(DataGuide/2)

DataGuide/2 DB2 via DRDA

Large server 2 Programmableworkstation(DataGuide/2)

CDF/MVS DB2

Large server 3 3270(DataGuide/MVS)

CDF/MVS DB2

DataGuide provides knowledge workers with easy-to-use functions to locateshared enterprise data, including informational objects. DataGuide/2 pre-sents a GUI for interacting with knowledge workers; this interface approachallows the knowledge worker to find informational objects through the use ofbusiness terms. Once the object is identified, DataGuide/2 can invoke infor-mational applications to retrieve and process the data. DataGuide/2 can alsobe used to provide a GUI to DataGuide/MVS, which uses CDF/MVS as ameta-data store.

DataGuidehelps finddata objects

DataGuide/2 supports the Information Catalog API documented in InformationWarehouse Architecture I. This support gives software vendors and inhouseapplication developer a methodology for manipulating meta-data managed byDataGuide/2. The methodology is based on the use of Information CatalogAPI commands. The DataGuide end-user interface is IBM′s implementationfor this function. The DataGuide/2 administrator uses a separate GUI inter-face to manage data and informational object definitions in DataGuide/2,together with a set of utilities to extract meta-data from various sources.

DataGuide/2supports anopen inter-face


There are two distinct uses of the term object: one is defined by thedesigners of GUIs, the other by object-oriented language designers (forexample, C+ + ). Typical examples of GUI objects are container objects; thefolder is a specific implementation of a container object. For example, DataObjects is a folder containing the objects of type text, graphics, or image.Typical object-oriented language concepts are objects, object classes, classhierarchies, and the associated behavior of these constructs. GUI termi-nology is used in the discussion of DataGuide.

DataGuide/2uses GUI

DataGuide/2 is discussed in terms of two user communities: end users(knowledge workers) who use DataGuide/2 to view meta-data, and (know-ledge) administrators who are responsible for populating, maintaining, andextending DataGuide/2 in response to knowledge worker demand. We beginwith the knowledge worker′s view of the product and then proceed to theadministrative issues.

8.2.2 Knowledge Worker Functions

Knowledgeworkers andadministratorsuseDataGuide/2

Knowledge workers use DataGuide to locate enterprise data, using business-oriented terminology in requests submitted from the desktop. DataGuide/2provides a GUI environment based on objects and actions to maximize theproductivity of the knowledge workers. Figure 28 on page 107 shows theinitial DataGuide/2 work area with the functions that the knowledge workercan execute under DataGuide/2. This work area is presented to the know-ledge worker upon opening of the DataGuide/2 icon from the desktop. Itholds the results of various tasks—creating collections and savingsearches—performed by knowledge workers. The initial work area can becustomized to contain the more commonly used objects for a particularknowledge worker.

DataGuide iscustomizablefor the know-ledge worker


Figure 28. DataGuide/2 Initial Panel

The functions that the knowledge worker can execute with DataGuide/2 are:

• Search• Launch applications• Create private collections• Display contact information• View current news• View glossary.

These functions are all geared to the knowledge worker. They addressrequirements for finding and accessing informational objects and working inan Information Warehouse environment, in general. These functions are nowaddressed individually.

8.2.3 Search

DataGuide/2 supports two search modes: keyword and navigational.Keyword search is used by knowledge workers who have some descriptiveword associated with the object of interest. The descriptive word can beapplied to the object type, properties of an object, and where an object wasused. For example, the knowledge worker may know that the name of areport contains the keyword “sales.” In this case, the object type is Chart,and Name is a property of the Chart object type for this informational object.The panels a knowledge worker would see when creating and saving asearch are as follows:

DataGuide/2has twosearch modes


Search specification Figure 29 on page 108Search results Figure 30 on page 109Search save Figure 31 on page 110New initial work area panel Figure 32 on page 111

Figure 29. Search Specification

Knowledge workers arrive at the Define Search panel by double clicking onthe New Search Icon on the initial work area panel. They then select theCharts entry under Object types. At this point, they can select the Nameproperty and enter a search argument in the Enter value for selected prop-erty field. They can also choose to leave the Value blank to execute anunqualified search. The unqualified search returns all object instances forthe selected object type.


Figure 30. Search Results

Knowledge workers are then presented with the Search Results - Icon Listdisplay. There are only three instances of the object type Charts in thisDataGuide/2′s meta-data store; the unqualified search returns those three.To save the search, knowledge workers first click on the Search resultsoption on the menu bar and then select the Save option on the pull-down.The result is shown in Figure 31 on page 110.


Figure 31. Save Search

Knowledge workers can enter a name for the saved search. They have theoption of choosing an icon for the saved search by clicking on the Find...button.


Upon completion of the Save Search function, the saved search appears onthe initial work area panel as a new icon, as shown in Figure 32. The savedsearch is now available to be executed whenever the knowledge workerneeds to find the set of informational objects identified by the search criteriain the saved search.

Figure 32. New Initial Work Area Panel

Navigation is a form of searching that does not require the knowledge workerto explicitly enter search criteria. Rather, a graphical tree representation ofdata, objects, and their relationships is traversed to locate informationalobjects. This approach allows knowledge workers using DataGuide/2 for thefirst time to become familiar with the organization of data and objects withinthe enterprise. Figure 33 on page 112 shows a navigational tree resultingfrom a search under the Subjects icon.

Navigationmakes first-t ime usersproductive


Figure 33. Navigation Tree

Navigational search—invoked through the Subjects icon—is used when infor-mation is searched for based on its grouping within the enterprise. Forexample, the knowledge worker might want to know which reports are part ofthe Annual Reports grouping. In this case, Report is the object type of aninformational object; the Annual Reports grouping is defined by an adminis-trator to reflect a business relationship.

Navigationalsearch isbased ongroupings

Knowledge workers can view descriptive information for selected objects inDataGuide/2 by double clicking with the left mouse button on the object.This descriptive information is called meta-data in the Information Warehousearchitecture. DataGuide/2 presents a container window having values foreach property of the informational object. The panel also offers the opportu-nity to launch a decision support tool.

Single clicking with the right mouse button on the object causes DataGuide/2to display a pop-up Actions panel. This panel offers the opportunity to launchan informational application against this object.

Descriptiveinformationshows objectproperties


8.2.4 Launch Applications

Launching applications involves invocation of a decision support tool using alocated query object or a complete decision support report as input.DataGuide/2 can invoke an online document display package, such asBookManager Read*, to display a manual containing a relevant description ofthe information found in the search. Another possibility is invoking an imagedisplay package, such as IBM ImagePlus*, to display an image of a retailproduct.

Launchingapplicationsintegratesdata locationand analysis

DataGuide manages the association of informational applications to informa-tional objects. The association between application instances and objects isat the object type level. That is, the process is a three-step process, asfollows:

1. Create the object type2. Associate a program with the object type3. Create object type instances.

Thus, multiple program instances are associated with the object type andmultiple object instances are associated with the object type. DataGuideallows any of these program instances to execute against any of the objecttype instances, although the informational object may have been generatedfor a specific program instance. When the knowledge worker chooses theStart Program action against an informational object, DataGuide displays alist box containing the program instances for that object type. It is theresponsibility of the administrator to ensure that the knowledge worker isgiven the proper direction, through an object property, to choose the correctprogram instance.

8.2.5 Create Collections

Applicationscan be linkedto objects

A collection is a graphic grouping of informational objects that knowledgeworkers logically view as one. Such a collection might include commontables, business grouping, queries, reports or graphs that the knowledgeworker often uses for analysis. Collections are defined by knowledgeworkers for their private use, whereas DataGuide groupings are defined byadministrators for use by multiple knowledge workers. Figure 34 onpage 114 shows a collection made up of two chart objects.

Collectionsmirror theway peoplethink


Figure 34. A Collection

8.2.6 Display Contact Information

A contact is the name and computer system ID or telephone number of thesteward for an informational object. The contact would be someone whocould enable access to the associated data or object. The Contact function isimportant for nonelectronic objects such as product samples. DataGuide isan excellent facility for cataloging nonelectronic as well as electronic objects.

8.2.7 View Current News

A person maybe accesspoint

News might include which new objects or data had been added, whichobjects had been replaced, or a schedule of upcoming maintenance activity.Information found in this facility could be moved to the search results col-lection for further work. News items apply to the DataGuide environment andnot necessarily to an individual informational object.

News keepsknowledgeworkerscurrent withtheir IW


8.2.8 View Glossary

The glossary contains definitions of terminology unique to the enterprise, itsindustry, and its technological environment. For example, certain commonbusiness terms and the normal business cycle might be documented here.Items in the glossary apply to the DataGuide environment and not neces-sarily to an individual informational object.

8.2.9 Administrator Functions

A glossarymakescommon ter-minology pos-sible

DataGuide administrators use a set of panels and functions unavailable to theknowledge worker to manage the DataGuide meta-data store itself. Theygenerally manage the environment, the knowledge workers′ access, and thecontents of the DataGuide meta-data store.

The functions available to the administrator include the following:

• Adapt DataGuide to the enterprise

− Create business groups− Associate informational application programs with objects− Define contact information− Create object instances

• Extending DataGuide

− Extend starter set properties− Create custom object types

• Manage the meta-data

− Import/export between DataGuide instances− Run extractors− Edit extractor results− Import extractor results.

Administra-tors manageDataGuide

Two simple functions are helpful to adapt DataGuide to the enterprise: cre-ating business groupings and associating informational application programswith informational object types. The business groupings define how informa-tional objects are grouped together, and business groupings vary betweenenterprises. A simple example on managing groupings uses relationaltables as the informational object. There are two possible perspectives on arelational table as an informational object: the relational table could beviewed as the lowest level informational object, or the columns in a tablecould be viewed as the lowest level. If the columns are considered ofinterest to the knowledge worker, then the administrator defines the table inthe Grouping category and the column in the Elemental category and thenmust define relationships between the table and column object types. If thecolumns are not considered of interest to the knowledge worker, the admin-istrator defines the table in the Elemental category and does not define thecolumn as an object type. Groupings require planning and have implicationsfor moving meta-data between DataGuide instances.

AdaptDataGuide tothe enterprise


Extensibility is also important to the Information Catalog solution. Enter-prises are expected to have their own variation on the structure of the meta-data at the meta-data entity level. That is, different enterprises want to keepdifferent attributes of specific meta-data entities. For example, an enterprisemay want to maintain the date of last update for meta-data describing aninformational object. A second enterprise may want to maintain that dateand an indicator for the status of the operational application at the time themeta-data was updated. The administrator would have to add this indicatoras an attribute of the meta-data object.

Extensibility iscrucial to theInformationCatalog

Meta-data management is critical to the effective use of DataGuide on anenterprisewide basis. Because DataGuide/2 is a client-server product tar-geted for the LAN as the primary server, it could become constrained to asingle LAN and the work group associated with that LAN. However,DataGuide/2 manages meta-data that describes informational data createdfrom operational data throughout the enterprise. The meta-data may there-fore be of interest to anyone in the enterprise in any work group. Therefore,the locality of a LAN-based Information Catalog must be reconciled withenterprisewide needs.

Meta-datamanagement:copying meta-data

Associating programs to informational object instances through the objecttype has an implication for the knowledge worker. The Chart object typemight have two report tools—for example, Query Manager andPAS—associated with it. Each Chart object instance—for example, 1993 Cor-porate Report and 1992 Division Report—has been built using one of thesetools. The DataGuide Start Program function gives the knowledge workersthe opportunity to choose the program for the informational object instance.Administrators must ensure that a property of the informational object hastext directing the knowledge worker in the decision support tool to choose.

8.2.10 Extending DataGuide/2

Associatingprogramsfacilitateslaunching

DataGuide administrators can create new object types or add properties toexisting object types. The DataGuide starter set creates an initial set ofobject types to help administrators build their object type set. The series ofpanels that follow show the basic steps in creating a new object type.Figure 35 on page 117 shows two panels: the initial DataGuide/2 panel,DataGuide/2 - Icon View, displayed when the administrator double-clicks onthe desktop DataGuide/2 icon, and the administrator′s panel, DataGuide/2Admin Utils - Icon View, displayed when the administrator double-clicks onthe DataGuide/2 Admin Utils icon. Knowledge administrators double-click onthe DataGuide/2 Knowledge Admin icon to display the DataGuide/2 - Iconspanel containing the Object types icon. The Object types icon is availableonly to the administrator and is used to maintain DataGuide/2 meta-data.

Extend theDataGuide/2model to suitthe enterprise


Figure 35. Initial and DataGuide Administrator Panels

Administrators open the Object types icon, shown in Figure 36, to get theicon list of all object types currently defined in this DataGuide/2 and theicons associated with those object types. The icon list is shown in Figure 37on page 118.

Figure 36. DataGuide Administrator Panel

Administrators open the New Object type icon to create a new object type oradd a property to an existing object type.


Figure 37. Object Types

Knowledge workers double-click on the New Object type icon, at which timeDataGuide/2 displays the Create Object Type panel (see Figure 38 onpage 119). They can now enter the Category, Object type name, and a Shortname for the new object type, MKTCHART.


Figure 38. Create Object Type

Administrators use the Create Object Type panel to define the object typeand to manage object type properties. Administrators can add, modify, andremove properties and they can define a universal unique identifier (UUI) asbeing made up of a specific set of properties. The UUI is required and isused by DataGuide to uniquely identify each instance of all object types.Administrators click on the Define UUI... button to perform this definition.

The administrators select Add from the Create Object Type panel to displaythe Add Property panel shown in Figure 39 on page 120.


Figure 39. Add Object Type Property

Administrators define new properties for the object type using the Add Prop-erty panel shown in Figure 39. They can specify a Property name, a Shortname, the Data type, the Size, and whether entry is required. Any propertyto be used as part of the UUI must have Entry required specified for it, byclicking on the Entry required check box.

8.2.11 Meta-data Management

Two aspects of the retail enterprise put special demands on the InformationCatalog and the management of meta-data: the need to consolidate meta-data from heterogeneous sources, and the need to physically disperse asingle logical collection of meta-data across multiple LAN work groups. Theexport/import interface provides the means for meeting both of theserequirements.

Meta-datamust bemobile to beuseful

Most retail enterprises have a need for multiple DataGuide/2 instances, if forno other reason than to maintain test and production Information Catalogs.The enterprise needs to move meta-data between DataGuide/2 instances onLANs. This movement may be from DataGuide/2 to DataGuide/2 or it may befrom DataGuide/2 on one LAN up to DataGuide/MVS and then down toanother DataGuide/2 on a LAN. The DataGuide/2 import/export interface sup-ports either path.

Use theexport/importstrategy tomanagemeta-data forthe enterprise


The Information Warehouse Architecture I defines an intermediate dataformat for meta-data. This approach breaks the meta-data movementprocess into three steps: extract from the source, storing on an intermediatebasis, and loading into DataGuide. The format of the intermediate storage ispublished as part of the Information Warehouse framework strategy, andDataGuide provides a tool to import the meta-data in this published formatinto DataGuide′s meta-data store. The CASE tool vendor or retail enterprisenow has only to write tools to extract from the source into the intermediateformat.

DataGuide/2import/exportinterfacedefines anopen, inter-mediateformat

The intermediate format also has implications in planning and project man-agement. Enterprises can begin Information Warehouse implementationprojects and build meta-data storage and end-user interface systems imme-diately. The export/import interface allows these projects to be migrated intothe DataGuide/2 at a later point in time. The only additional investment isthe tool to extract from the temporary meta-data store to the intermediateformat. Special consideration should be given to the DataGuide/2 model andthe relationships between the categories and object types. The meta-data inthe intermediate format needs a mechanism to connect meta-data definitions.

8.2.12 DataGuide Data Model

In this section, data and objects within DataGuide are described using a clas-sification schema borrowed from the world of GUI programming rather thanobject-oriented programming. For an explanation of object terminology inthe GUI environment and the object-oriented programming environment, seethe paragraph on Object terminology on page 106.

Theimport/exportprovides flexi-bility

The meta-data in DataGuide is defined within the hierarchy of Category,Object Type, and Objects. Categories represent a general classification ofobject types. Figure 40 on page 122 shows the categories that make up theDataGuide model. These categories, in turn, have a distinct set of actionsassociated with them. For example, application objects found in the Programcategory can be associated with objects from the Grouping category but notwith objects or data from the Contact category. Category-action-categoryrelationships are shown as double-ended arrows in the diagram.

Categoriesare a way ofclassifyingobjects


Figure 40. DataGuide Categories

DataGuide functions are constrained with respect to the object types withwhich they work. For example, the Where Used search capability can onlybe applied to objects types within the Grouping and Elemental categories.Actions associated with any one of the seven categories apply to all objectsin that category. The categories, with object types that are allowed in eachcategory, are as follows:

Grouping Group object types in this category can contain other grouptypes. The Group and Table object types are in the Groupingcategory. Groups form the highest level object in the DataGuideclassification system.

Contact The Contact category defines people responsible for objects.Contact objects can be associated with group and elementalobjects. The Contact object is in this grouping category.

Elemental Object Types in this category form the bottom nodes of a navi-gational tree. They can be contained by object types in theGrouping category but cannot contain other object types. Exam-ples of object types in this category include the Column, Query,Report, and Image object types.

Categoriesmanageobject-actionassociationsin DataGuide


Program The only object type allowed here is Program. New objecttypes cannot be defined to this category.

Dictionary This category serves to hold definitional support object typessuch as Glossary. This category is used to define business ter-minology.

Support The Support object defines supporting and informationalobjects.

New object types can be created within any category except the program cat-egory, and new objects can be created within an object type. Object typeshave properties that apply to all object instances for that object type. Theseproperties are analogous to attributes for entities or columns for tables andhave syntactic and semantic definitions. The syntactic definition takes theform of the property′s data type (for example, VARCHAR andCHARACTER(80)). The semantic definition is a business description of themeaning of the object (for example, the property UPDATIME is the systemgenerated time of the last update of one object instance). Object types arealso the target of associations with applications such as decision supportproducts. Only the DataGuide administrator has the authority to createobject types.

Categories,object types,objectinstancesmanage yourmeta-data

DataGuide administrators at the retail enterprise want to describe eachreport as being large, medium, or small in size. This helps knowledgeworkers decide whether or not they want to print the report on a local mini-printer or the large printer at the main office. They use the Chart object typeand create a new property called Size. They define this new property asbeing a character data type, with a length of 10. Each report is then definedas an instance of the object type report with the additional step of populatingthe Size property.

8.2.13 Interfaces

Customizemeta-data

The functions available to the knowledge worker and administrator throughthe respective end-user interfaces are also available through the DataGuideAPI. The API is available to any informational application that requires theservices of an Information Catalog. Knowledge workers can then interactwith the informational application with which they are familiar, and the infor-mational application makes the necessary interactions with DataGuide toobtain the necessary descriptive information. These API commands performsearches for meta-data or administrative functions such as creating objects,getting objects, and providing listings.

A user usesthe tool; thetool uses theDataGuide/2API

The many functions available through the API can be categorized into ObjectType, Object Instance, and Function services. Table 8 on page 124 summa-rizes these categories and their services.

Decisionsupport toolscan useDataGuide asa callableservice


Table 8. DataGuide Services

Service Category Services

Object type Create, delete, append, and get forobjects

Object instance Create, delete, update, and get forinstances

Function Search, Search-all, and Navigatefor finding meta-data

Both the object type and object instance service categories deal with thedirect manipulation of meta-data in DataGuide. The Function services aremore broad in their scope and cover services ranging from searching formeta-data to more system-oriented services such as export and import formoving meta-data into and out of DataGuide. The Listing service managescontacts, programs, object types, and groupings. Program invocation,commit and rollback, memory management, initiation and termination, andthe trace service address the behavior of DataGuide itself.

Servicesinclude meta-data searchand manage-ment

DataGuide API services are applicable to certain categories (see 8.2.12,“DataGuide Data Model” on page 121 for a discussion of DataGuide catego-ries). For example, the ability to navigate is restricted to object types in theGrouping category.

Services arespecific tocategories

Input/output structures are key to the use of DataGuide services by both theDataGuide end-user interface and other applications invoking DataGuide ser-vices. This DataGuide API data structure is self-defining and has a basicformat made up of three parts, as follows:

Header Structure identification and scopingDefinition Object property definitionsObject Object property values (not always required)

Input/outputstructuresenable theDataGuideAPI

The export/import interface supports the migration of meta-data fromnon-DataGuide sources to DataGuide and between DataGuide instances.This interface opens DataGuide to heterogeneous meta-data stores and sup-ports a distributed environment for DataGuide/2 and DataGuide/MVS. Enter-prises have meta-data in a variety of sources, including the structuredcolumns and the Remarks column of the DB2 catalogs. Enterprises haveinvested time and resource in CASE tools as well. They have created andstored meta-data in CASE tool encyclopedias and other forms of meta-datastores. The export/import interface is a way of consolidating the heteroge-neous meta-data into DataGuide.

Export/importinterfacemakesDataGuide anopen tool

The export/import interface takes the form of a documented format for meta-data. The DataGuide products are distributed with a tool that reads this doc-umented format and populates the respective DataGuide meta-data storeaccordingly. The documentation is available to any customer or softwarevendor. The retail enterprise′s information systems department can write atool to extract meta-data from the CASE tool encyclopedia used in its shop.This requires that the staff know the internal format of the meta-data in thatparticular CASE tool, or if provided by the CASE tool, interfaces and com-mands for extraction of meta-data. The enterprise then uses the DataGuideimport tool to load the reformatted meta-data into the DataGuide store.

The interfaceis open to allvendors


The more likely scenario has CASE tool vendors using the documentedformat to write the extract tool. The tool extracts meta-data from their partic-ular encyclopedia and stores it in the documented format in an intermediatemedium, most likely a flat file. The retail enterprise can then use theDataGuide import tool to load the meta-data into the DataGuide meta-datastore. This approach is preferable for the vendors, because it avoids disclo-sure of their internal meta-data format.

The informa-tion systemsdepartmentcan write theextract tool

The export/import interface can also be used to manage multiple instances ofDataGuide. This requirement is rooted in the LAN orientation of DataGuide/2and the nature of most retail enterprises. DataGuide/2 is designed toexecute on a LAN server with multiple programmable workstation clientsmaking use of its services. This configuration is very attractive to the workgroup or small project group.

Use the inter-face tomanageDataGuideinstances

Most retail enterprises are larger than this single work group and want toshare meta-data across work groups. Large retail enterprises desire acentral meta-data collection on a larger server with the capability to movethe meta-data down to the work group. DataGuide/MVS can be used for themeta-data collection on the large server. DataGuide supports the movementof meta-data between DataGuide/2 and DataGuide/MVS instances through theexport/import interface. Thus, DataGuide/2 offers the flexibility of the LANenvironment with the flexibility to support enterprise-level meta-data andsharing across the enterprise.

DG/MVS sup-ports central-izedmeta-data

DRDA can also be used to implement a centralized meta-data store.DataGuide/2 can access meta-data stored in a remote DB2 for MVS instance.It uses DRDA as the transport protocol to request meta-data. The flexibilityof DRDA allows the meta-data to be moved to any DB2 family member.

DRDA sup-ports central-izedmeta-data


Chapter 9. Conclusions

Three years have passed since Keen discussed the range and reach modelof information technology (see Shaping the Future: Business Design throughInformation Technology). The basic message was that competitive advantageand economy of operation would accrue to those enterprises that improvedtheir span of information technology communications to include their clients(reach) and enriched the functions offered to the clients (range). Along thepath to the client′s door, it was equally important to get connected to yoursuppliers and outside institutions such as banks. And the key architecturalfeature of such an information technology strategy was its openness: theability to use more than one vendor product for a particular function, not belocked into a proprietary solution that forced other parts of the solution toalso come from the same vendor.

Today, we are beginning to see many instances of how range is being imple-mented. In the example of one hypermart, providing information in a ware-house construct with preservation of the historical data allowed two things tohappen:

• Better analysis

The enterprise was better able to uncover sales anomalies and under-stand the driving factor behind the anomaly. This allowed it to refine itssales approach and achieve an immediate result.

• Improved information accessibility

Outside suppliers found value in accessing this information and paid thecompany back by assuming a part of their normal workload (maintainingstock levels) and eliminating parts of the business process (business re-engineering by cutting out a large portion of the order-to-invoice papertrail).

Such was the unknown power of making information accessible that the fullbenefit of an Information Warehouse solution was not understood on day 1 ofthe implementation. In the evolving world marketplace, information becomesthe new means of extending range and thereby gaining competitive advan-tage.

As a result of this shift from the traditional application-oriented informationtechnology approach, the role of the information systems specialist is alsochanging. In the past, information systems emphasis was on the deliveryand maintenance of business systems, with 80% of the resources spent onmaintenance. With the advent of powerful, independent LANs and work-stations, business unit dependency on information systems has shifted fromthe supply of business systems to the supply of information. In the new envi-

Chapter 9. Conclusions 127

ronment, the information systems specialist is expected to be a provider ofinformation rather than a provider of systems.

In this new world, the Information Warehouse architecture supplies the planfor automating the provisioning of this information. Products such asDataGuide allow end users to see which information is available. Tied toother workstation products through the key Information Warehouse interfacesand protocols, end users can begin to manipulate this information in newways. The new ways can often lead to better interpretation of informationand reengineering of the business processes that generate this information.In their new roles, information systems specialists maintain the inventory ofdata and relationships between data; the knowledge worker builds smallapplications on the fly.

As providers of the raw information, information systems specialists need tosend consistent copies of shared information to various points in the enter-prise. Given their limited people resources, such information flows must beautomated. This need leads to new products, such as DataPropagator Rela-tional and DataHub/2, that implement the automated flow of full copies anddifferential updates of source data to multiple informational objects. A keyfeature of such products is the use of the workstation as the point of control.Use of the workstation takes advantage of an object-oriented approach tomask the complexity required to maintain the new information network.

Finally, in the area of presenting that information to the knowledge worker,the Information Warehouse architecture establishes some importantapproaches to assist information systems specialists in keeping their infor-mation networks open and flexible:

• A reliance on an established data access language (SQL)• A view of functions as objects that allows new functions to be snapped in

over time or replaced. For example, in the DataGuide/2 product, a queryor report is merely an icon to the knowledge worker. If the decisionsupport product that ultimately runs the query or allows the end user toview the report is changed, that change and its impact on the knowledgeworker can be hidden by the information systems specialist who willreconfigure the underlying DataGuide/2 representation of that object.

The competition is calling and information is answering....


Appendix A. Models and Modeling

The solution presented in this book devotes significant attention to modelsand modeling. Modeling has been given much publicity over time and hasbeen considered crucial to a well-organized and efficient data processingorganization. However, modeling has, in general, failed to deliver on thepromised benefits of model-based application generation. Some of thisfailure is due to the sheer variety of modeling methodologies available, andsome is due to the esoteric language and complexity of the conceptsinherent in modeling. Perhaps the largest contribution to this failure is thelack of open interfaces and automation between the components and phasesof the application development life-cycle. In this appendix, we present theconcepts and benefits of modeling by a simple example and lay the ground-work for the role of the Financial Application Architecture* (FAA), InsuranceApplication Architecture* (IAA), and Retail Application Architecture* (RAA), indata processing in general and Information Warehouse implementation inspecific.

A.1 The Construction Model

Modelingorganizes thedata proc-essing envi-ronment

The building of a structure—a home, an office building, or other complexstructure—serves as a platform for discussing the benefits of a model andmodeling. A model is an abstract representation of a real world environ-ment. Modeling classifies certain aspects of building into things called enti-ties. The concept of entities immediately presents a challenge for relatingmodeling to a real-world environment. Entity is a term used to classifypeople, places, things, ideas, concepts, or events that are relevant to thebusiness. The key here is to understand the motivation for entities. Entitiesprovide a way to group things that have common characteristics or role withrespect to the business.

For example, within a construction company, entities include nails, boards,and windows; carpenters, plumbers, and electricians; contracts; trucks; andother components of the business. As a general categorization, entities is aconvenient catch-all for anything with which the construction projectleader—the general contractor (GC)—has to be concerned. The benefit is thatthe GC can look at one list of things needed to build the house.

Entity: classi-fying things

Appendix A. Models and Modeling 129

Further reflection reveals that these entities are not all the same, that somesubsets of these entities have common characteristics. If it is of benefit togroup all entities, then it is of more benefit to group them so that thecommon aspects of the subgroupings can be utilized.

A.1.1 Entity: Things

Entities canbe grouped tosimplify theiruse

The next step is to create entities within the larger-scope Entity, based onthese common aspects or ways of being used. For example, nails andboards have attributes in common: they are both things to be ordered,stored, and physically incorporated into the structure. We therefore create aspecific type of entity called Materials. Materials is a grouping of things thatare handled in a similar manner from a business perspective and typicallyhave common attributes. The attributes describe the nature of the entity. Inthis case, the attributes are the color, size, and other physical aspects of thething. The value here is that the GC can use one order sheet for all thingsthat fit into the Entity category Materials. The GC can use another form forall subcontracts for services. The GC′s job is now easier because the dif-ferent parts of the job can be generalized.

Entities makethe overallprocesssimpler

At this point, we have introduced two perspectives on things essential to thebusiness: the way these things fit into the business—what the business doeswith the thing—and the attributes or descriptions of these things. The benefitthen is that we can think of the things that are part of our business as ageneral group. We can also generalize the business activities performedagainst the things.

A.1.2 Entity: Agreements

The next set of entities is centered around the contract, or agreement, legalor otherwise, that is part of building the structure. Contracts are written,agreed to, and enforced. Contracts are different from nails and boards,which are ordered, installed, or are part of the physical structure. By sepa-rating these two sets, we can treat them appropriately from a business pointof view. What is of more interest here is that they can be treated consist-ently from a data processing point of view. That is, application code can bewritten to consistently operate on data about things defined as being thesame entity. Furthermore, application code written to operate on data aboutthings used in building a house is consistent with application code written tooperate on data about those same things used in building an office.

This approach suggests that contracts in the construction business, categor-ized as an entity called agreements, can be viewed from both a business anda data processing perspective as similar to contracts in the insuranceindustry, where they are policies.

Entities helpgeneralizedata proc-essing proc-esses


A.2 The Annual Report As a Model

We can then extend these concepts to the corporate statement. The annualreport presents the financial status of an enterprise in terms of its assets andliabilities. Both the assets and liabilities can be seen as entities, but this isstill rather ambiguous with respect to common experience. A better exampleis the subheading Plant and Property under assets. This is a financial viewof things the enterprise has as an asset. This perspective on the enterpriseis similar to the points we stressed as being a benefit for modeling, entities,and the general categorization philosophy of modeling. Regardless of theindustry within which the enterprise does business, the enterprise alwayshas some type of plant and property.

From an industry perspective, we can treat all plant and property as some-thing that has value, that exists. From a data processing perspective, we canexpect to write applications that sum up the present value of that plant andproperty. Furthermore, we can expect to write applications that depreciatethat plant and property over time. The treatment and expectations of thesebusiness objects are independent of the enterprise′s industry. We havegained perspective on a component of the business and have gained anopportunity to leverage data processing resources by this generalization,which is wholly compatible with the objectives of modeling.

A.3 Information Warehouse and Modeling

The annualreport is amodel

Models contain representations of the enterprise′s business in the form ofEntities and other modeling constructs. These constructs contain technicaland descriptive information about the business objects. The technical infor-mation includes data type and length and may in fact be used in limited waysby an application generator. The descriptive information is for reading andunderstanding purposes only for the model user. This descriptive informa-tion is called meta-data and is a crucial part of an the Information Warehouseenvironment.

Models are aformal repre-sentation

Meta-data explains the meaning of the object and the data processing objectin business terms. It helps the administrator know whether thebusiness/data processing object is needed for informational analysis. It alsohelps the knowledge worker understand the meaning of the object, once it isincorporated into the Information Warehouse implementation. This con-nection between modeling, the model, and the dictionary for the knowledgeworker (the Information Catalog) is dependent on a subset of the model infor-mation in the Information Catalog.

Meta-dataexplainsobjectmeaning

The descriptive information also reflects the reconciliation and enhancementprocess. It describes the business/data processing object as it exists. Theknowledge administrator and the business analyst know what the informa-tional data needs to look like for use in an informational environment. Theprocesses of decoding, reconciling, and enhancing operational data for usein an informational environment are based on these before and afterdescriptions.

Meta-datareflects dataenhancement

Appendix A. Models and Modeling 131

List of Abbreviations

ATM automated tellermachine

CPU central processingunit

DASD direct access storagedevice

DBA database adminis-trator

DIS data interpretationsystem

DRDA Distributed RelationalDatabase Architecture

EDI electronic data inter-change

EFT electronic fundstransfer

FAA Financial ApplicationArchitecture

GMROI gross margin returnon investments

GOI generic output inter-face

GSA General Store Appli-cation

HHT hand-held terminal

IAA Insurance ApplicationArchitecture

IBM International BusinessMachines Corporation

ITSO International Tech-nical Support Organ-ization

MIS management informa-tion system

PWS programmable work-station

ROA return on assets

RAA Retail ApplicationArchitecture

UPC universal productcode

List of Abbreviations 133

Index

A

access enablersAS V3 94DRDA 77Information Warehouse architecture 74purpose 77SQL mappers 74

actions panel 112airline industry 35application

informational 4architecture

industry 5Information Warehouse 5

C

C+ + 106client-server

DataGuide/2 116POS 31

D

dataaggregation 44

data replicationeffort 25

DataGuideapplication program interface 123categories 121client-server 116collections 113contact information 114DataGuide/MVS 105, 125export/import interface 124input/output structures 124interface enabling 75

DataGuide (continued)knowledge worker end-user

interface 105overview 105user types 106work area 106

DataHubcopy tool 83systems management 68

DB2data compression 99DB2 V3.1 91I/O parallelism 95in-store processor 63interface enabling 75platform flexibility 77remarks column 124S/390 Parallel Query Server 98

deploying product 75discovery 35Distributed Relational Database Architec-

tureSee DRDA

DRDAin Access Enablers 77

dril l-downin DataGuide/2 112

E

electronic funds transferrole 19

embedded SQL 76enabling product 75entity

classify 129

Index 135

G

GUIobjects 106

I

information systems departmentnew role 45

Information Warehouse architectureInformation Catalog API 105meta-data 104, 112objective 71

Information Warehouse frameworkconnectivity 68DataHub 68definition 67

informational applicationinterface enabling 75Personal AS/2 83

interfacesInformation Catalog 80Information Catalog export/import 81

K

knowledge workerdefinition 4

L

launch 104

M

meta-dataAPI 105CASE tool 124categories 123customization 123data processing terms 61definition 89, 112example 102export/ import 81export/import interface 120, 124extensibility 116extractor 81, 125grouping 115heterogeneous 80in organization asset data 88

meta-data (continued)Information Catalog 80interface 76, 80management 115, 120movement 116search 109services 124users 106

model-based application generation 129abstract 129

N

navigationdefinition 111

nonprogrammable terminalsin retail industry 23

O

objectterm usage 106

organization asset datacategories 87data and meta-data 88S/390 Parallel Query Server 82

P

Personal System/2in retail industry 23

processbusiness 37

R

reconciliationin retail industry 29

retail cycledefinition 60

retail industry studychallenges 14local data access 41primary resources 14


S

searchdialog 107keyword 107navigational 112

SLDM 63solution thread

objective 4SQL call level interface 76Store Logical Data Model

See SLDMstrategy

retail industry 23summary data

retail industry 41

Index 137

ITSO Technical Bulletin Evaluation RED000

Information Warehouse inThe Retail Industry

Publication No. GG24-4342-00

Your feedback is very important to help us maintain the quality of ITSO Bulle-tins. Please fill out this questionnaire and return it using one of the fol-lowing methods:

• Mail it to the address on the back (postage paid in U.S. only)• Give it to an IBM marketing representative for mailing• Fax it to: Your International Access Code + 1 914 432 8246• Send a note to [email protected]

Please rate on a scale of 1 to 5 the subjects below.(1 = very good, 2 = good, 3 = average, 4 = poor, 5 = very poor)

Overall Satisfaction ____

Organization of the bookAccuracy of the informationRelevance of the informationCompleteness of the informationValue of i l lustrations

____________________

Grammar/punctuation/spel l ingEase of reading and understandingEase of finding informationLevel of technical detailPrint quality

____________________

Please answer the following questions:

a) If you are an employee of IBM or its subsidiaries:

Do you provide bil lable services for 20% or more of your time? Yes____ No____

Are you in a Services Organization? Yes____ No____

b) Are you working in the USA? Yes____ No____

c) Was the Bulletin published in time for your needs? Yes____ No____

d) Did this Bulletin meet your needs? Yes____ No____

If no, please explain:

What other topics would you like to see in this Bulletin?

What other Technical Bulletins would you like to see published?

Comments/Suggestions: ( THANK YOU FOR YOUR FEEDBACK! )

Name Address

Company or Organizat ion

Phone No.

Cut or FoldAlong Line

Cut or FoldAlong Line

ITSO Technical Bulletin Evaluation RED000GG24-4342-00 IBML

Fold and Tape Please do not staple Fold and Tape

NO POSTAGENECESSARYIF MAILED IN THEUNITED STATES

BUSINESS REPLY MAILFIRST-CLASS MAIL PERMIT NO. 40 ARMONK, NEW YORK

POSTAGE WILL BE PAID BY ADDRESSEE

IBM International Technical Support OrganizationDepartment 471, Building 070B5600 COTTLE ROADSAN JOSE CAUSA 95193-0001

Fold and Tape Please do not staple Fold and Tape

GG24-4342-00

Printed in U.S.A.

INFORMATION WAREHOUSE IN THE RETAIL INDUSTRY

Documents

Transcript of INFORMATION WAREHOUSE IN THE RETAIL INDUSTRY