enos.itcollege.eeenos.itcollege.ee/~gseier/Sybase IQ admin guide.doc · Web view... PowerJ, Power...

Student Guide

Administering Adaptive Server IQ for the Data Warehouse

Version 1.0

2000 Sybase, Inc.

Part #EDB315-90-1242-10

I

Notice

Copyright 2000 Sybase, Inc. All rights reserved.

No part of this publication may be reproduced, transmitted, or translated in any form or by any means, electronic, mechanical, manual, optical or otherwise, without prior written permission from Sybase, Inc.

indicates registration in the United States of America.

Sybase Trademarks

Sybase, SYBASE (logo), ADA Workbench, Adaptable Windowing Environment, Adaptive Component Architecture, Adaptive Server, Adaptive Server Anywhere, Adaptive Server Enterprise, Adaptive Server Enterprise Monitor, Adaptive Server Enterprise Replication, Adaptive Server Everywhere, Adaptive Server IQ, Adaptive Warehouse, AnswerBase, Application Manager, AppModeler, APT Workbench, APT-Build, APT-Edit, APT-Execute, APT-FORMS, APT-Translator, APT-Library, ASEP, Backup Server, BayCam, Bit-Wise, Certified PowerBuilder Developer, Certified SYBASE Professional, Certified SYBASE Professional Logo, ClearConnect, Client-Library, Client Services, CodeBank, Column Design, ComponentPack, Connection Manager, CSP, Data Pipeline, Data Workbench, DataArchitect, Database Analyzer, DataExpress, DataServer, DataWindow, DB -Library, dbQueue, Developers Workbench, Direct Connect Anywhere, DirectConnect, Distribution Director, Electronic Case Management, Embedded SQL, EMS, Enterprise Application Server, Enterprise Application Studio, Enterprise Client/ Server, Enterprise Connect, Enterprise Data Studio, Enterprise Manager, Enterprise SQL Server Manager, Enterprise Work Architecture, Enterprise Work Designer, Enterprise Work Modeler, EWA, First Impression, Formula One, Gateway Manager, GeoPoint, ImpactNow, InfoMaker, Information Anywhere, Information Everywhere, InformationConnect, InstaHelp, InternetBuilder, iScript, Jaguar CTS, jConnect for JDBC, KnowledgeBase, Logical Memory Manager, MainframeConnect, Maintenance Express, MAP, MDI Access Server, MDI Database Gateway, media.splash, MetaWorks, MethodSet, MySupport, Net-Gateway, Net-Library, NetImpact, Next Generation Learning, O DEVICE, OASiS, OASiS logo, ObjectConnect, ObjectCycle, OmniConnect, OmniSQL Access Module, OmniSQL Toolkit, Open Client, Open ClientConnect, Open Client/ Server, Open Client/Server Interfaces, Open Gateway, Open Server, Open ServerConnect, Open Solutions, Optima++, Partnerships that Work, PB-Gen, PC APT Execute, PC DB-Net, PC Net Library, Power++, PowerJ, Power Through Knowledge, power.stop, PowerAMC, PowerBuilder, PowerBuilder Foundation Class Library, PowerDesigner, PowerDimensions, PowerDynamo, PowerScript, PowerSite, PowerSocket, Powersoft, PowerStage, PowerStudio, PowerTips, Powersoft Portfolio, Powersoft Professional, PowerWare Desktop, PowerWare Enterprise, ProcessAnalyst, Report Workbench, Report-Execute, Replication Agent, Replication Driver, Replication Server, Replication Server Manager, Replication Toolkit, Resource Manager, RW-DisplayLib, RW-Library, S Designor, S-Designor, SAFE, SAFE/PRO, SDF, Secure SQL Server, Secure SQL Toolset, Security Guardian, SKILS, smart.partners, smart.parts, smart.script, SQL Advantage, SQL Anywhere, SQL Anywhere Studio, SQL Code Checker, SQL Debug, SQL Edit, SQL Edit/TPU, SQL Everywhere, SQL Modeler, SQL Remote, SQL Server, SQL Server Manager, SQL SMART, SQL Toolset, SQL Server/CFT, SQL Server/DBM, SQL Server SNMP Sub Agent, SQL Station, SQLJ, STEP, SupportNow, Sybase Central, Sybase Client/Server Interfaces, Sybase Development Framework, Sybase Financial Server, Sybase Gateways, Sybase Learning Connection, Sybase MPP, Sybase SQL Desktop, Sybase SQL Lifecycle, Sybase SQL Workgroup, Sybase Synergy Program, Sybase Virtual Server Architecture, Sybase User Workbench, SybaseWare, SyberAssist, SyBooks, System 10, System 11, System XI (logo), SystemTools, Tabular Data Stream, The Enterprise Client/Server Company, The Extensible Software Platform, The Future Is Wide Open, The Learning Connection, The Model For Client/Server Solutions, The Online Information Center, Transact-SQL, Translation Toolkit, Turning Imagination Into Reality, UltraLite, UNIBOM, Unilib, Uninull, Unisep, Unistring, URK Runtime Kit for UniCode, Viewer, Visual Components, VisualSpeller, VisualWriter, VQL, WarehouseArchitect, Warehouse Studio, Warehouse WORKS, Watcom, Watcom SQL, Watcom SQL Server, Web.PB, Web.SQL, WebSights, Web Viewer, WorkGroup SQL Server, XA-Library, XA-Server and XP Server are trademarks of Sybase, Inc. or its subsidiaries.

All other company and product names used herein may be the trademarks or registered trademarks of their respective companies.

Restricted Rights Legend

Use, duplication or disclosure by the Government is subject to restrictions set forth in subparagraph (c)(1)(ii) of DFARS 52.227-7013 for the DOD and as set forth in FAR 52.227-19(a)-(d) for civilian agencies.

Sybase, Inc., 6475 Christie Avenue, Emeryville, CA 94608

How to Use This Guide

Student Guide

The pages of this Student Guide are divided into two sections: a slide and a set of student notes.

Your instructor will project the slide during class. The student notes provide additional notes to accompany the slide.

Putting It All Together

Some pages include only a code fragment or a code summary.

They are provided for your reference to help illustrate how the commands work in sequence.

Use the code fragments to complete the labs.

The instructor may or may not display these sections

Icon Samples

CautionPossible damage to equipment, data, or software

Critical noteVery important piece of information that should definitely be read

NoteInformation that expands upon or supplements the information in the main text

NTInformation specific to the Windows NT platform

Question

Recovery TipA suggestion for improving backup and recovery.

TipBest practice or "tips and tricks" information

TryIt

UNIXInformation specific to the UNIX platform

We Want to Hear From You

Feedback

Sybase EPTG address

Sybase EPTG fax Sybase EPTG E-mail

Sybase welcomes your feedback on our course materials. This includes corrections as well as suggestions for improvements.

Please mark typographical errors, formatting errors, errors of fact, or areas that need clarification and send copies of those pages to the Sybase Education Products and Technology Group (EPTG).

Be sure to include your name, phone number, address and/or e-mail address so that we can reach you should we need clarification about your concerns.

Sybase, Inc.

Education Products and Technology Group

6475 Christie Avenue

Emeryville, CA 94608

USA

Fax (510) 922-3559 [email protected]

Administering Adaptive Server IQ for

the Data Warehouse Student Guide

Module 1 Adaptive Server IQ Product Overview

Module Overview1-1

When to Use Adaptive Server IQ (ASIQ)?. Using ASIQ to Split DSS and OLTP

1-2

1-3

Distributed Data Mart1-4

Data Warehouse1-5

What Is ASIQ?1-6

What ASIQ Is Not1-7

Advantages of ASIQ: Overview1-8

Advantages of ASIQ: Speed1-9

Advantages of ASIQ: Economy1-10

Advantages of ASIQ: Flexibility1-11

What Principles Are at Work?1-13

Column-Wise Data Structures1-14

Bit-Wise Indexing1-15

ASIQ Multiplexing1-18

Benefits of ASIQ1-19

Summary1-20

Lab 1: ASIQ Product Overview1-21

Module 2 Architecture and Terminology

Module Overview2-3

ASIQ Platforms2-4

Module Map2-5

Adaptive Server Anywhere and Adaptive Server IQ2-6

Adaptive Server IQ Database Server2-7

ASIQ Server and Database (Without Multiplex)2-8

IQ Main Store2-9

Catalog Store2-10

IQ Temporary Store2-13

IQ Message Log2-14

Putting It All Together...ASIQ Database2-15

Module Map2-16

ASIQ Tables2-17

Utility Database2-18

ASIQ Indexes2-19

ASIQ Tables and ASIQ Indexes2-20

interfaces File and ASIQ Connectivity2-21

ASIQ Architecture2-22

2000 Sybase, Inc.

Administering Adaptive Server IQ for the Data Warehouse, Version 1.0

TOC-1

Table of Contents

Summary

Lab 2: Architecture

2-23 2-24

Module 3 ASIQ Environment and Installation

Module Overview3-1

Module Map3-2

Disk Space3-3

Virtual Memory3-4

Swap3-5

Basic Configuration Recommendations3-6

Hardware Recommendations3-7

Module Map3-8

Before Installing Software on UNIX3-9

Preparing for ASIQ Installation on the UNIX Platform3-10

Installing Adaptive Server IQ3-11

Setting Environment Variables3-12

Starting the ASIQ Server - UNIX3-15

Starting the ASIQ Server -Simple Method 3-16

Starting the ASIQ Server - UNIX3-17

Server Configuration File3-18

Stopping the Server3-19

Preparing for ASIQ Installation on the NT Platform3-20

Installing ASIQ Server Components3-21

Starting the Server3-22

Stopping the Server3-23

Installing ASIQ Server Client Components3-24

Module Map3-25

Validating the ASIQ Environment3-26

Lab 3-1: Installation 3-27

Module Map3-28

Setting Up ODBC3-29

Connection to ASIQ using DBISQL3-36

Starting DBISQL - GUI3-37

Ready to Connect3-38

Interactive SQL3-39

Connect Using Command Line DBISQL3-40

Connecting Through Sybase Central3-41

Sybase Central3-42

Sybase Central Features3-43

Module Map3-44

Database Configuration File for Open Client 123-45

Setting Up JDBC and Open Client3-46

Setting Up JDBC and Open Client3-48

Setting Up JDBC3-49

Connect Using ISQL3-50

ISQL Versus DBISQL3-51

sp_iqstatus3-52

TOC-2


2000 Sybase, Inc.

Table of Contents

Running sp_iqstatus Command to Check ASIQ3-53

Summary of Installation Steps3-54

Lab 3-2: Connectivity 3-55

Module 4 Creating Databases

Module Overview4-1

Database Review4-2

Database Overview4-3

Creating a Database4-4

Step 1: Select the Device Type4-5

Step 2: Estimate the Database Size4-6

Step 3: Create the Database4-7

Files Allocated When Creating Database4-8

Full Syntax4-9

Command Options4-10

Create Database: Example 14-14

Create Database: Example 24-15

Creating Databases: Sybase Central4-16

Before Starting the Server with the New Database4-18

Command-Line Switches4-19

Other Recommended Switches4-20

Command-Line or Configuration File Switches4-21

Communication Protocols4-22

Communication Protocol Switches4-23

Starting the Server with the New Database4-24

Creating a New DSN4-25

Viewing the New Database: Sybase Central4-26

sp_iqstatus4-27

Checking the Database with sp_iqstatus4-28

Enlarging a Database4-29

Enlarging a Database: Sybase Central4-30

Decreasing a Database4-31

Dropping a Database4-32

Other Useful Stored Procedures4-33

sp_iqdbsize4-34

Summary4-35

Lab 4: Databases 4-36

Module 5 ASIQ Index Types and Datatypes

Module Overview5-1

Module Map5-2

Types of Indexes Used by Typical RDBMSs5-3

ASIQ's Bitmap Indexes5-4

Review: ASIQ Index Characteristics5-5

2000 Sybase, Inc.


TOC-3

Table of Contents

Review: Four Basic Index Types5-6

Fast Projection (FP)5-7

Low Fast (LF)5-10

High Non Group (HNG)5-11

High Group (HG)5-12

Generic Method for Indexing Columns5-13

Combining Index Types5-14

Fast Indexes5-15

Step 1: By Default, an FP Index Is on Each Column5-16

Step 2: Create LF on Low-Cardinality Columns5-17

Step 3: Create HG on High-Cardinality Columns5-18

Step 4: Create HNG, If Needed5-19

Case Study5-20

Case Study: Solutions5-21

Module Map5-23

ASIQ Data Types5-24

User-Defined Datatypes5-27

Summary5-28

Lab 5: ASIQ Indexing and Datatypes 5-29

Module 6 Creating Tables and Indexes

Module Overview6-1

Steps for Deploying an ASIQ Installation6-2

Module Map6-3

Create Tables in the ASIQ Database6-4

CREATE TABLE6-5

Building Upon CREATE TABLE6-6

CREATE TABLE - IQ UNIQUE6-7

CREATE TABLE - UNIQUE6-8

CREATE TABLE - PRIMARY KEY6-9

Building Upon CREATE TABLE6-10

FOREIGN KEY Constraint6-11

Syntax: CREATE TABLE6-13

CREATE TABLE6-14

Creating a Table with Sybase Central6-16

Sybase Central: Steps 1-36-17

Sybase Central: Steps 4-66-18

Sybase Central6-19

Viewing a Table6-2 1

Altering Tables6-2 2

ALTER TABLE6-2 3

TRUNCATE TABLE6-2 4

DROP TABLE6-2 5

Module Map6-2 6

Local Temporary Tables: Method 16-2 7

Local Temporary Tables: Method 26-2 8

Module Map6-2 9

TOC-4


2000 Sybase, Inc.

Table of Contents

Global Temporary Tables6-30

Module Map6-31

View: Defined6-32

Creating a View: Benefits6-33

View: Using Partitioned Tables6-34

CREATE VIEW6-35

Views for Performance6-36

Steps for Deploying an ASIQ Installation6-37

Create the ASIQ Indexes 6-38

Review: Index Types6-39

Four Options for Creating Indexes6-40

CREATE INDEX: Syntax and Example6-41

CREATE INDEX: Sybase Central6-42

CREATE INDEX: Restrictions6-46

Creating Indexes in Parallel6-47

Viewing ASIQ Indexes with sp_iqindex6-49

Dropping Indexes6-50

Create and Maintain Scripts6-51

Summary6-52

Lab 6: Creating Tables and Indexes6-53

Module 7 ASIQ Memory Configuration

Module Overview7-1

Sybase ASIQ Memory Components7-2

Server Memory7-3

Buffers, Page Size, and Blocks7-4

Buffer Caches7-5

Why Tune Shared Buffer Caches?7-6

Main Buffer Cache7-7

Temporary Buffer Cache7-8

Viewing Buffer Cache Size7-9

"Active" Users7-10

Strategy for Sizing Caches7-11

Memory: The Big Picture7-12

Operating System Memory and Other Applications7-13

ASIQ Server7-14

ASIQ Memory Overhead7-15

Side Note on Setting Database Options7-17

load_memory_MB option7-18

ASIQ Main and Temp Buffer Caches7-19

Strategy for Sizing Buffer Caches7-20

Calculating Memory for Caches on UNIX7-2 1

Calculating Memory for Caches on NT7-2 2

Query Memory Requirements: Read-Only7-2 3

Strategy for Sizing Caches7-2 4

Setting Buffer Caches7-2 5

Viewing the New Buffer Caches7-28

2000 Sybase, Inc.


TOC-5

Table of Contents

Memory and Swap Usage7-29

Memory Performance7-30

Summary7-31

Lab 7: Memory Configuration7-32

Module 8 Loading Data from Files

Module Overview8-1

Data Sources8-2

Table Loading8-3

Data Loading8-4

Simple Example: Flat-File Load8-5

Adding Rows with LOAD TABLE8-6

LOAD TABLE Command: Partial Syntax8-7

Module Map8-8

load-specification / column-spec8-9

column-spec8-10

Inserting Using Fixed-Width Field Specification8-11

Inserting Variable-Length Fields 8-12

Example: Loading from a Character File8-13

Date and Datetime Considerations8-14

A Word About NULL8-16

Module Map8-17

FROM Clause8-18

Module Map8-19

load-options 8-20

DELIMITED BY Option8-21

Inserting Variable-Length Fields 8-22

QUOTES/ESCAPES Options8-23

WITH CHECKPOINT Option8-24

Putting It Together8-25

PREVIEW load-option - Input Layout Description8-26

ROW DELIMITED BY load_option8-27

Loading in Parallel8-28

More Complex LOAD TABLE Example8-29

Module Map8-30

What Happens During a load Session?8-3 1

Notification Messages8-3 2

Transaction Processing and LOAD TABLE8-3 3

Deleting Data8-3 4

DELETE Statement8-3 5

DELETE, then LOAD TABLE8-3 6

UPDATE Command8-3 7

Summary8-3 8

Lab 8: Loading Data from Files8-3 9

TOC-6


2000 Sybase, Inc.

Table of Contents

Module 9 Advanced Data Loading from Files

Module Overview9-1

Review: Options for Table Loading9-2

Review: LOAD TABLE Command9-3

Advanced Insert Load Options9-4

Module Map9-5

Multifile Load9-6

Example: Multifile Load9-8

Module Map9-9

Partial-Width Loads9-10

What Is a ROWID?9-11

Partial-Width Loads9-12

Example: Partial-Width Loads9-14

Example: Partial-Width Load - The Right Way9-15

Example: Partial-Width Load - The Wrong Way9-17

Module Map9-19

Adding dbspaces During a Load9-20

Good Loading Practices9-21

Summary9-22

Lab 9: Advanced Loading Data from File9-23

Module 10 Loading Data Using the INSERT Command

Module Overview10-1

Table Loading10-2

INSERT from SELECT Command10-3

INSERT VALUES Command10-4

Module Map10-5

Partial-Width Inserts 10-6

Example: Partial-Width Insert - The Right Way 10-7

Module Map10-9

Inserting from the IQ Main Store 10-10

Module Map10-13

Inserting Directly from a Foreign Database10-14

Using CIS10-17

Summary10-18

Lab 10: Loading Data Using the insert Command 10-19

Module 11 Using ASIQ

Module Overview11-1

ASIQ SQL11-2

Review: SELECT Statement11-3

SELECT Statement11-4

Module Map11-6

ASIQ-Specific SQL Features 11-7

2000 Sybase, Inc.


TOC-7

Table of Contents

KEY JOIN11-8

NATURAL JOIN11-9

OUTER JOIN11-10

Expressions in Subqueries11-11

IF Condition11-12

CASE Statement11-13

A Word on Quotes 11-14

Module Map11-15

ASIQ Numeric Functions11-16

ASIQ String Functions11-17

ASIQ Date and Time Functions11-18

ASIQ Datatype Conversion Functions11-19

Additional ASIQ Functions11-20

Module Map11-21

Exporting Data from ASIQ11-22

Exporting Data Using Redirection11-23

Exporting Data Using Redirection11-24

Exporting Data Using Data Extraction11-25

Options for Data Extraction11-26

Data Extraction11-28

Summary11-29

Lab 11: Using ASIQ 11-30

Module 12 ASIQ Operational Data Administration

Module Overview12-1

Module Map12-2

Transaction Processing12-3

What Is a Transaction?12-4

Transaction Commands 12-5

Chained and Unchained Transaction Modes12-8

Completing Transactions12-9

Module Map12-10

Locking12-11

Locks forDML12-12

Locks for DDL12-13

ANSI Isolation Levels12-14

Module Map12-15

Snapshot Versioning12-16

Versioning Level12-17

Table Versioning12-18

Versioning Example12-19

Option to COMMIT After Every SELECT12-24

ODBC AutoPreCommit12-25

Snapshot Versioning: Issues12-26

Module Map12-27

Data Security12-28

Users12-29

TOC-8


2000 Sybase, Inc.

Table of Contents

Users and Groups12-30

Special User12-31

Creating Users12-32

Creating Users with Sybase Central12-33

Special Groups12-35

Creating Groups12-36

Adding a User to a Group12-37

Table and View Permissions12-38

Stored Procedure Permissions 12-39

REVOKE Statement12-40

Permissions Hierarchy12-41

Database Object Names and Prefixes12-42

Summary12-43

Lab 12: ASIQ Operational Data Administration12-44

Module 13 Joins

Module Overview13-1

Types of Joins13-2

Ad-Hoc Joins13-3

When to Use Ad-Hoc Joins 13-4

When to Create a Join Index13-5

Ad-Hoc Joins Versus Join Indexes13-6

Creating a Join Index13-7

Step 1: Create Tables13-8

Step 2: Define the Relationships13-9

Join Hierarchy13-10

Types of Join Hierarchies13-11

Entity-Relationship Diagram13-12

Join Hierarchy with the Top Table13-13


Step 3: Create the Indexes13-15

Step 4: Load Data13-16


Step 5: Create the Join Index13-18


Step 6: Synchronize Join Indexes13-23

Multiple-Table Join Example13-25

To Create the Join Index13-26

Rules for Join Indexes13-27

Estimating Join Index Size13-28

Actual Join Index Size13-29

Dropping a Join Index13-30

Data Modification in a Join Index13-31

How Join Indexes Are Used13-32

Summary13-33

Lab 13: Joins 13-34

2000 Sybase, Inc.


TOC-9

Table of Contents

Module 14 Monitoring and Troubleshooting

Module Overview13-1

Module Map13-2

Monitoring Users13-3

Monitoring Loads/Deletes/Inserts13-4

Monitoring Loading Messages13-5

Monitoring Load Messages 13-7

Monitoring the Database13-8

dbcc_option13-12

Monitoring the Database13-14

Monitoring Buffer Caches13-17

Stopping the Buffer Cache Monitor13-21

System Utilities13-22

Module Map13-23

Troubleshooting Fields13-24

ASIQ Server Won't Start13-25

User Cannot Access the Database

or Database Objects13-26

Database Fills Up During the Load13-27

Processing Slows Down or Hangs13-28

Reporting Problems to Technical Support13-29

Summary13-30

Lab 14 -Monitoring and Troubleshooting 13-31

Module 15 Performance and Tuning

Module Overview15-1

Module Map15-2

IQ Query Engine15-3

IQ Query Engine Architecture15-5

IQ Query Optimizer15-6

IQ Query Optimizer15-9

Module Map15-10

Choosing Correct Index Types15-11

Database Options for Performance15-14

IQ Query Plans15-15

IQ Query Tree15-16

IQ Query Plans15-17

IQ Query Plan Detail15-23

QUERY_ROWS_RETURNED_LIMIT15-24

NOEXEC and ROWCOUNT Options15-25

IQ Query Plan -Elements15-26

More Query Options15-28

IQ Query Performance15-29

Module Map15-30

Loading Data from a Fixed-Length ASCII Flat File 15-31

TOC-10


2000 Sybase, Inc.

Table of Contents

Using Disk Striping15-32

Using Multiple dbspaces15-33

Spreading Access Across Separate Disks15-34

Improving INSERT/UPDATE/DELETE Performance15-35

Memory Allocation15-36

Limiting Number of Active Queries on the IQ Server15-38

Summary15-39

Lab 15: Performance and Tuning15-40

Module 16 Backup and Restore

Module Overview16-1

Backup Features16-2

Archive Devices16-3

Backup16-4

Types of Backups16-5

Running a Backup16-6

BACKUP DATABASE Syntax16-7

Example: Simple Backup16-8

SIZE and STACKER Options16-9

Example: Incremental Backup16-10

Backup: Notes and Restrictions16-11

Validating Your Database16-12

sp_iqcheckdb16-13

Restore Features16-14

Before You Restore16-15

Restore Database16-16

Restore Database: Full Restore16-17

Moving Database Files16-18

Restore Database16-20

Restore Database: Notes and Restrictions16-21

ASIQ Backup and Restore Performance16-22

Estimating Dump Size16-23

DBLOG Utility16-24

Option for Backing Up a Large Database16-25

Summary16-26

Lab 16: Backup and Restore 16-27

Module 17 Multiplexing Overview

Module Overview17-1

What Is IQ Multiplexing?17-2

Multiplex Architecture17-3

Multiplex Features17-4

ASIQ Architecture (Simplex)17-6

ASIQ Architecture (Multiplex)17-7

Write and Query Servers17-8

Communication between Servers17-9

2000 Sybase, Inc.


TOC-11

Table of Contents

Multiplex Operating Modes17-10

IQM Advantages17-11

Summary17-12

Lab 17: Multiplexing Overview17-13

Module 18 ASIQ Multiplex Hardware

Module Overview18-1

Module Map18-2

Storage Basics18-3

RAID Basics18-4

RAID Diagram18-5

RAID Levels18-6

RAID Striping18-7

RAID Connectivity18-8

Module Map18-9

Configuring Storage18-10

IQM and Shared Disk Farm18-11

IQ Multiplex Hardware ... Putting it together18-12

Summary18-13

There is no lab for this module18-14

Module 19 Creating a Multiplex Database

Module Overview19-1

Steps to Set up Multiplex19-2

Step 1 - Create the Write Server19-3

Create the IQ Write Server19-4


Step 2 - Create Reader (Query) Servers19-12

Create Query Server19-13

Directory Structure19-18


Step 3 - Synchronize19-20

Synchronize19-21

What we have so far19-22


Why Add A Temp Store?19-24

Step 4 - Add Temp Store for Writer19-25

Add Temp Store for Writer19-26


Step 5 - Add Main IQ Store dbspaces19-31

Start Server in Simplex Mode 19-32

Add Main IQ Store dbspaces19-33

Add Main Store Dbspaces19-38

Stop the Writer Server19-39

TOC-12


2000 Sybase, Inc.

Table of Contents


Step 6 - Add Temp Store for the Query Server19-41

Summary19-42


Module 20 Multiplex Operations

Module Overview20-1

Working in the Multiplex Environment20-2

Creating Database Objects20-3

Setting Memory20-4

Loading Data20-5

Running Queries20-6

Specifying Database Options20-7

Postsync.sql Script20-8


Setting Startup Parameters20-10

Starting and Stopping20-11

Select the Start Mode20-12

Stopping Multiplex Server20-13

Stopping Individual Servers20-14

Managing Servers20-15

Restarting Servers20-16


Adding and Dropping Dbspaces20-18

To Add A Permanent (Main) Dbspace20-19

To Create a Temporary Dbspace20-20

To Drop a Main or Temporary Dbspace20-21


Backup the Multiplex20-23

Restore the Multiplex20-24

Summary20-25


Appendix A Migrating Data from Prior Versions

Appendix OverviewA-1

Why Is Data Migration Necessary?A-2

Migration StepsA-3

Step 1 - Install Version 12.0A-4

Step 2 - Create 12.0 DatabaseA-5

Method 1A-6

Method 2A-7

IQ CREATE ASIQ TEMPLATEA-8

Edit ScriptsA-9

Database Creation ScriptA-10

2000 Sybase, Inc.


TOC-13

Table of Contents

Module MapA-15

Migration ScriptsA-16

Data Extract ScriptA-17

Data Load ScriptA-19

Editing ScriptsA-21

Running ScriptsA-22

Module MapA-23

Load the Data from your Original SourceA-24

Module MapA-26

Insert the Data from an IndexspaceA-27

Module MapA-28

Select Data into an ASCII fileA-29

bcp Data into an ASCII fileA-30

Migration NotesA-31

SummaryA-33

TOC-14Administering Adaptive Server IQ for the Data Warehouse, Version 1.02000 Sybase, Inc.

1

Adaptive Server IQ Product Overview


Version 1.0 2000 Sybase, Inc.


Module Overview

This module will provide a brief overview of Adaptive Server IQ (ASIQ)

Objectives

After completing this module, you should be able to:

List the key advantages of ASIQ

Describe the principles at work behind ASIQ

Explain what ASIQ is

Explain what ASIQ is not

This module answers the basic questions about how ASIQ's uniquetechnology can be used for decision support systems (DSSs), data marts, ordata warehousing.

Note This is an introductory module.

Later modules discuss the architecture as well as the indexes in greater detail.

2000 Sybase, Inc.Administering Adaptive Server IQ for the Data Warehouse, Version 1.01-1


When to Use Adaptive Server IQ (ASIQ)?

Decision support system (DSS) Distributed data mart Data warehouse

DSS

Data mart

Data warehouse

Decision support system.

Although some queries will be requested regularly, management often asks for unexpected aggregates of data that traditionally require individualized tuning for each query.

A special-purpose data warehouse focused on a single strategic or efficiency issue.

A data mart often serves as a single department's analytical base.

Extremely large database encompassing an enterprise-wide set of data.

A data warehouse can be used as the source of data for a company's data marts or for direct DSS queries.

1-2


2000 Sybase, Inc.


Using ASIQ to Split DSS and OLTP

Off-Loads DSS Query Burden from an OLTP Engine

Splitting DSS and OLTP

Reduces contention between readers and writers

Ad-hoc query applications, used in decision support systems

Queries are not predefined

Are often I/O intensive

OLTP

DSS

Ad-hoc query Reduce contention

Online transaction processing involves heavy inserting, updating, and deleting.

Typically, queries bring back one row or only a few rows. Indexes are used sparingly and a Database Administrator can tune for the limited types of queries performed.

Decision support systems are typically read-only data that has been filtered, preprocessed, scrubbed, or rolled up for analysis. The data may have originated in an OLTP database and may be as new as one day old.

A query that the system has no prior knowledge of and for which no explicit tuning can therefore be done.

Typically, readers block writers and writers block readers. Contention results. Separating scan-intensive DSS users and OLTP users is beneficial to both.

2000 Sybase, Inc.


1-3


Distributed Data Mart

Specialized Servers for Speciaized Needs

Data mart example

This example shows a number of clients accessing a specialized data mart, depending on the client's needs.

1-4


2000 Sybase, Inc.


Data Warehouse

Centralized Enterprise-Wide Data Warehouse

Example

This example shows a number of clients accessing a central, enterprise-wide data warehouse ASIQ Server.

2000 Sybase, Inc.


1-5


What Is ASIQ?

A Powerful Ad-Hoc Query Server

Designed to dramatically increase thespeed of DSS queries performed againstlarge amounts of data

Short list of features:

Vertical data storage and compression

Optimized access methods for all types of data

Uses bit-wise storage technology

Ad-hoc query optimizer

Standard open interfaces

Supported by Sybase Central

Access method Sybase Central

All of the features are discussed in detail in later modules.

An access method is really an index that is defined on each column.

Database management tool in a graphical user interface.

1-6


2000 Sybase, Inc.


What ASIQ s Not

An online transaction-processing engine

Use Adaptive Server Enterprise instead

A tool for tuning an existing database

A server for applications that require real-time updates

Use Adaptive Server Enterprise or Adaptive ServerAnywhere instead



Advantages of ASIQ: Overview

Speed

Designed for ad-hoc queries

Reduced I/O on retrieval

Fast to implement

Economy

Set up once for all queries

Excellent data compression characteristics

Reduced disk storage

Flexibility

Interoperates with multiple datasources

Works with off-the-shelf query tools

Speed

Economy

Flexibility

Consistently fast response time for complex ad-hoc queries. Accesses only the data relevant to the query, reducing I/O up to 98%. Accelerates iterative analysis and information.

Enables Database Administrators (DBAs) to tune data once rather than for each query.

Supports a standard relational framework, Sybase's Open Client interfaces, and ODBC, allowing DBAs to leverage existing skills and infrastructure.

Provides more answers to more questions for more users, enhancing your productivity.

Improves performance without adding hardware.

Compresses data, dramatically reducing disk requirements and cost.

Loading from various data sources, including SQL Server, Adaptive Server, and flat files. Can be directly accessed through OmniConnect, which includes a large number of gateways.

Works with a wide range of popular, off-the-shelf query tools, giving users their choice.

Handles any database schema, allowing business users to see data their way.

1-8


2000 Sybase, Inc.


Advantages of ASIQ: Speed

Scalability

Handles large amounts of data

Takes advantage of multiple processors for loading andqueries

Multiplexing allows a multiple server configuration

Speed of loading

Fast loads are a critical success factor fora data warehouse

Indexes can be incrementally loaded or

deleted without being dropped and recreated

Can load data while users are querying

Scalability example

Multiplexing

When ASIQ query workload increases by 20 times, average ASIQ response time increases by only 5 times.

Will be discussed in further detail in later modules.

2000 Sybase, Inc.


1-9


Advantages of ASIQ: Economy

Saves Disk Space by Storing Only the Index

Index sizes are kept small through a variety of data compression technologies

The entire database can be fully indexed and made available for ad-hoc queries in less space than it takes to store the raw data

Less space

No summary tables needed

Compact indexes

Typically 50-100% of the raw data size.

The indexes are the data; there is no need for both structures.

Summary tables are typically not needed because ASIQ returns queries so quickly without them.

More compact, bit-wise indexing requires less disk space than traditional b-tree indexes.

1-10


2000 Sybase, Inc.


Advantages of ASIQ: Flexibility

Flexible (open systems support)

Integrated with Open Client and Open Server

Integrated with ODBC, JDBC

To the end user

and DBA, ASIQ looks just like another RDBMS

(continued . . .)

Note

ODBC driver

The most recent list of tested tools can be found in the ReleaseBulletin that came with your software.

Some of the PC tools are BrioQuery, Business Objects, Forest &Trees, GQL, Holos, InfoMaker, Iridon Panorama, Microsoft Access,MicroStrategy and Cognos.

Special ODBC driver ships with ASIQ.

2000 Sybase, Inc.


1-11


Advantages of ASIQ: Flexibility

Can load data from a variety of sources Client

Example

The graphic above shows that data can be loaded directly from an ASE database.

Data can also be loaded from flat files as well as other data sources. See Module 8, "Loading Data from Files," Module 9, "Advanced Data Loading from Files" and Module 10, "Loading Data Using the INSERT Command."

1-12


2000 Sybase, Inc.


What Principles Are at Work?

Column-wise data structures

Bit-wise indexing

This is an overview foil. These principles are covered in the next few pages.



Column-Wise Data Structures

Stores Data Column-Wise Rather Than Row-Wise

Vertical partitioning of data

Returns those columns necessary for a query, reducing I/O

SELECT count(*) FROM customer WHERE gender = "M"

Column-wise

Row-wise

Column-wise data structures are easily searched without having to scan the full rows.

Vertical partitioning of the data means never performing a table scan. At most, ASIQ would perform a column scan.

Row-wise data structures, such as those used in traditional RDBMSs, can be searched only by reading an entire page and then locating the data on the page.

Because the columns being searched for are only a fraction of the total data read, there is excessive I/O.

1-14


2000 Sybase, Inc.


Bit-WiseIndexing

Benefits Both Low- and High-Cardinality Columns

Translates the distinct data elements of a column into a bit mask

Cardinality - The number of unique values in a column

Low-cardinality data: Fewer than 1500 unique values

High-cardinality data: 1500 unique values or more

Example:

High-Cardinality Data

Low-Cardinality Data

Primary keys

Day of week

Telephone numbers

Gender

Social security number

States

(continued . . .)

Bit-wise indexing

Unlike other indexing methods, ASIQ indexes are based on the cardinality of the data and the way the data will be used.

ASIQ has indexes for both low- and high-cardinality columns.

ASIQ indexing types are discussed in depth in Module 5, "ASIQ Index Types and Data Types".

2000 Sybase, Inc.


1-15


Bit-WiseIndexing

Low-Cardinality Example

The table on the left shows the unindexed data The table on the right shows a bit-wise index

(continued . . .)

Note

This is not an actual index.

It is merely a representation of how bit-wise indexing works for low-cardinality data.

This topic is discussed in detail in Module 5, "ASIQ Index Types and Data Types".

1-16


2000 Sybase, Inc.


Bit-Wise Indexing

High-Cardinality Example

Actual datain Binary

Data is stored vertically

High cardinality/ bit-wise indexing

Sybase has patented technology for indexing high-cardinality data using bitmaps.

2000 Sybase, Inc.


1-17


ASIQ Multiplexing

Allows a multi-server configuration

Designed for managing large query loads across multiple nodes

Supports many users, each executing complex queries against a shared database

IQ Temp Store

Catalog Store

IQ Temp Catalog IQ Temp Catalog Store Store Store Store

IQ Main IQ Temp Catalog Store Store Store

ASIQ Multiplexing

Must have ASIQ 12.4.2 or higher. Multiplexing is implements and managed through Sybase Central.

This topic is discussed in detail in Module 17, Multiplexing".

1-18


2000 Sybase, Inc.


Benefits of ASIQ

Reduced ongoing tuning of databases

Reduced need for Database Administrator resources after initialinstall

Reduced disk requirements

Rapid access to information

Users can access data with a variety of query tools

Indexes can be efficiently incrementally added or dropped

Without reloading data

ASIQ Version 12.4.2 can support a multi-server configuration,ASIQ Multiplex

For more information about the benefits of ASIQ, see www. sybase. com/products/dataware/iqindex. html.



Summary

ASIQ Is...

Leading technology that stores data vertically

Fast at ad-hoc query resolution

Bit-wise storage solves low- and high-cardinality queries

Economical on disk usage due to compressed storage formats

Flexible, interoperable, open system

ASIQ Is Not...

An OLTP database

For applications requiring real-time updates

A tool to tune an existing database

1 - 20Administering Adaptive Server IQ for the Data Warehouse, Version 1.02000 Sybase, Inc.


Lab 1: ASIQ Product Overview

Exercise Overview

TopicsDSS

.OLTP

High- and low-cardinality data

Row-wise and column-wise data structures

Bit-wise indexing

GoalAfter completing this lab, you should be able to:

Explain the basic principles of ASIQ

General tasks You will take a written quiz.Lab setupNo lab setup is required.


2

Architecture and Terminology




Module Overview

This module provides the necessary background information for performing installations and administration.

Objectives


Describe the ASIQ architecture

Define ASIQ terminology and concepts



ASIQ Platforms

Sun Solaris (SPARC)

IBM RISC System/6000 AIX

HP-UX HP 9000/800

Compaq UNIX

Windows NT

SGI IRIX

LINUX (coming soon)

Refer to the Adaptive Server IQ Installation and Feature Guide for detailed information on each platform.

Refer to the Adaptive Server IQ Release Bulletin for detailed information on product restrictions, operating system patches, new and changed features, and documentation errors.

2-2Administering Adaptive Server IQ for the Data Warehouse, Version 1.02000 Sybase, Inc.


Module Map

ASIQ Database Server

ASIQ Database

IQ Store

Catalog Store & Transaction Log

IQ Temporary Store

IQ Message Log

ASIQ Tables

ASIQ Indexes



Adaptive Server Anywhere and Adaptive Server IQ

Adaptive Server Anywhere (ASA) is a relational databasesystem that can exist with or without IQ

Small memory footprint

ANSI SQL plus additional functionality

Proven database technology

Adaptive Server IQ (ASIQ) exists with ASA

Patented storage technology

Vertical data storage

High-performance database query engine

Anywhere database

Anywhere tables

Enterprise database

T-SQL

An Adaptive Server Anywhere database.

Anywhere tables have a different format from IQ tables.

An Adaptive Server Enterprise (ASE) database. Also known as a SQL Server store.

Transact-SQL is a superset of Structured Query Language (SQL). It includes all ANSI standard SQL plus additional commands.

2-4


2000 Sybase, Inc.


Adaptive Server IQ Database Server

Combination of Adaptive Server Anywhere network server andASIQ Server

ASA contains the catalog, language parser, and connectivity

IQ pieces include data store and query engine

ASIQAllows multiple users to query a database at the same time,

while another user inserts or deletes data.



ASIQ Server and Database (Wthout Multiplex)

Database specifically created to store ASIQ indexes

Includes the following components:

IQ Main Store (for data)

Catalog Store & Transaction Log (for metadata)

IQ Temporary Store (for temporary data)

IQ Message Log (for log file)

Has one database per ASIQ Server

Components of an ASIQ database are discussed in detail in the following slides.

ASIQ Multiplex has a similar component structure and is addressed later.



IQ Main Store

Contains the data (indexes)

Compressed in blocks on disk

First physical file name is dbname.iq

User specifies the name for additional files

One IQ Main Store per database

Can consist of many physical files

Can be raw or file system

Also contains an internally managed transaction log

2000 Sybase, Inc.


2-7


Catalog Store

Contains information called metadata needed to manage an IQdatabase

Contains the system tables and stored procedures

Traditional horizontal storage system

Physical file names are:

dbname.db

dbname.log (Transaction Log)

One Catalog Store per database

Can consist of multiple devices (unusual)

File System ONLY

(continued . . .)

Metadata Describes the data in your databasefor example, the size and

datatype for each column in a table.

An ASA type datastore with 4K data pages.



Catalog Store

System Tables

In addition to the system tables, the Catalog Store contains ASIQ-specific system tables, for example:

SYSIQCOLUMN

SYSIQFILE

SYSIQINDEX

SYSIQINFO

SYSIQTABLE

(continued . . .)

SYSIQCOLUMN

SYSIQFILE

SYSIQINDEX

SYSIQINFO

SYSIQTABLE

Lists information on columns in every table or view in the IQ store.

Lists information on operating system files for the database. Lists internal information on indexes in the database.

Lists database characteristics in addition to those in the SYSINFO table.

Describes each table or view in the IQ Store.

For a complete listing of system tables and views, see the Adaptive Server IQ Reference Guide.

2000 Sybase, Inc.



Catalog Store

Transaction Log

Only used to log changes to the Catalog Store (not thewarehouse data)

Used to roll back (undo) or roll forward changes if necessary

Physical file name is dbname.log

m Very small file (~ 15 to 20MB ) Placement not an issue

Default

CREATE DATABASE command

Pruning

By default the dbname.log file is placed in the same directory as the .db file (but can be placed elsewhere).

You can turn logging off using the CREATE DATABASE command, but this is not recommended.

To prune this log, first stop the server and rename the dbname.log file, then restart the server.

2-10


2000 Sybase, Inc.


IQ Temporary Store

Work area for the database

Sorting data during loads

Query processing work area

Physical file name is dbname. iqtmp

One IQ Temporary Store per IQ database

Can consist of several physical files

Can be raw or file system

The IQ Temporary Store is used for both loads and queries. It is used to build HG indexes during loads. Queries use it to sort data during ORDER BY and join queries.

2000 Sybase, Inc.


2-11


IQ Message Log

Readable log file for the ASIQ database that contains:

Error messages

Status messages

Insert notification messages

Physical file name is dbname. iqmsg

One IQ Message Log per IQ database

File system only

2-12


2000 Sybase, Inc.


Putting It All Together...ASIQ Database

ASIQ Database Components (with No Multiplex)

2000 Sybase, Inc.


2-13


Module Map

ASIQ Database Server

ASIQ Database

IQ Store


IQ Temporary Store

IQ Message Log

. ASIQ Tables

ASIQIndexes



ASIQ Tables

ASIQ tables are logical tables defined in the Catalog Store

A table is really a group of columns

Three types of tables:

Base tables

Local temporary tables

Global temporary tables

These tables are discussed in great detail in Module 6, "Creating Tables and Indexes."

Base tablesThese tables:

Are sometimes called main or permanent tables.

Are stored in the main IQ Store and are a permanent part of thedatabase.

Must be dropped explicitly.

Use the CREATE TABLE statement to create a base table.

Local temporary tables Use the DECLARE LOCAL TEMPORARY TABLE statement. The table will exist until the connection ends or within a compound statement in which it is declared. This table is completely inaccessible to other users.

Global temporary tables Use the GLOBAL TEMPORARY option of the CREATE TABLE

statement. The table will exist in the database until it is explicitly removed. Each user will have a separate instance of the data in a global temporary table. Those rows are visible only to the connection that inserts them. They are deleted when the connection ends.

2000 Sybase, Inc.Administering Adaptive Server IQ for the DataWarehouse, Version 1.02-15


Utility Database

Created automatically during ASIQ installation

Used for connection purposes when no "real" database exists

Never holds data

DO NOT DELETE this database

When to use the utility database

To start the database server when no database is specified during the START ENGINE command.

To create, drop, or restore a database when you have no other database to which to connect.

To start the database server or connect to a database when other databases are corrupt or unavailable due to media failure.

2-16


2000 Sybase, Inc.


ASIQ Indexes

Each data column is assigned one or more data access methods(indexes)

Usually determined before initial load

Index type selection determined by:

Cardinality of data

Column usage

Four different types of indexes (discussed in Module 5):

Default Index (FP)

LF

HNG

HG

Cardinality Column usage

Low - Fewer than 1500 unique values. High - 1500 or more unique values.

Joins, GROUP BY, aggregation.

Index types are discussed in further detail in Module 5, "ASIQ Index Types and Data Types."

2000 Sybase, Inc.


2-17


ASIQ Tables and ASIQ Indexes

Tables Contain Indexes on Columns

An index is an access method

Each ASIQ index contains all the data for a column

A column has one or more indexes

A default index is placed on each column when the table iscreated

2-18


2000 Sybase, Inc.


interfaces File and ASIQ Connectivity

TDS clients and servers find each other via lookups in aninterfaces file

The interfaces file is required for:

Servers, to discover the net addresses they should listen tofor client connections

Clients, to discover where to direct connection requests

The interfaces file has different names on different platforms:

UNIX: $SYBASEinterfaces

Windows: %SYBASE%\ini\sql.ini

Novell Netware (clients only): interfac

TDS clients and CIS servers use the interfaces file to find anASIQ Server instance

ASIQASIQ uses the command line during startup and not the interfaces file.

TDS clients (for example, SQL Advantage) will need the interfaces file, but DBISQL and Sybase Central do not.



ASIQ Architecture

Putting It All Together...

2-20


2000 Sybase, Inc.


Summary

An ASIQ database contains:

IQ Main Store (contains the data)

Catalog Store & Transaction Log (contains metadata)

IQ Temporary Store (contains temporary data)

IQ Message Log (contains database log file)

The ASIQ table definition is stored in the Catalog Store

The data is stored in indexes in the IQ Main Store



Lab 2: Architecture

Exercise Overview

Topics Process architecture components and functions

ASIQ architecture

Terminology:

ASIQ database server

ASIQ database

IQ Store


IQ Temporary Store

IQ Message Log

ASIQ tables

ASIQ indexes

GoalsAfter completing this lab, you should be able to:

Define ASIQ architecture and terminology

General tasks Complete a matching game.

Complete a multiple-choice quiz.

Complete fill-in-the-blank questions.

Participate in an instructor-led discussion.

Lab setup No lab setup is needed.


3

ASIQ Environment and Installation




Module Overview

This module provides the necessary background information to install ASIQ and to connect to ASIQ.

Objectives


Prepare for an ASIQ installation

Install ASIQ

Verify the success of an ASIQ installation

This module is divided into two sections:

Environment & Installation.

Connectivity.



Module Map

System Requirements

Disk space

Memory

Swap

Installing ASIQ

Validating ASIQ Installation

Setting Up ODBC

Setting Up JDBC and Open Client



Disk Space

IQ binaries

UNIX: ~300MB (if all binaries are installed)

NT: ~120MB

Database size

IQ Main Store: 70 - 100% of raw data

IQ Temporary Store - Varies based on users and data

Data staging area

Space required to load raw data from file

Swap

2-3 times the amount of physical memory (RAM)

Database size Staging area

The IQ binary size will vary from platform to platform. Refer to the Adaptive Server IQ Installation and Feature Guide for a specific platform.

The size of the IQ Main Store depends on the types of IQ indexes used and the data itself.

If you are loading from another server, you do not need this additional space.

2000 Sybase, Inc.


3-3


Virtual Memory

Virtual = Physical Memory + Swap Disk Space

Recommended minimum of 512MB of RAM

When the system is out of physical memory, swapping occurs

Processes are copied (swapped) out of memory to disk

Inactive pages are written from memory (paged) to disk

ASIQ has its own buffering system

Decides how to page buffers in and out of memory

Uses the operating system's swap disk

More is better

Paging and swapping

VMSTATorSAR

Sybase recommends 256MB of RAM dedicated to ASIQ.

512MB allows for the operating system and other necessarycomponents.

As with any RDBMS software, ASIQ requires a lot of memory. The more memory you can allocate to ASIQ, the better your system performance will be.

There is always a fixed limit to the amount of memory in a system. As a result, operating systems sometimes can have only part of the data in memory and the rest on disk. Paging or swapping occurs when the operating system must go out to disk and retrieve any data before a memory request can be satisfied. The primary objective of good memory management is to avoid or minimize paging or swapping.

Use the UNIX VMSTAT command or the SAR command, if available, to get statistics on the number of running processes and the number of page-outs and swaps. Use this information to find out whether the system is paging excessively, then make any necessary adjustments. Try to put your swap files on special fast disks. For more information on VMSTAT, type MAN VMSTAT.

3-4


2000 Sybase, Inc.


Swap

Also Known As Swap Disk or Swap Partition

Recommendation: Swap should be set between 2-3 times thephysical memory

See specific operating system information about extendingswap

ASIQ uses shared memory for reads and writes

ASIQ pages out buffers to the swap disk

Insufficient swap space may result in:

System supporting fewer users

A large process may be stopped for lack of virtual memory

Swap files The most frequently used operating system files are swap files.

When memory is exhausted, the operating system swaps pages of memory to disk to make room for new data.

When the pages that were swapped are called again, other pagesare swapped, and the required memory pages are brought back.

This is very time-consuming for users with high disk-usage rates.In general, organize memory to avoid swapping and thus minimizeuse of operating system files.

Examining swapOn HP-UX: as root, type /etc/swapinfo

On Solaris:/usr/sbin/swap -s On Compaq: as root, type /sbin/swapon -s On NT: Control Panel->System->Performance



Basic Configuration Recommendations

System Requirement

Recommendation

Disk Space

IQ binaries (~120- ~300MB) + database size (70-100% raw data) + staging area + IQ temporary space + swap (2-3 times RAM)

RAM

Recommend 512MB (more is better)

IQ temporary space

The IQ binary size number can vary from platform to platform.

Check the Adaptive Server IQ Installation and Feature Guide for aparticular platform.

Default is set to 50% of database size.

3-6


2000 Sybase, Inc.


Hardware Recommendations

Number and Type of Processors

More is better

Faster loading with SMP

Better multiuser performance

Faster is better

233 MHz (or more)

IQ is more CPU intensive than I/O intensive

CPU intensiveThe reason ASIQ is more CPU intensive is because of compressing and

decompressing data.



Module Map

System Requirements

. Installing ASIQ

UNIX install

NT install


Setting Up ODBC




Before Installing Software on UNIX

Decide what components to install

ASIQ Server will likely be installed

Optional Components

Sybase Open Client Libraries

Only need if you will load data from another Sybase server or if you want to use the ISQL client

a Version 11 or Version 12, not both

a Should install Open Client 12 unless Sybase ASE 11.x server is running on the same UNIX server

Sybase Central on the UNIX server

You need Sybase Central only if you want to run it on theUNIX server

May be installed on a Windows 9598/Win NT client

2000 Sybase, Inc.Administering Adaptive Server IQ for the Data Warehouse, Version 1.03 - 9


Preparing for ASIQ Installation on the UNIX Platform

Before You Begin

1. Check for proper version of the OS

2. Install kernel patches (if any)

3. Set kernel parameters to suggested values

4. Configure user resource limits with ULIMIT -A, then reboot

5. Verify network functionality with TELNET host

For more information on the individual platforms, see Chapter 1, "Adjusting the Operating System Configuration," in the Adaptive Server IQ Installation and Configuration Guide.



Installing Adaptive Server IQ

UNIX

1. Set the environment variable $SYBASE to a file system for theASIQ binaries

2. Create the user "sybase"; home for the user "sybase" is$S YBASE

3. Mount the CD-ROM drive

4. Start the install utility, SYBINSTALL

5. Follow options on SYBINSTALL screen

Accept the default username and password for the utilitydatabase

After installation, you need to set additional environmentvariables

Mounting CD-ROM drive

Check the Adaptive Server IQ Installation and Feature Guide for the exact commands for your platform.

If you have an existing version of Open Client on your system, installing the Open Client libraries will add missing Open Client files and overwrite older files.

2000 Sybase, Inc.


3-11


Setting Environment Variables

$SYBASE

Set to the /sybase directory (should already be set)

Contains up to three subdirectories

a asiq_12

a sybase-central_3.2 (optional)

a ociq-11_0(optional) or ociq-12_0(optional)

Open Client Open Client 11 - ociq-11_0

Open Client 12 - ociq-12_0




$ASDIR

$ASDIR = $SYBASE/asiq_12

This directory contains the ASIQ Server binaries

bin/lib

charsets/res/

includescripts

java/tix/javaconnect/

m $LD_LIBRARY_PATH

$LD_LIBRARY_PATH = $ASDIR/lib

This directory contains critical ASIQ Server library files

(continued . . .)




$SYBASE_OCS

Required only if you install Open Client 12

$SYBASE_OCS = $SYBASE/ociq-12_0

$PATH

Modify your path to include the following:

a $ASDIRbin

This is for the ASIQ Server binariesa If Open Client 11 is installed:

$SYBASEbin

a If Open Client 12 is installed:

$SYBASE_OCS/bin

$ASLOGDIR (optional)

Specify a directory for ASIQ Server log messages

A new log message file will be created each time the serveris started



Starting the ASIQ Server - UNIX

ASIQ Server must be started with a database

After installation, you must start the ASIQ Server with eitherthe asiqdemo database or the utility database

asiqdemo database

Supplied with the software

Easiest way to start and test the server installation

utility database

Never holds data

A "database" is a database file with a .db extension

Example: asiqdemo.db

The ASIQ Server must be started from a writable directory.

The directory where the server is started becomes the default directory for all files created by the ASIQ Server.



Starting the ASIQ Server - Simple Method

Syntax:start_asiq dbname

start_asiq is located in $ASDIR/bin

Starts the server as a background process

Example:

% start_asiq asiqdemo.db

This example starts the server with the asiqdemo database, thisexample assumes you are in the $ASDIR/demo directory

The default start parameters will be used



Starting the ASIQ Server - UNIX

Preferred method of starting the server in UNIX

Syntax:

start_asiq @configuration_file.cfg dbname

m Example:

% start_asiq @$ASDIR/demo/asiqdemo.cfg $ASDIR/demo/ asiqdemo.db

This example starts the server with the asiqdemo database

The configuration file is used to specify options you wantwhenever you start your server

2000 Sybase, Inc.Administering Adaptive Server IQforthe DataWarehouse, Version 1.03-17


Server Configuration File

All Platforms

A file with a .cfg extension that contains server startingparameters

These parameters override default ASA engine startingparameters

Are critical parameters for ASIQ performance

Configuration file example:

-n eptgdev2_asiqdemo

-c 16m

-gd all

-gm 10

-gp 4 0 96

-x tcpip{port=2638}

Do not use commas to separate parameters in this file

Additional parameters are required to use Sybase Open Client.

Discussed in "Connectivity" module.

Example: asiqdemo.cfg

-n

eptgdev2_asiqdemo

Server name (machine name_dbname)

-c

16m

Catalog Store cache size

-gd

all

Database starting permissions

-gm

10

Number of connections (default is 10)

-gp

4096

Catalog Store page size

-x

tcpip{port=2638}

Network connection protocol; 2638 is the default port number

3-18


2000 Sybase, Inc.


Stopping the Server

UNIX

Run the STOP utility

stop_asiq

a This utility prompts the user to stop an ASIQ Server

dbstop

a Requires connection parameters

Example:

dbstop -c

"uid=DBA;pwd=SQL;eng=eptgdev2;dbn=asiqdemo"



Preparing for ASIQ nstallation on the NT Platform

Before You Begin

1.Check the OS

See the ASIQ Release Bulletin for latest OS releaseinformation

Special instructions apply to NT Enterprise servers usingmore than 2 GB of memory

2.Decide what components to install

ASIQ Server will probably be installed

Optional components

a Sybase Open Client Libraries - Version 11 or Version 12

Sybase Central on the NT server

3.Check for sufficient page file (swap) space

You will reboot ASIQ after software is installed

4.Decide on location for binaries

Approx. 110MB required for ASIQ Server binaries

Approx. 24MB for Open Client (optional)

Approx. 35MB for Sybase Central Java Edition (optional)



Installing ASIQ Server Components

Required for Operation As a Network Server

1. Log in using an account with Administrator privileges

2. Exit any Windows programs running on your machine

3. Insert CD-ROM into CD-ROM drive

4. Run setup.exe

5. Select products to be installed

ASIQ Server

Click Change button to view subcomponents

a Deselect unnecessary subcomponents (as desired)

16-bit ODBC drivers

C++ development components

Sybase Central Java Edition

Open Client

6. Accept the default login (DBA) and password (SQL) for theutility database (can be changed later)

7. Click Finish to complete installation

8. Restart computer

If you have an existing version of Open Client on your system, installing the Open Client libraries will add missing Open Client files and overwrite older files.

2000 Sybase, Inc.


3-21


Starting the Server

Start ASIQ

Click Start on the taskbar, select Programs->

Sybase->Adaptive Server IQ 12->Start ASIQ Demo Database Starts the asiqdemo database using the name of the server on which you are running ASIQ

or

At the NT command prompt:

asiqsrv12 @asiqdemo.cfg asiqdemo.db

This example assumes you are in the $ASDIR/demo directory

See the UNIX example in the student notes for contents of asiqdemo.cfg file.

To display the startup options available, enter the following at the NT command prompt:

asiqsrv12 /?

ASIQ Server must be started with a database, either the asiqdemo or utility database.

After creating a database, you can start the server with that database.



Stopping the Server

Stop ASIQ - NT

1. Double-click the icon in the Windows system trayto display the server window

2. Click Shutdown

Command-line options during startup control whether a server is stopped even if there are active connections

2000 Sybase, Inc.


3-23


Installing ASIQ Server Client Components

Required for Connection to a Network Server

1. Log in using an account with Administrator privileges

2. Exit any Windows programs running on your machine

3. Insert Client Components CD-ROM into CD-ROM drive

4. Run setup.exe

5. Select products to be installed

ASIQ 12 client components

a Decide which subcomponents to install (click Change)

16-bit ODBC drivers may not be needed

C++ development components (if needed)

Sybase Central Java Edition (for DBA)

Open Client 12 (if needed)

a Open Client used in this class a Not required at some sites

6. Click Finish to complete installation

7. Restart computer

Sybase Central and ASIQ are installed in separate folders unless you specify otherwise.



Module Map

System Requirements

Installing ASIQ


Setting Up ODBC




Validating the ASIQ Environment

Was the Installation Successful?

Examine the installation log files

Access the ASIQ Server using:

DBISQL

Sybase Central (for database administration)

ISQL

Third-party tools

First, set up ODBC and JDBC

Next section



Lab 3-1: Installation

Exercise Overview

Topics Installing ASIQ Server components


. Install ASIQ

Start the ASIQ Server using the sample database

General tasksIncrease virtual memory on your machine

Install the ASIQ Server components

Start the ASIQ Server

Define and document the ASIQ environment

Lab setup Sufficient disk space and shared memory must be available.

Must have the following CD:

Sybase Adaptive Server IQ for Microsoft Windows NT 4.0, version 12.0



Module Map

System Requirements

Installing ASIQ


. Setting Up ODBC

Connect using DBISQL (GUI)

Connect using DBISQL (Quite Mode)

Connect using Sybase Central




Setting Up ODBC

Start the ODBC Administrator

Start->Programs->Sybase->Adaptive Server IQ 12->ODBC Administrator

Create a DSN (Data Source Name) by clicking Add

(continued . . .)

2000 Sybase, Inc.


3-29


Setting Up ODBC

Add a New Source

Select the Adaptive Server IQ 12 Driver

Click Finish

(continued . . .)

3-30


2000 Sybase, Inc.


Setting Up ODBC

Name the Data Source

(continued . . .)

2000 Sybase, Inc.


3-31


Setting Up ODBC

Log In

Click the Login tab

Enter a user ID of DBA and a password of SQL

(continued . . .)

3-32


2000 Sybase, Inc.


Setting Up ODBC

Database Tab

Click the Database tab

Enter the database name of asiqdemo

Enter the full path for the asiqdemo.db (Catalog Store) file

Click Browse, if desired

Can specify server name and server start command file May want to uncheck the Auto start and Auto shut down

(continued

2000 Sybase, Inc.


3-33


Setting Up ODBC

Network Tab

Click the Network tab

Click TCP/IP

For remote servers; identify the host and port

Examples:

host=poobah:2345 host=157.133.75.36:2345

(continued . . .)

3-34


2000 Sybase, Inc.


Setting Up ODBC

Test the Connection

Click the ODBC tab

Click the Test Connection button

2000 Sybase, Inc.


3-35


Connection to ASIQ using DBISQL

Interactive SQL

DBISQL is an ODBC Client for communicating with ASIQ

Comes with the software

Preferred method for communicating with ASIQ

Useful for database administration

Can be used as a simple query tool

DBISQL has two modes

GUI

Quiet Mode (no GUI)



Starting DBISQL - GUI

Windows:

Start->Programs->Sybase->ASIQ 12->Interactive SQL

UNIX:

% dbisql -c "uid=DBA;pwd=SQL;eng=eptgdev2_asiqdemo"

Option

Description

""

Need to use double quotes

-c

Database connection parameters

uid

User id

pwd

Password

eng

Servername

2000 Sybase, Inc.


3-37


Ready to Connect

Interactive SQL

Start->Programs->Sybase->ASIQ 12->Interactive SQL

Specify the ODBC DSN you just created

3-38


2000 Sybase, Inc.


Interactive SQL

Enter commands in the bottom window

Function keys

F1 = Help for DBISQL

F7 = List tables (select table to list the columns)

F9 = Execute command (or click the Execute button)

F10 = Activate menus at top of window

Stop = Cancel command

2000 Sybase, Inc.


3-39


Connect Using Command Line DBISQL

DBISQL (Quiet mode)

Syntax:

dbisql -c keyword=value; . . . ' ] [-d] [-q] [-x]

Option

Description

-d

Command delimiter (defaut is ;)

-q

Quiet mode (no window or messages), used to submit scripts

-x

Syntax check only

Example:

dbisql -c uid=DBA;pwd=SQL;eng=eptgdev2_asiqdemo' -q load.sql

In this example load.sql is the name of the script file

Parameters

Always embedded in quotes.

Separated by a semicolon.

See Chapter 2 of the Adaptive Server IQ Administration Guide or Chapter 3 of the Adaptive Server IQ Reference Manual.

3-40


2000 Sybase, Inc.


Connecting Through Sybase Central

Graphical user interface to Sybase database products

Can use ODBC or JDBC to connect

This is a Java version of Sybase Central for use only withASIQ

To connect to ASIQ (NT):

1.Open the Adaptive Server IQ 12 folder

Click Start on the taskbar, select Programs->Sybase->ASIQ12->Sybase Central

2.Enter a login and password in the connection dialog boxTo connect to ASIQ (UNIX):

% cd $SYBASE/sybcentral % scjview



Sybase Central

Connecting to a Database

On the Sybase Central menu bar, selectTools->Connect->ASIQ

In the connection box, click ODBC and enter the data sourcename

3-42


2000 Sybase, Inc.


Sybase Central Features

Create an IQ database

Add or remove database devices

Create tables, views, and indexes

Create stored procedures

Add users and groups and manage permissions



Module Map

System Requirements

Installing ASIQ


Setting Up ODBC




Database Configuration File for Open Client 12

This step is required only if you access an ASIQ 12 with OpenClient 12

Add a parameter in the server configuration for the databasename

Use a second -n parameter for the database name

This database name is used in the interfaces file as the servername

Example:

-n eptgdev2_asiqdemo -c 16m -gd all -gm 10 -gp 4 0 96

-n asiqdemo

See the ASIQ Release Bulletin on Open Client Library changes




Set up a server entry using DSEDIT

Start->Programs->Sybase->Open Client Directory Services Editor

Click OK

(continued . . .)

3-46


2000 Sybase, Inc.



Under Server Object, click Add

(continued . . .)

2000 Sybase, Inc.


3-47



Enter the server name

Enter the network address for the server

(continued . . .)

3-48


2000 Sybase, Inc.


Setting Up JDBC

Dsedit should look similar to this:

You can ping the server to test the connection you made

2000 Sybase, Inc.


3-49


Connect Using ISQL

Start ISQL from command window

Example:

isql -UDBA -PSQL -Sasiqdemo [-i][-o]

You are connected to your database

No concept of USE DATABASE

Command terminator default is still "go"



ISQL Versus DBISQL

You cannot direct output from DBISQL

No equivalent to ISQL -o parameter

IQ has TEMP_EXTRACT options for SELECT

DBISQL command terminator is a semicolon (;)

DBISQL can start the server, if desired



sp_iqstatus

Shows information about the current database

Database name

Creation date

IQ Page size

Block size

IQ-specific stored procedure, which is valid only on ASIQ servers

If you are unable to connect and execute a command, check the "Troubleshooting" section of the Adaptive Server IQ Administration Guide.

IQ Page sizeAdaptive Server IQ displays the IQ Page size in addition to the

block size. For example:

Page Size: 1024/512blksz/2bpc



Running sp_iqstatus Command to Check ASIQ

2000 Sybase, Inc.


3-53


Summary of Installation Steps

Prepare for the ASIQ install

Disk space

RAM

Perform the installation

Follow steps outlined and use installation guide

Verify the installation

DBISQL

Sybase Central (for database administration)

ISQL

Run sp_iqstatus to show database information

For more information on the individual platform installations, see the appropriate Adaptive Server IQ Installation and Feature Guide.



Lab 3-2: Connectivity

Exercise Overview

Topics How to use the ODBC Administrator

Setting up a server entry using DSEDIT

Connecting to the server using Interactive SQL


Set up a DSN using the ODBC Administrator

Use DSEDIT set up JDBC

Connect to the server using Interactive SQL

General tasks Access the asiqdemo database using Interactive SQL and ODBC.

Run sp_iqstatus to list information about the asiqdemo database.

Install student files for later use.

Lab setup Must have installed the ASIQ Server in Lab 3-1.


4

Creating Databases



Creating Databases

Module Overview

This module describes how to create and administer an ASIQ database.

Objectives


Determine space requirements for a database

Create/drop a database

Add space to a database

Drop space from a database


Creating Databases

Database Review

Created using CREATE DATABASE command

IQ database

IQ Store Catalog Store Transaction Log IQ Temporary Store IQ Message Log File

Con

enos.itcollege.eeenos.itcollege.ee/~gseier/Sybase IQ admin guide.doc · Web view... PowerJ, Power...

Documents

Transcript of enos.itcollege.eeenos.itcollege.ee/~gseier/Sybase IQ admin guide.doc · Web view... PowerJ, Power...