The IDUG Solutions Journal August 1997 - Volume 4, Number 3

Solutions Journal: Letter From the President of IDUG

The IDUG Solutions Journal August 1997 - Volume 4, Number 3

Letter from the President of IDUG by Casey Young

Wow! Our North American IDUG conference, held in Chicago this May, was a huge success! More than 2,000 members gathered for four days of intensive learning while choosing from among 140 technical sessions. A sellout number of vendors filled the Exhibit Hall to overflowing -- they provided a second forum for serious learning. We saw IBM demonstrate the latest features of its Universal Database Version 5. We even saw Janet Perna play the piano. And we soothed our psyches as motivational speaker Keith Harrell encouraged us to take our first class plane seats -- even if our tickets showed us sitting in row 33B!

Speaking of first class, IDUG itself is a first class organization due almost entirely to its membership. A not-for-profit, independent, and user-driven organization, we are growing larger all the time. Many users have already volunteered to help us plan the 1998 North American Conference (yes, we've already begun). And, in case you missed the excitement in Chicago, please consider signing up for this fall's conferences in Europe, the Pacific Rim, or our newest offering, the Technical Symposium in Toronto, Canada.

IDUG continues to work with IBM to provide a "road map" to technical and marketing support. The road map becomes visible with "AskDB2," a new web site "hotline" for DB2 users which can be accessed directly from IDUG's web site (www.idug.org). Announced jointly by IDUG and IBM on page 39, the Q&A hotline promises an answer to your DB2 questions and/or a call back from an IBM representative within 48 to 72 hours. Meanwhile, we're updating our own IDUG Web site, as IDUG members around the globe donate their time to making these improvements. Thanks, everyone!

You hold in your hands one of the premiere journals in the world dedicated to DB2 education. The IDUG Solutions Journal is another example of what dedicated users can do to inform us all about how they've used DB2 to solve technical challenges. (Yes, you can share this information without violating company privacy rules.) Do you have a business solution using DB2 you'd like to share? Let us know! We can help you organize it for a Journal article. Perhaps you'd like to serve as a resource for the DB2 community. We'd love to hear from you about that as well.

http://arbaltxp-011870/member/journal/aug97/presidents_letter.html (1 of 2)5/5/2006 10:27:53 AM

http://arbaltxp-011870/idug/db2/askdb2.html

Solutions Journal: Letter From the President of IDUG

Although we know that DB2 is the best product out there for relational technology, IDUG has been concerned about IBM's marketing of DB2. We've shared these concerns and IBM has responded. During the last six months, it has expanded much effort to improve its marketing of DB2. Have you noticed? Seen any significant changes? Anything more you'd like to see done? Please let me know -- I'll be glad to convey your suggestions to IBM. Just write me at [email protected].

See you in Toronto!

About IDUG | Conferences | DB2 Resources | Regional User Groups | Solutions Journal | Vendor Directory

Home | Contact Us

International DB2 Users Group 401 N. Michigan Ave.

Chicago, IL 60611 (312) 644-6610

http://arbaltxp-011870/member/journal/aug97/presidents_letter.html (2 of 2)5/5/2006 10:27:53 AM

mailto:[email protected]

http://www.idug.org/html/about.asp

http://conferences.idug.org/

http://www.idug.org/idug/DB2/index.asp

http://www.idug.org/idug/rugs/default.asp

http://www.idug.org/idug/journal/index.asp

http://www.idug.org/idug/vendors/index.asp

http://www.idug.org/

http://www.idug.org/feedback.asp

Solutions Journal: Letter from the Executive Editor


Letter from the Executive Editor by Doug Stacey

Hello and welcome to our third issue of the IDUG Solutions Journal for 1997. It is with great excitement that we here at IDUG bring you this issue. We are coming off of a grand North American conference this spring in Chicago which set a new record for attendance and at which we debuted our new look for the Solutions Journal to rave reviews. This fall we'll see strong offerings for both the European and Asia Pacific conferences as well as a unique IDUG educational event in Toronto.

I really hope you too enjoyed the new Solutions Journal. We would like to hear any feedback you may have on our first "new" issue. You can email me directly or contact Linda Pearlstein, our managing editor. We continue to strive to bring you an improved product!

As you look over the table of contents, I think you will agree with me that what you now hold in your hands is the strongest issue we have ever published. Our focus this issue is performance; we have gathered an impressive list of authors: Joel Goldstein, Frank Ingrassia, Craig Mullins, Dan Kitay, and Bryan Smith are all top names when it comes to DB2 performance. Wow! I'm afraid this issue will be tough for us to top. But rest assured we are already working on a fine November issue.

In this issue, we are proud to begin a three-part series by Bryan Smith and colleagues titled "Peak Parallel Performance: Secrets a DBA Should Know." All are developers at IBM's Santa Theresa labs experienced in query parallelism and performance. This is definitely "straight from the horses' mouth," as the authors share their secrets for getting the best performance out of your DB2 for OS/390 parallel system.

Joel Goldstein takes a walk through the DB2 Statistics records in "Performance Metrics for DB2 Buffer Pools." Joel points out where to look and what to look for to get the most out of your buffer pools. In "The eX-Files: MVS Explain: the Truth is Out There," Frank Ingrassia explores the "hidden" plan tables in the DB2 catalog and reveals the useful information you may find there. (WARNING: This material is not approved or supported by IBM. Use at your own risk.)

http://arbaltxp-011870/member/journal/aug97/executive_editor.html (1 of 2)5/5/2006 10:28:04 AM

Solutions Journal: Letter from the Executive Editor

Richard Bolesta and Craig Mullins urge you to get out of the reactive performance management mode and do some proactive performance management planning in "DB2 Performance Planning and Engineering." Dan Kitay really hits a hot topic as applications like SAP R/3 drive the resurgence of the mainframe. He offers "Performance Enhancements for SAP R/3 and DB2 for OS/390."

Dan Luksetich shares his real life experience with partitions in "DB2/MVS Version 4.1--Trying Out Full Partition Independence."

We round out the issue with an excellent technical tip from Tim Lenahan, our usual columns from gurus Richard Yevich and Roger Miller, product reviews of some performance tools offered by BMC Software and PLATINUM technology, and interesting news from IDUG. Again, I hope you enjoy this issue and are as pleased with it as we are proud in putting it all together. See you next time.


Home | Contact Us


Chicago, IL 60611 (312) 644-6610

http://arbaltxp-011870/member/journal/aug97/executive_editor.html (2 of 2)5/5/2006 10:28:04 AM









Solutions Journal: Ask DB2


IDUG and IBM Introduce AskDB2

New Web site offers Q&A support for IDUG members

Geared for the DB2 user community, a new support hotline for IDUG members is being announced by IDUG and IBM.

Accessible immediately from the IDUG web site (www.idug.org), "AskDB2" is available to IDUG members world-wide; the service will supplement IBM DB2 sales and support resources available locally to users. "AskDB2" promises an answer to your DB2 question and/or a call back from a local IBM representative within 48 to 72 hours. Responding to members' needs, IDUG asked IBM to provide direct answers to questions about the DB2 family of products; such questions may address technical issues, functionality, trends, directions, and the like. Later this year IDUG and IBM intend to expand "AskDB2" to a Domino-based implementation accessible from any web browser. Q&A support will continue to be provided, but with the addition of resource lists/pointers, organized and searchable Frequently Asked Question (FAQ) responses, and links to information available on other web sites.

IDUG members are urged to continue utilizing existing IBM support services by regularly checking out IBM's DB2 web site at http://www.software. ibm.com/data for up-to-date information and fresh bulletins concerning the DB2 family. Or they can access the DB2 web site directly through the IDUG web site. So bring your DB2 question to www.idug.org andÉ "AskDB2"!


Home | Contact Us

http://arbaltxp-011870/member/journal/aug97/askdb2_announcement.html (1 of 2)5/5/2006 10:28:15 AM

http://arbaltxp-011870/idug/db2/askdb2.html

http://www.software.ibm.com/data









Solutions Journal: Ask DB2


Chicago, IL 60611 (312) 644-6610

http://arbaltxp-011870/member/journal/aug97/askdb2_announcement.html (2 of 2)5/5/2006 10:28:15 AM

Solutions Journal: SQL-Explorer


BMC Patrol SQL-Explorer

Manufacturer: BMC Software, Inc. 2101 CityWest Blvd. Houston, Texas 77042 Phone: 800/841-2031 or 713/918-8800 Fax: 713/918-8000 http://www.bmc.com

Customer company: Nordstrom, Inc. 1501 5th Avenue Seattle, WA 98101 Phone: 206/628-2111 Fax: 206/628-1795 http://www.nordstrom-pta.com

The company -- Nordstrom Inc. is a customer-focused retailer of upscale specialty apparel, shoes, and accessories with department stores and clearance outlets located in 14 states throughout the U.S. Their data operations are located at company headquarters in Seattle, Washington. Nordstrom uses Version 4 of DB2 for MVS/ESA running on IBM 9672 and 9021 mainframes. Their distributed systems run on an SNA network, with Windows NT 4.0 running on the clients. The product -- Available in MVS batch and Windows-based versions, PATROL SQL-Explorer is an SQL and DB2 plan analysis tool that can proactively correct performance problems before the application reaches production. It analyzes SQL statements and database structures to optimize the performance of applications, and can be employed as an ad hoc tuning tool for programmers as they develop SQL statements. DBAs can use it to facilitate change control for performance variables -- it allows them to identify and manage the performance impact of application changes.

PATROL SQL-Explorer enables full SQL extended plan analysis with enhanced, easy-to-understand descriptions of DB2 access pathinformation and recommendations for improving application efficiency, including relative cost calculations, comparisons, SQL text, plan table, and DB2 system catalog information. The problem -- As the DBA of a relatively large shop, I was spending more and more time analyzing SQL statements

http://arbaltxp-011870/member/journal/aug97/bmc_patrol.html (1 of 3)5/5/2006 10:28:25 AM

http://www.bmc.com/

http://www.nordstrom-pta.com/


because of application performance problems. This was a time-consuming, expensive, trial-and-error process. After recently working late hours reviewing a major application before implementation, I knew I needed a more automated way of determining what the application was supposed to be doing. Without PATROL SQL-Explorer, I would have to look in the plan table for the last access path information, then correlate it with statistics in the DB2 catalog. Also, because the catalog is always changing, today's statistics differ from those that applied when the application was first implemented. When the application is implemented again, it may perform poorly because the optimizer selected different access paths.

Strengths and weaknesses -- Our programmers can use PATROL SQL-Explorer to proactively check the syntax of SQL statements during application development to verify compliance with installation standards. As the DBA, it has helped me many times to identify and manage performance impacts of database structure changes before they are implemented in production on applications where optimal performance is crucial to our business.

The product's SQL Analysis feature enables powerful online analysis of individual SQL statements extracted from plans, packages, or DBRMs. Analyzing an application in batch creates a unique set of historical baseline snapshots containing key DB2 catalog statistics. Database access path information from the plan table can be put into context with the database and system environments at a given point in time. By correlating the two, I can get a complete picture of why an application performed the way it did.

PATROL SQL-Explorer provides two forms of analysis: using an existing plan table and using a dynamic explain. Using the existing plan table information, it automatically correlates access path information with the catalog statistics that were in place when the optimizer selected the access path. Additionally, by using the dynamic explain feature, Ican also see this type of information as it would be selected today. PATROL SQL-Explorer compares access paths, SQL text, and key catalog statistics that can impact the selection of access paths by the optimizer. These results are combined with the analysis results so you can determine what changes impacted performance.

The analysis is driven by a set of expert rules providing information showing where the performance problems are and what design changes are necessary. With PATROL SQL-Explorer, I can customize rules to enforce installation standards and detect SQL statements that should be avoided in certain circumstances. PATROL SQL-Explorer has two categories of recommendations: those related to SQL statements or bind options that do not comply with installation standards, and those related to the physical state of the database objects that can adversely influence the optimizer. The recommended solution can be as simple as having the programmer change an SQL statement, or I can run a RUNSTATS or REORG utility to provide better organization and information for the optimizer's use in selecting data access paths for the SQL statements.

PATROL SQL-Explorer enables you to exploit DB2's Extended Explain facilities. DB2



has additional plan tables that provide information from its Optimizer to help you understand each processing step within an SQL statement. PATROL SQL-Explorer automatically externalizes this information for you without incurring additional DB2 overhead, and without the need to manually define tables or change DB2 parameter settings.

The optimizer might change the access paths based solely on the differences in data types or lengths between the column and the host variable. However, it does not provide any warning to indicate that a data type mismatch occurred. PATROL SQL-Explorer produces a report that automatically finds all the DB2 plans or applications having a host variable mismatch. You can keep historical information from a previous plan or package batch analysis, and view detail about a statement or statement step, including versions, table or index statistics, and bind data. A list is provided of explainable statements from the selected object. Statements are displayed in statement number order but you can re-sort by cost or SQL text. You can view up to four versions of a statement simultaneously, and PATROL SQL-Explorer highlights any differences between the versions. It also provides features enabling you to create temporary "what if" scenarios to determine the impact of adding rows, indexes or rebinding a plan.

One limitation we see is that the "what if" feature is only available from the client, not from the ISPF interface. We would also like more integration between PATROL SQL-Explorer and BMC Software's APPTUNE product, which we use to identify problematic SQL statements. BMC has said it will address this integration in a future release.

Selection criteria -- We chose PATROL SQL-Explorer for its ease of use, especially for its capability to schedule MVS batch jobs for analysis without special training or the need to wade through numerous dialogs. We also like that it fully exploits DB2's extended plan table.


Home | Contact Us


Chicago, IL 60611 (312) 644-6610










Solutions Journal: Conference Report


News from IDUG

An estimated 2,000 attendees (a record) heard reports of DB2's continued vitality -- its 8,000 worldwide users, a surge in recent sales, and the promise, since fulfilled, of general availability for Version 5 of DB2 for OS/390 -- at the 1997 Annual North American IDUG Conference held in Chicago this May.

Choosing from among 140 sessions distributed among five technical threads, attendees gave presentations on performance their highest marks. Voted best overall speaker for his presentation on "achieving maximum performance in DB2 through Version 5" was Richard Yevich of RYC, Inc. Voted best user speaker for his talk on "estimating DB2 batch database runtimes" was Tim Lenahan of Allstate Insurance Co. (Versions of both presentations appear elsewhere in this issue of the IDUG Solutions Journal.) Presentations offering enterprise solutions won strong approval from attendees, with growing numbers choosing sessions on DB2 Universal Database and network computing.

Keynote speaker Janet Perna sported suspenders touting DB2 for OS/390 while giving a piano recital to dramatize DB2 Universal Database's implementation in object relational technologies. Not only data, but music itself, can be searched using this technology, noted Perna, who is general manager of database management in IBM's software solutions division. Assuring users that IBM has committed millions of dollars to marketing DB2 products, Perna reported outperforming analysts' projections for DB2 sales in both 1996 and 1997. Perna listed improvements in stored procedures, utility performance, Sysplex query parallelism, and terabyte-sized tables among DB2 for OS/390 Version 5's key enhancements. Later, IBM publicly honored IDUG and the 8,000 DB2 for OS/390 customers around the world with the presentation of a crystal eagle, designed to symbolize DB2's developmental code name "Eagle."

Projecting the total relational DBMS market at $4 billion in the year 2000, Gartner Group's Tony Percy foresaw success for IBM, Oracle and Microsoft, with Informix fading from the top tier of UNIX and NT DBMS vendors. NT DBMSs are the wave of the future, as are extended relational DBMSs, Percy believes; he also saw a bright future for data warehouses and operational data stores. In his general session address, Percy predicted only "temporary" success for multi-dimensional DBMSs and data marts, and a dim future for object oriented DBMSs.

http://arbaltxp-011870/member/journal/aug97/conference_report.html (1 of 2)5/5/2006 10:28:41 AM

Solutions Journal: Conference Report

Popular features of the conference included motivational speaker Keith Harrell, a nightly Speaker's Corner, where attendees could consult privately with speakers, and the exhibit hall, where more than 40 vendors offered attendees opportunities for private discussions and hands-on demonstrations. In his talk "attitude is everything," Harrell managed to both delight and entertain while imparting a serious message for enhancing one's life and career.

As reported elsewhere in this issue, the 1997 IDUG North American Award for Information Excellence went to Gene Brown of Rite Aid Corporation for his "groundbreaking" use of relational technology. The award is co-sponsored by Platinum technology, inc., and KPMG Peat Marwick.


Home | Contact Us


Chicago, IL 60611 (312) 644-6610

http://arbaltxp-011870/member/journal/aug97/conference_report.html (2 of 2)5/5/2006 10:28:41 AM









Solutions Journal: Design and Management for High-Performance Applications


Design and Management for High Performance Applications by Roger Miller

As soon as we become successful, the reward is often more to do. That may sound more like a fortune from a fortune cookie, but it is true for our applications as well as for our careers.

Today's high performance applications require a combination of large databases, high volumes, and high availability. The job becomes much simpler if any one of these requirements is eased. If the databases are not large, then the I/O is reduced, and recovery times are low. Managing for high performance and high availability requires some difficult tradeoffs. As our applications become successful, they face more challenges. Today the challenge is likely to be adding Web access, making the information more usable, and moving the operational data into a warehouse.

These days customers are running more than 1,000 batch updates per second, with concurrent online work. The challenges are to run even higher rates: thousands of transactions per second with concurrent batch, larger queries on many terabytes of data, and availability that is closer to 24 x 365.25 x 99.9999 percent. While we can move closer to 100 percent, achievable availability targets must leave some room for error -- at least until we are perfect!

Many "ities" are required to deliver high performance. These include integrity, scalability, availability, and even responsibility. Data integrity, performance, scalability, and availability are all shared responsibilities among products and customers: system administrators, database administrators, programmers, operators, and end users. To produce a successful application requires that all of us work well together. Here are some of the issues.

Continuous operations and high availability

Availability issues must be addressed individually. Planned outages generally take the most time, but unplanned outages can be very disruptive. To avoid problems, separate planning is necessary to address applications, data, and systems. Application concerns center around testing, change control, and backup versions. Data concerns include

http://arbaltxp-011870/member/journal/aug97/design_management.html (1 of 4)5/5/2006 10:28:51 AM


backup and restore processing, but also involve locking and concurrency. Systems issues include outages in the hardware and software infrastructure.

The bottom line is that continuous availability costs a significant amount of money, and you can't get what you don't pay for. Understanding the need for each application is crucial, but the reward for success is often a demand for much higher availability.

Performance for a spectrum of business needs

Performance has many dimensions: capacity, response time, price/performance, and systems management are the primary ones. These dimensions vary substantially in the various environments of DB2: transaction, batch, query, and utilities. The topic is so complex that, as you read this column, another red book is being written. Look for it soon on a CD-ROM and web page near you. Many of the red book's topics will be accurate for all of the DB2 family, but since performance and availability depends upon the platform, there will be differences.

Network computing

Network computing or client/server computing provides new challenges and new opportunities. Performance and availability can be masked or made worse by local computing facilities. Performance and availability demands change; indeed, such demands often increase in this environment, because access is more varied, and is provided to more people. Some distributed processing is similar to light transactions, while other work is like batch or intensive query. The demands of intensive text, image, and video are substantial, especially for the I/O subsystems.

Set objectives and monitor them

With many applications, one can avoid the need for written objectives and careful performance monitoring. But high performance applications demand more careful management. We need to establish a service level objective, and we need to know that we are meeting the criteria.

Set objectives for key processes, applications, plans, and packages. Track the summary data daily, and examine exceptions and longer times. Daily tracking can prevent problems and the extra work required to manage and solve them. Agree on the objective before any meeting held to discuss complaints. It will be useful for resolving unreasonable demands.

Turning now to DB2 for OS/390 for some specifics, the primary approach for monitoring and tuning from the DB2 perspective is outlined in the DB2 Administration Guide; see especially the chapters on "Planning Your Performance Strategy" and "Analyzing Performance Data." The remainder of Volume 2 of the Guide describes tuning from the DB2 perspective.



Monitoring the averages and exceptions from DB2 accounting reports makes problem analysis much easier. Having a history for performance makes it fairly simple to identify changes.

Stay current on DB2 releases and service levels -- it's another key to good performance. Ideally, stay current (within two to six months) on the full set of service, and apply high impact pervasive (HIPER) PTFs at least monthly. You'll have a better chance of finding a known and fixed problem, while avoiding the chance to encounter a bad PTF.

Those customers who are most dissatisfied with DB2 are those who are least current on releases and service. Those who are happiest and doing the best job are quite current. If the release you are using is fairly new, then staying current is even more important.

Track performance across the entire life cycle of an application. Begin your planning with some estimates that can be tracked and compared with test runs. Manage test performance, then update your estimates. As the application moves into production, the history of performance can be carried along. As I've said, when the application is successful, we will need to meet new challenges.

Remember that DB2 is not IMS, IDMS, Oracle, Informix, VSAM or ... The most common problem I see is using the design and programming techniques for some other product for an application that requires high performance. If you want to achieve high performance or high availability, you cannot use the design, programming, or operational techniques for another DBMS. Don't design for VSAM, IMS, or object DBMS, then implement in DB2 if you want high performance.

If, however, your objective is moderate performance, changing VSAM or IMS calls into SQL can be easier and can give adequate results. The single, general I/O module that can access any database or file is a common technique. But the loss of information and capability in the interface generally leads to significantly slower performance. Gaining skill in DB2 techniques requires education and experience. Use those skills -- and DB2's many unique features -- to identify ways to change your techniques, designs, and programs in order to gain the best availability and performance.

The basics of performance are simple, but they require constant attention and work. We must establish performance criteria, monitor the results against that criteria, apply maintenance regularly, and design for the systems we are running. If we don't, we will be managing the crisis instead of the performance.

Roger Miller is a lead DB2 strategist at IBM Santa Teresa Laboratory, working for the last 17 years on product strategy, design and development. He often helps customers use DB2 and makes presentations to user groups. Prior to joining IBM, Miller was an



IBM customer for then years, working on most facets of DB2, ranging from overall design issues to SQL, languages, install, security, audio standards, performance, concurrency, and availability.


Home | Contact Us


Chicago, IL 60611 (312) 644-6610










Solutions Journal: The eX-Files: MVX Explained


The eX-Files - MVS Explain by Frank Ingrassia

The DB2 Optimizer is easily the most intriguing piece of software to come out of IBM in the last 20 years. Translating sentence-oriented SQL into pure mathematics and computing the cost of a variety of reasonable access paths before choosing the most efficient one is a task of wonder and mystery too often left "under the covers." To this day, it is not clearly understood by most who toil in this industry. DB2 Explain can illuminate the process, and using all six of the Explain tables in MVS can take you to the next plateau of enlightenment. In this article, we'll explore that new frontier, mining some of the data buried within these tables.

Understanding the Plan_Table in Version 5

Access path selection is the primary responsibility of the DB2 Optimizer. Several access paths are available, mostly variations on indexed themes or tablespace scans. Inserts show a "blank" access path, as do Updates and Deletes "Where Current of Cursor" in the Accesstype field of the Plan _Table.

Tablespace scans are processed differently if the tablespace is segmented or non-segmented, although the difference is not discernible from the Plan_Table. Partitioned tablespaces may be accessed in parallel, and some of the partitions may be skipped during processing (more on this later).

Indexed access offers several choices, from the use of a single index (I), a single index with the `IN (list)' predicate (N), or the use of multiple indexes M), which includes at least two individual index accesses (MX), and the intersection if ANDing (MI), or the union if ORing (MU), of the resulting rid lists that qualify for the predicates. Presently, there is no indication of whether the index chosen is clustered or random. A special form of indexed access may be used with the MIN or MAX function against an indexed column, causing only the first index leaf page to be returned (I1). If all the columns are found in one or more indexes, then the query can be an Index Only access.

The number of columns matched in qualifying predicates against an index tells a lot about the effectiveness of that index as an access path choice. Any non-matching scans

http://arbaltxp-011870/member/journal/aug97/ex_files.html (1 of 2)5/5/2006 10:29:04 AM

Solutions Journal: The eX-Files: MVX Explained

have a Matchcols value of 0. Unique indexes and the =' predicate are a special form of matching index scan, but not examined here.

DB2 for OS/390 offers a choice of three possible join methods: nested loop (1), merge scan (2), and hybrid join (3).

Nested loop joins reduce the outer (Composite) table to a minimum, then joins its rows with the inner (New) table by scanning once for each qualified row in the outer table (on the join predicate). It works best with indexes on one or both tables.

Merge scan joins reduce the inner table first, sorting into join column order, and then join by scanning the inner table once for each qualifying row in the outer table. It is the best choice for non-indexed joins. Joining on more than one column negates Parallel CP and Sysplex join processing.

Hybrid joins scan (and possibly sort) the outer table, building an intermediate table that consists of the outer table rows and columns and the inner index's rids. Replacing the rids in a subsequent step and applying any remaining predicates for the inner table will yield the final results table.

The Join_Type column indicates that a Left, Right (both L) or Full (F) Outer join, or a standard inner join (blank), will be required.

There are four SortN_ and four SortC_ flags, representing sort activity caused by ORDER BY, GROUP BY, UNION, DISTINCT or JOIN operations. Of the SortN_ flags, only the SORTN_JOIN is used. It represents any sorts caused by a join that occurs on the "New," or inner, table. In the case of a hybrid join, this flag represents the sorting of the inner index's rid list, and not a table at all.

The SortC_ flags are used for the "composite," or outer, table of a join, and for all other activity occurring on the results table. For non-join sorts, the method is always 3, and the Tabno is 0, indicating the results table. Non-correlated subqueries will generally sort for both uniqueness and order, thus flagging the SORTC_ORDERBY and SORTC_UNIQ columns simultaneously. Unions and the use of a Select Distinct will often invoke the SORTC_UNIQ flag as well.

The SortC_ and SortN_Pgroup_ID columns identify the parallel group number involved in a "parallel" sort, normally invoked to join one or more tables that have the same number of parallel degrees. Again, C is for the composite, or outer, table and N is for the new, or inner, table in the join process.

Non-partitioned tables may be sorted into logical partitions for joining with a partitioned tablespace, if there are multiple DSNDB07 datasets (temporary storage database) and the ESA Sort hardware assist feature is installed.

http://arbaltxp-011870/member/journal/aug97/ex_files.html (2 of 2)5/5/2006 10:29:04 AM

Solutions Journal: IDUG News


News from IDUG

During the 9th annual business meeting of the North American Conference in Chicago this May, IDUG elected its 1997-1998 Board of Directors. Casey Young of RYC, Inc., Santa Cruz, CA, was re-elected President, with Rolf Dippold of SYSTOR AG/Swiss Bank Corp., Basel, Switzerland, elected as President-Elect.

IDUG's Board Vice Presidents and their term expirations are as follows:

● David Beulke, Spiegel, Inc., Westmont, IL (1998) ● Brenda Castiel, Ernst & Young, Los Angeles, CA (2000) ● Detlef Felser, UFD, Sippenaeken, Belgium (1999) ● Freddy Hansen, Kommunedata, Ballerup, Denmark (1999) ● Kathy Komer, Aetna Life & Casualty, Middletown, CT (1998) ● Tom Millar, Mtech Consulting Pty. Ltd., Sydney, Australia (1999) ● Elizabeth Moore, American General Life & Accident Ins., Nashville, TN (1998) ● Nicholette Ross, BMC Software, Houston TX (1998) ● Carol Runyon, Johnson & Johnson SLC, New Brunswick, NJ (1998) ● Doug Stacey, Comdisco, Inc., Rosemont, IL (1998).

Also, Margaret Poteat of KPMG Peat Marwick joined the Board as a non-voting organizational advisor, replacing Christine Ebken of KPMG Peat Marwick who stepped down after five years of service. Also stepping down from the Board this year was Marco Chou of Allstate Insurance Co. after serving his three-year term. IDUG President Casey Young thanked both Ebken and Chou for their years of commitment and service.

Call for Asia Pacific Award for Information Excellence Nominations

September 12, 1997, is the deadline for all nominations for the 1997 Asia Pacific Award for Information Excellence. Sponsored jointly by IDUG founding vendor members KPMG Peat Marwick and Platinum technology, inc., and administered by IDUG, the Award for Information Excellence is presented to a distinguished contributor to the advancement of relational technology. Eligible candidates include innovative senior managers involved with relational technology--CIOs, directors of MIS, DAs, or DBAs.

http://arbaltxp-011870/member/journal/aug97/idug_news.html (1 of 4)5/5/2006 10:29:20 AM


The award, created to convey respect and to provide an opportunity for peer recognition and career visibility, is presented annually at each IDUG conference: North America, Europe, and Asia Pacific. The 1996 Asia Pacific recipient was Tony Crewdsen of the New Zealand Police Force. This year's award will be presented at the 3rd Annual IDUG Asia Pacific Conference in Sydney, Australia.

Abstracts due Sept. 16 for May 1998 North American Conference in San Francisco

Speak at the IDUG North American Conference, May 10-14, 1998, in San Francisco, CA, and your conference registration fee will be waived. If your presentation is accepted, you'll have access to all conference activities, scheduled meals, proceedings, and a one-year IDUG membership. Focus of the conference is business solutions that are pertinent to the day-to-day jobs of DB2 professionals. Presentations must feature at least one member of the DB2 Product Family, comply with the IDUG mission statement, and include DB2 if other database engines are mentioned; they may be 70 or 90 minutes long.

To submit an abstract for consideration, simply complete a Call for Nominations form, available on the IDUG Web site at www.idug.org., and return it to IDUG Headquarters by September 16, 1997. Questions? Call IDUG. Rite Aid VP wins 1997 IDUG Award for Information Excellence Gene Brown of Rite Aid Corporation, the largest retail drugstore operator in the U.S., received the IDUG Award for Information Excellence at the ninth annual IDUG conference held this May in Chicago.

Rite Aid VP wins 1997 IDUG Award for Information Excellence

Gene Brown of Rite Aid Corporation, the largest retail drugstore operator in the U.S. received the IDUG Award for Information Excellence at the ninth annual IDUG conference held this May in Chicago.

Brown was recognized for his "groundbreaking use of relational technology, and for pioneering the use of the multi-member data sharing facilities offered by DB2," says Andrew Filipowski, president and CEO of Platinum technology, inc. PLATINUM and KPMG Peat Marwick LLP, co-founders of IDUG, also co-sponsor the award, given at each of IDUG's three annual conferences.

"Mr. Brown has a strong understanding of how database technology can be exploited to achieve business goals," said Christine Ebken, senior manager in KPMG's data warehousing practice.



Brown is vice president of Information Services (IS) Operations at Rite Aid, a $5 billion drug store chain. One of his major challenges was to maintain superb IS services through the company's rapid expansion and merger with Thrifty PayLess, which added 1,000 drug stores to the operation. "Taking months and/or years to enhance systems to allow for this growth was not acceptable," said Brown, who used data sharing to enable CPU-intensive processing to reside on another DB2 member, away from daily transaction processing. "This enables workload balancing and more effective DB2 operations; DB2 data sharing now allows for unlimited growth potential, which is what the business requires."

How to Set Up Regional Users Group Web Site

As more DB2 professionals gain access to the Internet, there are obvious benefits to using the Internet to publicize Regional Users Group (RUG) meetings. Here are some tips for RUGs who want to set up their own Web sites from Tim Johnson, webmaster for the Central Canada DB2 Users Group.

CompuServe is commonly used by IT professionals, and it's one of the simplest, least expensive options for a personal Web site. CompuServe charges $9.95 a month for 2Mb of space; it provides an application for creating and uploading Web pages. Local Internet Service Providers (ISP) can also provide inexpensive access.

Either way, RUGs can develop their Web site content using something as basic as "notepad" in MS-Windows, testing locally with a browser such as Netscape Navigator, and then finally uploading the documents to the ISP host.

Anyone familiar with the mainframe DCF format will immediately feel at home with HTML, which is a relatively straightforward text formatting language. Web browsers like Netscape Navigator and Microsoft's Internet Explorer provide menu options to view the source of the HTML documents, or save them to disk files, providing a ready source of examples. This, Johnson notes, is otherwise known as reusable code!

Beyond static HTML text, interactive forms require some programming, typically using some type of UNIX script language such as PERL; also required is that the ISP support the Common Gateway Interface (CGI). The Central Canada DB2 Users Group has implemented a few scripts to enable additions to its mailing list, and to update an online guest book.

The RUG listing on IDUG's home page includes hot links to the Web sites for a number of RUGs. In Johnson's opinion, the "coolest" RUG site is the Melbourne DB2 Users Group (http://www.mdug.org.au)., which boasts impressive, professionally designed graphics. Another example is the Los Angeles Area DB2 Users Group site (http://www.


http://arbaltxp-011870/idug/rugs/rugrost.html

http://www.mdug.org.au/

http://www.geocities.com/ResearchTriangle/2244/index.html


geocities.com/ResearchTriangle/2244/index.html). Then there's the Central Canada DB2 Users Group site which Johnson manages; it's at http://www.interlog.com/~ccdb2. "It's not too difficult or expensive to set up a fairly simple Web site, and the use of Web authoring tools can make this even simpler, masking even the HTML tags," concludes Johnson, who can be reached at [email protected].


Home | Contact Us


Chicago, IL 60611 (312) 644-6610


http://www.geocities.com/ResearchTriangle/2244/index.html

http://www.interlog.com/~ccdb2










Solutions Journal: Trying Out Full Partition Independence


DB2/MVS Version 4.1 - Trying Out Full Partition Independence by Dan Luksetich

Why partition?

People partition tablespaces for three primary reasons: to reduce physical dataset sizes, to organize a table into meaningful pieces, or to improve performance. Physical dataset size is important because, as one adds more data to a table, the underlying VSAM linear dataset becomes larger. As the dataset becomes larger, it takes longer to copy, reorg, or move it. Using partitioning, a dataset can be split in as many as 64 parts (with DB2 Version 5.1, this can be as many as 254 parts), effectively reducing a large, cumbersome dataset into several more manageable parts.

A tablespace can be partitioned according to key values specific to the data. For example, sales data for Chicago can be placed in one partition, and data for New York in another. Thus, if maintenance is needed only on the Chicago data, the New York data could still be available. This can dramatically increase availability.

High volume applications can create a heavy demand on DASD devices. Partitioning a tablespace allows you to distribute the tablespace among many partitions, which can be placed on distinct DASD devices. The subsequent access is thus distributed across many devices. With the advent of type 2 indexes, and full partition independence, partitioning becomes even more desirable.

Full partition independence

Full partition independence is one of the many features introduced with Version 4.1 of DB2. To users wishing to take advantage of partitioned tablespaces, this marked a wonderful improvement in the availability of these tablespaces. While the partition independence feature of Version 3.1 allowed for a DB2 utility such as REORG or LOAD to operate on a single partition of a tablespace, while SQL applications or other utilities could simultaneously access other partitions, the feature was limited only to the tablespace and partitioning index. Any non-partitioning indexes (NPIs) were completely unavailable while the utility executed. Full partition independence in Version 4.1 extends partition independence to all NPIs. Enabling this feature requires that at least the NPI for

http://arbaltxp-011870/member/journal/aug97/partition.html (1 of 7)5/5/2006 10:29:32 AM


which partition independence is desired be a Type 2 index.

Type 2 indexes are new in DB2 Version 4.1, and are implemented by means of a new index structure and index manager. IBM has stated clearly that Type 2 indexes are the index of the future for DB2, and no new features will be dependent on the Type 1 index manager.

When a non-partitioning Type 2 index is defined on a table that is partitioned, that index is considered "logically partitioned." Unlike the tablespace or partitioning index parts, which physically map to one dataset per partition, the entire non-partitioning index remains in one physical dataset (except in cases where the first dataset reaches the two GB size limit). The structure of the record identifier (RID) supports the logical partitioning. A RID referencing a row in a partitioned table contains the partition number as the high order value. This high order value is used to identify a particular RID as "belonging" to a specific partition. Thus the structure of the non-partitioning index remains the same, but the partitions within the index can be identified simply by examining the RID.

This logical partitioning of the NPI makes it possible to execute commands and utilities against a single partition in exactly the same way as you would against a partitioned tablespace or partitioning index.

Converting to Type 2 indexes

"It depends" has never been a better response than when it addresses the size increase of indexes as they are converted to Type 2. Typical increase in space when converting an index from Type 1 to Type 2 can range from 0 percent to 29 percent. Large growth can be directly attributed to the amount of unused space in the index, uniqueness, and the size of the key. For example, if you have a database that utilizes dumb unique integer keys, the resultant index entry length is 8 bytes per key, 4 for the value, and 4 for the RID. Converting that index to Type 2 results in an index entry length of 11 bytes per key, 4 bytes for the value, 4 bytes for the RID, 1 byte for the RID flags, and 2 bytes for the key map. This can result in a substantial increase in index size. Subpage structures in Type 1 indexes also plays a role, so results may vary.

Putting partition independence to the test

Testing has proven that full partition independence allows SQL applications to access the data, via an NPI, of all partitions not impacted by a utility execution. When an attempt was made to access the data affected by the utility, the application went into a lock wait on the NPI logical partition and eventually timed out. End users may deem this unacceptable, if the affected online application has an extremely low response time service level agreement. If the user initiates the affected partitions for utility access, the application will immediately receive an unavailable resource SQL code. This eliminates the problem of waiting for a lock, but now the application must differentiate between a -904 on a partition started for utility access and a legitimate resource problem.



Testing multiple utilities (Reorgs and Loads) against various partitions of the tablespace verified that full partition independence can be utilized to reduce outage windows by parallelizing utility processes. However, since the Type 2 NPI is still in a single dataset, care must be taken to avoid any sort of excessive access to NPIs during the utility build phase. In this case, the same bufferpool and DASD issues could be considered for an NPI that would be considered if a high volume application were accessing it. In other words, be careful how many parallel utilities you allow to operate concurrently against an NPI.

RECP*

In general, full partition independence is an important and powerful new feature that should be exploited. However, as with anything else, exploiting the feature without proper preparation can lead to complications.

The RECP* state was added late in the design of full partition independence was the RECP* state. This pending state means that a logical partition of a non-partitioning index is in recover pending, and the entire indexspace is unavailable for SQL access. This can occur, for example, when a LOAD utility process has failed, leaving a logical partition of a non-partitioning index in RECP; then the utility is terminated. IBM has indicated that the following situations should result in a RECP* state:

1. A data partition has been recovered to a prior point in time, and the logical partition of the NPI has not been recovered.

2. A RECOVER INDEX PART or a REORG INDEX PART is operating on a logical partition of an index, and an SQL application is using another logical partition of the same index.

3. A LOAD PART is termed, leaving logical partitions in RECP.

RECP* exists because of a potential inconsistency in SQL query results. If a LOAD utility was replacing data into a partition and failed during the build phase, it leaves index partitions that were not yet built in a RECP* state. Data has been added to the tablespace partition having a key value indexed via a non-partitioning index, but that same key value may also exist in other partitions. If the utility is terminated, the tablespace partition is accessible via a tablespace scan. Thus, rows of data with the specified key value will be returned as a result of the query. However, a query that accesses the same key value via the non-partitioning index will not return the same data. RECP* guarantees that this potential inconsistency will not occur.

IBM's example

"If partition 1 of the tablespace is loaded and the LOAD is terminated after the RELOAD phase, then partition 1 of the NPI is placed in RECP*. Let's say that after the -TERM UTIL but before recovering the indexes, we perform a query with an index match (e.g.,



SELECT * WHERE C2=B' and the NPI is defined on C2). We find a B' in the NPI which belongs to partition 2. However, after the LOAD of the partition there could be B's in C2 which belong to partition 1. The user would be able to select rows from the tablespace using a tablespace scan and find that there is more than one row where C2=B'. The results for the index match and the tablespace scan would be inconsistent. RECP* was introduced to preclude this inconsistency of SQL results."

The RECP* state was created based upon the assumption that the primary access to partitioned tables is via a unique partitioning index. In a situation where data is partitioned by a key value that is not used for SQL access, the possibility of an NPI being completely unavailable due to a RECP* state may not be acceptable. For example, a customer file could be partitioned by geographic area for billing purposes, but the online point of sales system accesses the data via a customer number which is defined as the key to a non-partitioning index. If an outage is scheduled for the Chicago region, during which the Chicago partition is replaced via a LOAD utility, all other regions can remain available. However, an expected outage for the Chicago region may unexpectedly create an outage for all other regions if an error leaves the NPI on the customer number in RECP*.

Proper utility planning can help prevent occurrences of RECP*, but if even the possibility of this type of outage is unacceptable, then full partition independence may not be acceptable -- rather, splitting the data into several tables will achieve the level of availability desired. In our data split by geographic area example, the Chicago data is placed in one table, and the New York data in another. Each table has one unique index on customer number, and the applications access each table depending upon the geographic area in which the customer belongs. So when maintenance is performed on the New York data, the Chicago data will be completely unaffected. This eliminates the need for a partitioning index, but also increases table administration and the complexity of the code that accesses the table.

At a recent Midwest Database User Group meeting, attendees were asked their opinion of the RECP* state when it involves a LOAD REPLACE PART utility failure and the subsequent termination of that utility. An overwhelming majority indicated that in that situation, they would prefer to create a more restrictive pending state at the tablespace partition level, rather than RECP* on the logical partition of the NPI. This particular situation would leave available the logical parts of the NPI that were unaffected by the load. The data partition, which was unavailable due to the LOAD utility, would remain unavailable. This change would increase availability in these situations for customers who access data via a unique non-partitioning index. The argument in this case is that any customer requiring the level of availability offered by full partition independence will very likely be planning for, and expecting, a utility outage for the partition on which they are performing some sort of maintenance.

When it occurs, RECP* represents a global outage to an NPI. This impacts partitions that were not part of a planned outage. A more restrictive state at the tablespace partition level will eliminate the need for RECP*, and not expand unavailability beyond



what was planned. IBM will review this as a requirement (REQ00064964).

Managing LOAD PART

At first, the load utility seems to be one of the easiest utilities to understand. It replaces or adds data to a table. However, when the complexities of partitioning are added, the functionality of the load utility can become confusing. With full partition independence, some additional issues come to light. For example, it was commonly understood that in order to replace part 5 of the table DAN.COOLTB, the following utility command was to be used:

LOAD RESUME YES...INTO TABLE

DAN.COOLTB REPLACE PART 5

This will not, however, give you full partition independence, since locks are taken at the tablespace level. Specifying resume at the tablespace level (right after the LOAD keyword) tells DB2 that you are acting at the tablespace level. This command syntax may also result in some unexpected pending states for any NPIs defined on the table.

PSRCP, or Page Set ReCover Pending, means that the entire indexspace is unavailable and must be recovered. This pending state can happen when an ABEND occurs during the build phase of a load job, while an NPI is being built, and LOAD RESUME YES is specified at the tablespace level. A recent IBM APAR (PN90623) addressed the issue that PSRCP could be reset only by a RECOVER utility. LOAD REPLACE at the tablespace level will now also reset PSRCP. Another common belief was that either RESUME or REPLACE was specified at the tablespace level, and that RESUME NO was the default if nothing was specified. As it turns out, RESUME NO at the tablespace level can be overridden by what is specified at the partition level. Thus, in the previous example, the proper command syntax is:

LOAD...INTO TABLE DAN.COOLTB REPLACE PART 5

Using this command in a LOAD statement, you can take advantage of full partition independence, and avoid the possibility of PSRCP on any NPIs. This is clearly documented in the DB2 Version 4.1 Utility Reference.

In order to support the IBM LOAD utility's use of insert and delete processing, an NPI must be assigned freespace to support the shifting of key values. Initial testing has shown that page splits can be expensive. By allocating enough freespace before the data is loaded or reorganized, page splits can be avoided as data moves or grows. The freespace must be specified before loading or reorganizing the entire table, or before a conversion to the Type 2 index structure.



Aggregate SQL

Users of data warehouses and decision support systems share a new concern about the relative accuracy of SQL results during the replacement of data, as during a load utility. For example, if an NPI on a table containing information on automobiles was keyed on, say, the color blue, an SQL statement summarizing data based upon the color blue may temporarily return relatively inaccurate results during a LOAD REPLACE of a single partition. The word "relatively" is used here because the results returned are accurate and consistent, but may not actually represent an expected result.

This was not an issue prior to DB2 Version 4.1, because the entire NPI was unavailable during utility processing, and any SQL query accessing the NPI would fail. This occurs because, during the build phase of a load replace part utility, each NPI will be swept, and any key entries for the affected partition will be deleted. Then the new key values for that partition will be inserted into the index.

Since the logical partitioning of the NPI is based upon RID values that are dependent upon key values, it is only possible to prevent SQL from accessing a particular partition via the NPI by interrogating the RID once the key entry has been accessed. The result is that SQL performing aggregate processing on non-unique key values, or SQL utilizing range predicates (e.g. WHERE CUST_ID > 1000, and there is a unique NPI on CUST_ID), will return accurate results or fail due to the unavailability of a logical partition. However, if the SQL executes after the index entries for the logical partition have been deleted, and no index entries that could impact the query have been inserted, the query would be successful, but the results would not reflect the new or old index entries.

This is an issue for tables that might be part of a decision support system that replaces data as part of a revolving data refresh process. Proper scheduling should avoid this possibility, but unscheduled ad hoc queries do run some risk.

Conclusion

In most situations, full partition independence works as promised, and is a very powerful and important enhancement to DB2. The most significant enhancement is in the REORG PART utility. Since this utility simply replaces RIDs in the NPIs, the performance and dependability are outstanding, and concerns about RECP* and index accuracy and growth are non-existent. However, special considerations should be taken when the LOAD REPLACE PART utility is utilized to avoid those possible occurrences of RECP*.

Dan Kuksetich is a consultant with Software Alternatives Incorporated, Manhattan, IL. He can be reached at 312.409.0497, or via email at [email protected]





Home | Contact Us


Chicago, IL 60611 (312) 644-6610










Solutions Journal: Peak Parallel Performance


Peak Parallel Performance - Secrets that a DBA Should Know by Bryan Smith, Ted Blank, and Kathryn Zeldenstein

This article is the first of three that will focus on achieving the best performance from query parallelism on DB2 for OS/390. Achieving optimal query performance requires a strategy from all key areas in your shop. It does no good to provide enough buffer pool storage if the application programmer doesn't enable parallelism. And if the application programmer codes a query that begs for parallelism, what good is it if access to the data is funneled through a single I/O path?

In this first article, we look at query parallelism from a DBA's perspective, giving an overview of what parallelism is, what is realistic to expect from it, and what you can do to help make query parallelism a winner in your shop.

Query parallelism concepts and effects

Query parallelism occurs when the database manager splits a single query and each of the split-off parts does some of the processing for the query. The goal of query parallelism is to reduce the elapsed time of queries by giving those queries more parallel resources. What do we mean by parallel resources? We mean more instances of a given resource, such as multiple I/O paths or multiple processors. If the DBMS can assign more processing and I/O power to a query, its overall elapsed time is reduced.

At what cost parallelism?: Parallelism is not free, but it can be managed. It is up to you to decide if the time to process a complex query is more or less valuable than a certain amount of computing resources. DB2's query CP parallelism, on average, requires less than ten percent processor overhead for short-running queries, and less than one percent for longer running queries. This is how DB2 can scale processor-intensive work in a linear fashion across as many processors as are available.

The decision of how much computing resources to give to queries should be based partly on where you intend to run those queries. Will they coexist with on-line transactions, or will they run on a system dedicated and tuned for complex query operations?

http://arbaltxp-011870/member/journal/aug97/peak_parallel.html (1 of 7)5/5/2006 10:29:54 AM


For OLTP, you probably avoid using 100 percent of processing resources, because this can lead to severe queuing of transaction work and unacceptable end user response times.

However, being 100 percent busy with OLTP only is much different than being 70 percent busy with OLTP and 30 percent busy with query work. Giving transaction work a higher priority than the query work does not harm transaction performance. The query work simply soaks up the remaining processing resources, thereby making full use of idle resources.

How DB2 determines the degree of parallelism

DB2's cost-based optimizer does degree determination with one specific goal: to minimize overhead while achieving the maximum benefit. Remember this goal. We'll be coming back to it later to explain some of DB2's actions. It is helpful to compare DB2's approach to degree determination with that offered by other database management systems. Most other DBMSs appear to choose a degree of parallelism based upon the number of ways the data is partitioned, or they require you to select a degree of parallelism when you define the table or execute a query.

This degree is used for all accesses to the object, regardless of how large the object is or how the object is accessed (scan, sorts, joins, and so forth). It requires that you perform what may be a very complicated analysis to determine an optimal degree of parallelism. Why should you do the work when an effective Optimizer can do that analysis for you?

What about choosing a degree of parallelism based on the number of partitions? It certainly makes life easier not to have to do complex analysis of query costs to determine the degree. However, as we examine DB2's approach to degree determination, you will see that the "number of partitions" approach, although simple, is really too simple to give you the maximum benefit from parallelism, because it ignores such issues as data skew and what we call the nature of the query (its tendency to be I/O-intensive or processor-intensive).

DB2's optimization strategy -- the optimal degree: Remember, the goal of parallelism is to minimize overhead while achieving the maximum benefit. With DB2, it is possible to achieve the maximum benefit even when data becomes skewed over time. We'll show you an example of how data skew can affect the parallel degree determination, but first let's look at the basic process that DB2 follows for determining how much parallelism a query, or part of a query, receives.

At bind or prepare time, DB2:

1. Chooses the best access path (DB2's normal access path selection) 2. Post-processes the access path (this is called post-optimization):



1. Identify parts of the access path that can benefit from parallelism 2. If part of the query is a good candidate to run in parallel, determine the

degree of parallelism. A host variable can cause this part of the process to be deferred to execution time.

Components of degree determination

DB2 considers the following factors when determining how many parallel operations are optimal for a query or part of a query:

● The number of partitions ● The estimated I/O cost of the largest physical partition (To help DB2 do a good

job, you must keep accurate statistics in the DB2 catalog.) ● Estimated processing cost based on the complexity of the SQL, the speed of the

central processors (CPs), and the number of those CPs that are online.

As a simple example, assume that a table space has ten partitions in which the data is evenly balanced. DB2 determines that the I/O cost to read each partition is five seconds, and that the processor cost for the entire query is 60 seconds. If the system is configured with four processors, there is no way to reduce the query elapsed time below 15 seconds, because the query will then be bound by the number of available processors. Setting the parallel degree at more than four does not reduce the overall elapsed time; it actually increases the overhead for the query without the benefit of further reducing elapsed time.

The effect of data skew: It is difficult to maintain perfectly balanced partitions in the real world. DB2's Optimizer understands data skew. In Figure 1, partition 3 has the greatest amount of data, which means that partition 3 is the limiting factor for an I/O-intensive scan, and the amount of data in partition 3 is the amount of work each parallel task can perform. DB2 divides the total amount of work into equal work ranges based on the size of partition 3, as shown in Figure 1.

The nature of the query: Recall that DB2 looks at estimated I/O costs and processor costs when determining the parallel degree. The more I/O-intensive the query is (scans, for example), the more likely that the degree of parallelism will approach the number of partitions. This is because DB2 assumes that each partition is supported by a separate I/O path.

For example, assume that you have a configuration like that shown in Figure 2; here you have a table space with 40 partitions on a system with ten processors (either within a single SMP machine, or as the sum total of the processors in a data sharing group).

For an extremely I/O-intensive query, DB2 chooses a degree of parallelism that approaches 40. For an extremely processor-intensive query, such as one that applies many predicates or requires a sort, DB2 chooses a degree of parallelism that



approaches the number of processors (in this case, ten). If a degree of 20 is chosen, for example, it means that the query is somewhat balanced in its I/O and processor needs.

DB2's technique: How good is DB2 at choosing an optimal degree of parallelism? IBM has tried to outsmart the Optimizer by forcing a certain degree of parallelism. IBM compared the elapsed time results of queries run with a forced degree of parallelism with those in which the Optimizer chose a degree. In almost all cases, DB2 chose a degree of parallelism that resulted in the greatest elapsed time reduction. And for those queries in which DB2 did not make the best choice, the Optimizer was refined. Supply resources, demand performance

We have seen how DB2 chooses a parallel degree by looking at I/O resources, processor resources, and using those resources to achieve the maximum benefit. To configure your system to get the most from parallelism, you can take advantage of this knowledge. You can use a supply-and-demand model for determining a system configuration (number of processors, number of table space partitions, number of I/O paths, and so forth) to meet a predetermined performance objective for a particular query.

The model is based on the idea that the total amount of resources consumed by the parallel query typically remains close to the amount used when the same query runs sequentially -- it is simply consumed at a higher rate for a shorter period of time. Increasing parallelism reduces elapsed time only if the system is capable of supplying resources at a new, higher rate. This supply-demand model is shown in Figure 2.

Increasing the supply of resources: There are two basic ways to increase the supply of resources to a query:

● Increase the number of processors in the system (or systems in the sysplex) to supply more processor time per elapsed second.

● Increase the number of I/O paths and table space partitions to increase the rate at which pages can be read from disk.

By analyzing the processor time and GETPAGE count of a query, we can estimate the rate at which these resources would have to be supplied in order to complete the query in less time.

The procedure for analyzing an existing simple query follows:

1. Identify the new performance objective for the query 2. Understand the current performance and resource requirements of the query 3. Identify the current limits to parallelism (resource bottlenecks) 4. Calculate how much more parallelism would be required to allow the query to

finish in less time 5. Calculate the additional number of processors and partitions required to support



the new degree of parallelism

Using the methodology: an example

How can you configure a system to allow a hypothetical query to reach a specific performance objective? This example assumes you know how many pages per second your processor can handle and how many pages per second the I/O subsystem can feed back to DB2. (For existing queries, you can gather this data from performance monitor reports. Use an estimating tool if the query does not yet exist.)

Here's the information we have:

The query: The query is complicated; you estimate it will take one msec of third generation CMOS processor time per page retrieved in a tablespace scan.

The data: There are 10 million pages in the tablespace.

The goal: This query must complete in 30 minutes or less.

The process: Determine the I/O and processor constraints for the query; then increase the supply of those resources so that the query can meet its performance objective.

The assumptions: Here are the numbers we are using:

Processor: 1 msec per page (1,000 pages per second) I/O: 450 pages per second per partition, assuming one part per disk and five to six parts per controller (This number is based on RAMAC II)

From these numbers, use the formulae in Figure 3 on page 8 to determine how much time the query would take to run, based on single instances of processor resource and I/O resource.

The analysis: From the formula for processor time, you can determine that one processor (if that's all you had) could do the job in 10,000 seconds, or about 2.8 hours. That's about 5.6 times too long! Six processors could do the job in about 28 minutes, which is just under our goal. Thus you will need at least six processors.

What about the data from disk? Each disk can supply about 450 pages per second. 10,000,000/450=22,222 seconds, which is about 12.3 times longer than the objective. Round this up to a minimum of 13 partitions. (When determining the number of partitions you need, don't forget that a single partition in DB2 can be no larger than 4 gigabytes (about one million 4KB pages).

The result: Thirteen partitions on a six-way third-generation CMOS processor is about



the smallest configuration that could process the query in under 30 minutes. As a bonus, both the processor and I/O subsystem are fully utilized for the entire query, so no resources are wasted.

The conclusion: With only a few pieces of information, you can do optimal partitioning and system sizing to meet specific performance objectives:

● The amount of processor time required per page ● The number of pages per second the I/O subsystem can deliver per disk ● The number of pages to be scanned ● Most importantly, the performance objective for the query!

Partition everything

In years past, the rule of thumb was to partition large tables. With the advent of query parallelism, and the increasing possibility of parallel access paths, that advice has extended to small tables, too. Assume, for example, you have a query that requires a nested loop join in which the inner table is large and the outer table is small. If the smaller outer table isn't partitioned, then a sequential access path is chosen. If, on the other hand, the smaller outer table is partitioned, let's say with five partitions, then it is possible to obtain access to the large inner table with five parallel tasks. Figure 4 on page 9 shows the potential elapsed time benefits from this approach.

Playing your part(ition) to maximize performance

In this article, we have shown that a key factor in maximizing parallel query performance is how the data is partitioned and how many I/O paths you can give to those partitions. In our next article, we'll describe strategies application programmers can use to make the most of query parallelism.

Bryan F. Smith is a developer for DB2 on OS/390 product at IBM's Santa Teresa Laboratory. He was the technical leader for the query parallelism work in Versions 4 and 5. These days, he works on additional DSS/BI enhancements to the product.

For the last three years, Ted Blank has been a performance analyst for DB2 for OS/390 at IBM's Santa Teresa Laboratory. His most recent DB2 work was in parallel query performance. Prior to this he worked for ten years in the MVS design and performance area.

Kathryn Zeidenstein is a publications writer at IBM's Santa Teresa Laboratory. She has worked on DB2 for OS/390 for the past 10 years, specializing in such areas as distributed database, data sharing, and query parallelism. She is the author of Data



Sharing: Planning and Administration and is currently working on the object-oriented enhancements to DB2.


Home | Contact Us


Chicago, IL 60611 (312) 644-6610










Solutions Journal: Performance Enhancements for SAP R/3


Performance Enhancements for SAP R/3 and DB2 for OS/390 - An Introduction

by Daniel D. Kitay

SAP's R/3 product has enjoyed phenomenal growth in recent years as more and more companies have begun implementing it. More importantly, it has grown along with the growth of mission critical production environments supporting thousands of users. R/3 comprises a Basis system to which application modules are added to provide a full range of business processes and transactions for almost every type of enterprise.

In April 1997, SAP and IBM began a phased rollout of the R/3 Database Server for System 390 using the OS/390 operating system and DB2 for OS/390 Version 5 (DB2) optimized for R/3. The R/3 Database Server becomes generally available in the third quarter of 1997. SAP and IBM released SAP-certified benchmark results showing record-breaking results. SAP customers considering large SAP R/3 servers, especially those that already use OS/390 and DB2, can now take advantage of the mainframe and DB2's proven ability to handle large scale mission-critical applications. Some of SAP's 1,600 R/2 customers will be especially interested in this migration path to R/3.

This article provides a technical introduction to SAP's R/3 Basis system and its three-tiered client/server architecture; it examines how DB2 Version 5 supports R/3 applications, and highlights DB2's key enhancements and optimizations to help ensure high availability and performance of R/3 applications. It is assumed the reader knows little of SAP R/3 but is familiar with DB2.

What is SAP and the R/3 system?

SAP AG is a Walldorf, Germany based company providing commercial applications software to business, education, and government. SAP leads the client/server software market worldwide and is the fifth largest independent software company. Its two basic products are R/2 and R/3. R/3 stands for Real-time System, Version 3. Its predecessors, R/1 and R/2, are also real-time systems but only for the mainframe environment.

R/3 provides a set of business application software modules designed for the client/server market. Between 1992 and 1996, the number of R/3 installations grew from zero

http://arbaltxp-011870/member/journal/aug97/performance_enhancements.html (1 of 6)5/5/2006 10:30:06 AM


to 5,000! This compares with over 1,600 mainframe-based R/2 installations.

R/3 system is the name associated with one or more R/3 instances, each defined with the same three-letter system name during the installation process.

R/3 instances are the entities forming the runtime parts of an R/3 system. Instances provide the central or distributed R/3 services of an R/3 system. The terms application server and application instance are synonymous with R/3 instance. An instance defined with the message and enqueue service can be referred to as the central instance. During an installation, R/3 instances are numbered sequentially, starting with 00, 01, 02, and so on.

R/3 system architecture overview

The R/3 system is based on a multi-layered client/server architecture. The database server (for example, DB2 Version 5 on OS/390) contains the company's data, and the application instance executes the business logic and service requests from users and background jobs. Each application instance consists of several different types of work processes, such as dialog, update, enqueue, batch, and spool.

In a three-tier R/3 system, there are usually several application instances, each residing on a separate physical machine or on different nodes of a multi-processor system (for example, the IBM RS/6000 Scalable POWERparallel system (SP)). No matter how many instances there are, one of them is always configured as the central instance, providing messaging and enqueue services for all application instances in a single R/3 system.

A user initiates a transaction through R/3's graphical user interface (sapgui). The sapgui is used to execute business transactions whose transaction code determines the type of transaction from among the several different R/3 application modules.

The application instance executes the R/3 transactions and either reads data from local memory within the instance (buffered data) or communicates with the database server to retrieve the data. Each instance has a dispatcher that schedules an incoming user request with an available dialog work process.

As Figure 1 shows, the SAP architecture has two layers: basis and application. The basis layer contains the R/3 system middleware, including a developer workbench, system administrator tools and transactions, and a number of basis services for processing dialog and update requests, enqueuing, and messaging. This middleware provides application independence from the system interfaces of the operating system, database system, and communication system used.

Each R/3 instance can provide different functions (dialog, update, batch) or each can be defined as a logical grouping of users, such as by department or by location. To balance workloads, application instances can be defined to support different application areas,



such as sales and distribution or financial accounting. Because a large number of different application instances can work in parallel with each other, accessing data stored in the database server, R/3 is very scaleable.

While the basis layer is written in the C and C++ languages, the application layer is written in SAP's proprietary Advanced Business Application Programming (ABAP/4) language. R/3's runtime system includes a number of different work processes (see Figure 2). The R/3 system administrator can configure the active number of each work process type on each application instance. Operation modes can also be defined for shifts throughout each day; a number of work processes can be active for each operation mode. For example, during a nightly batch cycle, fewer dialog and more background processes can be created to better handle changing workload types.

The system services include the dispatcher (distributes requests to the responsible work process), message service (one per R/3 system used to handle internal message exchange), and enqueue service (provides central lock management within R/3 in addition to the database server's lock management scheme). The Gateway service (three processes) provides external communication to other R/3 systems, R/2 systems, and non-SAP applications. The spool service supports output spooling for a printer or fax device in conjunction with the locally provided spool system of the operating system.

R/3 and DB2 for OS/390

The R/3 database server contains application data, control and customization data, and the R/3 repository. The R/3 repository contains data and process models, application programs, the ABAP/4 dictionary, help documentation, and more. The R/3 tables in DB2 are organized very differently in DB2 than they were defined in Oracle, for example. A single Oracle instance manages only one database containing all of the R/3 tablespaces and tables. A single DB2 subsystem contains many databases. To better support R/3, there are roughly 100 R/3 databases, and a larger number of table and index spaces. This improves R/3 performance and locking concurrency. By default, most of these tablespaces are segmented; two of them are partitioned. But the database administrator could partition some of the segmented tablespaces if the size warrants it.

To run the database server on OS/390 will require OS/390 Release 3, a S/390 processor, SAP R/3 Release 3.0x or later, and a RS/6000 system or PC Server with Windows NT as application servers. In addition, IBM and SAP are cooperating with other vendors to extend support of the S/390 database server for R/3 to other NT and UNIX platforms.

To support DB2 for OS/390, the Database Server Layer of R/3 (DBSL) has been ported. This component resides on the application server and interacts with the database via remote SQL. For optimal performance of the remote SQL, R/3 uses a new IBM high speed protocol built upon standard protocols in TCP/IP. In order to support high performance and bandwidth connections between the application servers and the



database, high performance connectivity hardware is used. In a RS/6000 configuration, ESCON channels are used to connect to the S/390 database server. Where an IBM PC running Windows NT is used, the connection is via one or more OSA2 cards in the S/390 with a FDDI LAN attachment to the application server platforms.

Performance enhancements

The S/390 database server for R/3 can be implemented on a single processor complex, or on a parallel sysplex (DB2 data sharing environment). SAP R/3 is very different from most high performance, high availability applications today. For example, R/3 uses dynamic SQL, and it needs data in ASCII order and format. Because of this, enhancements were made in both Versions 4 and 5 of DB2 to better support R/3 workloads. The major changes include:

● Dynamic statement cache to improve performance ● ASCII tables to improve performance ● Isolation level read stability ● Data sharing optimizations to allow R/3 to scale better (use row level locking, for

example) ● Online REORG and faster utilities to improve continuous operation and availability ● Optimization of DB2 locking to support R/3's locking model ● SQL RENAME of tables reduces R/3 release migration time ● SQL STRIP function can eliminate trailing blanks and improves performance ● LIKE with ESCAPE clause allows parameter markers for dynamic SQL.

Dynamic statement cache

To optimize the performance of R/3 transaction, DB2 provides the ability to cache a previously prepared SQL statement for subsequent use. When the same dynamic SQL is issued repeatedly, the cost of running these statements can be significantly reduced. DB2 Version 5 provides a global dynamic statement cache that eliminates or reduces the number of dynamic SQL binds that must occur. A subsystem option that should be activated on DB2's supporting R/3 activates this cache so that different application processes can share the same prepared statements. A new bind option also lets you retain a prepared statement past a commit. The application programmer doesn't have to reissue a PREPARE statement, resulting in fewer messages being transmitted in a client/server environment using dynamic SQL. Since R/3 uses dynamic SQL exclusively, "the statement level cache makes a monster of a difference," according to Roger Miller, lead DB2 strategist at IBM.

ASCII tables

R/3 application instance platforms require data to be in ASCII format and order. DB2 Version 5 now allows data to be stored in either EBCDIC or ASCII. At CREATE DATABASE, TABLESPACE, or TABLE time, a CCSID ASCII keyword can be specified



or a default preference can be set as an installation parameter. DB2's direct support of ASCII code pages avoids the FIELDPROC and translation overhead. However, joining of EBCDIC and ASCII tables is not supported.

Locking

A significant number of changes were made in the area of DB2 locking and concurrency control. The locking model for R/3 is very different from that of most DB2 applications. Rather than using cursor stability and static SQL plans and packages, R/3 uses a mixture of isolation levels, dynamic SQL, and no packages at all. With R/3, the pendulum swings from DBMS-provided integrity to much more application-provided integrity. Many of the reads are performed with uncommitted reads. In other cases, read locks are retained until commit with RS locking. To optimize performance, additional tuning was done with the BIND options for keeping update locks and for releasing the last lock.

SAP-certified performance benchmark results

Earlier this year, SAP certified record-breaking IBM test results showing that 3,400 SAP Sales and Distribution (SD) Parallel Benchmark users can run R/3 Release 3.0E with DB2 for OS/390. An average dialog response time of 1.50 seconds per transaction was achieved with a throughput of 1,065,000 dialog steps per hour. The R/3 Parallel SD Benchmark is a new benchmark defined by SAP; IBM is the first to use it. Different parallel architectures support different types of inter-communication between nodes. SAP's Parallel Benchmark is designed to better measure these varying characteristics and to give customers a more realistic representation of R/3 scalability in different parallel environments.

R/3 systems management

R/3 is a powerful and complex system to implement and manage. To ensure effective systems management of R/3 applications and provide adequate service levels, a number of areas within the R/3 basis system must be managed, monitored, and tuned. R/3 supplies its own set of systems management tools with its Computing Center Management System (CCMS). CCMS is an integral part of the R/3 Basis component, is accessible from within the sapgui, and provides a robust set of functions for monitoring and administering R/3 including:

● Background job scheduling ● Workload distribution and balancing ● Operation mode definition and switching support ● Printing and spool services ● System and profile configuration, startup, and shutdown ● Database administration, such as backup functions ● Performance monitoring of R/3 applications, database, operating system, and

network



● Alert generation, customer-tailorable performance thresholds, and exception analysis

● Problem analysis via access to each R/3 system log ● Performance database for historical trending.

Conclusion

Customers who choose DB2 and OS/390 as the R/3 Database Server can take advantage of those systems' proven ability to handle large-scale mission critical applications with large numbers of concurrent users. OS/390 customers, including R/2 customers, can also take advantage of their existing platform experience and system infrastructure.

Regardless of the database platform chosen, achieving high performance in R/3 applications requires careful planning prior to production implementation. It also requires ongoing monitoring of the performance of the R/3 applications, Basis components, operating systems, and the database management system.

Dan Kitay is an R&D Manager in Candle Corporation's Westlake Village, CA office. He has extensive experience with DB2 performance and administration, and more recently with Oracle, SAP R/3, and Lotus Notes. He can be reached via email at [email protected].


Home | Contact Us


Chicago, IL 60611 (312) 644-6610











Solutions Journal: Performance Metrics for DB/2 Buffer Pools


Performance Metrics for DB2 Buffer Pools by Joel Goldstein

A wealth of buffer pool performance information is available in DB2 system statistics records; these records provide useful metrics for a) measuring performance and b) determining both when tuning is necessary and the effects of tuning changes. It is extremely important to maintain long-term performance history so the effects of workload changes and tuning efforts can be evaluated. The major impact thresholds of Sequential Prefetch Threshold (SPTH), Data Manager Threshold (DMTH), and Immediate Write Threshold (IWTH) should be familiar to everyone; however, the IWTH counter is also incremented when synchronous writes occur for reasons other than having 97.5 percent of the buffers unavailable. So, if SPTH and DMTH have not been hit within any given statistics interval or performance monitoring period, IWTH can be ignored.

Current Buffers Active (CBA) is an often misunderstood field. It does not provide a high usage count, and it is captured only at that microsecond in time when the statistics record is created, or when the online monitor requests the statistics data from DB2. Rather, CBA indicates the number of buffers that are unavailable because they contain updated pages and/or pages that are allocated to a read process that has not completed. Thus, CBA is of interest only when it begins to represent a significant percentage of the number of buffers in the pool; you will frequently see a high value drop quite rapidly as the Vertical Write Queue (VDWQT) and/or Deferred Write Queue (DWQT) thresholds are reached.

DB2 maintains two write queues for updated pages: a Vertical Write Queue per object, with a default of 10 percent (of the number of pages in the pool), and the Deferred Write Queue (DWQT) for the entire pool. The default for DWQT is 50 percent. Reaching either of these thresholds will trigger deferred (asynchronous) writes of those pages, and will also increment the respective counter. Many shops should lower both of these thresholds, after they calculate the average number of pages/write. Observations made at a large number of installations show a page/write ratio for the transaction processing environment between two and five (yours, of course, may vary significantly from this range). This means that the updated pages for objects are widely dispersed, so DB2 must schedule multiple physical writes each time these thresholds are reached.

http://arbaltxp-011870/member/journal/aug97/performance_metrics.html (1 of 5)5/5/2006 10:30:18 AM


DB2 is capable of writing up to 32 pages (for one object) with a single I/O; it will schedule (dequeue) up to 128 pages for one object using one write engine. However, it will only write control intervals within a range of 150 relative block addresses with a single I/O to reduce the duration of write events. Thus, it is usually beneficial to lower the VDWQT to trigger the asynchronous write events earlier in the process. This smooths the write intensity for normal processing and especially at checkpoint intervals. The DWQT should normally be lowered in conjunction with the VDWQT when many objects in a pool are updated frequently. Lowering these thresholds does not usually increase the total number of write I/Os during a day when the average number of pages/write is in the single digit range. Naturally, this threshold may require a variety of settings along with other thresholds such as VPSEQT, HPSEQT, and pool sizes for batch versus online update workloads. The desired performance relationship between these two counters is a large number of VDWQT hits, and a low number of DWQT hits.

When a buffer pool is dedicated to, or contains the DSNDB07 files, it is important that sort performance not be degraded because of a lack of buffer pool resources. The VDWQT and DWQT thresholds for a dedicated sort/work pool frequently need to be altered from the original defaults in order to maintain good performance. If sorts usually complete using the memory sort space and the buffer pool pages, then raising these thresholds may avoid unnecessary writes. However, when sorts require heavy usage of the physical files, and there is a significant amount of DASD contention, these thresholds should be lowered in order to avoid hitting SPTH, DMTH, and IWTH.

Backing this pool with an Hiper Pool (HP) may or may not provide performance improvements. While most sites that backed the sort/work VP with an HP did not see any improvement in performance, a few have done so with noticeable performance improvements. Thus, this is simply a "try it and measure it" situation. If an HP helps, you gain. If it doesn't, simply set the size back to zero.

Another issue critical to sort performance is the number and placement of the physical datasets. Each must be on a different physical device. Thus, if you have many physical datasets, and are unable to separate them onto different devices, you will be better off with fewer datasets of larger size. Summing up, a large number of datasets increases the probability of filling the pool with updated pages and actually hitting the upper critical thresholds before the asynchronous writes can complete.

The Buffer Pool system hit ratio is a significant performance indicator, and should be used in conjunction with the overall I/O rate/sec. It is important to use the proper system hit ratio calculation (see the chapter on "Performance" in the DB2 for Administration Guide) by adding the number of Synchronous I/Os to the number of Pages Read by Sequential Prefetch, List Prefetch, and Dynamic Prefetch; then divide this sum by the number of Getpage Requests.

System Hit Ratio Calculation:



GP-(Synchio+SeqPrefPgs+DynPrefPgs+LPrefPgs) -------------------------------------------

GP

A prefetch request often reads fewer pages than the maximum prefetch quantity, when some of the pages in the requested block are already in the pool. It is possible for the system hit ratio formula to produce a negative hit ratio; this is not necessarily a critical situation. While it can be caused by severe pool thrashing, it is more commonly the result of large amounts of dynamic prefetch, and, occasionally, sequential prefetch -- when many more pages are read into the pool than the application actually references. The older application hit ratio (that only considers I/Os and not the number of pages read) may be relevant when there is very little prefetch activity (of any type) -- then it is quite close or equal to the system hit ratio. The I/O rate/sec is probably the most important metric for determining the effect of tuning efforts; it can be calculated by adding together all the I/O events, then dividing by the duration of the statistics or measurement interval in seconds.

Although not a common occurrence, Prefetch Disabled-No Read Engine, or Write Engine Unavailable, can have a major impact. The number for both read and write engines is presently fixed at 300; however, this can change at IBM's discretion. Running out of read or write engines has occurred several times over the years at installations where I consult, but could not be repeated in order to determine its cause. It often indicates I/O subsystem contention -- when read and/or write I/Os cannot complete fast enough and when all the "engines" have requests queued waiting for completion.

Aside from application design and SQL coding, buffer pool tuning is the single greatest tuning lever available to improve application performance. The existence of sixty pools will, I hope, dispel the myth that using one large pool is the best way to optimize performance. Moreover, even though many pools are available, it does not help to use too many -- this will often waste memory and make it difficult to monitor performance. Most installations should be able to achieve good performance with six to eight pools, while some large environments may need a dozen.

One of the critical factors for success is the proper grouping of objects into different pools based on the method of access (random or sequential), and the average/maximum number of pages an object uses within a pool. You can generally expect performance improvement from pool tuning to be a 15 percent or more reduction in transaction elapsed times. While the sum of pool sizes in most sites is too small, some installations have achieved this level of performance improvement while reducing the total amount of memory allocated to buffer pools.

Hiper Pools are an effective way to leverage less expensive expanded storage in order to obtain better overall hit ratios and eliminate I/O. But many installations have implemented HPs with minimal benefit -- not because they don't work, but because basic pool tuning was not performed. It is important to understand which kinds of objects can benefit from larger amounts of pool memory, whether it is all VP, or a mix of VP and HP.



Very large objects that are accessed using sequential prefetch will benefit little if any. Very large objects with very random access will not usually see great improvements. Objects with very high update rates will also not enjoy much improvement from an HP. When HPs are implemented it becomes just as important to evaluate their benefit as to track overall performance. One extremely effective formula is:

PagesRdHP --------------

PagesWrittenHP

This formula shows the amount of benefit an HP can provide. While low percentages usually imply little benefit, the number of pages read into the VP from the HP should be divided by the measurement period to determine the I/Os per second that the HP saved. If this number is very small, the HP is not providing much benefit. For very large volume systems, on the other hand, percentage improvements may reach the low teens, and may still save several I/Os per second.

Many reporting facilities are available for tracking performance. The example shown in Figure 1, is a batch report from one of the widely used online monitors. The report tells us that the percentage of pages read into the VP from the HP is extremely low -- only 2.7 percent. If we looked only at the number of pages read back from the HP, and divided by 3,600 seconds for one hour, we would find a savings of 37 I/Os per second. However, if we add the pages read back in asynchronously both with and without ADMF, then divide that sum by 32 (for a normal prefetch quantity), then add the synchronous read, and divide this sum by 3,600, we enjoy a savings of 7 I/Os per second. While this I/O saving might be significant, it must be viewed in the context of the 84 megabytes of expanded storage used to provide this savings.

The paging rate/second for each pool (page-ins for read+page-ins for write) should be calculated, as well as the rate for all the pools together. While high rates indicate that memory may be over-committed, it is important to understand that this is only for the pools; the rate for the entire Database Manager address space may be considerably higher. It is usually necessary to obtain RMF workload reports to determine the actual paging rates for the address space. The elapsed time interval of reports, whether from RMF or other DB2-based batch reporting facilities, must always be considered. Long reporting intervals may often mask serious performance spikes by combining many intervals having good performance with one or more poor ones. In short, the longer the reporting interval, the greater the smoothing or averaging effect.

This article has scratched only the surface of monitoring and tuning buffer pool performance. It is essential to remember that every installation and workload is different. While some base performance metrics or guidelines apply throughout the world, most are dependent upon the specific system, workload, and application.



Joel Goldstein has been involved with the performance management and capacity planning of onilne database systems, including DB2, for more than 20 years. Known widely as a performance expert, consultant, and instructor, Goldstein speaks frequently at DB2 user groups and Computer Measurement Group meetings around the world. In addition to authoring more than two dozen articles on DB2 performance issues, he has served as editor for Enterprise Systems Journal, Relational Journal, and CMG Transactions. Goldstein is president of Responsive Systems, which specializes in strategic planning, capacity planning, and performance tuning of online database systems. He can be reached at 732.972.1261.


Home | Contact Us


Chicago, IL 60611 (312) 644-6610










Solutions Journal: Models for Estimating Performance


Models for Estimating Performance by Richard Bolesta & Craig S. Mullins

Most organizations monitor and tune the performance of DB2-based applications. But the performance management steps are almost always reactive. A user calls with a response time problem. A tablespace maxes out on extents. The batch window extends into the day. You've been there.

Even the proactive steps taken against completed applications in production might be considered reactive. The application is written and most changes will require some code to be re-written. Event-driven tools exist on the market that can make performance management easier by automatically taking pre-defined actions when pre-specified alerts are triggered. But even these tools must operate against a completed and running application.

A critical additional step needs to be taken: planning for the performance of an application before it is completed. This article will discuss methods to enable proactive performance engineering as applications are being developed.

Defining DB2 performance

You must have a firm definition of DB2 performance before you can learn ways to plan for efficiency. Think, for a moment, of DB2 performance using the familiar concepts of supply and demand. Users demand information from DB2. DB2 supplies information to those requesting it. The rate at which DB2 supplies the demand for information can be termed DB2 performance.

Five factors influence DB2's performance: workload, throughput, resources, optimization, and contention.

The workload that is requested of DB2 defines the demand. It is a combination of online transactions, batch jobs, and system commands directed through the system at any given time. Workload can fluctuate drastically from day to day, hour to hour, and even minute to minute. Sometimes workload can be predicted (such as heavy month-end processing of payroll, or very light access after 5:30 p.m., when most users have left for

http://arbaltxp-011870/member/journal/aug97/performance_planning.html (1 of 7)5/5/2006 10:30:30 AM


the day), but at other times it is unpredictable. The overall workload has a major impact on DB2 performance.

Throughput defines the overall capability of the computer to process data. It is a composite of I/O speed, CPU speed, and the efficiency of the operating system.

The hardware and software tools at the disposal of the system are known as the resources of the system. Examples include memory (such as that allocated to bufferpools or address spaces), DASD, cache controllers, and microcode.

The fourth defining element of DB2 performance is optimization. All types of systems can be optimized, but DB2 is unique in that optimization is primarily accomplished internal to DB2.

When the demand (workload) for a particular resource is high, contention can result. Contention is the condition in which two or more components of the workload are attempting to use a single resource in a conflicting way (for example, dual updates to the same data page). As contention increases, throughput decreases.

DB2 performance can be defined as the optimization of resource use to increase throughput and minimize contention, enabling the largest possible workload to be processed.

In addition, DB2 applications regularly communicate with other MVS sub-systems, which must also be factored into performance planning. Many factors influence not only the performance of DB2, but also the performance of these other MVS subsystems.

Proactive performance engineering

Although reactive performance management will always be required (since systems and applications change over time), proactive performance engineering can be used to minimize reactive monitoring and tuning. Proactive performance engineering is a methodology for constructing performance models; it is done before code or database construction, and focuses on an application's run and response times. It provides a rigorous process that focuses on quantifiable results and helps to eliminate re-designing, re-coding, and retrofitting during implementation for performance requirements. Because proactive performance engineering occurs before implementation, multiple scenarios need not be coded to determine which meets the performance requirements. Figure 1 shows the relationship between the phase in which a problem is detected in an application and the cost to fix the application or database.

A performance model of an application is an analytical representation of the application's resource consumption. The model is used to estimate workloads generated by the proposed design. The resource consumption is broken down by CPU requirements, DASD requirements, and data transfer volume. The results are disbursed across a time



frame and then applied to the projected hardware platform.

Performance models will analyze the effect of all the SQL in a system based on an estimated arrival rate in terms of hardware capacity. (Most industry experts agree that approximately 80 percent of performance problems originate in the application code and database design.) Once the estimated workload has been determined, it is applied to the projected hardware configuration through queuing theory.

Proactive performance engineering is different from analyzing and optimizing single queries. Since individual queries may optimize at the expense of other queries, every query cannot be optimized. In turn, a model will show the overall effect of all the queries and how they affect each other's performance. The model allows you to optimize for overall performance objectives.

Developing a performance model is an iterative process. Designs and information supplied by various groups must be reviewed and updated as changes are made. To proactively build a performance model, management must take an active role in bringing together DBAs, application designers, and capacity planners to share information and address any business requirement issues that may affect the performance criteria.

Regardless of the project development stage, the following methodology can be used as a simple framework for developing a performance model. The description that follows is necessarily brief, given space constraints. A complete description of the Proactive Performance Engineering framework can be downloaded from PLATINUM technology's web site at http://www.platinum.com/products/ppewhite.htm.

The process of building a performance model has five phases: planning, information gathering, model construction, model validation, and model analysis.

Planning

The planning phase is the most important to the success of a modeling project. During this phase you will set the scope and performance objectives and establish communication channels between all participating parties. The planning steps include:

● Setting performance objectives -- specific and concise objectives are required to avoid confusion and scope creep when the model is analyzed. Two examples of performance objectives are: two-second response time for adding an order, and completion of a batch application in under five hours.

● Establishing communication channels -- all parties involved should be brought together and their responsibilities for the project should be stated clearly to the team.

● Setting the scope of the modeling project -- to limit the effort to a reasonable amount of work and indicate the level of detail for the model and the parts of the design that will be modeled.


http://www.platinum.com/products/ppewhite.htm


● Determining business priorities -- to help focus on the components that should be modeled.

Information gathering

Information requirements for a performance model can be broken down into three categories: process, data, and hardware. Business factors can be incorporated into both the process and data categories.

● Business metrics is the information that gives you insight into the processing requirements of the application.

● Database information is the metadata of your database. In this step you must identify the required tables for the model (possibly obtaining the information from the DB2 Catalog if the design has been implemented) and determine the tables sizes based upon business factors.

● Process information describes what the application will do. The information includes the functions (programs, plans, packages, DBRMs (Database Request Modules), and the like), an estimate for the frequency of execution for each function, a determination of the unique actions to be taken against the database (i.e., the SQL), and an estimation of the access path for each.

● Hardware information includes the type and class of mainframe being used and the devices attached to it (DASD, for example). The constraints and performance parameters for each piece of hardware need to be factored into the plan.

Model construction

A model can be constructed in several ways. You can use a method as simple as paper and pencil (for only the most simple models) or a sophisticated modeling tool. Spreadsheets can be used for small models; modeling tools should be used for everything else. The building of a model will now be described, without regard to implementation technology.

The model must assign the tables and indexes to DASD devices. As multiple tables are assigned to the same device, the model will use the I/O volumes to calculate service and wait times.

The next phase is calculation. Steps include:

● Calculate the execution frequency using process information -- for example, an order entry system may process 2,000 orders a day, with each order containing 25 line items. The execution frequency of the process that validates line items is 50,000 (2,000 x 25).

● Use algorithms to calculate resource requirements -- once the execution frequency has been calculated, the resource requirements for each action must be calculated. This can be based on rules of thumb (ROTs) in absence of a



modeling tool, but ROTs must be carefully applied or an inaccurate model may result.

● Calculate total workload -- multiply resource requirements for each action by its execution frequency to determine the total workload of the model.

● Calculate service and wait times for the workload -- by applying the workload to the hardware configuration, service times can be calculated. Queuing algorithms must be used to calculate wait times. The sum of service and wait times will provide response and run times.

Model validation

The model should be validated before moving on to the analysis phase. There is no reason to spend time analyzing a model that does not reflect the current designs or one that will not meet the objectives. The two major steps for model validation are:

● Review the model ● Check for order of magnitude errors.

Model analysis

Model analysis is the process of reviewing the results of the calculation, comparing it to the objectives, and gaining insight into what drives performance of the design. First, compare the results against the performance objectives. If you meet the objectives, you will still want to continue the analysis and also build what-if scenarios.

Simple analysis can be accomplished quickly. Determine where the greatest resource consumption is in the system. Consider tuning strategies based on the results of the performance model and the information gathered as input to the model.

Steps that can result from model analysis include:

● Load balancing of DASD devices ● Modification to database design ● Tweaking of application design and code ● SQL re-writes ● Adding indexes ● Hardware changes such as adding memory or capacity (this result is unlikely).

Finally, once you have completed an initial analysis, you should build several what-if scenarios. These scenarios provide insight into how the application will handle changes. The following are just a few examples of what-if options:

● Increase table sizes ● Increase transaction rates



● Add additional indexes ● Increase execution frequencies.

Successfully using performance models

The key to using performance models is understanding that performance models are based on averages and therefore require multiple scenarios. After a model has been completed, compare the results of a performance model with the implemented system. Determine the differences and adjust the model. You can also tune a model so the next model will be more accurate. Finally, performance models are not a panacea; if a system is over capacity due to large transaction volumes, proactive performance engineering will not be your silver bullet for solving the capacity problem. It can, however, alert you to the situation before the system is implemented.

Summary

Constructing a performance model is different from proactive performance engineering. Proactive performance engineering addresses performance requirements during the design stage without losing focus on the business requirements. It incorporates business requirements and rules; this makes the modeling effort cost effective. The results will only inspire management to carry out bigger and better modeling projects.

Richard J. Bolesta is a manager of product development at PLATINUM technology, inc. with responsibility for the development of performance estimation technology. With more than 14 years experience in the information technology industry, Bolesta has been a speaker at IDUG conferences as well as at the Computer Measurement Group, Enterprise Expo, and local user groups.

Craig S. Mullins is vice president of marketing and operations for the database tools division of PLATINUM technology, inc. He is also the author of the popular book, DB2 Developer's Guide, which will soon be available in its third edition; the book will include tips, techniques, and guidelines for DB2 through Version 5.


Home | Contact Us












Chicago, IL 60611 (312) 644-6610


Solutions Journal: Platinum Subsystem Analyser


Platinum Subsystem Analyzer by Garyn Pekarski

Manufacturer: Platinum technology, inc. 1815 S. Meyers Road Oakbrook Terrace, IL 60181 Phone: 630/620-5000 Fax: 630/691-0718 Email: [email protected] http://www.platinum.com

The organization -- The State of Maryland Department of Human Resources manages the State's welfare and child support services. Just over two years ago, the Department began developing a new database system designed to streamline and improve the processes associated with its welfare and child support systems -- primarily managing data on recipients, and automating the disbursement of funds. The Department acquired a database system developed by the State of Connecticut, and began modifying it to accommodate the needs of Maryland's agencies.

The result was two major applications: the Client Automated Resources and Eligibility System (CARES), which handles processes related to welfare services, and the Child Support Enforcement System (CSES). These applications include programs that establish and track eligibility to receive funds, automate the disbursement of funds, initiate and manage procedures to recoup funds when overpayments or other errors are detected, and manage many other processes.

CARES and CSES are written in DB2 for MVS, Version 4.1, running under CICS on an IBM ES/9000 9064 mainframe. Combined, the applications consist of 600 DB2 tables and 8,000 datasets, all of which take up approximately 4 GB of storage. Users of these applications generate approximately 100,000 CICS transactions per day.

The problem -- Development of the CARES and CSES systems has been a gradual process, with data from the various Maryland jurisdictions being added county by county. As more counties have been brought online, database performance issues have become

http://arbaltxp-011870/member/journal/aug97/platinum.html (1 of 4)5/5/2006 10:30:42 AM


a growing concern. Two of our major issues related to performance include: the need for acceptable response time to online database requests; and the need for efficient batch processing, so that all batch jobs are completed within the overnight window allocated for these jobs. In December of 1996, we formed a DB2 management group that focuses exclusively on performance issues. I am one of three team leaders for that group; five other full-time IT staff members work with me on a variety of developing performance initiatives.

When we began looking at performance issues, we realized we needed tools that would allow us to quickly see the performance bottlenecks in our environment. We also wanted to be able to capture a history of database activity, so we could see how our systems changed over time, analyze trends, and detect problems in the making. We wanted tools that would allow us to move from reactive to proactive performance management. And, we needed tools that would supply all these functions, but would not cause significant system overhead. After evaluating several products, we selected Subsystem Analyzer and Detector from PLATINUM technology.

The products -- PLATINUM Subsystem Analyzer gathers a wide range of detailed performance information on database page activity at the dataset, table, and index levels. It tracks all physical I/O processes and activity by DASD volume, collects response time statistics by volume and by dataset, and provides useful overviews of buffer pool activity, including details on hit rates. It collects data in variable slices of time which can be narrowed or broadened as needs may dictate: from minutes to hours in length. It allows you to save this data in an historical archive, which lets you compare activity during the same time slices on different days, and identify trends such as increased I/O that may reveal developing problems before they begin to greatly affect performance.

We use Subsystem Analyzer in conjunction with PLATINUM Detector, which provides performance statistics at the SQL level. Once we've identified time slices that appear problematic, we can use Detector to drill down further into that time interval to examine the underlying SQL that may be at the root of the problems.

We began using Subsystem Analyzer in April 1997, and almost immediately it showed us several major performance issues that were occurring during our overnight batch processing. The product showed that, in spots, the system was never able to get repeat hits on data pages, which was causing a steady decline in some buffer pool hit ratios. Drilling down into the datasets, we also observed that the amount of active data in a few objects was spreading out and getting thinner over time. In fairly quick order, the product showed us that dataset and page activity on only a dozen tables (out of 600) was responsible for the bulk of the problems. It also showed us that the data portion, not the index portion, of the datasets was responsible. Equally important, we were also able to see at the same time where we were doing especially well, and communicate all this information to our application developers in a meaningful way.

Strengths and weaknesses -- One of Subsystem Analyzer's nicest features is the way



it displays the information it gathers: high-level information is presented by database, then you can drill down into tablespaces, tables, volumes, and indexes. On the first screen that is displayed, I can see at a glance the percentage of total I/O load attributed to a database in a specific time slice, see what the buffer pool hit ratios are, and see other important statistics. This is tremendously useful: without Subsystem Analyzer, I would have to gather this data using several other tools and compile it into one report. With Subsystem Analyzer, I get the major information I need in one place. Subsystem Analyzer is the only tool we found that could provide highly detailed system statistics presented in a concise, easy-to-interpret format.

Subsystem Analyzer is a very intuitive product: I was using it effectively within hours of installation. Its top-down design allows you to navigate from macro- to micro-views of the system with ease. Online help is available, but I haven't needed to use it much at all, and I've never needed to look at the documentation. It's also a very economical product to use: its resource utilization is very low, so it provides great visibility into the system without causing a great deal of strain on resources.

Still, there are a few things I would like to see improved. There are some differences in the content and behavior of the collected data when displayed in the historical archive as opposed to the displays in the current real time interval. For example, one real time display shows me buffer pool content and includes high-water mark statistics on the pool pages consumed by a table or index. These useful measurements do not appear in the archived intervals. Also, the bridges between Detector's Plan/SQL displays and Subsystem Analyzer's volume, dataset and pool displays work smoothly and intuitively in real time intervals, but become difficult to traverse in historical mode. I find myself working in split-screen between the two when reviewing archived intervals. Still, on balance, I find the pairing of these two products to be a very powerful analysis tool.

Using Subsystem Analyzer, it's much easier to find the answers to a performance analyst's two most salient questions: What is happening in the system, and why is it happening?

Garyn Pekarski is a DBA team leader of the DB2 performance group at the State of Maryland Department of Human Resources. He has been a consultant since 1978, and has served as a DBA since 1990. His specialities include modeling, design, and performance tuning for large scale DB2 applications. His experience also includes large scale projects in the engineering, financial, retail, and insurance disciplines.










Home | Contact Us


Chicago, IL 60611 (312) 644-6610




Solutions Journal: Technical Talk


Performance is a Design Issue by Richard Yevich

History lesson: 1960's major IS issue? Performance. 1970's major IS issue? Performance. 1980's and 1990's and 1997's major IS issue? Performance. I think there's a theme here. But is solving performance today different than in the 60's? Yes. In fact, the difference began around 1984, the dawning of the "age of Aquarius," or relational database reality.

1984 was the time that major performance issues left the realm of system's programming (yes, I was one), and moved into the application arena in terms of database design (logical and physical), and application coding (navigational versus relational sets). Okay, we left 20 percent of the issues to the sysprogs; but what about the remaining 80 percent? Well, today one of the RBOCs has a billing system with 1500+ tables, navigationally built, with each major table having approximately 4 I/O layer subroutines, none functional, (4 = select, update, insert, delete), and a client base of perhaps 30 million customers; they wanted to pretend they aren't going to have performance problems. (No, I will not tell you what RBOC to switch to!) This is what happens when major performance guidelines are ignored -- improper design. In the case of the ROBC, they "bought" a legacy navigational design and made it worse by ignoring performance design issues. It can be fixed, but it's the old "pay me now or pay me lots more later." More on that issue later.

On the other hand, a government IS department is implementing an operational data warehouse (yes, they can and do exist), with an initial load primary account table of 6 billion rows (700+ gigabytes), with multiple indexes built brand new on Version 5, and they've gone from conception to production in 14 months. This government agency decided legacy migration equals bad performance, so they started over with performance design. In addition, they are using state of the art DBMS technology, lots of complex large SQL statements, and stored procedures with result sets, but only minimal impact on application programming.

Both my examples are based on real, not mythical, organizations. Both are implementing their systems this year. One has mainframe server-based management, and one has networked Unix -based management. One thought it could actually solve its performance

http://arbaltxp-011870/member/journal/aug97/technical_talk.html (1 of 3)5/5/2006 10:30:54 AM


problems with hundreds and hundreds of little servers connected in a distributed network (been in a bottleneck on an interstate lately?). The other knows that sysplexed CMOS with the right DBMS works. What, you ask, is the point? You'll find it in this column -- these two companies will dramatize point and counter-point.

Performance is a design issue, in the sense that it is designed in a logical database design, redone at physical database design, and designed into the application first at the functional level (main versus subroutine versus stored procedures), and then at the set level (SQL SQL SQL). Version 5 of DB2 brings maturity to some of these issues, but the others have been there since 1984 (Was that a novel? Are we really roving on Mars?!). The new 1 TB partitioned tablespaces in Version 5 will help immensely. But how many have used partitioned tablespaces to improve performance? Think about a 40 row divisional control table that is occasionally joined with a large partitioned tablespace. Wouldn't it make sense to use 40 partitions of one row each for the divisional table? Try it in test and watch what happens to your EXPLAINS -- it triggers parallelism and other such goodies.

Partitioned tablespaces benefited from page range scans starting with Version 4; it really exploded with this in Version 5. Imagine a query that we used to say shouldn't be written; now it becomes recommended strategy in order to use parallelism in page range scanning discrete non-contiguous partitions, instead of tablespace scans. Our government client says yes; the other guys think SQL is an I/O layer. (Remember those old hats with the propellers on top?)

POINT: Performance design really begins at the management level, and that is not in DB2 Version 5.

And what about indexes? In some cases, Type 2 indexes have yielded up to 70 percent locking reductions. Well, couldn't I add a few other indexes that I need for access strategies -- the ones I avoided before, due to the overhead? Yes, but these are known as physical design changes; some companies don't understand the fact that those things don't hurt (don't affect data integrity or functional processes). Also, many of those columns added to non-unique indexes in order to improve the delete performance problems of the past should now be removed. But that, again, falls into "changing" the physical design and its inherent misunderstood impacts.

Could using temporary tables, stored procedures, and result sets allow us to greatly reduce/improve coding and change control problems? Yes! (If you are allowed to use them - government agency yes, RBOCÉwell you know.) For true performance, it is important to stay current with technology, change with technology, and be ready to change again. If we had done that in the past, we wouldn't have so many legacy applications that cannot be moved forward. If you don't believe me, take a look at what Java/JDBC/Net.Data/browsers in the enterprise could mean.

Let's return to my point that performance design includes removing change control



problems by using stored procedures: if it is coded in one place, and really written correctly with correct SQL, a single if necessary change could affect every user (all 35,000 of them) instantly. I would not have to change 47 transactions, 53 batch programs, 17 periodic programs forgotten about, just to make a physical design improvement. A quality stored procedure could access both the OLTP and the warehouse databases (and legacy systems and distributed data) for those difficult analytical queries that determine how to set business strategy in order to make money. What a novel idea -- using the computer to assist business strategy rather than just processing paper! (Which, do you suppose, of our sample customers did what?)

Using stored procedure results sets really helps with those difficult queries and business strategies. If they are coded in that stored procedure, then I could adapt all my processes to any required changes in the business process or in the data required or whatever. This is easy when it is in one place, with powerful set processing SQL, with embedded if-logic (CASE statements), stuck in table expressions down-leveled in outer joins.

Performance designed in is cheap and easy. Performance after production is still easy but expensive. That is the now or later cost risk paradigm.

Richard Ywevich is an internationally recognized consultant, lecturer, and teacher, known for his expertise in relational database design, application design, and DB2 performance. He is president of RYC, Inc., a Biscayne, FL firm specializing in information technology consulting, data modeling, and advanced education. Yevich may be reached via email at [email protected] or through his Web home page at http://www.ryci.com.


Home | Contact Us


Chicago, IL 60611 (312) 644-6610



http://www.ryci.com/

http://www.ryci.com/









Solutions Journal: Technical Tip


Models for Estimating Performance by Tim Lenahan

Performance, performance, performance! It's the first cry heard once a system is in production. Unfortunately, less attention is given to performance before a system runs in production. Many applications design and code a system without any advance thought of what production performance will be. Systems are frequently benchmarked in test with little correlation to production behavior. Although production statistics can be simulated, other factors such as sequential detection are more difficult to predict from a test run.

This technical tip offers techniques that can be used to provide better understanding of expected elapsed and CPU times. Although most applicable in the batch environment, the majority of these techniques can also be carried over to the online world.

To employ these techniques, a basic understanding of CPU and I/O consumption is assumed. Each SQL statement must be broken down into autonomous pieces. A three table join will be broken down into three separate statements for estimation.

Although this is not completely accurate, it provides an easy way to estimate each statement. Subqueries are broken down into separate statements as well. The inner and outer portions of the subquery are each treated as individual statements. A good understanding of both correlated and non-correlated subqueries is essential to understanding just how many times both the inner and outer queries will execute.

After breaking each statement into autonomous statements, make an approximate estimate for the CPU and I/O for each statement. The purpose of these estimates is to develop a ballpark understanding of a process. Some processes have executed for hours when the process author had anticipated minutes, while other processes have executed in minutes when the author expected hours.

CPU estimates

To estimate the CPU consumed, two items are required: 1) the approximate number of instructions used to execute the SQL contained in the process, and 2) the number of instructions per second a single CPU engine can process. To estimate the number of

http://arbaltxp-011870/member/journal/aug97/technical_tip.html (1 of 4)5/5/2006 10:31:10 AM


instructions consumed to process each SQL statement, multiply the number of instructions per SQL statement times the number of times the statement is executed. This should be performed for all SQL statements in the entire process, including both read and update statements. General CPU estimates are as follows:

Simple Fetch 3,500-9,000 Singleton Selects 12,000-40,000 Update/Delete/Insert 40,000-90,000

The actual number of instructions consumed can vary widely, depending on such factors as access path, number of indexes, number of columns selected, number of predicates, and so forth. Using a rough CPU estimate will usually help one to develop a good CPU consumption model.

Processor engine speed is affected by the number of engines within the box. General estimates for engine speeds are as follows:

ES9000 - 50 mips CMOS I - 8 mips CMOS II - 18 mips CMOS III - 36 mips

I/O estimates

I/O has historically been the primary contributor to elapsed time. Although this began to change when OSAM sequential buffering was introduced in IMS many years ago, it is still essential to develop an I/O model. I/O can be divided into two types: random (synchronous) and sequential (asynchronous). Random I/O represents buffer manager requesting a single page from the I/O subsystem. If the database is significantly larger than the buffer pool, a random request will usually require a physical I/O. Each physical I/O can consume an average of 20ms (.020) elapsed time to be returned to the buffer manager. This time may be reduced by cache storage on the I/O controller. For proper estimation an average number must be used; for most shops, 20ms works well.

Sequential I/O can provide significantly different performance; it is exploited by DB2 during several processes, including but not limited to: list prefetch, sequential prefetch, and sequential detection. Sequential I/O represents a request for several consecutive pages in a pageset. The actual number is determined by the number of pages in the virtual buffer pool. For all but the smallest of subsystems, this is normally 32 pages per request. When a pageset is to be read sequentially, the amount of time to locate a specific page is irrelevant. The speed at which sequentially read pages are passed to the buffer pool is determined by the density on the DASD disk and the rotational speed of the device. Most modern DASD subsystems have many variables that affect outer tracks of the device. RAID devices introduce additional variables that preclude using a simple mathematical formula for determining the rate at which sequentially read data can be



returned.

For these reasons, I recommend that you perform simple benchmarks in your own environment. This can be as simple as running a query against a large production table using a predicate that is not indexed. This will cause a scan of the table with minimal CPU consumption; it should be performed during the time that the production jobs will be running. The average rate can be expressed in terms of megabytes per second. Most shops can expect an average rate of 4 to 5 meg per second without exploiting I/O parallelism.

Putting the pieces together

After a general understanding for estimating both CPU and I/O is achieved, an elapsed time estimate can be developed. Each process (job) can have both synchronous and asynchronous tasks. The two primary contributors to synchronous times are random I/O and CPU time. Both should be calculated as described above. Because both of these functions are synchronous, they are added together to form one "time line." Each pageset that is processed sequentially constitutes another "time line." As described earlier, each asynchronous "time line" is calculated by dividing the size of the pageset by the scan rate (normally 4 to 5 meg per second). The elapsed time of the process is determined by whichever "time line" is longer. This exercise results in a rough estimate for both CPU and elapsed time and helps to determine whether the process is I/O or CPU bound.

Serving multiple purposes

Although this process is most beneficial prior to coding, it also provides a very useful breakdown for processes that are already coded. When reducing elapsed time is the primary concern, it is essential to understand which components are contributing to the current time. The breakdown of CPU and I/O can provide an excellent foundation for performance analysis for both online and batch processes.

Tim Lenahan has worked in the database arena for the last 13 years, during which he's worked at four Fortune 500 companies. His special expertise is in performance; he has an extensive background in architecture, database design, and database implementation in IMS, DB2, and Oracle. Lenahan has received the "Best User" speaker awards during the last FIVE North American IDUG conferences. He writes and speaks frequently on database issues.










Home | Contact Us


Chicago, IL 60611 (312) 644-6610




The IDUG Solutions Journal August 1997 - Volume 4, Number 3

Documents

Transcript of The IDUG Solutions Journal August 1997 - Volume 4, Number 3