Senegacnik PGA Memory Management Oracle9i 10g Clanek

AADDVVAANNCCEEDD MMAANNAAGGEEMMEENNTT OOFF WWOORRKKIINNGG AARREEAASS

IINN OORRAACCLLEE 99II//1100GG

Joze Senegacnik, private researcher

BACKGROUND

The reason that I have picked the management of working areas in Oracle 9i/10g as the topic for this presentation was several questions posed in the Server Performance Forum on Oracle Metalink. The problem was related to the new Oracle feature for automatic memory management introduced in Oracle 9i and the amount of memory one can define for the PGA and for a single working area. Obviously there are, like in many other cases, certain possibilities in a Oracle database to overcome the default limitations defined for memory allocation of PGA. In this paper we will discuss the possibilities how to define the appropriate working areas for very large join or sort operations. We will see how setting of different regular and hidden initialization parameters affects the execution plan prepared by CBO and the Query Execution SQL Memory Manager (QESMM).

THE PGA

Process Global Area or often known also as Program Global Area resides in the process private memory and contains global variables and data structures that are accessable to all modules of the Oracle server (kernel) code, i.e contains data and control information for a server process. It is not shared between processes and therefore each server process has its own PGA which contains only a process specific inforamtion. The concept of shared memory structures contained in System Global Area (SGA) and private process global area is described on Figure 1: SGA, PGA and Server Processes. Beacuse PGA is a private global area there is no ned to protect this memory structures with latches or mutexes. The sum of PGA memory allocated by all server

processes attached to an Oracle instance is also referred to as the aggregated PGA memory allocated by the instance

controlled by PGA_AGGREGATE_TARGET parameter. It is important to know, that each time a cursor (SQL

statement) is executed, a new runtime memory is allotted in PGA memory region of the server process executing that

cursor.

The contents of the PGA varies depending on whether we are using DEDICATED or SHARED server mode, but generally we can use the following description of PGA parts:

• Session Memory: The memory allotted to hold logon information and other session details. When a shared server model is used, this kind of information is stored in SGA because it needs to be persistent between calls.

• SQL Execution Memory: The memory allotted for the execution of SQL statements The SQL Execution Memory has a persistent and a run time area:

• The persistent area requires persisting across multiple executions of the same SQL statement and contains the information such as bind details, data type conversion, etc. This persistent area is de-allocated only when the cursor is closed. When the shared server processes model is used the persistent area is a part of the SGA (part of Large pool if properly configured).

• The runtime area contains information used while a SQL statement is being executed. Its size depends on the number and size of rows being processed as well as the type and complexity of SQL statement as. It is de-allocated when the execution completes. For shared sever processes, the run time area is resides in the PGA for DML/DDL operation and in the SGA for queries. The complex queries that perform a lot of sort, group by, hash-join and other operations warehousing environment require a large run time SQL execution memory.

Figure 1: SGA, PGA and Server Processes

AUTOMATIC PGA MEMORY MANAGEMENT

Prior to automatic PGA memory management a DBA had to define different working areas for sorting, hash-joins, bitmap merges etc. by setting *_AREA_SIZE initialization parameters. In an OLTP environment this was a relatively simple task because the SQL statements process only a small number of rows. The only exception is long running reports which process a large number of rows. But it was a complex task in a DSS environment because a lot of complex long running queries are executed and their performance to a great extent depends on the amount of memory that is available for the execution of different SQL operators such as

• Sort-based operators, such as ORDER BY, GROUP BY, ROLLUP, and window functions in analytical functions

• Hash-join

• Bitmap merge

• Bitmap create

• Write buffers used by bulk load operations

Oracle Instance

SGA

Redo Log Buffer

Shared Pool

Data Dictionary

Library

Cache

DBWR SMON PMON CKPT LGWR Others

Java Pool

Large Pool

Database Buffer Cache

ARCH

PGA

Server process

1

PGA

Server process

2

PGA

Server process

3

PGA

Shared

Server process 1

PGA

Shared

Server process 2

PGA_AGGREGATE_TARGET

Database Administration

3 Paper # 411

This part of memory allocated to execute the operators is known as working area.

Each SQL operator requires an amount of memory to complete the execution. There are three different possible modes of execution:

• Optimal execution – when the size of the working area is big enough to accommodate the input data and the auxiliary memory structures.

• One-pass execution – when the size of the working area is smaller than optimal size the response time increases, because an extra pass has to be performed over part of the input data.

• Multi-pass execution – when the size of a work area is far too small compared to the input data size then multiple passes over the input data are required. This could dramatically increase the response time of the SQL operator because it is using temporary segment on disk. Even for this execution mode minimum memory is required.

As we can see from the above explanation a bigger working area can improve the performance of a particular SQL operator at a cost of higher memory consumption. On Figure 2: Effect of Used Memory on the Response Time we can see the graphical representation of the influence of required (actually used) memory on the response time. Therefore the amount of acquired memory defines the possible execution mode.

Figure 2: Effect of Used Memory on the Response Time

PGA_AGGREGATE_TARGET INITIALIZATION PARAMETER

As we already discussed in the previous section, the PGA_AGGREGATE_TARGET (P_A_T) initialization parameter specifies the target aggregate PGA memory available to all server processes attached to the instance. Setting P_A_T to a nonzero value has the effect of automatically setting the WORKAREA_SIZE_POLICY parameter to AUTO. Conversely, setting P_A_T to 0 automatically sets the WORKAREA_SIZE_POLICY parameter to MANUAL. This means that SQL work areas are sized using the *_AREA_SIZE parameters.

Response time

Memory used Minumum

Multiple-

passes

1 pass cache

optimal


4 Paper # 411

Oracle kernel attempts to respect the P_A_T parameter and tries adapting the size of work areas to ensure the maximum performance benefits and the most efficient utilization of memory. P_A_T parameter can be dynamically changed what gives the DBA the opportunity to add or reduce the amount of memory used for all PGAs. It is important to know that in Oracle9i the automatic work area memory management supports only dedicated server connections, while it has no effect for shared server connections. In Oracle10g the automatic work area memory management controls both dedicated and shared connections.

HOW DOES AUTOMATIC SQL EXECUTION MEMORY MANAGEMENT WORK?

The paper SQL Memory Management in Oracle9i [1] briefly explains the mechanism of automatic SQL execution memory management introduced first time in Oracle9i. The second available source of information is the technical paper »Oracle9i Memory Management: Easier Than Ever« [2] that goes further and give some new details. On the web, there are some presentations which more or less use the information from the above sources and give additional information [3],[4]. The Automatic SQL Execution Memory Management or sometimes referred to as Query Execution Memory Manager (QESMM) feature uses a feedback mechanism to dynamically regulate the amount of memory used by various active SQL operations. The mechanism is depicted on Figure 3: The Automatic Work Area Management (QESMM). The text which very nicely describes the mechanism of computation of SQL memory target and global memory bound is available in [2], written by an Oracle employee Sushil Kumar. Generally the mechanism works in the following way: When a SQL operation starts it registers its work area profile through local memory manager services. The work area profile contains the characteristics of working area like the type of operation (e.g. sort, hash-join, group-by), the current minimum, one-pass and cache memory requirements, the degree of parallelism for parallel operations and the amount of PGA currently in use. Local memory manager stores this profile in SGA. All active profiles are constantly updated to reflect the current memory requirement and actual consumption. As all these profiles are stored in one place the kernel can quickly and accurately determine the instance wide PGA memory needs and actual memory consumption. When the SQL operation completes the working area is deregistered from the set of active work areas (v$sql_workarea_active). On the instance level there is a service called global memory manager which is executed as part of one of the background processes (later we will see this is the CKPT process). Its role is to indirectly dictate the size of each active work area through a mechanism called global memory bound which is published at regular intervals, generally every three seconds. The global memory bound is calculated from the number and characteristics of all currently active work area profiles stored in SGA. The global memory bound constrains the size of working areas. This mechanism works like this: the higher the global memory bound is, the more is memory available to each working area and vice versa. With this mechanism the kernel is able to restrict the total PGA memory usage below the PGA_AGGREGATE_TARGET parameter. Therefore when the overall memory consumption is low the memory bound is high and the working areas can grow up to their maximum limit. When many concurrent SQL operations are consuming a lot of memory the global memory bound is lowered to reflect the current memory consumption and to enforce the overall memory consumption below the PGA_AGGRGATE_TARGET limit. The global memory bound is calculated in two steps. In the first step the kernel determines an internal target called SQL Memory target which reflects the actual usage of PGA memory which can’t be dynamically resized (i.e. persistent area + the run-time area, the memory used for PL/SQL tables etc.) and adds some reserve for unforeseen factors like delays in adapting to the newly published global memory bound. When a lower global memory limit is published the active working areas need some time to adapt their sizes to the newly published memory limit. Actually they have to reduce their memory consumption. In Oracle9i the PGA memory for the shared servers (MTS) is not automatically tuned. In Oracle10g also the PGA for shared servers is automatically tuned when PGA:AGGREGATE_TARGET parameter is set to nonzero value and WORKAREA_SIZE_POLICY is set to AUTO. In Oracle9i the memory used by shared servers is treated like memory which can’t be dynamically resized. When the SQL memory target is determined it is translated to the local memory limit called global memory bound and published in regular 3 seconds interval. There may be situation when a global memory bound can become stale because of the sudden increase of memory consumption. The SQL operation should detect such extreme


5 Paper # 411

situation and perform a foreground memory bound computation. However, such situations should be rare and the background computation at 3 seconds interval should be adequate most of the time.

Figure 3: The Automatic Work Area Management (QESMM)

The locally memory manager uses the current value of the global memory bound and the work area’s profile to determine the amount of PGA memory which can be used by this work area. This size is called the expected size. During the calculation of the expected size the following rules are applied (according to [2]):

• The expected size can never be less than the minimum memory requirement of the operation and more than its cache size. Therefore at least the minimum memory requirement will be always allocated what can result in memory over-allocations in the case when the PGA_AGGREGATE_TARGET is set too low.

WA1

HJ

SQL1

aggregated persistent

area + a part of the run

time area of all server

processes

SQL2 SQLn

PGA aggregate

WORK AREA TABLE (X$QESMMIWT)

WP1

v$sql_workarea

WP2 WP3 WP4 WP5 WPn ...

...

WA2

SORT

WA3

BITMAP

LOCAL

MEMORY

MANAGER

GLOBAL

MEMORY

MANAGER

(CKPT)

SGA

PGA_AGGREGATE_TARGET

SQL Memory Target (auto target) 1

st step

2nd

step (every 3 seconds bound is published)

Free

list

Free

list

Global

Memory

Bound

P

1

P

2


6 Paper # 411

• When the global memory bound is between minimum and cache requirement, the expected size will be equal to the global memory bound.

• Sort operations are the only exception to the previous rule. The expected size for the sort operation will be equal to one pass size if the bound is less than its cache size. This is due to the fact that the sort operation does not benefit from more than one pass memory unless the whole operation can be performed in cache.

• The parallel operations get the expected size multiplied by the degree of parallelism.

• No single operation is allowed to use all available memory. Therefore, the expected size can never be more than certain limit. This limit was quite restrictive in Oracle9i/10gR1 where it was 5% of the overall target for serial and 30% for parallel operations. This limit was changed in Oracle10gR2 what we will see later in this paper.

Each SQL operation periodically checks the expected size and adapts the work area size accordingly to the specified value. By this mechanism the globally memory bound limit is respected although not instantly but after some time.

Figure 4 graphically represents how globally memory bound impacts the size of the work area (so-called expected size).

Figure 4: Workarea expected size – size that can be made available to workarea

As we can see the global memory bound is set at 60MB. The work areas W1 and W2 will run in optimal mode as their optimal size is below global memory bound. The work areas W3 will get only one-pass memory because sort operation don’t benefit from more memory unless they can get the cache size. The W4 will get 4 times 60M because the parallel operations get the expected size multiplied by DOP. As we will see later on there are certain limitations to this rule.

THE WORKAREA TABLE

While the working area is active it is registered in the work area table (at least all working areas bigger then 64kB are visible in the work area table). The view V$SQL_WORKAREA_ACTIVE reports the active working areas and is based on the

one

pass

10M

20M

30M

40M

50M

60M

70M

80M

90M

100M

W1

SORT

DOP=2

W2

HJ

DOP=1

W3

SORT

DOP=2

W4

HJ

DOP=4

Global Memory

Bound

min

one

pass

one

pass

one

pass

min

min

min


7 Paper # 411

fixed table X$QESMMIWT. The work area table has 67 entries - this number I found on all databases during the preparation of this paper. Each work area is included in one work area table entry that is protected by a latch. The work area table can be dumped to the trace file using the following commands:

SQL> oradebug setmypid SQL> oradebug dump workareatab_dump 3

or within the current session with:

SQL> alter session set events 'immediate trace name workareatab_dump level 3';

The command always dumps the whole work area table regardless if the current session has any active work area or not. Therefore we always get the list of currently active working areas for the instance.

The contents of the work area table dump depends on the dumping level. During the preparation of this paper the following values for dump level were observed that produce different dumping results:

Level Description

1 Only a list of values is dumped

2 Level 1 + work area table with summary information

3 Level 2 + details for all active work areas

The work area table dump contains two parts. The first part contains the values of different parameters, while the second part contains the list of all work area table entries. For each entry the number of currently active together with maximum number ever registered in this table entry (most likely from since instance startup) is reported (i.e. max=3). When dumping the work area table at level 3 we get also the details for each active work area that is currently registered in the work area table entry. We will se later that the work areas smaller than 64kB are not registered (see Metalink Note: 223730.1). Some dynamic work areas are not registered either. The reader will be able to observe this fact while we will step through execution of one SQL statement later on in this paper.

An important fact to know is that each entry in the work area table is protected by a “Child SQL memory manager workarea list latch”. We will discuss the latch protection in the subsequent chapter. Each entry in the working area table has its own freelist that is maintained during registration/deregistration of working areas in that table entry. Dumping Work Area Table (level=3) ===================================== Global SGA Info --------------- global target: 10 MB auto target: 4 MB max pga: 200 MB pga limit: 0 MB pga limit known: 0 pga limit errors: 0 pga inuse: 261 MB pga alloc: 279 MB pga freeable: 0 MB pga freed: 0 MB pga to free: 0


8 Paper # 411

pga auto: 6 MB pga manual: 0 MB pga alloc (max): 339 MB pga auto (max): 159 MB pga manual (max): 0 MB # workareas : 18 # workareas(max): 43 Work Area Table (67 entries) ---------------------------- Entry 0: 0 are active (max=3) Entry 1: 0 are active (max=3) Entry 2: 0 are active (max=3) ... ... Entries 3 through 61 are omitted ... Entry 62: 1 are active (max=3) Work area 2FD9B334 wid=0, trcId=-1 sid=130, ser#=1, flags=11 (AUTO|SGA|ADVICE) qc_sid=65535, qc_instid=65535, slv_grp=1 curSize=488, maxSize=488, #maxPasses=0 tempSegStateObj=00000000, tempSegObjType=0 shared work area descriptor (2F856080): flags:12 ( AUTO SGA ) opType:9(HASH-JOIN) opId=35 proxy=00000000 isize=1907K, osize=3497K, rowlen=12, flags=01 opt=270, 1pass=4, mpass=9, lastMem=7832KB maxTempSegSize=4096KB, lastTempSegSize=0KB Entry 63: 0 are active (max=4) Entry 64: 1 are active (max=3) Work area 2FFF103C wid=0, trcId=-1 sid=138, ser#=1, flags=43 (AUTO|SGA|ADVICE|DATASTATS_SET) qc_sid=65535, qc_instid=65535, slv_grp=1 curSize=7864, maxSize=7864, #maxPasses=0 tempSegStateObj=00000000, tempSegObjType=0 shared work area descriptor (2F856080): flags:12 ( AUTO SGA ) opType:9(HASH-JOIN) opId=35 proxy=00000000 isize=1907K, osize=3497K, rowlen=12, flags=01 opt=270, 1pass=4, mpass=9, lastMem=7832KB maxTempSegSize=4096KB, lastTempSegSize=0KB Entry 65: 0 are active (max=2) Entry 66: 0 are active (max=3)

As we can see from the above dump the work area profiles can be shared among different sessions. The sessions with SID 130 and 138 both use the same shared work area profile whereas the actual working memory alotted to this shared work area used for execution of SQL operator is different and only belongs to the current session. There are many abbreviations used in this dump. The Appendix A contains explanations for some of them.

SQL MEMORY MANAGER LATCHES

The global SQL memory manager and work area table entries are protected by latches.

The following command dumps the current state of latches and from the listing from the trace file below we can see the locations where the child latches are held. To improve the readability of this listing the child latches from #66 through #2 inclusive are omitted. The SQL memory manager latch most likely protects the global memory bound computation.


9 Paper # 411

Each Child SQL memory manager workarea list latches protect the single work area table entry and is used during registration/de-registration of working area in the work area table and during bound computation. Each work area table entry contains its own free list.

SQL> ALTER SESSION SET EVENTS 'immediate trace name latches level 1'; .... 03c4fd70 SQL memory manager latch level=1 Location from where latch is held: qesmmIDeamonCb: Context saved from call: 1 state=free, wlstate=free 03c4fde8 Parent SQL memory manager workarea list latch level=2 Location from where latch is held: No latch: Context saved from call: 0 state=free, wlstate=free 1f265798 Child SQL memory manager workarea list latch level=2 child#=67 Location from where latch is held: qesmmIRefreshBound: Context saved from call: 6 state=free, wlstate=free .... Child latches #66 through #2 inclusive are omitted 1f263698 Child SQL memory manager workarea list latch level=2 child#=1 Location from where latch is held: qesmmIRefreshBound: Context saved from call: 6 state=free, wlstate=free ...

In the QESMM trace we will observe locations when these child lathes are acquired. We can also observe that each entry in the Work Area Table is protected by one “SQL memory manager workarea list latch” child latch.

Figure 5: Work area table with work areas and SQL Memory Manager workarea list latches

WORK AREA TABLE

WDS

...

Child SQL memory manager

workarea list latches (0..66)

Entry 0

Entry 66

Table Entries

...

0

1

66

...

WDS WDS WDS

WDS

Entry 1

FREE

LIST

FREE

LIST

FREE

LIST


10 Paper # 411

ORACLE TRACE AND QESMM DAEMON

During the preparation of the paper I found Jonathan Lewis comment on one forum that the QESMM daemon is most likely the CKPT process. His explanation was based on the X$MESSAGES fixted table that contains the following row where the destination is CKPT process.

SQL> select description,dest from x$messages where description like 'SQL Memory%'; DESCRIPTION DEST ---------------------------------------------------------------- ------------------ SQL Memory Management Calculation CKPT

Another idea I got while I was searching for the deamon that recalculates the memory bound every three seconds was to use the Oracle Trace facility wich was significantly improved. Event without any additional setting in Oracle 10gR2 I was able to see immediately that the daemon is really the CKPT process.

SQL> column time format 99999999 SQL> column data format a130 SQL> set pages 999 lines 150 SQL> column cur_seq new_value cur_seq SQL> SELECT MAX(seq#) cur_seq FROM x$trace; SQL> SELECT TIME time,data FROM x$trace WHERE data LIKE '%SQL Memory%' AND seq# > &cur_seq ORDER BY seq#;

On a highly loaded system the results were the following: TIME DATA --------- --------------------------------------------------------------------------------------------- 14204070 KSBCTI: (CKPT) : (timeout action) : acnum=[129] comment=[SQL Memory Management Calculation] 14204432 KSBCTI: (CKPT) : (timeout action) : acnum=[129] comment=[SQL Memory Management Calculation] 14204784 KSBCTI: (CKPT) : (timeout action) : acnum=[129] comment=[SQL Memory Management Calculation] 14205091 KSBCTI: (CKPT) : (timeout action) : acnum=[129] comment=[SQL Memory Management Calculation] 14205433 KSBCTI: (CKPT) : (timeout action) : acnum=[129] comment=[SQL Memory Management Calculation] 14205752 KSBCTI: (CKPT) : (timeout action) : acnum=[129] comment=[SQL Memory Management Calculation]

Wile the system was only lightly loaded the results were the following: TIME DATA --------- --------------------------------------------------------------------------------------------- 9169937 KSBCTI: (CKPT) : (timeout action) : acnum=[129] comment=[SQL Memory Management Calculation] 9170239 KSBCTI: (CKPT) : (timeout action) : acnum=[129] comment=[SQL Memory Management Calculation] 9170539 KSBCTI: (CKPT) : (timeout action) : acnum=[129] comment=[SQL Memory Management Calculation] 9170839 KSBCTI: (CKPT) : (timeout action) : acnum=[129] comment=[SQL Memory Management Calculation] 9171140 KSBCTI: (CKPT) : (timeout action) : acnum=[129] comment=[SQL Memory Management Calculation] 9171440 KSBCTI: (CKPT) : (timeout action) : acnum=[129] comment=[SQL Memory Management Calculation] 9171741 KSBCTI: (CKPT) : (timeout action) : acnum=[129] comment=[SQL Memory Management Calculation]

KSB stands for Kernel Services Background. Observe the TIME column in the above listing. Approximately every 3 seconds (TIME columns is reported in centicesonds and has the same value as is reported by V$TIMER) the CKPT process recalculates (in reality it checks if this is necessary) the bound. During the first snapshot the system was highly loaded and thus the interval is slightly more than 3 seconds.


11 Paper # 411

I found the same description in the X$KSBTABACT fixed table that in Oracle10gR2 contains 139 distinct actions. According to my current understanding this is a table of all possible actions performed by a certain background process. SQL> select PROCESS_NAME_KSBTABACT PROC, ACTION_DESCRIPTION_KSBTABACT DESCRIPTION from X$KSBTABACT where ACTION_DESCRIPTION_KSBTABACT like '%SQL Memory%'; PROC DESCRIPTION ------------- ---------------------------------- CKPT SQL Memory Management Calculation

ALLOCATIONS OF QESMM MEMORY STRUCTURES

The allocation of the memory for different QESMM structures can be observed V$SGASTAT fixed view or in X$KSMFSV. In Oracle10gR2 the V$SGASTAT contains much more records. The fixed view X$KSMFSV contains the variables located in the fixed SGA. Querying both views returns the following results:

SQL> select * from v$sgastat where name like 'qesm%' or name like 'work area%'; POOL NAME BYTES ------------ -------------------------- ---------- shared pool qesmmaInitialize: ta_qesm 264 shared pool qesmmaInitialize: oa_qesm 112 shared pool qesmmaInitialize: pa_qesm 11088 shared pool work area tab 271216 shared pool qesmmaInitialize: ia_qesm 264 shared pool qesmmaInitialize: 112

SQL> SELECT ksmfsnam name,ksmfstyp "TYPE", ksmfssiz "SIZE" FROM X$KSMFSV WHERE ksmfsnam LIKE '%qesmm%' NAME TYPE SIZE ---------------------- --------- ---- qesmmISgaLatch_ ksllt 104 qesmmISgaLatch1_ uword 4 qesmmISgaLatch2_ uword 4 qesmmISgaLatch3_ uword 4 qesmmISgaLatch4_ uword 4 qesmmILstLatch_ ksllt 104 qesmmILstLatch1_ uword 4 qesmmILstLatch2_ uword 4 qesmmILstLatch3_ uword 4 qesmmILstLatch4_ uword 4 qesmmILstLatch5_ uword 4 qesmmILstLatch6_ uword 4 qesmmISga_ qesmmISg 264 qesmmIdeamon_ ksbsa 8 qesmmwm_ sword 4 qesmmcow_ sword 4 qesmmc1w_ sword 4 qesmmcmw_ sword 4 qesmmMmonPgaMaxSize_ ksbsa 8 qesmmMmonSlvActResize_ sword 4 qesmmaSga_ qesmmaSg 280


12 Paper # 411

DIAGNOSTIC TOOLS

To trace the memory management and different SQL operators one can use the following events:

To dump sort statistics at level 10 which is the most detailed level one can run:

SQL> ALTER SESSION SET EVENTS '10032 trace name context forever, level 10';

Here are the results from the trace file:

---- Sort Parameters ------------------------------ sort_area_size 15788032 sort_area_retained_size 15788032 sort_multiblock_read_count 1 max intermediate merge width 962 ---- Sort Statistics ------------------------------ Input records 1000000 Output records 1000000 Total number of comparisons performed 6729172 Comparisons performed by in-memory sort 6729172 Total amount of memory used 15788032 Uses version 2 sort Does not use asynchronous IO ---- End of Sort Statistics -----------------------

In order to dump sort intermediate run statistics at the most detailed level 10 one can set event 10033.


To dump hash-join statistics:


The output of the hash-join statistics is in APPENDIX D.

There are certain events that can be used for testing and tracing of SQL Memory Management like events 10133 and 10134.

One can also set event 10319 at level 2 to trace top-level PGA allocation and de-allocation.

SQL> alter session set events '10319 trace name context forever, level 2';

In the trace file we can observe how PGA is allocated/de-allocated:

PGA ALLOC used=16102K alloc=16370K sz=0002180 typ=FRE name='callheap' PGA ALLOC used=16104K alloc=16370K sz=0002168 typ=FRE name='callheap' PGA ALLOC used=16106K alloc=16370K sz=0002180 typ=FRE name='callheap' PGA ALLOC used=16108K alloc=16370K sz=0002180 typ=FRE name='callheap' PGA ALLOC used=16110K alloc=16370K sz=0002180 typ=FRE name='callheap' ...


13 Paper # 411

PGA ALLOC used=15850K alloc=16178K sz=0065476 typ=FRE name='session heap' PGA ALLOC used=15914K alloc=16242K sz=0065476 typ=FRE name='session heap' PGA ALLOC used=15978K alloc=16306K sz=0065476 typ=FRE name='session heap' PGA ALLOC used=16042K alloc=16370K sz=0065476 typ=FRE name='session heap' ... PGA FREE used=16108K alloc=16370K sz=0002180 typ=FRE name='callheap' PGA FREE used=16106K alloc=16370K sz=0002180 typ=FRE name='callheap' PGA FREE used=16104K alloc=16370K sz=0002180 typ=FRE name='callheap' PGA FREE used=16102K alloc=16370K sz=0002168 typ=FRE name='callheap' PGA FREE used=16099K alloc=16370K sz=0002180 typ=FRE name='callheap' .... PGA FREE used=15978K alloc=16370K sz=0065476 typ=FRE name='session heap' PGA FREE used=15914K alloc=16370K sz=0065476 typ=FRE name='session heap' PGA FREE used=15850K alloc=16370K sz=0065476 typ=FRE name='session heap' PGA FREE used=15786K alloc=16370K sz=0065476 typ=FRE name='session heap' PGA FREE used=15722K alloc=16370K sz=0065476 typ=FRE name='session heap' PGA FREE used=15658K alloc=16370K sz=0065476 typ=FRE name='session heap'

...

RUNTIME EXECUTION OF ONE SQL STATEMENT

The QESMM has one hidden initialization parameter named _smm_trace which one can use to produce a trace of the actions QESMM performs while excuting a SQL statement. I tried to use this parameter on Oracle9iR2, but in that version it is somehow disabled. On Oracle10gR1 and release 2 as well the trace is produced. I experimented a little bit with levels. The research showed the values that produce different level of trace use different bits or their combination. The most detailed trace is obviously produced at level 65535.

Below is an excerpt from a trace file produced with by setting _smm_trace=65535 at session level and event 10046 at level 8. The text from event 10046 trace is marked in blue colour while the QESMM trace text is black colour. The relevant parts of text are marked bold and the colour can even hange to red. All the lines are marked with line numbers to be easily identified when referred to in the text.

First comes the text of the SQL statement which was executed during the parse phase of a create index command and will be used to explain how both memory managers – local and global work.

SELECT i.obj#, i.ts#, i.FILE#, i.BLOCK#, i.intcols, i.TYPE#, i.flags, i.property, i.PCTFREE$, i.INITRANS, i.MAXTRANS, i.blevel, i.leafcnt, i.distkey, i.lblkkey, i.dblkkey, i.clufac, i.cols, i.analyzetime, i.samplesize, i.dataobj#, NVL (i.DEGREE, 1), NVL (i.INSTANCES, 1), i.rowcnt, MOD (i.pctthres$, 256), i.indmethod#, i.trunccnt, NVL (c.unicols, 0), NVL (c.DEFERRABLE# + c.valid#, 0), NVL (i.spare1, i.intcols), i.spare4, i.spare2, i.spare6, DECODE (i.pctthres$, NULL, NULL, MOD (TRUNC (i.pctthres$ / 256), 256) ), ist.cachedblk, ist.cachehit, ist.logicalread FROM ind$ i, ind_stats$ ist, (SELECT enabled, MIN (cols) unicols, MIN (TO_NUMBER (BITAND (defer, 1))) DEFERRABLE#, MIN (TO_NUMBER (BITAND (defer, 4))) valid# FROM cdef$ WHERE obj# = :1 AND enabled > 1 GROUP BY enabled) c WHERE i.obj# = c.enabled(+) AND i.obj# = ist.obj#(+) AND i.bo# = :1 ORDER BY i.obj#

The run-time execution plan of the statement from V$SQL_PLAN_STATISTICS_ALL is:


14 Paper # 411

ID PID OPERATION POLICY LAST_EXEC LAST_MEM_USED ------- --------------------------------------------- ------- --------- -------------- 1 0 SORT ORDER BY AUTO OPTIMAL 2048 2 1 HASH JOIN OUTER AUTO OPTIMAL 305152 3 2 NESTED LOOPS OUTER 4 3 TABLE ACCESS CLUSTER IND$ 5 4 INDEX UNIQUE SCAN I_OBJ# 6 3 TABLE ACCESS BY INDEX ROWID IND_STATS$ 7 6 INDEX UNIQUE SCAN I_IND_STATS$_OBJ# 8 2 VIEW 9 8 SORT GROUP BY AUTO OPTIMAL 8192 10 9 TABLE ACCESS BY INDEX ROWID CDEF$ 11 10 INDEX RANGE SCAN I_CDEF2

We can use this run-time execution plan to get a general idea how many working areas the QESMM will allocate during the execution of this SQL statement. Now let us step through the trace file. The trace starts with a few lines of event 10046 trace – lines 001 through 004. 001 PARSING IN CURSOR #3 len=789 dep=2 uid=0 oct=3 lid=0 tim=902551359381 hv=3159716790 ad='44677684' 002 Text of the SQL statement is omitted here and is available above 003 END OF STMT 004 PARSE #3:c=0,e=1845,p=0,cr=0,cu=0,mis=1,r=0,dep=2,og=4,tim=902551359375

The SQL statement is now parsed ( it was a hard parse because mis=1) and the execution phase now begins. We must be aware of the fact that the kernel writes these lines in the trace file only when the execution of the phase is completed. We will see this very clearly for the EXEC phase which is reported only at line 031. During EXEC phase the required shared work areas are allocated. The word “qksmm” which is used in the subsequent lines as the prefix of the names for some of the executed functions stands for memory management services for the SQL compiler according to the Metalink Note: 175982.1 titled “ORA-600 lookup Error Categories. The expression “qesmm” is in the same document described as run time support for sql execution.

In the run-time execution plan we can see that 3 work areas are allocated for steps with ID 9, 2 and 1 respectively. The work areas are allocated in the order of the execution of the SQL statement.

005 qksmmAllocCContext: alloc CMM context (dyn=0) 006 qksmmAllocCContext: alloc context (shr=4171883C/mut=00000000) 007 qksmmAllocSharedWorkArea: alloc shared workarea desc (ctx=4171883C) 008 allocate work area 41A29B20 using heap0 009 qksmmMakeCompTimeWDef: done creating wds=41A29B20 010 flags:52 ( AUTO SGA NOSTATS ) 011 opType:1(GROUP BY (SORT)) opId=9 proxy=00000000 012 isize=64K, osize=64K, rowlen=15, flags=02 013 opt=0, 1pass=0, mpass=0, lastMem=0KB 014 maxTempSegSize=0KB, lastTempSegSize=0KB

As the first step a CMM context (probably CMM stands for Cursor Memory Management – most likely the CMM is also the local memory manager) is allocated. In next step the first work area for the GROUP BY operator is allocated using heap0 of the PGA (we are using here DEDICATED server). Here we must mention the hidden parameter _use_realfree_heap which dictates how Oracle allocates CGA and UGA heaps. When set to true they are allocated as independent top-level heaps (heap 0), not as a PGA sub-heap. We will discuss this parameter a little bit more later on. The size of the work area is 64K (isize - input and osize - output size) and no statistics will be calculated for this work area (most likely this is the meaning of the field flags: NOSTATS).

The observed possible values of flags field for the work area profile are: MANUAL |AUTO, SGA|DYN, NOSTATS|...

The meaning of other columns is: opt for optimal size of the workarea, 1pass for one required one pass memory, mpass for multi-passes memory requirement, maxTempSegSize is the maximum size of the temporary segment (in temporary tablespace) and the lastTempSegSize is the size of the temporary segment for the last execution.


15 Paper # 411

In the section from line 015 to line 031 two additional work areas are opened within the same CMM context (hence we are executing only one statement). The work area for HASH-JOIN operator will be included in the statistics calculation.

015 qksmmAllocSharedWorkArea: alloc shared workarea desc (ctx=4171883C) 016 allocate work area 41A29448 using heap0 017 qksmmMakeCompTimeWDef: done creating wds=41A29448 018 flags:12 ( AUTO SGA ) 019 opType:9(HASH-JOIN) opId=2 proxy=00000000 020 isize=64K, osize=64K, rowlen=223, flags=02 021 opt=0, 1pass=0, mpass=0, lastMem=0KB 022 maxTempSegSize=0KB, lastTempSegSize=0KB 023 qksmmAllocSharedWorkArea: alloc shared workarea desc (ctx=4171883C) 024 allocate work area 41A294A0 using heap0 025 qksmmMakeCompTimeWDef: done creating wds=41A294A0 026 flags:52 ( AUTO SGA NOSTATS ) 027 opType:8(SORT (v2)) opId=1 proxy=00000000 028 isize=64K, osize=64K, rowlen=186, flags=02 029 opt=0, 1pass=0, mpass=0, lastMem=0KB 030 maxTempSegSize=0KB, lastTempSegSize=0KB 031 EXEC #3:c=0,e=35168,p=0,cr=0,cu=0,mis=1,r=0,dep=2,og=4,tim=902551395030

Now the EXEC phase is completed and all the subsequent trace information comes from the FETCH phase. The initialization of work areas now starts. We can see that the function names executed onwards start with the prefix “qesmm” what stands for the runtime support functions for SQL statement execution.

032 qesmmCStartWorkArea: starting wds=41A29448 033 qesmmCInitialize: start initializing cursor work areas 034 SQL String: 035 Text of the SQL statement is omitted (is the same as the above text)

For each of the three work areas the qesmmCRegisterWorkArea (which probably stands for CursorRegisterWorkArea) function is called to register the work area in local memory manager. Immediately after that the qksmmAdjustStatistics function is called to adjust the statistics in the appropriate SGA memory structures (like V$SQL_PLAN_STATISTICS which is based on X$QESRSTATALL fixed table). After that the memory requirements for this working area are calculated with qksmmEstimMemoryRequirement function. This function returns the estimated memory requirements for all possible types of execution: one pass execution (mem1Pass), optimal execution (memCache) as well as for the multipass execution which requires minimal memory (memMin). This value also means the minimum possible memory which should be available to this statement to be successfully completed.

In the lines 036-040 the work area 41A29B20 is registered, in the lines 041-045 the work area 41A29448 and in the lines 046-050 the work area 41A294A0. 036 qesmmCRegisterWorkArea: cursor register wds=41A29B20 wdm=09FCE5A8 037 qksmmAdjustStatistics(enter): isize=64 osize=64 rlen=15 038 qksmmAdjustStatistics(exit): isize=64 osize=64 rlen=15 039 qksmmEstimMemoryRequirement(enter): wds=41A29B20, isize=64, osize=64, rlen=15 040 qksmmEstimMemoryRequirement(exit): memMin=72, mem1Pass=72, memCache=72 041 qesmmCRegisterWorkArea: cursor register wds=41A29448 wdm=09FCE648 042 qksmmAdjustStatistics(enter): isize=64 osize=64 rlen=223 043 qksmmAdjustStatistics(exit): isize=64 osize=64 rlen=223 044 qksmmEstimMemoryRequirement(enter): wds=41A29448, isize=64, osize=64, rlen=223 045 qksmmEstimMemoryRequirement(exit): memMin=448, mem1Pass=3070, memCache=3070 046 qesmmCRegisterWorkArea: cursor register wds=41A294A0 wdm=09FCE6E8 047 qksmmAdjustStatistics(enter): isize=64 osize=64 rlen=186 048 qksmmAdjustStatistics(exit): isize=64 osize=64 rlen=186


16 Paper # 411

049 qksmmEstimMemoryRequirement(enter): wds=41A294A0, isize=64, osize=64, rlen=186 050 qksmmEstimMemoryRequirement(exit): memMin=72, mem1Pass=72, memCache=72

In the following line 051 the function qesmmCComputeCursorBound computes the cursor bound. The input values to this function are PGA max size maxSize (defined with _pga_max_size hidden parameter), the amount of PGA already allocated pgaInUse and the amount of memory used in PGA for other memory structures (not for work area) - pgaOther. 051 qesmmCComputeCursorBound: pgaInUse=362 pgaOther=362 maxSize=204800 target=136292 052 qesmmCComputeCursorBound(exit): bound=136292 053 qesmmCInitialize: done initializing cursor work areas

Now the work areas are initialized. In subsequent line 054 the work area is registered in the work area table used by the global memory manager and a check for over large drifting is performed. 054 qesmmIRegisterWA: registering wd=41A29448 055 qesmmIRecordDrifting: recording 3070 drift (Tau=716427, Tres=0) 056 Dumping Global SGA descriptor (cnt=9209) 057 Tau=699M Res=0 Bnd=102400 Drift=0 Gcnt=9209 Qcnt=0 058 qesmmIRecordDrifting: exit (3070 drift, too much=0) 059 wid=0, trcId=-1 sid=137, ser#=591, flags=11 (AUTO|SGA|ADVICE) 060 qc_sid=65535, qc_instid=65535, slv_grp=1 061 curSize=0, maxSize=0, #maxPasses=0 062 tempSegStateObj=00000000, tempSegObjType=0 063 shared work area descriptor (41A29448): 064 flags:12 ( AUTO SGA ) 065 opType:9(HASH-JOIN) opId=2 proxy=00000000 066 isize=64K, osize=64K, rowlen=223, flags=02 067 opt=0, 1pass=0, mpass=0, lastMem=0KB 068 maxTempSegSize=0KB, lastTempSegSize=0KB

The local memory manager (see line 077) reports the registration of the working area in the line 069. The area is registered within a work area table that resides in SGA under wi=41DA7E04. The detailed information for this work area is dumped. This information is the same as we get in the work area table dump at level 3. 069 qesmmIRegisterWA: done registering wd=41A29448, wi=41DA7E04 070 status=ACTIVE 071 Operation statistics (41DA7E2C): 072 op=HASH-JOIN sib=1 curSize=(0,0) curPass=(0,0) 073 temp seg type=0 temp seg state object=00000000 074 cap=3070 waSize=0 075 isz=64, osz=64, rlen=223, min=448, 1pass=3070, cache=3070, flgs=02 076 shared work area descriptor (41A29448): 077 registered at the IMM iwds=41DA7E04 078 qesmmCStartWorkArea: done starting wds=41A29448, wdm=09FCE648 wi=41DA7E04

Now starts the data input and the HASH-JOIN operator checks for the possible maximum size of the work area (qesmmCGetMaxWASize). 079 qesmmCGetDataStat: wds=41A29448 isize=64 osize=0 rlen=223 080 qesmmCGetMaxWASize(enter): wds=41A29448 081 Operation statistics (41DA7E2C): 082 op=HASH-JOIN sib=1 curSize=(0,0) curPass=(0,0) 083 temp seg type=0 temp seg state object=00000000 084 cap=3070 waSize=0 085 isz=64, osz=64, rlen=223, min=448, 1pass=3070, cache=3070, flgs=02 086 qesmmCGetMaxWASize(exit): ret=102400KB 087 Operation statistics (41DA7E2C): 088 op=HASH-JOIN sib=1 curSize=(0,0) curPass=(0,0)


17 Paper # 411

089 temp seg type=0 temp seg state object=00000000 090 cap=3070 waSize=0 091 isz=64, osz=64, rlen=223, min=448, 1pass=3070, cache=3070, flgs=02

Now the HASH-JOIN operator calculates the expected size of the work area (the function reports parameters used in the function invocation and in next step the returned parameters are reported as well. 092 qesmmCGetExpectedWASize(enter): wds=41A29448 093 Operation statistics (41DA7E2C): 094 op=HASH-JOIN sib=1 curSize=(0,0) curPass=(0,0) 095 temp seg type=0 temp seg state object=00000000 096 cap=3070 waSize=0 097 isz=64, osz=64, rlen=223, min=448, 1pass=3070, cache=3070, flgs=02 098 qesmmCGetExpectedWASize(exit): bound=102400 exp_size=3070 ret=3070 099 Operation statistics (41DA7E2C): 100 op=HASH-JOIN sib=1 curSize=(0,0) curPass=(0,0) 101 temp seg type=0 temp seg state object=00000000 102 cap=3070 waSize=0 103 isz=64, osz=64, rlen=223, min=448, 1pass=3070, cache=3070, flgs=02

Finally in the line 104 the qesmmCGetMemRequirement reports the memory requirement for the operator HASH-JOIN operator. 104 qesmmCGetMemRequirement: wds=41A29448 min=448 onepass=3070 cache=3070

In line 105 the size of work area is set to the value calculated in the previous steps (slightly less than that). 105 qesmmCSetWorkAreaSize: wds=41A29448 size=2977

Then the function qesmmCUpdateCurSize (probably qesmmCursorUpdateCursorSize) function updates the size of the memory allocated to the cursor. The difference between old and new parameter is the allocated or de-allocated memory (what we will see later) for this work area. The work area is described here with the shared work area descriptor (wds) 106 qesmmCUpdateCurSize: wds=41A29448, wdm=09FCE648 (old=0, new=201) 107 qesmmCUpdateCurSize: wds=41A29448, wdm=09FCE648 (old=201, new=201) 108 qesmmCUpdateCurSize: wds=41A29448, wdm=09FCE648 (old=201, new=321) 109 qesmmCSetDataStat: wds=41A29448 isize=0 osize=0 rlen=145

In lines 110 and 111 the qesmmCEndOfInput reports the end of input for HASH-JOIN operator and then the memory manager updates the cursor size with the real value of the allocated memory to this work area (qesmmCUpdateCurSize). 110 qesmmCEndOfInput: wds=41A29448 111 qesmmCUpdateCurSize: wds=41A29448, wdm=09FCE648 (old=321, new=321)

The actual memory requirement is subsequently used for recalculation of memory requirements in the local memory manager and for the correction of the expected work area size. All these values are externalized in V$SQL_WORKAREA and V$SQL_PLAN_STATISTICS_ALL fixed view and are used for the subsequent executions when the same execution plan is reused with a soft pars or just by re-executing the SQL statement. 112 qesmmCChgWARequirement(enter): wds=41A29448 min=1764 onepass=1764 cache=1764 113 Operation statistics (41DA7E2C): 114 isz=1, osz=64, rlen=145, min=448, 1pass=3070, cache=3070, flgs=02 115 qesmmCChangeWorkAreaRequirement(exit) 116 Operation statistics (41DA7E2C): 117 isz=1, osz=64, rlen=145, min=1764, 1pass=1764, cache=1764, flgs=02 118 qesmmCGetExpectedWASize(enter): wds=41A29448


18 Paper # 411

119 Operation statistics (41DA7E2C): 120 op=HASH-JOIN sib=1 curSize=(321,321) curPass=(0,0) 121 temp seg type=0 temp seg state object=00000000 122 cap=3070 waSize=2977 123 isz=1, osz=64, rlen=145, min=1764, 1pass=1764, cache=1764, flgs=02 124 qesmmCGetExpectedWASize(exit): bound=102400 exp_size=1764 ret=1764 125 Operation statistics (41DA7E2C): 126 op=HASH-JOIN sib=1 curSize=(321,321) curPass=(0,0) 127 temp seg type=0 temp seg state object=00000000 128 cap=3070 waSize=2977 129 isz=1, osz=64, rlen=145, min=1764, 1pass=1764, cache=1764, flgs=02 130 qesmmCUpdateCurSize: wds=41A29448, wdm=09FCE648 (old=321, new=321) 131 qesmmCUpdateCurSize: wds=41A29448, wdm=09FCE648 (old=321, new=321) 132 qesmmCChgWARequirement(enter): wds=41A29448 min=1638 onepass=1638 cache=1638 133 Operation statistics (41DA7E2C): 134 isz=1, osz=64, rlen=145, min=1764, 1pass=1764, cache=1764, flgs=02 135 qesmmCChangeWorkAreaRequirement(exit) 136 Operation statistics (41DA7E2C): 137 isz=1, osz=64, rlen=145, min=1638, 1pass=1638, cache=1638, flgs=02 138 qesmmCSetWorkAreaSize: wds=41A29448 size=1639 139 qesmmCUpdateCurSize: wds=41A29448, wdm=09FCE648 (old=321, new=321)

The first work area with wds=41A29B20 used for GROUP BY (SORT) operator is now closed in lines 140 to 151 with qesmmCCloseInternal (CursorCloseInternal) function. Because it was opened with NOSTATS flag it was not registered in work area table (v$sql_workarea_active) and therefore it is not used in the PGA_ADVICE calculation. 140 qesmmCCloseInternal: closing wds=41A29B20, wdm=09FCE5A8 final=0 141 pga_advice: CMM off (adv=1, auto=1, min=0) => skipped 142 qksmmAdjustStatistics(enter): isize=1 osize=1 rlen=15 143 qksmmAdjustStatistics(exit): isize=1 osize=1 rlen=15 144 status=CLOSED 145 Operation statistics (09FCE5B8): 146 op=GROUP BY (SORT) sib=1 curSize=(0,0) curPass=(0,0) 147 temp seg type=0 temp seg state object=00000000 148 cap=72 waSize=0 149 isz=1, osz=1, rlen=15, min=72, 1pass=72, cache=72, flgs=02 150 shared work area descriptor (41A29B20): 151 qesmmCCloseInternal: done closing wds=41A29B20, wdm=09FCE5A8

The previously allocated memory for the work area used for the HASH-JOIN operator is now de-allocated and returned to the operating system. We can observe the change in the memory usage (de-allocation of memory) by monitoring the updated cursor size (qesmmCUpdateCurSize) which has changed the new value to 0. 152 qesmmCCloseInternal: closing wds=41A29448, wdm=09FCE648 final=1 153 qesmmCUpdateCurSize: wds=41A29448, wdm=09FCE648 (old=321, new=0) 154 wid=0, trcId=-1 sid=137, ser#=591, flags=11 (AUTO|SGA|ADVICE) 155 qc_sid=65535, qc_instid=65535, slv_grp=1 156 curSize=0, maxSize=321, #maxPasses=0 157 tempSegStateObj=00000000, tempSegObjType=0 158 shared work area descriptor (41A29448): 159 flags:12 ( AUTO SGA ) 160 opType:9(HASH-JOIN) opId=2 proxy=00000000 161 isize=64K, osize=64K, rowlen=223, flags=02 162 opt=0, 1pass=0, mpass=0, lastMem=0KB 163 maxTempSegSize=0KB, lastTempSegSize=0KB


19 Paper # 411

In lines 164 to 167 the local memory manager un-registers the work area from the workarea table (qesmmIUnRegisterWA). 164 qesmmIUnRegisterWA: unregister wi=41DA7E04, wd=41A29448 165 pga_advice: IMM off (adv=1, auto=1, min=0) => skipped 166 Freeing element 41DA7E04 from entry 42A64F98, freelist 44B7E8A8 167 qesmmIUnRegisterWA: done unregistering wi=00000000 168 qksmmAdjustStatistics(enter): isize=1 osize=64 rlen=145 169 qksmmAdjustStatistics(exit): isize=1 osize=64 rlen=145

In lines 170 through 171 a negative drift (the memory was de-allocated) is registered. 170 qesmmCRegisterDrift: recording -3070 drift 171 qesmmCRegisterDrift: exit (3070 drift, too much=0)

Finally the work area used for HASH-JOIN operator is unregistered and the cursor is closed within the local memory manager. The kernel now reports the FETCH line for cursor #3 which retrieves one row and denotes that the fetch phase is finished. The last work area will be closed only when all rows will be fetched from that area. 172 qesmmCUnRegisterWorkArea: cursor unregister ctM=09FCE584 wds=41A29448 wdm=09FCE648 173 status=TERMINATED 174 Operation statistics (09FCE658): 175 op=HASH-JOIN sib=1 curSize=(0,321) curPass=(0,0) 176 temp seg type=0 temp seg state object=00000000 177 cap=3070 waSize=1639 178 isz=1, osz=64, rlen=145, min=1638, 1pass=1638, cache=1638, flgs=02 179 shared work area descriptor (41A29448): 180 qesmmCCloseInternal: done closing wds=41A29448, wdm=09FCE648 181 FETCH #3:c=0,e=306004,p=0,cr=8,cu=0,mis=0,r=1,dep=2,og=4,tim=902551701163

In the subsequent lines the event 10046 trace now reports the parsing, execution and fetch phase for another cursor #4. Again a CursorMemoryManagement context is opened ant 2 rows are fetched in three subsequent FETCH calls. 182 PARSING IN CURSOR #4 len=67 dep=2 uid=0 oct=3 lid=0 tim=902551709397 hv=2280069326 ad='44687c30' 183 select pos#,intcol#,col#,spare1,bo#,spare2 from icol$ where obj#=:1 184 END OF STMT 185 PARSE #4:c=0,e=781,p=0,cr=0,cu=0,mis=1,r=0,dep=2,og=4,tim=902551709389 186 qksmmAllocCContext: alloc CMM context (dyn=0) 187 qksmmAllocCContext: alloc context (shr=40F20720/mut=00000000) 188 EXEC #4:c=0,e=5572,p=0,cr=0,cu=0,mis=1,r=0,dep=2,og=4,tim=902551715246 189 FETCH #4:c=0,e=88,p=0,cr=3,cu=0,mis=0,r=1,dep=2,og=4,tim=902551715457 190 FETCH #4:c=0,e=25,p=0,cr=2,cu=0,mis=0,r=1,dep=2,og=4,tim=902551715540 191 FETCH #4:c=0,e=12,p=0,cr=1,cu=0,mis=0,r=0,dep=2,og=4,tim=902551715599

Now the local memory manager closes the last work area used for the final sort operation. Closing is possible because the subsequent FETCH call for cursor #3 returns no more rows (r=0). The statistics for the SORT operator is updated as well, and the function qesmmISqlWaHistogramInc probably increments SQL work area histogram. 192 qesmmCCloseInternal: closing wds=41A294A0, wdm=09FCE6E8 final=0 193 pga_advice: not registered => transient 194 qesmmISqlWaHistogramInc(): wid=0 wg=2 extra=0 opt=11486 opass=0 mpass=0 195 qksmmAdjustStatistics(enter): isize=2 osize=2 rlen=186 196 qksmmAdjustStatistics(exit): isize=2 osize=2 rlen=186 197 status=CLOSED 198 Operation statistics (09FCE6F8): 199 op=SORT (v2) sib=1 curSize=(0,2) curPass=(0,0)


20 Paper # 411

200 temp seg type=0 temp seg state object=00000000 201 cap=72 waSize=0 202 isz=2, osz=2, rlen=186, min=72, 1pass=72, cache=72, flgs=02 203 shared work area descriptor (41A294A0): 204 qesmmCCloseInternal: done closing wds=41A294A0, wdm=09FCE6E8 205 FETCH #3:c=0,e=307,p=0,cr=0,cu=0,mis=0,r=0,dep=2,og=4,tim=902551716001

Now the final housekeeping work follows from line 206 through 220. In line 206 qesmmCCleanup function is called to clean up cursor #3 work areas. Again the size of the allotted memory for cursor #3 is updated. Because the working area 41A294A0 was created with NOSTATS flag set (see line 026) this work area was transient and as such not registered within the work area table. However, the function qesmmISqlWaHistogramInc (probably globalSqlWorkareaHistogramIncrement) is executed to increment the statistics used for the PGA advice view (V$PGA_TARGET_ADVICE). 206 qesmmCCleanup: cleaning up cursor work areas 207 qesmmCCloseInternal: closing wds=41A294A0, wdm=09FCE6E8 final=1 208 qesmmCUpdateCurSize: wds=41A294A0, wdm=09FCE6E8 (old=0, new=0) 209 pga_advice: not registered => transient 210 qesmmISqlWaHistogramInc(): wid=0 wg=2 extra=0 opt=11487 opass=0 mpass=0 211 qesmmCUnRegisterWorkArea: cursor unregister ctM=09FCE584 wds=41A294A0 wdm=09FCE6E8 212 status=TERMINATED 213 Operation statistics (09FCE6F8): 214 op=SORT (v2) sib=1 curSize=(0,2) curPass=(0,0) 215 temp seg type=0 temp seg state object=00000000 216 cap=72 waSize=0 217 isz=2, osz=2, rlen=186, min=72, 1pass=72, cache=72, flgs=02 218 shared work area descriptor (41A294A0): 219 qesmmCCloseInternal: done closing wds=41A294A0, wdm=09FCE6E8 ....

All these shared work areas can be used later on for the execution of the same SQL statement.

In the above trace we saw also several values which were not explained. Their meaning is not as obvious as for the majority of them. However, certain facts where observed which put them in some kind of relationship. ctM=09FCE584 – probably cursor memory management context wds=41A29B20 wds=41A29448 wds=41A294A0 wdm=09FCE5A8 wdm=09FCE648 wdm=09FCE6E8 size=160 bytes wi=41DA7E04 os=09FCE5B8 os=09FCE658 os=09FCE6F8

The wds stands for shared workarea descriptor, while os stands for operating statistics.

All these pointers point into an area in SGA which is not externalized through X$ fixed tables.

DYNAMIC WORKING AREAS

If we change the default system sort algorithm back to the old algorithm (Oracle10gR2 marks it with SORT (v1)) then we can observe dynamically created working areas which are created by a different function qksmmMakeDynamicWDef and are immediately closed after creation. They are created with AUTO DYN NOSTATS flags, where DYN stands for dynamic.

SQL> alter session set "_newsort_enabled"=FALSE;

... qksmmMakeDynamicWDef: creating wds=0B2DE574 isProxy=0 proxy=00000000 ctx=1D2BC3CC


21 Paper # 411

flags:4a ( AUTO DYN NOSTATS ) mut=0B2DF700 heap=0B16AAA8 opType:7(SORT (v1)) opId=65535 proxy=00000000 isize=64K, osize=64K, rowlen=10, flags=02 opt=0, 1pass=0, mpass=0, lastMem=0KB maxTempSegSize=0KB, lastTempSegSize=0KB qksmmMakeRunTimeWDef: done creating wds=0B2DE574 qesmmCCloseInternal: closing wds=0B2DE574, wdm=0B2DF700 final=1

...

MANUALLY MANAGED WORKING AREAS

When we set WORKAREA_SIZE_POLICY to MANUAL at session level then the allocated working areas have different flags. The manual work areas are also registered in V$SQL_WORKAREA_ACTIVE (X$QESMMIWT).

SQL> alter session set WORKAREA_SIZE_POLICY = MANUAL;

... qesmmIRecordDrifting: exit (0 drift, too much=0) wid=0, trcId=-1 sid=145, ser#=33, flags=2 (MANUAL|SGA) qc_sid=65535, qc_instid=65535, slv_grp=1 curSize=0, maxSize=0, #maxPasses=0 tempSegStateObj=00000000, tempSegObjType=0 shared work area descriptor (1D6274DC): flags:50 ( SGA NOSTATS ) opType:8(SORT (v2)) opId=1 proxy=00000000 isize=22460K, osize=22460K, rowlen=23, flags=02 opt=0, 1pass=0, mpass=0, lastMem=0KB maxTempSegSize=0KB, lastTempSegSize=0KB qesmmIRegisterWA: done registering wd=1D6274DC, wi=1E6C6EBC status=ACTIVE Operation statistics (1E6C6EE4): op=SORT (v2) sib=1 curSize=(0,0) curPass=(0,0) temp seg type=0 temp seg state object=00000000 cap=0 waSize=0 isz=22460, osz=22460, rlen=23, min=72, 1pass=72, cache=72, flgs=02 shared work area descriptor (1D6274DC): registered at the IMM iwds=1E6C6EBC qesmmCStartWorkArea: done starting wds=1D6274DC, wdm=0A8797EC wi=1E6C6EBC qesmmCUpdateCurSize: wds=1D6274DC, wdm=0A8797EC (old=0, new=2) qesmmCUpdateCurSize: wds=1D6274DC, wdm=0A8797EC (old=2, new=4) ... qesmmCUpdateCurSize: wds=1D6274DC, wdm=0A8797EC (old=46, new=48) qesmmCRegisterTempSeg: wds=1D6274DC, wdm=0A8797EC (0A8885F8) – sort spills to disk qesmmCUpdateCurSize: wds=1D6274DC, wdm=0A8797EC (old=48, new=1) qesmmCUpdateCurSize: wds=1D6274DC, wdm=0A8797EC (old=1, new=9) ...

When using MANUAL mode the old *_AREA_SIZE parameters are used to determine the size of different working areas. In this case the PGA_AGGREGATE_TARGET is greater than 0 but is not used for this session.

GLOBAL BOUND COMPUTATION

The CKPT process checks every 3 seconds for required global memory bound background re-computation. Here is an excerpt from a CKPT process trace file. To be able to trace background processes, what the CKPT process is, I had to start the whole instance in a trace mode (_smm_trace=65535)

...


22 Paper # 411

qesmmIRefreshBound: enter Dumping Global SGA descriptor (cnt=48) PgIu=15M PgAl=(30M, 30M) PgFr=(0,0M,0M) PgAu=(0M, 0M) PgMa=(1M, 0M) Cache=(0M, 6M) OnePass=(0M, 6M) Tau=166M Res=546 Bnd=40960 Drift=0 Gcnt=48 Qcnt=0 qesmmIRefreshBound(): advice enable (free=10230 max=269, pos=85) Making snapshot of work area table (maxElem=110) new: op=0778B11C ind=0 nbWa=1 qc=65535 #sib=1 End of snapshot (NbElem=1 NbWa=1 totAutoCache=546) Dumping Global SGA descriptor (cnt=48) PgIu=15M PgAl=(30M, 30M) PgFr=(0,0M,0M) PgAu=(0M, 0M) PgMa=(1M, 0M) Cache=(0M, 6M) OnePass=(0M, 6M) Tau=166M Res=546 Bnd=40960 Drift=0 Gcnt=48 Qcnt=0 qesmmIRefreshBound: exit, don't need to recompute bound qesmmaSimulateLog(enter): maxTime=59352 peek recType=max ==> done (log empty) qesmmaSimulateLog(exit): maxTime=59352 timeout=0

...

FOREGROUND BOUND RECOMPUTE

When the memory drift between the sum of expected sizes and the sum of the actual sizes of all active working areas is too big, then a foreground re-compute is required. The foreground global bound re-compute is more an exception then a rule and I had to use relatively low values for P_A_T to enforce a foreground re-compute during my experiments. According to the [1] the foreground bound re-compute occurs when the PGA memory consumption on the system is close to the maximum limit set by the PGA_AGGREGATE_TARGET parameter and the memory drift crosses 10% tolerance threshold around the SQL memory target.

Below is the excerpt from the trace file: ... qesmmIRefreshBound: need to recompute bound qesmmComputeBound: start (nbOp=5, nbWa=5, target=4096 useCap=1 minSz=128, serCap=2048, parCap=5120) Input TabOp: bound info (074E90FC): op=GROUP BY (SORT), min=184, 1pass=184, cache=184, capSize=184, sib=1 bound info (074E91B4): op=GROUP BY (SORT), min=232, 1pass=560, cache=560, capSize=1984, sib=1 bound info (074E926C): op=HASH-JOIN, min=2998, 1pass=2998, cache=2998, capSize=2999, sib=1 bound info (074E9324): op=HASH-JOIN, min=930, 1pass=930, cache=930, capSize=3970, sib=1 bound info (074E93DC): op=HASH-JOIN, min=448, 1pass=5178, cache=7766, capSize=7766, sib=1 Start (it=1): nbWa=5 nbLft=5 bnd=819 memTg=4096 memLeft=4096 start loop: bi[0]=074E90FC mem=184 nextIncr=0 remove: bi[0]=074E90FC totMem=184 totIncr=0 memLeft=4096 memTarget:4096 -> 3912 swap bi[0]=074E90FC with bi[4]=074E93DC lft=4 nbWa=4) end loop 0: lft=4 nbWa=4 start loop: bi[0]=074E93DC mem=819 nextIncr=64 keep bi[0]=074E93DC, nextMemLeft=0, pos=0 end loop 1: lft=4 nbWa=4 start loop: bi[1]=074E91B4 mem=560 nextIncr=0 remove: bi[1]=074E91B4 totMem=560 totIncr=0 memLeft=4096 memTarget:3912 -> 3352 swap bi[1]=074E91B4 with bi[3]=074E9324 lft=3 nbWa=3) end loop 1: lft=3 nbWa=3 start loop: bi[1]=074E9324 mem=819 nextIncr=64 keep bi[1]=074E9324, nextMemLeft=0, pos=1 end loop 2: lft=3 nbWa=3 start loop: bi[2]=074E926C mem=819 nextIncr=64 keep bi[2]=074E926C, nextMemLeft=0, pos=2


23 Paper # 411

end loop 3: lft=3 nbWa=3 End (it=1): memTg=3352 res=744 memLeft=0 Start (it=2): nbWa=3 nbLft=3 bnd=1117 memTg=3352 memLeft=0 start loop: bi[0]=074E93DC mem=1117 nextIncr=64 bi[0]=074E93DC lrgest so far keep bi[0]=074E93DC, nextMemLeft=0, pos=0 end loop 1: lft=3 nbWa=3 start loop: bi[1]=074E9324 mem=930 nextIncr=0 remove: bi[1]=074E9324 totMem=930 totIncr=0 memLeft=0 memTarget:3352 -> 2422 swap bi[1]=074E9324 with bi[2]=074E926C lft=2 nbWa=2) end loop 1: lft=2 nbWa=2 start loop: bi[1]=074E926C mem=1117 nextIncr=64 keep bi[1]=074E926C, nextMemLeft=0, pos=1 end loop 2: lft=2 nbWa=2 End (it=2): memTg=2422 res=1674 memLeft=0 Start (it=3): nbWa=2 nbLft=2 bnd=1211 memTg=2422 memLeft=0 start loop: bi[0]=074E93DC mem=1211 nextIncr=64 bi[0]=074E93DC lrgest so far keep bi[0]=074E93DC, nextMemLeft=0, pos=0 end loop 1: lft=2 nbWa=2 start loop: bi[1]=074E926C mem=1211 nextIncr=64 keep bi[1]=074E926C, nextMemLeft=0, pos=1 end loop 2: lft=2 nbWa=2 All use max. We are done End (it=3): memTg=0 res=4096 memLeft=0 qesmmComputeBound: exit-3: reserved=4096 bound=1211 qesmmIRefreshBound: bound recomputed ...

HIDDEN QESMM PARAMETERS

There are several hidden initialization parameters that govern the behaviour of automatic SQL memory management (QESMM). Some of them I changed very intensively during my tests, but there are plenty of them I haven’t tested as I ran out of time and had to finish this presentation. Obviously there is a huge research work left for the future. Therefore I only comment some of the parameters whereas some of them are explained in more details either here or elsewhere in the paper.

_SMM_ADVICE_ENABLED

This parameter is used to enable/disable the calculation for v$pga_advice view. The flags don’t contain ADVICE flag and in the QESMM trace we can see lines like :

pga_advice: IMM off (adv=0, auto=1, min=0) => skipped

_SMM_ADVICE_LOG_SIZE

This parameter is used to overwrite the default size of the PGA advice workarea history log. I haven’t tested it.

_SMM_AUTO_COST_ENABLED

If set to TRUE then use the AUTO size policy cost functions. The parameter was not tested during my research.

_SMM_AUTO_MAX_IO_SIZE & _SMM_AUTO_MIN_IO_SIZE

Maximum /minimum IO size (in KB) used by sort/hash-join in auto mode. I haven’t experimented with these two parameters either.

_SMM_BOUND


24 Paper # 411

This parameter one can use to overwrite the memory manager automatically computed bound. I performed very limited testing with this parameter. When once set one can’t easily reset its calculation back to automatic computation. The only way I found is to reset it is by changing the P_A_T parameter.

_SMM_CONTROL

Provides controls on the memory manager. For values >= 32 the query returns ORA-942 error because one TKMM schema table is missing. Most likely TKMM stands for Trace Kernel Memory Manager and there are some functions available for advance tracing and for testing different scenarios. My testing of this parameter was not intensive and this was merely the only difference I noticed.

_SMM_FREEABLE_

The value in KB of the instance freeable PGA memory to retain. I haven’t done any tests with this parameter.

_SMM_ISORT_CAP

This is a new hidden parameter in Oracle10gR2 which defines the size of work area for old (v1) insertion sort. The default value is 100MB. The new sort is marked as (v2) sort.

Consider the following case: SQL> alter session set "_newsort_enabled"=FALSE; SQL> set pause on -- to prevent de-allocation of work area SQL> select c1 from t_big2 t order by c1;

WORKAREA_ADDRESS OPERATION_TYPE OPERATION_ID ACTUAL_MEM_USED MAX_MEM_USED TEMPSEG_SIZE ---------------- -------------- ------------ --------------- ------------ ------------ 1E119960 SORT (v1) 1 102318080 102318080 19922944

If we change this parameter to 262,060KB then we get the following result: SQL> alter system set "_smm_isort_cap"= 262060; System altered. SQL> @smm PARAMETER VALUE ------------------------------ ------------------ pga_aggregate_target 3072000 _pga_max_size 524120 _smm_isort_cap 262060 _smm_max_size 262060 _smm_min_size 1024 _smm_px_max_size 1536000 SQL> select c1 from t_big2 t order by c1; And from the another session we run the following query: SQL> SELECT workarea_address, operation_type, operation_id, actual_mem_used, max_mem_used, tempseg_size FROM v$sql_workarea_active ORDER BY max_mem_used DESC; WORKAREA_ADDRESS OPERATION_TYPE OPERATION_ID ACTUAL_MEM_USED MAX_MEM_USED TEMPSEG_SIZE ---------------- -------------- ------------ --------------- ------------ ------------


25 Paper # 411

1E119960 SORT (v1) 1 265445376 265445376 9437184

_SMM_MAX_SIZE

This parameter defines the maximum work area size in auto mode (serial) and defaults to 50% of _pga_max_size. In Oracle 9i/10gR1 the maximum value is 0.1GB (100MB), while in 10gR2 it can go up to 0.25GB as we will see later. After one of the instance restarts I found that the _pga_max_size and _smm_max_size had even bigger values with P_A_T set to 4GB.

_SMM_MIN_SIZE

This parameter defines the minimum work area size in auto mode. I have never changed it during my experiments.

_SMM_PX_MAX_SIZE

This parameter defines the maximum work area size in auto mode (global). In Oracle9i/10gR1 this parameter is used to constraint the size of work area for parallel slaves for DOP > 6. In those versions the default value for this parameter is 30% of P_A_T.

According to my tests in Oracle10GR2 the parameter defaults now to 50% of P_A_T and it is used to constraint the size of work area for parallel slaves for DOP > 5. For more details see the graph later in this paper.

_SMM_RETAIN_SIZE

This parameter defines the work area retain size in SGA for shared server sessions (0 for AUTO). During my experiments I haven’t changed this parameter as I was using a DEDICATED mode almost all of the time.

_SMM_TRACE

This parameter is used to turn on/off tracing for SQL memory manager. I found that different values produce different details. The minimum value to produce a trace is 8 (at least in my tests that was the case). During the experiments I found that levels 8 - 15 produce a very limited trace, while levels over 32 to 63 and 112-127 produce even more detailed trace. I used level 65535 to produce traces in this paper.

_PGA_MAX_SIZE

The parameter defines the maximum size of the PGA memory for one process.

_USE_ISM_FOR_PGA

This parameter governs whether ISM (Intimate Shared Memory) can be used for allocation of large extents. I haven’t tested this parameter, but Tanel Poder mentions it in his presentation about memory management [11].

_PGA_LARGE_EXTENT_SIZE & _UGA_CGA_LARGE_EXTENT_SIZE

These parameters are used to set the size of the PGA large extent size and UGA/CGA large extent size respectively for initial mmap() allocation function [11].

_REALFREE_HEAP_MAX_SIZE & _REALFREE_HEAP_MODE & _REALFREE_HEAP_PAGESIZE_HINT


26 Paper # 411

By setting these parameters one can influence the realfree heap size, the realfree_heap_mode and can hint the realfree page size. The last one defines the amount of memory which is allocated at a time for PGA growth. I have performed no tests with these parameters so I mention them here just for the reader’s information.

Output of strace command of a server process on Linux shows the following: … mmap2(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_NORESERVE, 7, 0xb4) = 0x40c2f000 mmap2(0x40c2f000, 65536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 7, 0) = 0x40c2f000 mmap2(0x40c3f000, 65536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 7, 0) = 0x40c3f000 …

Oracle uses here a trick with MAP_NORESERVE first allocating a virtual address space with no swap space reservation. In the subsequent calls memory is mapped in smaller sizes defined by _realfree_heap_pagesize_hint. The memory is mapped with MAP_FIXED flag which gives the opportunity to map the virtually memory to a specific process address space.

_USE_REALFREE_HEAP

The memory for working areas can be allocated as heap 0 when the parameter is set to TRUE. This means that the memory is allocated as a separate heap and not as a sub-heap of the PGA.

INFLUENCE OF P_A_T SIZING ON HIDDEN QESMM PARAMETERS

Changing the value of P_A_T parameter also changes the value of the related QESMM hidden parameters whose names start with _SMM* and the hidden parameter _PGA_MAX_SIZE as well.

According to the Metalink Note: 147806.1 [6] PGA_AGGREGATE_TARGET limits both the global PGA consumption and the size of a work area i.e. the memory allocated to a single SQL operator is limited to:

• for serial operations: min(5% PGA_AGGREGATE_TARGET, 100MB) (_smm_max_size hidden parameter)

• for parallel operations 30% PGA_AGGREGATE_TARGET / DOP for parallel operations where DOP stands for degree of parallelism (_smm_px_max_size) parameter

This is true for Oracle91 and Oracle10gR1. In the latest release 10.2 Oracle changed these default values. The memory allocated to a single SQL operator (_smm_max_size) is limited to:

• for P_A_T <= 500MB the parameter _smm_max_size = 20% of P_A_T

• for P_A_T between 500MB and 1000MB the parameter _smm_max_size = 100M

• for P_A_T betweeen 1001MB and 2560MB (2.5GB) the parameter _smm_max_size = 10% of P_A_T

• for P_A_T > 2560MB (2,5GB) the parameter _smm_max_size = 262,060 MB (~0,25GB)

• I have seen even cases when these values were even bigger after the instance was restarted with P_A_T set at 4GB.

The maximum value for parallel operations changed from 30% to 50% PGA_AGGREGATE_TARGET/DOP. Also the DOP changed. When DOP <=5 then _smm_max_size is used, otherwise _smm_px_max_size/DOP limits the maximum memory usage.

As a result of changed maximum values for single SQL operator the maximum value for single process PGA (_pga_max_size) changed as well. In Oracle 9i/10gR1 the default limit was 200M. In Oracle 10.2 the PGA limit is still 2*_smm_max_size. As the result the maximum default value for the PGA sizes is ~0.5GB.


27 Paper # 411

The graph on Figure 6: P_A_T and Related QESMM sizes shows the differences in the sizing of work areas in Oracle9i, Oracle10gR1 and Oracle10gR2.

P_A_T and Related SMM Sizes

0

200

400

600

800

1000

1200

1400

10

90

17

0

25

0

33

0

41

0

49

0

57

0

65

0

73

0

81

0

89

0

97

0

105

0

113

0

121

0

129

0

137

0

145

0

153

0

161

0

169

0

177

0

185

0

193

0

201

0

209

0

217

0

225

0

233

0

241

0

249

0

P_A_T size in MB

SM

M S

ize

in

MB

0,000

0,200

0,400

0,600

0,800

1,000

1,200

1,400

_S

MM

_M

IN_

SIZ

E in

MB

_PGA_MAX_SIZE_9iR2/10gR1 _SMM_MAX_SIZE_9iR2/10gR1 _SMM_PX_MAX_SIZE_9iR2/10gR1

_PGA_MAX_SIZE_10gR2 _SMM_PX_MAX_SIZE_10gR2 _SMM_MAX_SIZE_10gR2

_SMM_MIN_SIZE

Figure 6: P_A_T and Related QESMM sizes

From this graph we can see that the memory available for the single SQL operator (see the value for _smm_max_size_10gR2) is allocated much more aggressively when P_A_T is defined below 500M. For P_A_T values bigger than 1000M the same parameter is again allocated more aggressively and the size of PGA grows accordingly (_smm_max_size*2). Obviously Oracle recognized that the default values used in previous versions were somehow too low and although there was enough memory available on the system the SQL operators had to spill on disk. All these values were calculated automatically only by changing the P_A_T parameter.

While I was preparing this graph I found that sometimes the calculated values can even be different from those mentioned above.

SQL> alter system set pga_aggregate_target=2560M; System altered. SQL> @smm PARAMETER VALUE ------------------------------ -------------------- pga_aggregate_target 2621440 _pga_max_size 524120 _smm_max_size 262060 _smm_min_size 1024


28 Paper # 411

_smm_px_max_size 1310720

SQL> alter system set pga_aggregate_target=2500M; System altered. SQL> @smm PARAMETER VALUE ------------------------------ -------------------- pga_aggregate_target 2560000 _pga_max_size 360332 _smm_max_size 180166 _smm_min_size 1024 _smm_px_max_size 1280000

As we can see the difference in P_A_T is only 60M while the PGA max size and SMM_MAX_SIZE are significantly lower in the second case. I haven’t found any good explanation for such behaviour. Maybe the readers will observe a similar behaviour on their systems.

This new more aggressive working area size policy enables execution of SQL operators that require a lot of working memory. Without setting any hidden parameters I could perform a big sort with DOP=5 (5 siblings) on a table with 10M rows with only one column varchar2(6): SQL> SELECT workarea_address, operation_type, operation_id, actual_mem_used, max_mem_used, tempseg_size FROM v$sql_workarea_active ORDER BY max_mem_used DESC; WORKAREA_ADDRESS OPERATION_TYPE OPERATION_ID ACTUAL_MEM_USED MAX_MEM_USED TEMPSEG_SIZE ---------------- -------------- ------------ --------------- ------------ ------------ 1E37734C SORT (v2) 3 216162304 216162304 1E37734C SORT (v2) 3 100427776 100427776 1E37734C SORT (v2) 3 78235648 78235648 1E37734C SORT (v2) 3 78106624 78106624 1E37734C SORT (v2) 3 53011456 53011456

The memory usage in the above report is reported in bytes.

DYNAMIC PERFORMANCE VIEWS FOR MONITORING MEMORY USAGE

The following performance views can be used to monitor the memory consumption for different purposes in PGA. Some of them we will briefly discuss.

• V$PGASTAT

• V$PROCESS

• V$PROCESS_MEMORY

• V$SQL_WORKAREA_HISTOGRAM

• V$SQL_WORKAREA_ACTIVE

• V$SQL_WORKAREA

• V$MYSTAT

• V$SESSTAT


29 Paper # 411

• V$PX_SESSTAT

• V$PGA_TARGET_ADVICE

V$PGASTAT

The V$PGASTAT view is based on X$QESMMSGA fixed view and reports different parameters and statistics for QESMM. The X$QESMMSGA fixed view reports 37 values while the V$PGASTAT reports only 19. Therfore one can query directly X$QESMMSGA fixed view to get all possible information. The most important reported values are marked with red colour:

SQL> select QESMMSGANM name, QESMMSGAVL value, QESMMSGAMU multiplier 2 from x$qesmmsga; NAME VALUE MULTIPLIER ---------------------------------------------------------------- ------------- ---------- aggregate PGA target parameter 3145728 1024 aggregate PGA auto target 2815272 1024 global memory bound 262060 1024 total expected memory 546 1024 drifting from expected memory 0 1024 PGA max size parameter 524120 1024 total PGA inuse 17794 1024 total PGA allocated 442665 1024 maximum PGA allocated 471922 1024 total freeable PGA memory 0 1024 workarea SGA retain size 40 1024 process count 36 1 max processes count 43 1 PGA memory freed back to OS 0 1024 percentage freeable memory to be released 0 100 total PGA used for auto workareas 102 1024 maximum PGA used for auto workareas 408091 1024 total PGA used for manual workareas 0 1024 maximum PGA used for manual workareas 0 1024 over allocation count 0 1 bytes processed 25112377 1024 extra bytes read/written 0 1024 cache hit percentage 10000 1 recompute count (queries) 0 1 recompute count (total) 1745 1 last recompute time 90 1 maximum recompute time 15004 1 cumulated IMM deamon time 212158 1 maximum IMM deamon time 186222 1 cumulated v$pga_advice simulation time 355011 1 workarea simulated count 1648 1 interrupt v$pga_advice simulation count 2 1 skip v$pga_advice simulation count 0 1 BUG count in v$pga_advice 0 1 PGA current limit 1048240 1024 PGA limit known 1 1 PGA limit errors 0 1

37 rows selected.

The meaning of some of them is described in the Oracle10gR2 Performance tuning guide, chapter 7. The “hidden values” display the time used by IMM (most likely the global memory manager) as well as the last and maximum global memory


30 Paper # 411

bound re-compute time. The values in this fixed table constantly change and reflect their current values. The most important values are marked with red colour:

• aggregate PGA target parameter – the value of PGA_AGGREGATE_TARGET parameter

• aggregate PGA auto target – SQL memory target – P_A_T minus used memory for other categories in PGAs

• global memory bound – the current value for the global memory bound

• PGA max size parameter – the value of _pga_max_size hidden parameter

The Metalink notes [5],[6],[7] discuss how one can tune the PGA_AGGREGATE_TARGET parameter. The Enterprise Manager has a graphical tool to display the values from V$PGA_TARGET_ADVICE view and set P_A_T parameter to an appropriate value.

Figure 7: OEM and PGA_TARGET_ADVICE

V$PROCESS

V$PROCESS performance view contains different parameters about memory used for PGA. According to Metalink the reported memory usage can sometimes differ from values in v$sesstat due to a bug. SQL> select PID,SPID,PGA_USED_MEM,PGA_ALLOC_MEM,PGA_FREEABLE_MEM,PGA_MAX_MEM 2 from v$process where PGA_MAX_MEM > 20*1024*1024; PID SPID PGA_USED_MEM PGA_ALLOC_MEM PGA_FREEABLE_MEM PGA_MAX_MEM ---------- ------------ ------------ ------------- ---------------- ----------- 24 3688 37029761 37193545 0 37193545 25 724 56279705 56461129 0 56461129 34 2296 17446089 17925961 0 27625289 35 2368 9616309 10061641 0 27821897


31 Paper # 411

36 3144 9400193 9799497 0 27625289 37 1724 1499537 2590537 0 27887433 38 3636 27084533 27363145 0 27625289 7 rows selected.

V$PROCESS_MEMORY

There is a new performance view V$PROCESS_MEMORY in Oracle10gR2 that reports the memory usage for the following categories: SQL, PL/SQL, OLAP, JAVA, Freeable and Other. We can use this view to exactly determine the amount of memory used for each category by process or globally. The view reports allocated memory in bytes. SQL> select PID,CATEGORY,ALLOCATED,USED,MAX_ALLOCATED 2 from v$process_memory 3 where nvl(used,0) > 0 4* order by MAX_ALLOCATED desc; PID CATEGORY ALLOCATED USED MAX_ALLOCATED ---------- --------------- ---------- ---------- ------------- 25 SQL 178172800 178045640 178172800 24 SQL 104383328 104280992 104383328 35 SQL 15763252 15640196 26414992 37 SQL 9045692 9010060 26383500 34 SQL 8997992 8973276 26363908 ... 15 PL/SQL 77768 16384 102436 14 PL/SQL 79684 16408 79684 11 PL/SQL 39748 17900 62048 16 PL/SQL 48512 16408 55508 17 PL/SQL 48512 16408 52664 ... SQL> select category,sum(used) used 2 from v$process_memory 3 group by category; CATEGORY USED --------------- ---------- PL/SQL 359500 SQL 263225068

CONCLUSIONS ABOUT SIZING PGA AND QESMM RELATED PARAMETERS

Obviously the default value for PGA maximum size of 200M and 50% of that value as maximum value for the single SQL operator were causing performance bottlenecks in previous releases of the Oracle database. Therefore Oracle changed the defaults in release 10.2 and made them much more aggressive.

In Oracle9i/10gR1 when one has to perform a big sort or hash-join or bitmap operation, he can change the hidden parameters like _PGA_MAX_SIZE, _SMM_PX_MAX_SIZE for parallel execution when DOP > 6 or _SMM_MAX_SIZE to be able to use the available, still unused memory on the system for large SQL operators.

It is important to recall that _PGA_MAX_SIZE parameter defines the maximum size of process’ global area (PGA). Because an SQL statement can have more than one SQL operator like sort, group-by, hash-join, etc. the size of working area for single SQL operator is limited by default to 50% of _PGA_MAX_SIZE. Changing this parameter results also in change of _SMM_MAX_SIZE, but I noticed that sometimes the default of 50% is not used. Consider the following:

SQL> @smm


32 Paper # 411

PARAMETER VALUE ------------------------------ -------- pga_aggregate_target 4194304 _pga_max_size 524120 _smm_max_size 262060 _smm_min_size 1024 _smm_px_max_size 2097152 SQL> alter system set "_pga_max_size"=2G; System altered. SQL> @smm PARAMETER VALUE ------------------------------ ---------- pga_aggregate_target 4194304 _pga_max_size 2097152 _smm_max_size 838860 _smm_min_size 1024 _smm_px_max_size 2097152

We can see that by changing _PGA_MAX_SIZE to 2G the _SMM_MAX_SIZE doesn’t get the default of 50%. Now we set a new value for P_A_T to be 8G big. SQL> alter system set pga_aggregate_target=8G; System altered. SQL> @smm PARAMETER VALUE ------------------------------ --------------- pga_aggregate_target 8388608 _pga_max_size 2097152 _smm_max_size 1048576 _smm_min_size 1024 _smm_px_max_size 4194304

The ratio _PGA_MAX_SIZE / _SMM_MAX_SIZE changes to the default ratio although the value for _PGA_MAX_SIZE remains unchanged and is still user-defined. Therefore one must be careful when playing with these parameters and has to see their values. Even more funny things happen when you produce a CBO trace with event 10053 at level 1 or with a new hidden parameter _optimizer_trace avaialble in Oracle10gR2. We can easily get the valid values for this parameter by issuing:

SQL> alter session set "_optimizer_trace"=xxx; ERROR: ORA-00096: invalid value XXX for parameter _optimizer_trace, must be from among HINT, ENVIRONMENT, PHYSICAL, LOGICAL, MEDIUM, NONE, HIGH, ALL, LOW

It is beyond the scope of this paper to discuss the contents of the trace file when certain level is used. The fact is that the CBO trace contains the current values of QESMM and other parameters that influence how CBO behaves. The parameters and especially the changed parameters are listed in a separate section. Here is the case: SQL> alter session set events '10053 trace name context forever, level 1'; Session altered. SQL> explain plan for select c1 from t_big2 t order by c1; Explained.


33 Paper # 411

In the CBO trace we can observe the following: ... *************************************** PARAMETERS USED BY THE OPTIMIZER ******************************** ************************************* PARAMETERS WITH ALTERED VALUES ****************************** _pga_max_size = 524120 KB ************************************* PARAMETERS WITH DEFAULT VALUES ****************************** pga_aggregate_target = 3145728 KB _smm_auto_min_io_size = 56 KB _smm_auto_max_io_size = 248 KB _smm_min_size = 1024 KB _smm_max_size = 262060 KB _smm_px_max_size = 1572864 KB ...

CBO reports _pga_max_size as changed parameter, but I have restarted the instance with only changed parameter P_A_T set to 3G in init.ora. The P_A_T is not listed among parameters with altered values. Interesting!

NEW SORT ALGORITHM IN ORACLE10GR2

Oracle10gRw introduced a new sort algorithm which is using less memory and CPU resources [9],[10]. A hidden parameter _newsort_enabled = {TRUE|FALSE} governs whether the new sort algorithm will be used. I tested the new sort algorithm in one create index command and compared the results with creating the same index in Oracle10gR1 version on the same box. Below are the excerpts from tkprof output.

INDEX CREATION ON ORACLE10GR1 create index t_big_i1 on t_big(normal) compute statistics call count cpu elapsed disk query current rows ------- ------ -------- ---------- ---------- ---------- ---------- ---------- Parse 1 0.00 0.00 0 2 0 0 Execute 1 20.96 34.94 4009 4050 2620 0 Fetch 0 0.00 0.00 0 0 0 0 ------- ------ -------- ---------- ---------- ---------- ---------- ---------- total 2 20.96 34.95 4009 4052 2620 0 Misses in library cache during parse: 1 Optimizer mode: ALL_ROWS Parsing user id: 75 Elapsed times include waiting on following events: Event waited on Times Max. Wait Total Waited ---------------------------------------- Waited ---------- ------------ db file scattered read 259 0.10 1.74 log file switch completion 4 0.36 0.95 log buffer space 9 1.02 1.49 direct path write 5 0.00 0.00 log file sync 1 0.00 0.00 SQL*Net message to client 1 0.00 0.00 SQL*Net message from client 1 0.02 0.02


34 Paper # 411

NAME VALUE ------------------------------- ---------- session uga memory 217648 session uga memory max 42049144 session pga memory 825932 session pga memory max 43817548 workarea memory allocated 0 workarea executions - optimal 6 workarea executions - onepass 0 workarea executions - multipass 0 SQL> alter system set pga_aggregate_target=2620600K; System altered. SQL> @smm PARAMETER VALUE ------------------------------ -------------------- pga_aggregate_target 2620600 _pga_large_extent_size 1024 _pga_max_size 524120 _smm_advice_enabled TRUE _smm_advice_log_size 0 _smm_auto_cost_enabled TRUE _smm_auto_max_io_size 248 _smm_auto_min_io_size 56 _smm_bound 0 _smm_control 0 _smm_freeable_retain 5120 _smm_isort_cap 102400 _smm_max_size 262060 _smm_min_size 1024 _smm_px_max_size 1310300 _smm_retain_size 0 _smm_trace 0 _use_ism_for_pga TRUE

INDEX CREATION ON ORACLE10GR2

create index t_big_i1 on t_big(normal) compute statistics call count cpu elapsed disk query current rows ------- ------ -------- ---------- ---------- ---------- ---------- ---------- Parse 1 0.01 0.05 0 2 0 0 Execute 1 3.86 19.12 3178 4103 3253 0 Fetch 0 0.00 0.00 0 0 0 0 ------- ------ -------- ---------- ---------- ---------- ---------- ---------- total 2 3.87 19.17 3178 4105 3253 0 Misses in library cache during parse: 1 Optimizer mode: ALL_ROWS Parsing user id: 61 Elapsed times include waiting on following events: Event waited on Times Max. Wait Total Waited ---------------------------------------- Waited ---------- ------------ db file scattered read 211 0.17 9.93 direct path write 269 0.00 0.00


35 Paper # 411

SQL*Net message to client 1 0.00 0.00 SQL*Net message from client 1 0.03 0.03 Values from V$SESSTAT: NAME VALUE ------------------------------- ---------- session uga memory 352672 session uga memory max 29549616 session pga memory 646740 session pga memory max 30989908 workarea memory allocated 0 workarea executions - optimal 11 workarea executions - onepass 0 workarea executions - multipass 0 When a shared server mode is used the allocation of memory is slightly different: NAME VALUE ------------------------------- ---------- session uga memory 201748 session uga memory max 298740 session pga memory 327368 session pga memory max 30801608 workarea memory allocated 0 workarea executions - optimal 11 workarea executions - onepass 0 workarea executions - multipass 0 Index creation

Parameter 10gR1 10gR2 Difference % Diff

CPU time 20.96 3.86 17.10 -81.6%

session uga memory max (bytes) 42,049,144 29,549,616 12,499,528 -29.7%

session pga memory max (bytes) 43,817,548 30,989,908 12,827,640 -29.3%

workarea max size (bytes) 36,611,072 28,819,456 7,791,616 -21.3%

CONCLUSION

When I submitted the proposal for this paper I was not aware that in the latest release (Oracle10gR2) a lot of problems related to inappropriate automatic sizing of working areas for SQL operators will be already resolved. However, as I am convinced that a lot of existing problems are already resolved, one might still require to set them accordingly to very special needs. In this case the reader is able to find a lot detailed information in this paper and the referenced literature that is publicly available on the web.

Finally I must admit that a lot of detailed material in this paper is based on assumptions which may be proved wrong later either by me or any body else. So the reader should use the information contained in this paper according to this warning.

ACKNOWLEDGEMENTS:

I would like to give thanks to my friend Neil Smith for his help in making all the linguistic corrections of this paper.


36 Paper # 411

BIBLIOGRAPHY:

1. Dageville, Benoît and Zait, Mohamed, »SQL Memory Management in Oracle9i«, Proceedings of the 28th VLDB Conference, Hong Kong, China, 2002

2. Kumar, Sushil: »Oracle9i Memory Management: Easier Than Ever«, Oracle Open World 2002, [http://www.oracle.com/technology/products/manageability/database/pdf/OOW2002MemMgmtV2.pdf]

3. Kutrovsky, Christo, »Working with Automatic PGA«, [http://www.pythian.com/documents/Working_with_Automatic_PGA.ppt]

4. Ramaswamy, Ramesh: »Optimising Oracle9i Instance Memory«, [http://www.quest.com/presentations/Optimising_Oracle9i_Instance_Memory.pdf ]

5. Oracle Metalink Note 223730.1 »Automatic PGA Memory Managment in 9i« [http://metalink.oracle.com/metalink/plsql/showdoc?db=NOT&id=223730.1]

6. Oracle Metalink Note 147806.1 »Oracle9i New Feature: Automated SQL Execution Memory Management« [http://metalink.oracle.com/metalink/plsql/showdoc?db=NOT&id=147806.1]

7. Oracle Metalink Note 148346.1 »Oracle9i Monitoring Automated SQL Execution Memory Management« [http://metalink.oracle.com/metalink/plsql/showdoc?db=NOT&id=148346.1]

8. Patch 3130972 »ENH: Allow over 1Gb private memory in realfree allocator« [http://metalink.oracle.com/metalink/plsql/showdoc?db=NOT&id=3130972.8]

9. An Oracle White Paper : »Sort Performance Improvements in Oracle Database 10g Release 2«, June 2005, [http://www.oracle.com/technology/products/bi/db/10g/pdf/twp_general_sort_performance_10gr2_0605.pdf]

10. An Oracle White Paper : »DSS Performance in Oracle Database 10g Release 2«, July 2005, [http://www.oracle.com/technology/deploy/performance/pdf/twp_dss_performance_with_10gr2.pdf]

11. Pőder, Tanel: »Memory Management and Latching Improvements in Oracle Database 9i and 10g«, Oracle Open World 2005, [https://www35.cplan.com/cv_106/sessions_catalog.jsp]

12. Lewis, Jonathan: »Undocumented secrets – or untested fairytales ?«, [http://www.jlcomp.demon.co.uk/untested.html]

13. Lewis, Jonathan: »The Snark research mechanism ?«, [http://www.jlcomp.demon.co.uk/snark.html] 14. Dyke, Julian, Diagnostics and Dumping commands, [http://www.juliandyke.com] 15. Shee, Richmond: »If Your Memory Serves You Right…«, IOUG Live! 2004, April 2004, Toronto, Canada

[http://download-uk.oracle.com/oowsf2004/1080.pdf]

ABOUT THE AUTHOR

Joze Senegacnik is an internationally recognized speaker. He is a regular speaker in all Slovenian Oracle Users Group Conferences (SIOUG). In past years he was also a regular speaker at American International Oracle Users Group (IOUG) conferences and Hotsos Symposium as well. Since 2005 he is a member of the highly respected OakTable Network. He conducts well known DBA & Developers Tuning Workshops together with the Oracle University Training Center Ljubljana, Slovenia. His contact address is: [email protected].

APPENDIX A: ABBREVIATIONS

This paper contains the excerpts from the trace files with a lot of abbreviations. In this place I would like to explain some of them as far as I was able to decode them. This is only a speculation and I may be totally wrong. Therefore I ask the reader to take this interpretation with a lot of precaution.

#maxPasses=0 --> maximu passes when not runing in optimal mode 1pass --> one pass Bnd=0 --> bound Cache=(0M, 0M) --> cache size, cache(max)


37 Paper # 411

curSize=488 --> current size Drift=0 --> memory drift flags=11 --> flags Gcnt=0 --> recompute count (total)- increased for each global computation isize --> input size lastMem --> last size of memory used lastTempSegSize --> last size of the temporary segment maxSize=488 --> maximum size maxTempSegSize --> maximum size of the temporary segment mpass --> multipasses OnePass=(0M, 0M) --> onePass size, onePass(max) size opId --> operation id opt --> size for optimal executution opType --> operator type like hash-join, sort, ... osize --> output size PgAl=(19M, 19M) --> PGA allocated, PGA allocated max PgAu=(0M, 0M) --> PGA auto, PGA auto(max) PgFr=(0,0M,0M) --> PGA to free, PGA freed, PGA freeable PgIu=8M --> PGA in use PgMa=(0M, 0M) --> PGA manual, PGA manual(max) qc_instid --> query coordinator instance id in parallel query (when 65535 then DOP=1) qc_sid --> query coordinator SID in parallel query (when 65535 then DOP=1) Qcnt=0 --> recompute count (queries) Res=0 --> reserved rowlen --> row length ser#=1 --> serial from v$session sib=1 --> number of siblings (parallel execution) – sib > 1 when DOP > 1 sid=130 --> session id SID from v$session slv_grp=1 --> slave group Tau=4M --> auto target tempSegObjType --> temporary segment object type tempSegStateObj --> temporary segment state object

APPENDIX B: FOREGROUND BOUND RECOMPUTE ... qesmmIRecordDrifting: recording 6489 drift (Tau=4096, Tres=0) Dumping Global SGA descriptor (cnt=3769) Tau=4M Res=0 Bnd=409600 Drift=0 Gcnt=3769 Qcnt=571 qesmmIRecordDrifting: exit (6489 drift, too much=1) Dumping Global SGA descriptor (cnt=3769) Tau=4M Res=0 Bnd=409600 Drift=6489 Gcnt=3769 Qcnt=571 qesmmIQueryRefreshBound(enter): wi=1EF74AD4 drift=6489 Dumping Global SGA descriptor (cnt=3769) PgIu=15M PgAl=(29M, 49M) PgFr=(0,0M,0M) PgAu=(0M, 0M) PgMa=(3M, 0M) Cache=(0M, 29M) OnePass=(0M, 29M) Tau=4M Res=0 Bnd=409600 Drift=6489 Gcnt=3769 Qcnt=571 We got the latch. State before recompute Dumping Global SGA descriptor (cnt=3769) PgIu=15M PgAl=(29M, 49M) PgFr=(0,0M,0M) PgAu=(0M, 0M) PgMa=(3M, 0M) Cache=(0M, 29M) OnePass=(0M, 29M) Tau=4M Res=0 Bnd=409600 Drift=6489 Gcnt=3769 Qcnt=571 qesmmIRecordDrifting: recording 6489 drift (Tau=4096, Tres=0) Dumping Global SGA descriptor (cnt=3769) Tau=4M Res=0 Bnd=409600 Drift=6489 Gcnt=3769 Qcnt=571 qesmmIRecordDrifting: exit (12978 drift, too much=1) Dumping Global SGA descriptor (cnt=3769) Tau=4M Res=0 Bnd=409600 Drift=12978 Gcnt=3769 Qcnt=571 Too much drifting, we need to recompute bound qesmmIRefreshBound: enter


38 Paper # 411

Dumping Global SGA descriptor (cnt=3769) PgIu=15M PgAl=(29M, 49M) PgFr=(0,0M,0M) PgAu=(0M, 0M) PgMa=(3M, 0M) Cache=(0M, 29M) OnePass=(0M, 29M) Tau=4M Res=0 Bnd=409600 Drift=12978 Gcnt=3769 Qcnt=571 qesmmIRefreshBound(): advice disabled Making snapshot of work area table (maxElem=110) new: op=088790FC ind=0 nbWa=1 qc=65535 #sib=1 End of snapshot (NbElem=1 NbWa=1 totAutoCache=6489) Dumping Global SGA descriptor (cnt=3769) PgIu=15M PgAl=(29M, 49M) PgFr=(0,0M,0M) PgAu=(0M, 0M) PgMa=(3M, 0M) Cache=(6M, 29M) OnePass=(5M, 29M) Tau=4M Res=0 Bnd=409600 Drift=12978 Gcnt=3769 Qcnt=571 qesmmIRefreshBound: need to recompute bound qesmmComputeBound: start (nbOp=1, nbWa=1, target=4096 useCap=1 minSz=128, serCap=409600, parCap=5120) Input TabOp: bound info (088790FC): op=HASH-JOIN, min=448, 1pass=6107, cache=6489, capSize=6489, sib=1 Start (it=1): nbWa=1 nbLft=1 bnd=4096 memTg=4096 memLeft=4096 start loop: bi[0]=088790FC mem=4096 nextIncr=64 bi[0]=088790FC lrgest so far keep bi[0]=088790FC, nextMemLeft=0, pos=0 end loop 1: lft=1 nbWa=1 All use max. We are done End (it=1): memTg=0 res=4096 memLeft=0 qesmmComputeBound: exit-3: reserved=4096 bound=4096 qesmmIRefreshBound: bound recomputed Dumping Global SGA descriptor (cnt=3769) PgIu=15M PgAl=(29M, 49M) PgFr=(0,0M,0M) PgAu=(0M, 0M) PgMa=(3M, 0M) Cache=(6M, 29M) OnePass=(5M, 29M) Tau=4M Res=4096 Bnd=4096 Drift=12978 Gcnt=3769 Qcnt=571 After recompute Dumping Global SGA descriptor (cnt=3770) Tau=4M Res=4096 Bnd=4096 Drift=0 Gcnt=3770 Qcnt=572 qesmmIQueryRefreshBound: exit wid=0, trcId=-1 sid=156, ser#=1791, flags=11 (AUTO|SGA|ADVICE) qc_sid=65535, qc_instid=65535, slv_grp=1 curSize=0, maxSize=0, #maxPasses=0 tempSegStateObj=00000000, tempSegObjType=0 shared work area descriptor (1DA9D574): flags:12 ( AUTO SGA ) opType:9(HASH-JOIN) opId=6 proxy=00000000 isize=1907K, osize=3497K, rowlen=12, flags=01 opt=0, 1pass=1, mpass=0, lastMem=3305KB maxTempSegSize=3072KB, lastTempSegSize=3072KB

APPENDIX C: BACKGROUND BOUND RECOMPUTE

This is an excerpt from the trace file of the CKPT process (at different time than the foreground bound compute in different condtions.). ... qesmmIRefreshBound: need to recompute bound qesmmComputeBound: start (nbOp=14, nbWa=14, target=224195 useCap=1 minSz=430, serCap=86016, parCap=215040) Input TabOp: bound info (0823B11C): op=SORT (v2), min=448, 1pass=3626, cache=26281, capSize=26281, sib=1 bound info (0823B1D4): op=HASH-JOIN, min=546, 1pass=546, cache=546, capSize=3234, sib=1 bound info (0823B28C): op=HASH-JOIN, min=4690, 1pass=4690, cache=4690, capSize=4691, sib=1 bound info (0823B344):


39 Paper # 411

op=GROUP BY (SORT), min=448, 1pass=448, cache=990, capSize=990, sib=1 bound info (0823B3FC): op=GROUP BY (SORT), min=1216, 1pass=36208, cache=36208, capSize=36208, sib=1 bound info (0823B4B4): op=GROUP BY (SORT), min=1216, 1pass=36208, cache=36208, capSize=36208, sib=1 bound info (0823B56C): op=GROUP BY (SORT), min=448, 1pass=448, cache=990, capSize=990, sib=1 bound info (0823B624): op=GROUP BY (SORT), min=1216, 1pass=36208, cache=36208, capSize=36208, sib=1 bound info (0823B6DC): op=GROUP BY (SORT), min=448, 1pass=448, cache=990, capSize=990, sib=1 bound info (0823B794): op=SORT (v2), min=448, 1pass=457, cache=3193, capSize=3193, sib=1 bound info (0823B84C): op=GROUP BY (SORT), min=1216, 1pass=36208, cache=36208, capSize=36208, sib=1 bound info (0823B904): op=HASH-JOIN, min=4690, 1pass=4690, cache=4690, capSize=6489, sib=1 bound info (0823B9BC): op=GROUP BY (SORT), min=1216, 1pass=36208, cache=36208, capSize=36208, sib=1 bound info (0823BA74): op=HASH-JOIN, min=4690, 1pass=4690, cache=4690, capSize=4691, sib=1 Start (it=1): nbWa=14 nbLft=14 bnd=16013 memTg=224195 memLeft=224195 start loop: bi[0]=0823B11C mem=16013 nextIncr=64 bi[0]=0823B11C lrgest so far keep bi[0]=0823B11C, nextMemLeft=0, pos=0 end loop 1: lft=14 nbWa=14 start loop: bi[1]=0823B1D4 mem=546 nextIncr=0 remove: bi[1]=0823B1D4 totMem=546 totIncr=0 memLeft=224195 memTarget:224195 -> 223649 swap bi[1]=0823B1D4 with bi[13]=0823BA74 lft=13 nbWa=13) end loop 1: lft=13 nbWa=13 start loop: bi[1]=0823BA74 mem=4690 nextIncr=0 remove: bi[1]=0823BA74 totMem=4690 totIncr=0 memLeft=224195 memTarget:223649 -> 218959 swap bi[1]=0823BA74 with bi[12]=0823B9BC lft=12 nbWa=12) end loop 1: lft=12 nbWa=12 start loop: bi[1]=0823B9BC mem=16013 nextIncr=64 keep bi[1]=0823B9BC, nextMemLeft=0, pos=1 end loop 2: lft=12 nbWa=12 start loop: bi[2]=0823B28C mem=4690 nextIncr=0 remove: bi[2]=0823B28C totMem=4690 totIncr=0 memLeft=224195 memTarget:218959 -> 214269 swap bi[2]=0823B28C with bi[11]=0823B904 lft=11 nbWa=11) end loop 2: lft=11 nbWa=11 start loop: bi[2]=0823B904 mem=4690 nextIncr=0 remove: bi[2]=0823B904 totMem=4690 totIncr=0 memLeft=224195 memTarget:214269 -> 209579 swap bi[2]=0823B904 with bi[10]=0823B84C lft=10 nbWa=10) end loop 2: lft=10 nbWa=10 start loop: bi[2]=0823B84C mem=16013 nextIncr=64 keep bi[2]=0823B84C, nextMemLeft=0, pos=2 end loop 3: lft=10 nbWa=10 start loop: bi[3]=0823B344 mem=990 nextIncr=0 remove: bi[3]=0823B344 totMem=990 totIncr=0 memLeft=224195 memTarget:209579 -> 208589 swap bi[3]=0823B344 with bi[9]=0823B794 lft=9 nbWa=9) end loop 3: lft=9 nbWa=9 start loop: bi[3]=0823B794 mem=3193 nextIncr=0 remove: bi[3]=0823B794 totMem=3193 totIncr=0 memLeft=224195 memTarget:208589 -> 205396 swap bi[3]=0823B794 with bi[8]=0823B6DC lft=8 nbWa=8) end loop 3: lft=8 nbWa=8 start loop: bi[3]=0823B6DC mem=990 nextIncr=0 remove: bi[3]=0823B6DC totMem=990 totIncr=0 memLeft=224195


40 Paper # 411

memTarget:205396 -> 204406 swap bi[3]=0823B6DC with bi[7]=0823B624 lft=7 nbWa=7) end loop 3: lft=7 nbWa=7 start loop: bi[3]=0823B624 mem=16013 nextIncr=64 keep bi[3]=0823B624, nextMemLeft=0, pos=3 end loop 4: lft=7 nbWa=7 start loop: bi[4]=0823B3FC mem=16013 nextIncr=64 keep bi[4]=0823B3FC, nextMemLeft=0, pos=4 end loop 5: lft=7 nbWa=7 start loop: bi[5]=0823B4B4 mem=16013 nextIncr=64 keep bi[5]=0823B4B4, nextMemLeft=0, pos=5 end loop 6: lft=7 nbWa=7 start loop: bi[6]=0823B56C mem=990 nextIncr=0 remove: bi[6]=0823B56C totMem=990 totIncr=0 memLeft=224195 memTarget:204406 -> 203416 swap bi[6]=0823B56C with bi[6]=0823B56C lft=6 nbWa=6) end loop 6: lft=6 nbWa=6 End (it=1): memTg=203416 res=20779 memLeft=0 Start (it=2): nbWa=6 nbLft=6 bnd=33902 memTg=203416 memLeft=0 start loop: bi[0]=0823B11C mem=26281 nextIncr=0 remove: bi[0]=0823B11C totMem=26281 totIncr=0 memLeft=0 memTarget:203416 -> 177135 swap bi[0]=0823B11C with bi[5]=0823B4B4 lft=5 nbWa=5) end loop 0: lft=5 nbWa=5 start loop: bi[0]=0823B4B4 mem=33902 nextIncr=64 keep bi[0]=0823B4B4, nextMemLeft=0, pos=0 end loop 1: lft=5 nbWa=5 start loop: bi[1]=0823B9BC mem=33902 nextIncr=64 keep bi[1]=0823B9BC, nextMemLeft=0, pos=1 end loop 2: lft=5 nbWa=5 start loop: bi[2]=0823B84C mem=33902 nextIncr=64 keep bi[2]=0823B84C, nextMemLeft=0, pos=2 end loop 3: lft=5 nbWa=5 start loop: bi[3]=0823B624 mem=33902 nextIncr=64 keep bi[3]=0823B624, nextMemLeft=0, pos=3 end loop 4: lft=5 nbWa=5 start loop: bi[4]=0823B3FC mem=33902 nextIncr=64 keep bi[4]=0823B3FC, nextMemLeft=0, pos=4 end loop 5: lft=5 nbWa=5 End (it=2): memTg=177135 res=47060 memLeft=0 Start (it=3): nbWa=5 nbLft=5 bnd=35427 memTg=177135 memLeft=0 start loop: bi[0]=0823B4B4 mem=35427 nextIncr=64 bi[0]=0823B4B4 lrgest so far keep bi[0]=0823B4B4, nextMemLeft=0, pos=0 end loop 1: lft=5 nbWa=5 start loop: bi[1]=0823B9BC mem=35427 nextIncr=64 keep bi[1]=0823B9BC, nextMemLeft=0, pos=1 end loop 2: lft=5 nbWa=5 start loop: bi[2]=0823B84C mem=35427 nextIncr=64 keep bi[2]=0823B84C, nextMemLeft=0, pos=2 end loop 3: lft=5 nbWa=5 start loop: bi[3]=0823B624 mem=35427 nextIncr=64 keep bi[3]=0823B624, nextMemLeft=0, pos=3 end loop 4: lft=5 nbWa=5 start loop: bi[4]=0823B3FC mem=35427 nextIncr=64 keep bi[4]=0823B3FC, nextMemLeft=0, pos=4 end loop 5: lft=5 nbWa=5 All use max. We are done End (it=3): memTg=0 res=224195 memLeft=0 qesmmComputeBound: exit-3: reserved=224195 bound=35427 qesmmIRefreshBound: bound recomputed ...


41 Paper # 411

APPENDIX D: HASH-JOIN STATISTICS

This statistics was produced by setting event 10104 ate level 10. *** 2005-10-25 10:01:37.218 *** ACTION NAME:() 2005-10-25 10:01:37.148 *** MODULE NAME:(SQL*Plus) 2005-10-25 10:01:37.148 *** SERVICE NAME:(SYS$USERS) 2005-10-25 10:01:37.148 *** SESSION ID:(124.804) 2005-10-25 10:01:37.148 kxhfInit(): enter kxhfInit(): exit *** RowSrcId: 3 HASH JOIN STATISTICS (INITIALIZATION) *** Join Type: INNER join Original hash-area size: 5179587 Memory for slot table: 3686400 Calculated overhead for partitions and row/slot managers: 1493187 Hash-join fanout: 8 Number of partitions: 8 Number of slots: 30 Multiblock IO: 15 Block size(KB): 8 Cluster (slot) size(KB): 120 Minimum number of bytes per block: 8160 Bit vector memory allocation(KB): 128 Per partition bit vector length(KB): 16 Maximum possible row length: 97 Estimated build size (KB): 1 Estimated Build Row Length (includes overhead): 18 # Immutable Flags: Not BUFFER(execution) output of the join for PQ Evaluate Left Input Row Vector Evaluate Right Input Row Vector # Mutable Flags: IO sync kxhfSetPhase: phase=BUILD kxhfAddChunk: add chunk 0 (sz=32) to slot table kxhfAddChunk: chunk 0 (lbs=0BAD7D14, slotTab=0BAD7E68) successfuly added *** 2005-10-25 10:02:12.960 kxhfSetPhase: phase=PROBE_1 qerhjFetch: max build row length (mbl=12) *** RowSrcId: 3 END OF HASH JOIN BUILD (PHASE 1) *** Revised row length: 12 Revised build size: 944KB kxhfResize(enter): resize to 12 slots (numAlloc=10, max=30) kxhfResize(exit): resized to 12 slots (numAlloc=10, max=12) Slot table resized: old=30 wanted=12 got=12 unload=0 *** RowSrcId: 3 HASH JOIN BUILD HASH TABLE (PHASE 1) *** Total number of partitions: 8 Number of partitions which could fit in memory: 8 Number of partitions left in memory: 8 Total number of slots in in-memory partitions: 10 Total number of rows in in-memory partitions: 81388 (used as preliminary number of buckets in hash table) Estimated max # of build rows that can fit in avail memory: 245160 ### Partition Distribution ### Partition:0 rows:10242 clusters:1 slots:1 kept=1 Partition:1 rows:10024 clusters:1 slots:1 kept=1 Partition:2 rows:10168 clusters:1 slots:1 kept=1 Partition:3 rows:10332 clusters:2 slots:2 kept=1 Partition:4 rows:9996 clusters:1 slots:1 kept=1 Partition:5 rows:10276 clusters:1 slots:1 kept=1 Partition:6 rows:10336 clusters:2 slots:2 kept=1 Partition:7 rows:10014 clusters:1 slots:1 kept=1 *** (continued) HASH JOIN BUILD HASH TABLE (PHASE 1) ***


42 Paper # 411

Revised number of hash buckets (after flushing): 81388 Allocating new hash table. *** (continued) HASH JOIN BUILD HASH TABLE (PHASE 1) *** Requested size of hash table: 16384 Actual size of hash table: 16384 Number of buckets: 131072 Match bit vector allocated: FALSE kxhfResize(enter): resize to 16 slots (numAlloc=10, max=12) kxhfResize(exit): resized to 16 slots (numAlloc=10, max=16) freeze work area size to: 3084K (16 slots) *** (continued) HASH JOIN BUILD HASH TABLE (PHASE 1) *** Total number of rows (may have changed): 81388 Number of in-memory partitions (may have changed): 8 Final number of hash buckets: 131072 Size (in bytes) of hash table: 524288 kxhfIterate(end_iterate): numAlloc=10, maxSlots=16 *** (continued) HASH JOIN BUILD HASH TABLE (PHASE 1) *** ### Hash table ### # NOTE: The calculated number of rows in non-empty buckets may be smaller # than the true number. Number of buckets with 0 rows: 96194 Number of buckets with 1 rows: 7 Number of buckets with 2 rows: 29674 Number of buckets with 3 rows: 2 Number of buckets with 4 rows: 4625 Number of buckets with 5 rows: 1 Number of buckets with 6 rows: 524 Number of buckets with 7 rows: 0 Number of buckets with 8 rows: 41 Number of buckets with 9 rows: 0 Number of buckets with between 10 and 19 rows: 4 Number of buckets with between 20 and 29 rows: 0 Number of buckets with between 30 and 39 rows: 0 Number of buckets with between 40 and 49 rows: 0 Number of buckets with between 50 and 59 rows: 0 Number of buckets with between 60 and 69 rows: 0 Number of buckets with between 70 and 79 rows: 0 Number of buckets with between 80 and 89 rows: 0 Number of buckets with between 90 and 99 rows: 0 Number of buckets with 100 or more rows: 0 ### Hash table overall statistics ### Total buckets: 131072 Empty buckets: 96194 Non-empty buckets: 34878 Total number of rows: 81388 Maximum number of rows in a bucket: 10 Average number of rows in non-empty buckets: 2.333505 kxhfInit(): enter kxhfInit(): exit *** RowSrcId: 5 HASH JOIN STATISTICS (INITIALIZATION) *** Join Type: INNER join Original hash-area size: 11291247 Memory for slot table: 9338880 Calculated overhead for partitions and row/slot managers: 1952367 Hash-join fanout: 8 Number of partitions: 8 Number of slots: 76 Multiblock IO: 15 Block size(KB): 8 Cluster (slot) size(KB): 120 Minimum number of bytes per block: 8160 Bit vector memory allocation(KB): 512 Per partition bit vector length(KB): 64 Maximum possible row length: 65 Estimated build size (KB): 6 Estimated Build Row Length (includes overhead): 45


43 Paper # 411

# Immutable Flags: Not BUFFER(execution) output of the join for PQ Evaluate Left Input Row Vector Evaluate Right Input Row Vector # Mutable Flags: IO sync kxhfSetPhase: phase=BUILD kxhfAddChunk: add chunk 0 (sz=128) to slot table kxhfAddChunk: chunk 0 (lbs=0BEBF864, slotTab=0BEBF9B8) successfuly added kxhfSetPhase: phase=PROBE_1 qerhjFetch: max build row length (mbl=44) *** RowSrcId: 5 END OF HASH JOIN BUILD (PHASE 1) *** Revised row length: 39 Revised build size: 6106KB kxhfResize(enter): resize to 56 slots (numAlloc=56, max=76) kxhfResize(exit): resized to 56 slots (numAlloc=56, max=56) Slot table resized: old=76 wanted=56 got=56 unload=0 *** RowSrcId: 5 HASH JOIN BUILD HASH TABLE (PHASE 1) *** Total number of partitions: 8 Number of partitions which could fit in memory: 8 Number of partitions left in memory: 8 Total number of slots in in-memory partitions: 56 Total number of rows in in-memory partitions: 162776 (used as preliminary number of buckets in hash table) Estimated max # of build rows that can fit in avail memory: 312480 ### Partition Distribution ### Partition:0 rows:20484 clusters:7 slots:7 kept=1 Partition:1 rows:20048 clusters:7 slots:7 kept=1 Partition:2 rows:20336 clusters:7 slots:7 kept=1 Partition:3 rows:20664 clusters:7 slots:7 kept=1 Partition:4 rows:19992 clusters:7 slots:7 kept=1 Partition:5 rows:20552 clusters:7 slots:7 kept=1 Partition:6 rows:20672 clusters:7 slots:7 kept=1 Partition:7 rows:20028 clusters:7 slots:7 kept=1 *** (continued) HASH JOIN BUILD HASH TABLE (PHASE 1) *** Revised number of hash buckets (after flushing): 162776 Allocating new hash table. *** (continued) HASH JOIN BUILD HASH TABLE (PHASE 1) *** Requested size of hash table: 32768 Actual size of hash table: 32768 Number of buckets: 262144 Match bit vector allocated: FALSE kxhfResize(enter): resize to 62 slots (numAlloc=56, max=56) kxhfResize(exit): resized to 62 slots (numAlloc=56, max=62) freeze work area size to: 9259K (62 slots) *** (continued) HASH JOIN BUILD HASH TABLE (PHASE 1) *** Total number of rows (may have changed): 162776 Number of in-memory partitions (may have changed): 8 Final number of hash buckets: 262144 Size (in bytes) of hash table: 1048576 kxhfIterate(end_iterate): numAlloc=56, maxSlots=62 *** (continued) HASH JOIN BUILD HASH TABLE (PHASE 1) *** ### Hash table ### # NOTE: The calculated number of rows in non-empty buckets may be smaller # than the true number. Number of buckets with 0 rows: 224458 Number of buckets with 1 rows: 0 Number of buckets with 2 rows: 0 Number of buckets with 3 rows: 8 Number of buckets with 4 rows: 34830 Number of buckets with 5 rows: 0 Number of buckets with 6 rows: 0 Number of buckets with 7 rows: 7 Number of buckets with 8 rows: 2686


44 Paper # 411

Number of buckets with 9 rows: 0 Number of buckets with between 10 and 19 rows: 154 Number of buckets with between 20 and 29 rows: 1 Number of buckets with between 30 and 39 rows: 0 Number of buckets with between 40 and 49 rows: 0 Number of buckets with between 50 and 59 rows: 0 Number of buckets with between 60 and 69 rows: 0 Number of buckets with between 70 and 79 rows: 0 Number of buckets with between 80 and 89 rows: 0 Number of buckets with between 90 and 99 rows: 0 Number of buckets with 100 or more rows: 0 ### Hash table overall statistics ### Total buckets: 262144 Empty buckets: 224458 Non-empty buckets: 37686 Total number of rows: 162776 Maximum number of rows in a bucket: 20 Average number of rows in non-empty buckets: 4.319270 Disabled bitmap filtering: filtered rows=0 minimum required=50 out of=1000 qerhjFetch: max probe row length (mpl=0) *** RowSrcId: 5, qerhjFreeSpace(): free hash-join memory kxhfRemoveChunk: remove chunk 0 from slot table Disabled bitmap filtering: filtered rows=0 minimum required=50 out of=1000 qerhjFetch: max probe row length (mpl=0) *** RowSrcId: 3, qerhjFreeSpace(): free hash-join memory kxhfRemoveChunk: remove chunk 0 from slot table<

Senegacnik PGA Memory Management Oracle9i 10g Clanek

Documents

Transcript of Senegacnik PGA Memory Management Oracle9i 10g Clanek