The Persistence of Memory (Issues) Brian Hitchcock OCP 8, 8i, 9i DBA Sun Microsystems...
-
Upload
frederick-charles -
Category
Documents
-
view
229 -
download
2
Transcript of The Persistence of Memory (Issues) Brian Hitchcock OCP 8, 8i, 9i DBA Sun Microsystems...
The Persistence of Memory (Issues)
Brian HitchcockOCP 8, 8i, 9i DBA
Sun Microsystems
Brian Hitchcock April 2, 2004 Page 1
www.brianhitchcock.net
NoCOUG
Brian Hitchcock April 2, 2004 Page 2
www.brianhitchcock.net
What’s The Issue?
Shared pool– What is it?– How does it work?– How to monitor and tune it?– What can go wrong?
Sorting– Same four questions
Validity of load testing
NoCOUG
Brian Hitchcock April 2, 2004 Page 3
www.brianhitchcock.net
Why Not Cover All of SGA?
Too much to discuss Shared pool
– Caused problems for me recently– Very different behavior from buffer cache
Sorting– Source of recent performance problem– Major change in management 8i to 9i
NoCOUG
Brian Hitchcock April 2, 2004 Page 4
www.brianhitchcock.net
Relevant Memory Structures
SGA in memory– Db buffer cache (db_block_buffers 8i, db_cache_size 9i)– Redo log buffers (log_buffer)– Shared pool
Dictionary cache Library cache Control structures
PGA in memory Sort area
Shared pool synonymous with library cache
NoCOUG
Brian Hitchcock April 2, 2004 Page 5
www.brianhitchcock.net
Memory Structures
Physical Memory of DB Machine
SGA
Db BufferCache Shared Pool
Redo LogBuffer
PGA
…
Individual Sessions…
OS etc.
NoCOUG
Brian Hitchcock April 2, 2004 Page 6
www.brianhitchcock.net
Shared Pool
What is it?– Part of SGA (shared global area)– Stores SQL and PL/SQL that is being executed
After parsing, SQL stored in shared pool
– Allows reuse of SQL and PL/SQL Prevents need to re-parse same SQL
– Controlled by Shared_pool_size
NoCOUG
Brian Hitchcock April 2, 2004 Page 7
www.brianhitchcock.net
Shared Pool init.ora Parameters
Shared_pool_size– Overall size of shared pool
Shared_pool_reserved_size– Part reserved for large memory allocations
Shared_pool_reserved_size_min_alloc– Minimum size of large memory allocations– Abbreviated SPRS_min_alloc for my sanity
Dedicated server processes assumed– MTS different, PGA allocations made in SGA etc.
NoCOUG
Brian Hitchcock April 2, 2004 Page 8
www.brianhitchcock.net
Shared Pool Structure
Shared Pool
Dictionary Cache Library Cache Reserved Pool
Shared_pool_size
Shared_pool_reserved_size
_shared_pool_reserved_size_min_alloc
NoCOUG
Brian Hitchcock April 2, 2004 Page 9
www.brianhitchcock.net
What Does Library Cache Do?
SQL or PL/SQL to be executed must be parsed
Once parsed, is stored in shared pool If same SQL or PL/SQL is executed again, it
can be used from shared pool without parsing Parsing is very compute intensive Storing parsed SQL, PL/SQL reduces parsing
– Improves performance
NoCOUG
Brian Hitchcock April 2, 2004 Page 10
www.brianhitchcock.net
How Does It Work?
SQL to be executed– Scan shared pool looking for free memory in library cache– Need enough contiguous free memory to store the SQL
statement– If not found
Flush all unused, unpinned memory and merge Scan again
– If still not found Issue ORA-04031 error “Unable to allocate % bytes of shared memory SQL is not executed
NoCOUG
Brian Hitchcock April 2, 2004 Page 11
www.brianhitchcock.net
Implications
Shared pool– Large enough to allocate memory for all SQL requests that
will execute at any one time– Over time, becomes fragmented– When scanning, if larger than needed piece found
Broken into size needed and small free segment Over time, lots of small pieces Over time, very few big contiguous pieces
– If large memory allocation requested May not have any large pieces available May not be able to flush enough to create large piece
NoCOUG
Brian Hitchcock April 2, 2004 Page 12
www.brianhitchcock.net
Shared SQL
Before tuning shared pool– Is application SQL shared?– Bind variables?– PL/SQL packages?
Vendor apps– May or may not be good about this– You can’t do much about the SQL– Some init.ora parameters may help
Force sharing, cache cursors etc.
NoCOUG
Brian Hitchcock April 2, 2004 Page 13
www.brianhitchcock.net
Why Not Just Make It Larger?
Performance– Even if all SQL in shared pool is not in use– Must scan all of it looking for free space– If not enough found, flush, merge, scan again– Shared pool has single shared pool latch
Longer scan time holds latch longer Other requests wait longer Latch wait shows up in STATSPACK wait events
NoCOUG
Brian Hitchcock April 2, 2004 Page 14
www.brianhitchcock.net
Shared Pool Latches
Shared pool latch– Serializes memory allocations in library cache
Library cache latch– Serializes access to objects in library cache
Row cache objects latch– Latch for dictionary cache
Latch waits for any of these– More shared SQL?– Investigate shared pool size
NoCOUG
Brian Hitchcock April 2, 2004 Page 15
www.brianhitchcock.net
What About Too Small?
Performance– Less scan time– Can’t hold all SQL executing at any one time– Lots of flushes– Flushing takes time, holds the latch– Performance suffers
NoCOUG
Brian Hitchcock April 2, 2004 Page 16
www.brianhitchcock.net
So, What To Do?
Shared pool size tradeoff– Smaller
Faster to scan before flushing More flushes Less SQL stored in parsed form, more parsing
– Bigger Slower to scan Fewer flushes Slower to flush More SQL stored in parsed form, less parsing
NoCOUG
Brian Hitchcock April 2, 2004 Page 17
www.brianhitchcock.net
In Perfect World
All SQL reused All SQL fits in shared pool Shared pool reaches steady state
– No flushes– No 04031 errors– No one tries to execute large on-off SQL at random
intervals– Everyone is happy
I haven’t been to the perfect world– Brochures look nice!
NoCOUG
Brian Hitchcock April 2, 2004 Page 18
www.brianhitchcock.net
In The Real World
Some, perhaps most SQL is reused Some is not
– Large SQL statements request large memory allocations
– Shared pool flushed Free up memory of SQL not in use Merge small pieces of memory
– Frequently used SQL (not currently in use) Must be re-parsed and reloaded Slows performance
NoCOUG
Brian Hitchcock April 2, 2004 Page 19
www.brianhitchcock.net
The Perfect World II
Assuming there will be some large SQL once in a while
– Reserved pool– Portion of shared pool for large allocations– Only large allocations go here– Prevents disruption of shared pool– Prevents smaller requests fragmenting reserved
pool
NoCOUG
Brian Hitchcock April 2, 2004 Page 20
www.brianhitchcock.net
Reserved Pool?
For allocation request bigger than SPRS_min_alloc
– If not enough free space in shared pool– Allocation goes to reserved pool– Scan reserved pool looking for enough memory– If not found, flush unused, merge, scan again– If not found, 04031 error– Normal shared pool not disturbed
No flushing No reloading of frequently used SQL
NoCOUG
Brian Hitchcock April 2, 2004 Page 21
www.brianhitchcock.net
But Really…
If you think about it… Shared pool needs to be big enough
– Over longest time between db restarts– Max number of simultaneous users– Each user submitting the largest SQL
Reserved pool needs to hold– Same criteria as shared pool but for allocations
larger than SPRS_min_alloc Is this really possible?
NoCOUG
Brian Hitchcock April 2, 2004 Page 22
www.brianhitchcock.net
How To Tune Shared Pool?
Monitor free space in shared pool– Large free space may not be a good thing
Memory that is never used Many small pieces, no large pieces
– Ideally Small amount of free space during max load
– Monitor ORA-04031 errors None, or very few, very infrequently Only caused by large requests that shouldn’t be made Not sent to alert log, select request_failures from
v$shared_pool_reserved
NoCOUG
Brian Hitchcock April 2, 2004 Page 23
www.brianhitchcock.net
Tuning Shared Pool
If shared pool– Has free space over time– No ORA-04031 errors
Consider reducing shared pool
Reduce scan time
– Only if latch waits are an issue STATSPACK report
Start large and reduce if needed– My opinion
NoCOUG
Brian Hitchcock April 2, 2004 Page 24
www.brianhitchcock.net
STATSPACK -- Shared Pool
STATSPACK report– Load profile
Parses Hard parses (low %)
– Instance efficiency percentages Soft parses (high %)
– Shared pool statistics Memory usage (high %) SQL with executions >1 (high %) Memory for SQL w/exec >1 (high %)
NoCOUG
Brian Hitchcock April 2, 2004 Page 25
www.brianhitchcock.net
STATSPACK – Shared Pool
STATSPACK report– Latch sleep breakdown
Shared pool Library cache Row object cache
– Dictionary cache stats Pct misses (< 2%)
– Library cache activity Pct misses (very low)
– Shared pool advisory (9i)
NoCOUG
Brian Hitchcock April 2, 2004 Page 26
www.brianhitchcock.net
9i Improvements
V$shared_pool_advice (9.2) Output seen in STATSPACK report
– Shared pool advisory
Similar to v$db_cache_advice Shows possible benefit of larger shared pool
NoCOUG
Brian Hitchcock April 2, 2004 Page 27
www.brianhitchcock.net
Shared Pool Details 8i vs 9i
Shared_pool_size– Static in 8i– Dynamic in 9i
Must also set sga_max_size Defaults to current sga size
– Shared_pool_reserved_size 8i, static, default 10% shared pool, min 5000 bytes 9i, static, default 5% shared pool, min 4400 bytes
– Shared_pool_reserved_size_min_alloc Hidden parameter in 8i, 9i Default to 4000 bytes (both 8i, 9i)
NoCOUG
Brian Hitchcock April 2, 2004 Page 28
www.brianhitchcock.net
Bigger Reserved Pool?
To increase– Reserved pool taken from shared pool size– Must increase both
Shared pool, shared_pool_size Reserved pool, shared_pool_reserved_size
NoCOUG
Brian Hitchcock April 2, 2004 Page 29
www.brianhitchcock.net
Real World Case 1
Priority: must not waste memory– Start testing with very small shared pool– Lots of ORA-04031 errors– Slowly increase shared pool until errors stop– Many days spent– Running with 300mb shared pool– App vendor recommends 400mb shared pool
True optimization takes a lot of time
NoCOUG
Brian Hitchcock April 2, 2004 Page 30
www.brianhitchcock.net
Real World Case 1a
ORA-04031 errors– Not sent to alert log– Vendor app doesn’t trap oracle error– App users report app error
Can’t connect
– App users don’t see oracle error– Only DBA can see if any have occurred
Select request_failures from v$shared_pool_reserved
– This delayed finding root cause of “can’t login”
NoCOUG
Brian Hitchcock April 2, 2004 Page 31
www.brianhitchcock.net
Real World Case 2
Priority: must not waste time– Existing app, running for months– Suddenly throws lots of ORA-04031 errors– No time, just increase shared pool– Happens twice– Problem may be fixed in 8174 patch– Finally running with 2gb shared pool– No performance problems seen
NoCOUG
Brian Hitchcock April 2, 2004 Page 32
www.brianhitchcock.net
Who Is Right?
It depends Different customers Different priorities Many apps don’t need supreme db
performance Large shared pool may cause long scan times
– If overall app performance doesn’t suffer– Who cares?
Time spent carefully tuning may be wasted
NoCOUG
Brian Hitchcock April 2, 2004 Page 33
www.brianhitchcock.net
What About Dictionary Cache?
Ignored so far– Part of shared pool– Caches system table info– Not much to tune
Make larger by increasing shared pool size– STATSPACK report shows
Dictionary cache stats Pct misses should be < 2% Assumes steady state under load
– Row cache objects latch waits Need for larger dictionary cache
NoCOUG
Brian Hitchcock April 2, 2004 Page 34
www.brianhitchcock.net
Sorting
Needed when– Creating index– SQL that uses
Distinct Order by Group by
Involves multiple passes– Merge results
NoCOUG
Brian Hitchcock April 2, 2004 Page 35
www.brianhitchcock.net
What’s The Issue?
Sorting done– In memory up to sort_area_size– On disk in temporary tablespace (TEMP)
Sorts to disk much slower– Contends with other disk activity
Want all sorting done in memory– More memory helps even if some sorting still
goes to disk
NoCOUG
Brian Hitchcock April 2, 2004 Page 36
www.brianhitchcock.net
Sorting Structure
PGA
…
Individual Sessions…
Sort_area_size
NoCOUG
Brian Hitchcock April 2, 2004 Page 37
www.brianhitchcock.net
Sorting In Memory
How much memory needed?– Level 1 (least memory)
Many multi-pass sorts to disk– Level 2 (more memory)
More memory than level 1 Only single-pass sorts to disk
– Level 3 (most memory) Lots more memory No sorts to disk of any kind
– Level 2 may be best option
NoCOUG
Brian Hitchcock April 2, 2004 Page 38
www.brianhitchcock.net
Sorting In Memory
For dedicated server processes (non-MTS)– When user SQL needs to sort– Memory allocated in PGA
Up to sort_area_size
– After sort completes Memory above sort_area_size_retained released for
reuse by same process Sort_area_size_retained defaults to sort_area_size Memory released to OS after process completes
NoCOUG
Brian Hitchcock April 2, 2004 Page 39
www.brianhitchcock.net
Sorting Issues
Lots of users, lots of sorting– Total memory needed is large
Worst case Max number users x sort_area_size
– Users can alter session to increase sort area size Restrict “alter session” privilege?
– Total memory needed Could be very large Could cause swapping
NoCOUG
Brian Hitchcock April 2, 2004 Page 40
www.brianhitchcock.net
Tune Sorting?
STATSPACK report– Sorts to memory, disk– TEMP tablespace I/O stats– Want all sorts in memory
Increase sort_area_size– Reduce sorts to disk– Reduce TEMP I/O– Monitor total memory used
NoCOUG
Brian Hitchcock April 2, 2004 Page 41
www.brianhitchcock.net
STATSPACK -- Sorting
Instance activity stats– Sorts (disk)– Sorts (memory)– Sorts (rows)
Tablespace IO stats File IO stats
NoCOUG
Brian Hitchcock April 2, 2004 Page 42
www.brianhitchcock.net
Sorting Issues
Similar to shared pool– Can’t really predict how much sorting will happen
at any given time– SQL that requires large sort area will interfere
with all other SQL in the database– Can’t increase sort_area_size without worrying
about total physical memory
NoCOUG
Brian Hitchcock April 2, 2004 Page 43
www.brianhitchcock.net
Sorting For Specific Sessions
Instead of tuning for all sessions– Identify sessions that always need large sorts– Those sessions use alter session to assign larger
sort_area_size– All other sessions use smaller sort_area_size
NoCOUG
Brian Hitchcock April 2, 2004 Page 44
www.brianhitchcock.net
Sorting in 9i
Instead of sort_area_size Pga_aggregate_target
– Set limit on total PGA for all users– Any user sort can use all of this PGA limit– Maximum memory for sorting controlled– Makes tuning much simpler
Increase pga_aggregate_target Reduce multiple pass sorts to disk Reduce single pass sorts to disk Reduce TEMP I/O
NoCOUG
Brian Hitchcock April 2, 2004 Page 45
www.brianhitchcock.net
Sorting in 9i
Pga_aggregate_target– Workarea_size_policy must be auto (default)– When set, *_area_size parameters ignored– Dynamic
Pga_target_advice– Show benefit of increased pga target– Makes tuning easier
NoCOUG
Brian Hitchcock April 2, 2004 Page 46
www.brianhitchcock.net
9i Sorting Structure
PGA
…
Individual Sessions…
Pga_aggregate_target
NoCOUG
Brian Hitchcock April 2, 2004 Page 47
www.brianhitchcock.net
Real World Case
App users login– Sorting– Load test
Max users Want to run test quickly Ramp up user logins fast
– Login SQL causes large sort– Stresses sort_area_size– Will we ever see this in production?
NoCOUG
Brian Hitchcock April 2, 2004 Page 48
www.brianhitchcock.net
Real World Case
Details– Load test 800 users– Sort_area_size 2M– Some sorts going to disk– Increase sort_area_size?
800x2m = 1.6gb Too much memory! Can’t change sort_area_size
– Live with performance impact of sorts to disk
NoCOUG
Brian Hitchcock April 2, 2004 Page 49
www.brianhitchcock.net
Load Testing?
Was it valid?– Done quickly– Simulate worst load possible
800 users
– Does this simulate real-world experience? Will 800 users ever connect in this time frame?
– Causes strain on TEMP tablespace– Sort area size tuned for 800 users at once
Could be bigger for smaller number of users
NoCOUG
Brian Hitchcock April 2, 2004 Page 50
www.brianhitchcock.net
Summary
Shared pool, sorting– Hard to tune perfectly– Need to monitor over time and adjust
Sorting– 9i features very good
Pga_aggregate_target– More memory helps
Even if some sorting still goes to disk
Load testing can make you tune incorrectly