IBM Java Garbage Collection Tuning

IBM Software Group

WebSphere Support Technical Exchange

Java Garbage CollectionBest Practices for Sizing and Tuning the Java Heap

Chris Bailey

IBM Software Group

WebSphere Support Technical Exchange 2

Objectives Overview

Selecting the Correct GC Policy

Sizing the Java heap

Questions/Answers

IBM Software Group


Garbage Collection Performance GC performance issues can take many forms

Definition of a performance problem is user centric User requirement may be for:

Very short GC pause times Maximum throughput A balance of both

First step is ensure that the correct GC policy has been selected for the workload type

Helpful to have an understanding of GC mechanisms

Second step is to ensure heap sizing is correct

Third step us to look for specific performance issues

IBM Software Group


Selecting the Correct GC Policy

IBM Software Group


Understanding Garbage Collection Responsible for allocation and freeing of:

Java objects, Array objects and Java classes

Allocates objects using a contiguous section of Java heap Ensures the object remains as long as it is in use or live

Determination based on a reference from another live object or from outside of the Heap

Reclaims objects that are no longer referenced Ensures that any finalize method is run before the object is

reclaimed

IBM Software Group


Object Allocation Requires a contiguous area of Java heap

Driven by requests from: The Java application JNI code

Most allocations take place in Thread Local Heaps (TLHs)Threads reserve a chunk of free heap to allocate from

Reduces contention on allocation lock Keeps code running in a straight line (fewer failures) Meant to be fast

Available for objects < 512 bytes in size Larger allocates take place under a global heap lock

These allocations are one time costs out of line allocateMultiple threads allocating larger objects at the same time willcontend

IBM Software Group


Object Reclamation (Garbage Collection) Occurs under two scenarios:

An allocation failure An object allocation is requested and not enough contiguous memory is available

A programmatically requested garbage collection cycle call is made to System.GC() or Runtime.GC() the Distributed Garbage Collector is running call to JVMPI/TI is made

Two main technologies used to remove the garbage: Mark Sweep Collector Copy Collector

IBM uses a mark sweep collector or a combination for generational

IBM Software Group


Global Collection Policies Garbage Collection can be broken down into 2 (3) steps

Mark: Find all live objects in the system Sweep: Reclaim unused heap memory to the free list Compact: Reduce fragmentation within the free list

All steps are in a single stop-the-world (STW) phaseApplication pauses whilst garbage collection is done

Each step is performed as a parallel task within itself

Four GC Policies, optimized for different scenarios-Xgcpolicy:optthruput optimized for batch type applications-Xgcpolicy:optavgpause optimized for applications with responsiveness

criteria-Xgcpolicy:gencon optimized for highly transactional workloads-Xgcpolicy:subpools optimized for large systems with allocation

contention

IBM Software Group


Parallel Mark Sweep Collector, with compaction avoidance Created to make use of additional processors on server systems Designed to increase performance for SMP and not degrade performance for uni-processor systems

Optimized for Throughput Best policy for batch type applications

Consists of a single flat Java heap:

0 GB 2 GB

Heap Base Heap LimitHeap Size

LOA

Parallel GC (optthruput)

IBM Software Group


Parallelism achieved through the use of GC Helper ThreadsParked set of threads that wake to share GC workMain GC thread generates the root set of objectsHelper threads share the work for the rest of the phasesNumber of helpers is one less than the number of processing

unitsSo helper threads and main GC thread equals the number of

processing unitsConfigurable using -Xgcthreads

GC Helper Threads

IBM Software Group


Parallel Mark/Parallel Sweep view of GC

IBM Software Group


Reduces and makes more consistent the time spent inside Stop theWorld GC Reduction usually between 90 and 95%

Achieved by carrying out some of the STW work whilst application is running 1.4.2: Concurrent Marking 5.0: Concurrent Marking and Concurrent Sweeping

Slight overhead on thruput for greatly reduced STW times Policy is ideal for systems with responsiveness criteria

eg. Portal applications

Concurrent GC (optavgpause)

IBM Software Group


Parallel and Concurrent Mark/Sweep

Concurrent Kickoff

IBM Software Group


Concurrent Mark hidden object issue Higher heap usage

IBM Software Group


Higher heap usage

because not all garbage removed

Concurrent Mark hidden object issue

Dangling pointer!

IBM Software Group


Similar in concept to that used by Sun and HP Parallel copy and concurrent global collects by default

Motivation: Objects die young so focus collection efforts on recently created objects Divide the heap up into a two areas: new and old Perform allocates from the new area Collections focus on the new area Objects that survive a number of collects in new area are

promoted to old area (tenured)

Ideal for transactional and high data throughput workloads

Generational and Concurrent GC (gencon)

0 GB 2 GB

Heap Base Heap LimitHeap Size

LOANursery (new) Space Tenured (old) SpaceAllocate Survivor

IBM Software Group


Allocate Space Survivor Space

Nursery is split into two spaces (semi-spaces)Only one contains live objects and is available for allocationMinor collections (Scavenges) move objects between spacesRole of spaces is reversed

Nursery/Young Generation

Survivor Space Allocate Space

Movement results in implicit compaction

Nursery (new) Space Copy Collection

IBM Software Group


Subpooling (subpool) Goals:

Reduce allocation lock contention by distributing free memory into multiple lists

Reduce allocation contention through use of atomic operations instead of a heap lock

Prevent premature garbage collections by using a best fit (or closer to best fit) policy instead of address ordered

Ideal for very large SMP systems where large amounts data is being allocated where there is heap lock contention

IBM Software Group


Looking for Heap Lock Contention All locks can be profiled using Java Lock Analyzer (JLA)

http://www.alphaworks.ibm.com/tech/jla(AlphaWorks)

Provides time accounting and contention statistics for Java and JVM locks

Functionality includes: Counters associated with contended locks Total number of successful acquires Recursive acquires times a thread acquires a lock it

already owns Number of times a thread blocks because a monitor is

already owned Cumulative time the monitor was held.

IBM Software Group


JLA Sample ReportSystem (Registered) Monitors

%MISS GETS NONREC SLOW REC TIER2 TIER3 %UTIL AVER-HTM MON-NAME87 5273 5273 4572 0 710708 18487 1 95408 JITC Global_Compile lock9 6870 6869 631 1 113420 2976 0 11807 Heap lock5 1123 1123 51 0 11098 286 1 248385 Binclass lock0 1153 1147 5 6 1307 33 0 47974 Monitor Cache lock0 46149 45877 134 272 36961 877 1 6558 JITC CHA lock0 33734 23483 19 10251 6544 150 1 17083 Thread queue lock0 5 5 0 0 0 0 0 9309689 JNI Global Reference lock0 5 5 0 0 0 0 0 9283000 JNI Pinning lock0 5 5 0 0 0 0 0 9442968 Sleep lock0 1 1 0 0 0 0 0 0 Monitor Registry lock0 0 0 0 0 0 0 0 0 Evacuation Region lock0 0 0 0 0 0 0 0 0 Method trace lock0 0 0 0 0 0 0 0 0 Classloader lock0 0 0 0 0 0 0 0 0 Heap Promotion lock

Java (Inflated) Monitors

%MISS GETS NONREC SLOW REC TIER2 TIER3 %UTIL AVER-HTM MON-NAME15 68 68 10 0 2204 56 2 11936405 test.lock.testlock1@A09410/A094182 42 42 1 0 186 5 0 300478 test.lock.testlock2@D31358/D313600 70 70 0 0 41 1 0 7617 java.lang.ref.ReferenceQueue$Lock@920628/920630

IBM Software Group


JLA: Fields in the report

IBM Software Group


Choosing the Right GC Policy Four GC Policies, optimized for different scenarios

-Xgcpolicy:optthruput optimized for batch type applications-Xgcpolicy:optavgpause optimized for applications with

responsiveness criteria-Xgcpolicy:gencon optimized for highly transactional

workloads-Xgcpolicy:subpools optimized for large systems with allocation

contention

How do I know whether to use optavgpause or gencon? Monitor GC activityLook for certain characteristics

IBM Software Group


Monitoring GC Activity Use of Verbose GC logging

only data that is required for GC performance tuning Graph Verbose GC output using GC and Memory Visualizer (GCMV) from ISA

Activated using command line options-verbose:gc-Xverbosegclog:[DIR_PATH][FILE_NAME],X,Y

where: [DIR_PATH] is the directory where the file should be written [FILE_NAME] is the name of the file to write the logging to X is the number of files to Y is the number of GC cycles a file should contain

Performance Cost: (very) basic testing shows a 2% overhead for GC duration of 200ms

eg. if application GC overhead is 5%, it would become 5.1%

IBM Software Group


Important Characteristics for Choosing GC Policy

Rate of Garbage CollectionHigh rates of object burn point to large numbers of transitional objects, and

therefore the application may well benefit from the use of gencon

Large Object Allocations?The allocation of very large objects adversely affects gencon unless the nursery is

sufficiently large enough. The application may well benefit from optavgpuse

Large heap usage variationsThe optavgpause algorithms are best suited to consistent allocation profilesWhere large variations occur, gencon may be better suited

Rule of thumb: if GC overhead is > 10%, youve most likely chosen the wrong one

IBM Software Group


Rate of Garbage Collectionoptavgpause gencon

Gencon could handle a higher rate of garbage collectionCompleting the test quicker

Gencon had a smaller percentage of time in garbage collection Gencon had a shorter maximum pause time

IBM Software Group


Rate of Garbage Collection

Gencon provides less frequent long Garbage Collection cycles Gencon provides a shorter longest Garbage Collection cycle

IBM Software Group


Large Object Allocations (Very) Large Object allocations affects the gencon GC policy

If object is larger than the Nursery size, the object is immediately tenured Removes the benefit of generational heaps Still has the additional overhead of running generational

If object is fits in the nursery but fills it, frequent nursery collects will have to occur Too frequent nursery collects mean objects are likely to survive and need copying Copying is an expensive process

If (Very) Large Objects are being used, a sufficiently large enough nursery is required

IBM Software Group


Sizing the Java Heap

IBM Software Group


Sizing the Java Heap Maximum possible Java heap sizes

The correct Java heap size

Fixed heap sizes vs. Variable heap sizes

Heap Sizing for Generational GC

IBM Software Group


Maximum Possible Heap Size 32 bit Java processes have maximum possible heap size

Varies according to the OS and platform used Determined by the process memory layout

64 bit processes do not have this limit Limit exists, but is so large it can be effectively ignored Addressability usually between 2^44 and 2^64 Which is 16+ TeraBytes

IBM Software Group


An Operating System process like any other application: Subject to OS and architecture restrictions 32bit architecture has an addressable range of:

2^32 which is 0x00000000 0xFFFFFFFF which is 4GB

Not all addressable space is available to the application The operating system needs memory for:

The kernel The runtime support libraries

Varies according to Operating System How much memory is needed and where that memory is located

0 GB 4 GB

0x0 0xFFFFFFFF

2 GB

0x80000000

1 GB 3 GB

0x40000000 0xC0000000

Java Process Memory Layout

IBM Software Group


Memory Available to the Java Process On Windows:

On AIX:

0 GB 4 GB

0x0 0xFFFFFFFF

2 GB

0x80000000

1 GB 3 GB

0x40000000 0xC0000000

0 GB 4 GB

0x0 0xFFFFFFFF

2 GB

0x80000000

1 GB 3 GB

0x40000000 0xC0000000

Operating System Space

Libraries

Kernel Libraries

IBM Software Group


Java Process Restrictions Not all Java Process space is available to the Java application

The Java Runtime needs memory for: The Java Virtual Machine Backing resources for some Java objects

This memory area as well as some other allocations, is part of the Native Heap

Memory not allocated to the Java Heap is available to the native heap

Available memory space Java heap = native heap

Effectively, the Java process maintains two memory pools

IBM Software Group


The Native Heap Allocated using malloc() and therefore subject to memory

management by the OS

Used for Virtual Machine resources, eg: Execution engine Class Loader Garbage Collector infrastructure

Used to underpin Java objects: Threads, Classes, AWT objects, ZipFiles

Used for allocations by JNI code

IBM Software Group


Native Heap available to Application On Windows

On AIX (1.4.2 with small heaps)

0 GB 4 GB

0x0

2 GB

0x80000000

1 GB 3 GB

0x40000000 0xC0000000

Operating System Space

Libraries

Java Heap

0xFFFFFFFF

0 GB 4 GB

0x0 0xFFFFFFFF

2 GB

0x80000000

1 GB 3 GB

0x40000000 0xC0000000

Kernel LibrariesJava Heap

VM Resources

VM Resources

Native Heap

Native Heap

IBM Software Group


Layout with Large Java Heaps on AIX Applies to heaps > 1GB in size and Java 5.0

Java heap becomes allocated using mmap()

Segments used start at 0xC and work downwards

understanding memory layout important for monitoring

0 GB 4 GB

0x0 0xFFFFFFFF

2 GB

0x80000000

1 GB 3 GB

0x40000000 0xC0000000

Kernel Libraries

VM Resources

0x7

Native Heap0xD0x3

Java Heap

IBM Software Group


Linux:

z/OS:

Memory Layout for Linux

0 GB 4 GB

0x0

2 GB

0x80000000

1 GB 3 GB

0x40000000 0xC0000000

KernelJava Heap

0xFFFFFFFFVM Resources

Native Heap

PAGE_OFFSETTASK_SIZE

0 GB

0x0

2 GB

0x7FFFFFFF

1 GB

0x40000000

Java Heap

VM Resources

IBM Software Group


Theoretical and Advised Max Heap Sizes

The larger the Java heap, the more constrained the native heap Advised limits to prevent native heap from becoming overly

restricted, leading to OutOfMemoryErrors

Exceeding advised limits possible, but should be done only when native heap usage is understood

Native heap usage can be measured using OS tools:Svmon (AIX), PerfMon (Windows), RMF (zOS) etc

1.8GB1.8GB/3GB

2.5GB3 GBHugemem Kernel

Advised MaximumMaximum PossibleAdditional OptionsPlatform

1.3GB1.7GBz/OS

1.5GB1.8GBWindows

1.5GB2 GBLinux2.5GB3.25 GBautomaticAIX

IBM Software Group


Moving to 64bit Moving to 64bit remove the Java heap size limit

However, ability to use more memory is not free 64bit applications perform slower

More data has to be manipulated Cache performance is reduced

64bit applications require more memory Java Object references are larger Internal pointers are larger

Major improvements to this in Java 6.0 due to compressed pointers

IBM Software Group


The correct Java heap size GC will adapt heap size to keep occupancy between 40% and 70%

Heap occupancy over 70% causes frequent GC cycles Which generally means reduced performance

Heap occupancy below 40% means infrequent GC cycles, but cycles longer than they needs to be

Which means longer pause times that necessary Which generally means reduced performance

The maximum heap size setting should therefore be 43% larger than the maximum occupancy of the applicationMaximum occupancy + 43% means occupancy at 70% of total heap

Eg. For 70MB occupancy, 100MB Max heap required, which is 70MB +43% of 70MB

IBM Software Group


Long Garbage Collection Cycles

Too Frequent Garbage Collection

The correct Java heap sizeM

e

m

o

r

y

Time

70%

40%

Heap Occupancy

Heap Size

IBM Software Group


Fixed heap sizes vs. Variable heap sizes Should the heap size be fixed?

i.e. Minimum heap size (-Xms) = Maximum heap size (-Xmx)?

Each option has advantages and disadvantages As for most performance tuning, you must select which is right for the particular

application

Variable Heap Sizes GC will adapt heap size to keep occupancy between 40% and 70%

Expands and Shrinks the Java heap Allows for scenario where usage varies over time

Where variations would take usage outside of the 40-70% window

Fixed Heap Sizes Does not expand or shrink the Java heap

IBM Software Group


Heap Expansion and Shrinkage Act of heap expansion and shrinkage is relatively cheap

However, a compaction of the Java heap is sometimes required Expansion: for some expansions, GC may have already

compacted to try to allocate the object before expansion

Shrinkage: GC may need to compact to move objects from the area of the heap being shrunk

Whilst expansion and shrinkage optimizes heap occupancy, it (usually) does so at the cost of compaction cycles

IBM Software Group


Conditions for Heap Expansion Not enough free space available for object allocation after GC has

complete Occurs after a compaction cycle Typically occurs where there is fragmentation or during rapid

occupancy growth (i.e., application startup)

Heap occupancy is over 70% Compaction unlikely

More than 13% of time is spent in GC Compaction unlikely

IBM Software Group


Conditions for Heap Shrinkage Heap occupancy is under 40%

And the following is not true: Heap has been recently expanded (last 3 cycles) GC is a result of a System.GC() call

Compaction occurs if: An object exists in the area being shrunk GC did not shrink on the previous cycle

Compaction is therefore likely to occur

IBM Software Group


Introduction to Xmaxf and Xminf The Xmaxf and Xminf settings control the 40% and 70% occupancy

bounds -Xmaxf: the maximum heap space free before shrinkage (default is 0.6

for 40%) -Xminf: the minimum heap space before expansion (default is 0.3 for

70%)

Can be used to move optimum occupancy window if required by the application

eg. Lower heap utilization required for more infrequent GC cycles

Can be used to prevent shrinkage -Xmaxf1.0 would mean shrinkage only when heap is 100% free Would completely remove shrinkage capability

IBM Software Group


Introduction to Xmaxe and -Xmine The Xmaxe and Xmine settings control the bounds of the size of

each expansion step -Xmaxe: the maximum amount of memory to add to the heap

size in the case of expansion (default is unlimited) -Xmine: the minimum amount of memory to add to the heap

size in the case of expansion (default is 1MB)

Can be used to reduce/prevent compaction due to expansion Reduce expansions by setting a large -Xmine

IBM Software Group


GC Managed Heap Sizing

Long Garbage Collection Cycles

To Frequent Garbage Collection

M

e

m

o

r

y

Time

-Xminf

-Xmaxf

Heap Occupancy

Heap Size

Expansion (>= -Xmine)

IBM Software Group


Fixed or Variable?? Again, dependent on application

For flat memory usage, use fixed For widely varying memory usage, consider variable

Variable provides more flexibility and ability to avoid OutOfMemoryErrors

Some of the disadvantages can be avoided: -Xms set to lowest steady state memory usage prevents

expansion at startup -Xmaxf1 will remove shrinkage -Xminf can be used to prevent compaction before

expansion -Xmine can be used to reduce expansions

IBM Software Group


Nursery Tenured

Options Are: Fix both nursery and tenured space

Allow them to expand/contract

General Advice: Fix the new space sizeSize the tenured space as you would for a flat heap

Heap Sizing for Generational GC

IBM Software Group


Sizing the Nursery Copying from Allocate to Survivor or to Tenured space is expensive

Physical data is copied (similar to compaction with is also expensive Ideally survival rates should be as low as possible

Less data needs to be copied Less tenured/global collects that will occur

The larger the nursery: the greater the time between collects the less objects that should survive However, the longer a copy can potentially take

Recommendation is to have a nursery as large as possible Whilst not being so large that nursery collect times affect the

application responsiveness

IBM Software Group


Summary GC Policy should be chosen according to application scenario

Java heap should ideally be sized for between 40 and 70% occupancy

Min=Max heap size is right for some applications, but not for others

IBM Software Group


Additional WebSphere Product Resources Discover the latest trends in WebSphere Technology and implementation, participate in

technically-focused briefings, webcasts and podcasts at: http://www.ibm.com/developerworks/websphere/community/

Learn about other upcoming webcasts, conferences and events: http://www.ibm.com/software/websphere/events_1.html

Join the Global WebSphere User Group Community: http://www.websphere.org

Access key product show-me demos and tutorials by visiting IBM Education Assistant: http://www.ibm.com/software/info/education/assistant

View a Flash replay with step-by-step instructions for using the Electronic Service Request (ESR) tool for submitting problems electronically: http://www.ibm.com/software/websphere/support/d2w.html

Sign up to receive weekly technical My support emails: http://www.ibm.com/software/support/einfo.html

IBM Software Group


Additional Java Product Resources Obtain Java Documentation:

https://www.ibm.com/developerworks/java/jdk/docs.html

Download the IBM Java SDKs:https://www.ibm.com/developerworks/java/jdk/index.html

Find and download Java tooling:http://www.ibm.com/software/websphere/events_1.html

Troubleshoot Java with the IBM Guided Activity Assistant:http://www-01.ibm.com/support/docview.wss?uid=swg27010135

Troubleshoot Java with the Guided Troubleshooting InfoCenterhttp://publib.boulder.ibm.com/infocenter/javasdk/tools/topic/com.ibm.java.doc.tools.welcome/tools/welcome/welcome.html

Discuss IBM Java:http://www.ibm.com/developerworks/forums/forum.jspa?forumID=367

IBM Software Group


Questions and Answers

IBM Java Garbage Collection Tuning

Documents

Transcript of IBM Java Garbage Collection Tuning