CSE 436—Software Development Models Ron K. Cytron cytron/cse436/ 16 October 2006.
Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi...
-
Upload
jennifer-ramsey -
Category
Documents
-
view
213 -
download
0
Transcript of Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi...
![Page 1: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/1.jpg)
Getting Real, Getting Dirty(without getting real dirty)
Ron K. CytronJoint work with Krishna Kavi
University of Alabama at Huntsville
April 2001
Dante Cannarozzi, Sharath Cholleti, Morgan Deters, Steve Donahue
Mark Franklin, Matt Hampton, Michael Henrichs, Nicholas Leidenfrost, Jonathan Nye, Michael Plezbert, Conrad Warmbold
Center for Distributed Object ComputingDepartment of Computer Science
Washington University
Funded by the National Science Foundation under grant 0081214
Funded by DARPA under contract F33615-00-C-1697
![Page 2: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/2.jpg)
Outline
• Motivation• Allocation• Collection• Conclusion
![Page 3: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/3.jpg)
Traditional architecture and object-oriented programs
• Caches are still biased toward Fortran-like behavior
• CPU is still responsible for storage management
• Object-management activity invalidates caches– GC disruptive– Compaction
CPU + cache
M
M M
M
L2 cache
![Page 4: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/4.jpg)
An OO-biased design using IRAMs(with Krishna Kavi)
• CPU and cache stay the same, off-the-shelf
• Memory system redesigned to support OO programs
CPU + cache
M
M M
M
L2 cache
Logic
IRAM
![Page 5: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/5.jpg)
IRAM interface
M
M M
M
Logic
IRAM
CPU + cache
L2 cache
malloc
addr
Stable address for an object allows better cache behavior
Object can be relocated within IRAM, but its address to the CPU is constant
![Page 6: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/6.jpg)
IRAM interface
M
M M
M
Logic
IRAM
CPU + cache
L2 cache
putfield/getfield
value
Object referencing—tracked inside IRAM–-supports garbage collection
![Page 7: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/7.jpg)
IRAM interface
M
M M
M
Logic
IRAM
CPU + cache
L2 cache
gccompactprefetch
Goal: relegate storage-management functions to IRAM
![Page 8: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/8.jpg)
Macro accesses
M
M M
M
Logic
IRAM
CPU + cache
L2 cache
p.getLeft().getNext()
*(*(p+12)+32)
Observe: code sequences contain common gestures (superoperators)
![Page 9: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/9.jpg)
Gesture abstraction
M
M M
M
Logic
IRAM
CPU + cache
L2 cache
p.getLeft().getNext()
*(*(p+12)+32)
M143(x):
*(*(x+12)+32)
Goal: decrease traffic between CPU and storage
![Page 10: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/10.jpg)
Gesture application
M
M M
M
Logic
IRAM
CPU + cache
L2 cache
Macro 143 (p)
M143(x):
*(*(x+12)+32)p.getLeft().getNext()
![Page 11: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/11.jpg)
Gesture application
M
M M
M
Logic
IRAM
CPU + cache
L2 cache
Macro 143 (p)
M143(x):
*(*(x+12)+32)
p.getLeft().getNext()
p.getLeft().getNext()
![Page 12: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/12.jpg)
Automatic prefetchingGoal: decrease traffic between CPU and storage
M
M M
M
Logic
IRAM
CPU + cache
L2 cache
Fetch p
p
![Page 13: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/13.jpg)
p.getLeft().getNext()
Automatic prefetchingGoal: decrease traffic between CPU and storage
M
M M
M
Logic
IRAM
CPU + cache
L2 cache
Fetch p
p
![Page 14: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/14.jpg)
Challenges
• Algorithmic– Bounded-time methods for allocation and
collection– Good average performance as well
• Architectural– Lean interface between the CPU and IRAM– Efficient realization
![Page 15: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/15.jpg)
Storage Allocation (Real Time)
• Not necessarily fast• Necessarily predictable• Able to satisfy any reasonable request
– Developer should know “maxlive” characteristics of the application
– This is true for non-embedded systems as well
![Page 16: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/16.jpg)
How much storage?
• curlive—the number of objects live at a point in time
• curspace—the number of bytes live at a point in time
Handles
Object Space
![Page 17: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/17.jpg)
Objects concurrently live
![Page 18: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/18.jpg)
How much object space?
![Page 19: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/19.jpg)
Storage Allocation—Free List
• Linked list of free blocks
• Search for desired fit
• Worst case O(n) for n blocks in the list
![Page 20: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/20.jpg)
Worst-case free-list behavior
• The longer the free-list, the more pronounced the effect
• No a priori bound on how much worse the list-based scheme could get
• Average performance similar
Slowdown of List-Based Allocator
3.15
72.84
0
20
40
60
80
180 3000
Number of objects allocated
![Page 21: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/21.jpg)
Knuth’s Buddy System
• Free-list segregated by size
• All requests rounded up to a power of 2
256
128
64
32
16
8
4
2
1
![Page 22: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/22.jpg)
Knuth’s Buddy System (1)
256
128
64
32
16
8
4
2
1
• Begin with one large block
• Suppose we want a block of size 16
![Page 23: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/23.jpg)
Knuth’s Buddy System (2)
• Begin with one large block
• Recursively subdivide
256
128
64
32
16
8
4
2
1
![Page 24: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/24.jpg)
Knuth’s Buddy System (3)
256
128
64
32
16
8
4
2
1
• Begin with one large block
• Recursively subdivide
![Page 25: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/25.jpg)
Knuth’s Buddy System (4)
256
128
64
32
16
8
4
2
1
• Begin with one large block
• Recursively subdivide
![Page 26: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/26.jpg)
Knuth’s Buddy System (5)
256
128
64
32
16
8
4
2
1
• Begin with one large block
• Yield 2 blocks size 16
![Page 27: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/27.jpg)
Knuth’s Buddy System (6)
• One of those blocks can be given to the program
256
128
64
32
16
8
4
2
1
• Begin with one large block
• Yield: 2 blocks size 16
![Page 28: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/28.jpg)
Worst-case free-list behavior
• The longer the free-list, the more pronounced the effect
• No a priori bound on how much worse the list-based scheme could get
• Average performance similar
Speedup of Buddy over List
3.15
72.84
0.89 0.910
1020304050607080
180 3000
Number of objects allocated
Worst
Avg
![Page 29: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/29.jpg)
Spec Benchmark Results
Speedup of Buddy over List
0
0.2
0.4
0.6
0.8
1
1.2
![Page 30: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/30.jpg)
Buddy System
• If a block can be found, it can be found in log(N), where N is the size of the heap
• The application cannot make that worse
![Page 31: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/31.jpg)
Defragmentation
• To keep up with the diversity of requested block sizes, an allocator may have to reorganize smaller blocks into larger ones
![Page 32: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/32.jpg)
Defragmentation—Free List
• Free-list permutes adjacent blocks
• Storage becomes fragmented, with many small blocks and no large ones
Blocks in memory
Free list
![Page 33: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/33.jpg)
Defragmentation—Free List
• Free-list permutes adjacent blocks
• Two issues:
Blocks in memory
Free list
– Join adjacent blocks
![Page 34: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/34.jpg)
Defragmentation—Free List
• Free-list permutes adjacent blocks
• Two issues:
Blocks in memory
Free list
– Reorganize holes (move live storage)
– Join adjacent blocks
![Page 35: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/35.jpg)
Defragmentation—Free List
• Free-list permutes adjacent blocks
• Two issues:
Blocks in memory
Free list
– Reorganize holes
• Organization by address can help [Kavi]
– Join adjacent blocks
![Page 36: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/36.jpg)
• The blocks resulting from subdivision are viewed as “buddies”
• Their address differs by exactly one bit
• The address of a block of size 2 differs with its buddy’s address at bit n
Buddies—joining adjacent blocks
…0…
…1…
n
![Page 37: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/37.jpg)
Knuth’s Buddy System (6)
256
128
64
32
16
8
4
2
1
![Page 38: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/38.jpg)
Knuth’s Buddy System (5)
256
128
64
32
16
8
4
2
1
• When a block becomes free, it tries to rejoin its buddy
• A bit in its buddy tells whether the buddy is free
• If so, they glue together and make a block twice as big
![Page 39: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/39.jpg)
Knuth’s Buddy System (4)
256
128
64
32
16
8
4
2
1
![Page 40: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/40.jpg)
Knuth’s Buddy System (3)
256
128
64
32
16
8
4
2
1
![Page 41: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/41.jpg)
Knuth’s Buddy System (2)
256
128
64
32
16
8
4
2
1
![Page 42: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/42.jpg)
Knuth’s Buddy System (1)
256
128
64
32
16
8
4
2
1
![Page 43: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/43.jpg)
Two problems
• Oscillation—Buddy looks like it may split, glue, split, glue—isn’t this wasted effort?
• Fragmentation—What happens when Buddy can’t glue but has space it would like to combine?
![Page 44: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/44.jpg)
256
128
64
32
16
8
4
2
1
Buddy—oscillation
![Page 45: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/45.jpg)
256
128
64
32
16
8
4
2
1
Buddy—oscillation
![Page 46: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/46.jpg)
256
128
64
32
16
8
4
2
1
Buddy—oscillation
![Page 47: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/47.jpg)
256
128
64
32
16
8
4
2
1
Buddy—oscillation
![Page 48: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/48.jpg)
256
128
64
32
16
8
4
2
1
Buddy—oscillation
![Page 49: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/49.jpg)
256
128
64
32
16
8
4
2
1
Buddy—oscillation
![Page 50: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/50.jpg)
Problem is lack of hysteresis
• Some programs allocate objects which are almost immediately deallocated.– Continuous, incremental approaches to
garbage collection only make this worse!• Oscillation is expensive: blocks are glued
only to be quickly subdivided again
![Page 51: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/51.jpg)
Estranged Buddy System
• Variant of Knuth’s idea• When deallocated, blocks are not eager to
rejoin their buddies• Evidence of value [Kaufman, TOPLAS ’84]• Slight improvement on spec benchmarks• Algorithmic improvement over Kaufman
![Page 52: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/52.jpg)
Buddy-Busy and Buddy-Free
2
Blocks whose buddies are busy
Blocks whose buddies are free
k
![Page 53: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/53.jpg)
Estranged Buddy—Allocation
Allocation heuristic
1. Buddy-busy
2. Buddy-free
3. Glue one level below, buddy-free
4. Search up (Knuth)
5. Glue below
Buddy-busy
Buddy-free
2k
2k+1
2k-1
![Page 54: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/54.jpg)
How well does Estranged Buddy do?(contrived example)
Size-8150 objects
0
2
4
6
8
10
200 300 400 500 600 700 800
Knuth
Estranged
![Page 55: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/55.jpg)
Estranged Buddy on SpecSpeedup of Estranged Buddy over Knuth
00.20.40.60.8
11.21.41.6
com
pres
sjes
s
raytr
ace db
javac
mpe
gaud
iom
trtjac
k
![Page 56: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/56.jpg)
Recall: two problems
• Oscillation—Buddy looks like it may split, glue, split, glue—isn’t this wasted effort?– Typically not, but can be
• Fragmentation—What happens when Buddy can’t glue but has space it would like to combine?
![Page 57: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/57.jpg)
Buddy System—Fragmentation
• Internal fragmentation from rounding-up of requests to powers of two
• Not really a concern these days• Assume a program can run in maxlive
bytes• How much storage needed so Buddy
never has to defragment?• What is a good algorithm for Buddy
defragmentation?
![Page 58: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/58.jpg)
Buddy Configurations
8
4
2
1
Allocated
Free
![Page 59: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/59.jpg)
Buddy Configurations
8
4
2
1
Allocated
Free
![Page 60: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/60.jpg)
Heap FullAllocated
Free8
4
2
1
![Page 61: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/61.jpg)
Buddy can’t allocate size-2 blockAllocated
Free8
4
2
1
![Page 62: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/62.jpg)
How Big a Heap for Non-Blocking Buddy (M = maxlive)?
• Easy bound: M log M• Better bound: M k,
where k is the number of distinct sizes to be allocated
• Sounds like a good bound, but it isn’t
• Defragmentation may be necessary
256
128
64
32
16
8
4
2
1
M bytes each level
![Page 63: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/63.jpg)
Managing object relocation
• Every object has a stable handle, whose address does not change
• Every handle points to its object’s current location
• All references to objects are indirect, through a handle
![Page 64: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/64.jpg)
Buddy Defragmentation
• When stuck at level k– No blocks free above
level k– No glueable blocks
free below level k– Assume maxlive still
suffices• Example: k=6, size 64
not available
256
128
64
32
16
8
4
2
1
![Page 65: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/65.jpg)
Defragmentation Algorithm
32
64
16
![Page 66: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/66.jpg)
Defragmentation Algorithm
32
64
16
swap
![Page 67: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/67.jpg)
Defragmentation Algorithm
32
64
16
glue
![Page 68: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/68.jpg)
Defragmentation Algorithm
32
64
16
![Page 69: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/69.jpg)
Defragmentation Algorithm
• Recursively visit below to develop two buddies that can be glued
• Analogous to the recursive allocation algorithm
• Still, choices to be made….studies underway
![Page 70: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/70.jpg)
Need 4 bytes
8
4
2
1
Allocated
Free
Move 3 bytes? Move 1 byte?
![Page 71: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/71.jpg)
Recall: two problems• Oscillation—Buddy looks like it may split,
glue, split, glue—isn’t this wasted effort?– Typically not, but can be
• Fragmentation—What happens when Buddy can’t glue but has space it would like to combine?– New algorithm to defragment Buddy– Selective approach—should beat List– Optimizations needed
![Page 72: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/72.jpg)
Towards an IRAM implementation
• VHDL of Buddy System complete– DRAM clocked at 150 MHz– 10 cycles per DRAM access
• Need 7 accesses per level to split blocks• For 16Mbyte heap—24 levels
– 1680 cycles worst case: 11us– 168x slower than a read
• Can we do better?
![Page 73: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/73.jpg)
Two tricks
• Find a suitable free-block quickly• Return its address quickly
![Page 74: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/74.jpg)
Finding a suitable free block
• No space at 16, but 16 points to the level above it that has a block to offer
256
128
64
32
16
8
4
2
1
![Page 75: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/75.jpg)
Finding a suitable free block
• Every level points to the level above it that has a block to offer
• Pointers are maintained using Tarjan’s path-compression
• Locating pointers are not stored in DRAM
256
128
64
32
16
8
4
2
1
![Page 76: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/76.jpg)
Alternative free-block finder
• Path-compression may be too complex for hardware
• Instead, track the largest available free block
256
128
64
32
16
8
4
2
1
![Page 77: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/77.jpg)
Alternative free-block finder
• Path-compression may be too complex for hardware
• Instead, track the largest available free block
• Tends to break up large blocks and favor formation of small ones
256
128
64
32
16
8
4
2
1
![Page 78: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/78.jpg)
Fast return for malloc
• Want 16 bytes• Zip to the 64 display• WLOG we return the
first part of that block immediately to the requestor
64
32
16
![Page 79: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/79.jpg)
• Want 16 bytes• Zip to the 64 display• WLOG we return the
first part of that block immediately to the requestor
• Adjustment to the structures happens in parallel with the return
64
32
16
Fast return for malloc
![Page 80: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/80.jpg)
Improved IRAM allocator• ~10 cycles fast return• ~1000 cycles to recover, worst case• Is this good enough?
– Compare software implementation• ~1000 cycles worst case• ~600 cycles average on spec benchmarks
– Hardware can be much faster– Depends on recover time
![Page 81: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/81.jpg)
Do programs allow us to recover?
• Run of jack—JVM instructions between requests
• 56% of requests separated by at least 100 JVM instructions• Assume 10x expansion, JVM to native code• For the 56%, we return in 10 cycles• Code motion might improve others
Min Median Max
3 181 174053
![Page 82: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/82.jpg)
Garbage Collection
• While allocators are needed for most modern languages, garbage collection is not universally accepted
• Generational and incremental approaches help most applications
• Embedded and real-time need assurances of bounded behavior
![Page 83: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/83.jpg)
Why not garbage collect?
• Some programmers want ultimate control over storage
• Real-Time applications need bounded-time overhead– RT Java spec relegates allocation and
collection to user control– Isn’t this a step back from Java?
![Page 84: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/84.jpg)
Marking Phase—the problem
• To discover the dead objects, we use calculatus eliminatus– Find live objects– All others are dead
![Page 85: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/85.jpg)
Marking Phase—the problem
• To discover the dead objects, we– Find live objects
stack heap
• Pointers from the stack to the heap make objects live
![Page 86: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/86.jpg)
Marking Phase—the problem
• To discover the dead objects, we– Find live objects
• Pointers from the stack to the heap make objects live
• These objects make other objects live
stack heap
![Page 87: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/87.jpg)
Marking Phase—the problem
• To discover the dead objects, we– Find live objects– Sweep all others
away as dead
stack heap
![Page 88: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/88.jpg)
Marking Phase—the problem
• To discover the dead objects, we– Find live objects– Sweep all others
away as dead– Perhaps compact the
heap
stack heap
![Page 89: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/89.jpg)
Problems with “mark” phase
• Takes an unbounded amount of time• Can limit it using generational collection
but then it’s not clear what will get collected
• We seek an approach that spends a constant amount of time per program operation and collects objects continuously
![Page 90: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/90.jpg)
Two Approaches
• Variation on reference counting• Contaminated garbage collection [PLDI00]
![Page 91: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/91.jpg)
Reference Counting
• An integer is associated with every object, summing– Stack references– Heap references
• Objects with reference count of zero are dead
stack heap
1
2
2
1
11
0
00
0
![Page 92: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/92.jpg)
Problems with Reference Counting
• Standard problem is that objects in cycles
stack heap
1
2
1
1
11
0
00
0
![Page 93: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/93.jpg)
• Standard problem is that objects in cycles (and those touched by such objects) cannot be collected
stack heap
1
2
1
1
11
0
00
0
Problems with Reference Counting
![Page 94: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/94.jpg)
• Standard problem is that objects in cycles (and those touched by such objects) cannot be collected
• Contaminated gc will collect such objects
• Overhead of counting can be high
• Untyped stack complicates things
stack heap
1
2
1
1
11
0
00
0
Problems with Reference Counting
![Page 95: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/95.jpg)
The Untyped Stack
• The stack is a collection of untyped cells
• In JVM, safety is verified at class-load time
• No need to tag stack locations with what they contain
• Leads to imprecision in all gc methods
stack heap
Heap
Address?
![Page 96: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/96.jpg)
Idea
• When a stack frame pops, all of its cells are dead
• Don’t worry about tracking cell pointers• Instead, associate an object with the last
stack frame that can reference the object
![Page 97: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/97.jpg)
Reference Counting Approach
• s is zero or one, indicating none or at least one stack reference to the object
• h precisely reflects the number of heap references to the object
• If s+h=0 object is dead
s h
![Page 98: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/98.jpg)
Our treatment of stack activity
• Object is associated with the last-to-be-popped frame that can reference the object
stack
1 0
![Page 99: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/99.jpg)
Our treatment of stack activity
• Object is associated with the last-to-be-popped frame that can reference the object
• When that frame pops– If object is returned,
the receiving frame owns the object
stack
1 0
![Page 100: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/100.jpg)
Our treatment of stack activity
• Object is associated with the last-to-be-popped frame that can reference the object
• When that frame pops– Otherwise the object
is dead
stack
0 0
![Page 101: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/101.jpg)
Our reference-counting implementation
• The objects associated with the frame are linked together
stack heap
0
1
2 0
1
frame
1
![Page 102: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/102.jpg)
Our reference-counting implementation
• The objects associated with the frame are linked together
• When a stack frame pops, all of its cells no longer can point at anything
stack heap
0
1
2 0
1
frame
1
![Page 103: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/103.jpg)
Our reference-counting implementation
• The objects associated with the frame are linked together
• When a stack frame pops, all of its cells no longer can point at anything
• This object is unlinked, but still thought to be live
stack heap
0
1
2 0
1
frame
1
![Page 104: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/104.jpg)
Our reference-counting implementation
• The objects associated with the frame are linked together
• When a stack frame pops, all of its cells no longer can point at anything
• This object is dead and is collected
stack heap
0
1
2 0
1
frame
1
![Page 105: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/105.jpg)
Our reference-counting implementation
• The objects associated with the frame are linked together
• When a stack frame pops, all of its cells no longer can point at anything
• This object is also dead
stack heap
0
2 0
1
frame
1
![Page 106: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/106.jpg)
Our reference-counting implementation
• The objects associated with the frame are linked together
• When a stack frame pops, all of its cells no longer can point at anything
• This object is still live
stack heap
1 0
1
frame
1
![Page 107: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/107.jpg)
Our reference-counting implementation
• The objects associated with the frame are linked together
• When a stack frame pops, all of its cells no longer can point at anything
• Now the frame is gone
stack heap
1 0
1frame
1
![Page 108: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/108.jpg)
Our reference-counting implementation
• The objects associated with the frame are linked together
• When a stack frame pops, all of its cells no longer can point at anything
• This object was linked to its frame all along
stack heap
1 0
1frame
1
![Page 109: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/109.jpg)
Our reference-counting implementation
• The objects associated with the frame are linked together
• When a stack frame pops, all of its cells no longer can point at anything
• This object was linked to its frame all along
• When heap count becomes zero, the object is scheduled for deletion in that frame
stack heap
1 0
1frame
0
![Page 110: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/110.jpg)
Our reference-counting implementation
• The objects associated with the frame are linked together
• When a stack frame pops, all of its cells no longer can point at anything
• This object was linked to its frame all along
• When heap count becomes zero, the object is scheduled for deletion in that frame
• When frame pops, all are dead
stack heap
1 0
1frame
0
![Page 111: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/111.jpg)
Our reference-counting implementation
• The objects associated with the frame are linked together
• When a stack frame pops, all of its cells no longer can point at anything
• This object was linked to its frame all along
• When heap count becomes zero, the object is scheduled for deletion in that frame
• When frame pops, all are dead
stack heap
0 0
1frame
0
![Page 112: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/112.jpg)
Our reference-counting implementation
• The objects associated with the frame are linked together
• When a stack frame pops, all of its cells no longer can point at anything
• This object was linked to its frame all along
• When heap count becomes zero, the object is scheduled for deletion in that frame
• When frame pops, all are dead
stack heap
0 0
0frame
0
![Page 113: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/113.jpg)
Our reference-counting implementation
• The objects associated with the frame are linked together
• When a stack frame pops, all of its cells no longer can point at anything
• This object was linked to its frame all along
• When heap count becomes zero, the object is scheduled for deletion in that frame
• When frame pops, all are dead
stack heap
0 0
0frame
0
![Page 114: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/114.jpg)
Reference Counting
• Predictable, constant overhead for each JVM instruction– putfield decreases count at old pointed-to
object, increases count at new pointed-to object
– areturn associates object with stack frame if not already associated below
• How well does it do? We shall see!
![Page 115: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/115.jpg)
Contaminated Garbage Collection
• Need to collect objects involved in reference cycles without resorting to marking live objects
• Idea– Associate each object with a stack frame
such that when that frame returns, the object is known to be dead
– Like escape analysis, but dynamic
![Page 116: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/116.jpg)
Contaminated garbage collection
• Initially each object is associated with the frame in which it is instantiated
stack
A
C
B
D
E
![Page 117: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/117.jpg)
Contaminated garbage collection
• Initially each object is associated with the frame in which it is instantiated
• When B references A, A becomes as live as B
stack
A
C
B
D
E
![Page 118: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/118.jpg)
Contaminated garbage collection
• Initially each object is associated with the frame in which it is instantiated
• Now A, B, and C are as live as C
stack
A
C
B
D
E
![Page 119: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/119.jpg)
Contaminated garbage collection
• Initially each object is associated with the frame in which it is instantiated
• Even though D is less live than C, it gets contaminated
• Should something reference D later, all will be affected
stack
A
C
B
D
E
![Page 120: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/120.jpg)
Contaminated garbage collection
• Initially each object is associated with the frame in which it is instantiated
• Static finger of life• Now all objects
appear to live forever
stack
A
C
B
D
E
![Page 121: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/121.jpg)
Contaminated garbage collection
• Initially each object is associated with the frame in which it is instantiated
• Static finger of life• Now all objects
appear to live forever• Even if E points away!
stack
A
C
B
D
E
![Page 122: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/122.jpg)
Contaminated garbage collection
• Every object is a member of an equilive set– All objects in a set are scheduled for
deallocation at the same time– Sets are maintained using Tarjan’s disjoint
union/find algorithm• Nearly constant amount of overhead per
operation
![Page 123: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/123.jpg)
Contaminated GC
• Each equilive set is associated with a frame
stack
![Page 124: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/124.jpg)
Contaminated GC
• Each equilive set is associated with a frame
• Suppose an object in one set references an object in another set (in either direction)
stack
![Page 125: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/125.jpg)
Contaminated GC
• Each equilive set is associated with a frame
• Suppose an object in one set references an object in another set (in either direction)
• Contamination!• The sets are unioned
stack
![Page 126: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/126.jpg)
Contaminated GC
• Each equilive set is associated with a frame
• When a frame pops, objects associated with it are dead
stack
![Page 127: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/127.jpg)
Contaminated GC
• Each equilive set is associated with a frame
• When a frame pops, objects associated with it are dead
stack
![Page 128: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/128.jpg)
Contaminated GC
• Each equilive set is associated with a frame
• When a frame pops, objects associated with it are dead
stack
![Page 129: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/129.jpg)
Summary of methods
Reference counting• Can’t handle cycles• Handles pointing at
and then away
Contaminated GC• Tolerates cycles• Can’t track pointing
and then pointing away
Both techniques:
•Incur cost at putfield, areturn
•(Nearly) constant overhead per operation
![Page 130: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/130.jpg)
Implementation details
• SUN JDK 1.1 interpreter version• Many subtle places where references are
generated: String.intern(), ldc instruction, class loader, JNI
• Each gc method took about 3 months to implement
• Can run either method or both in concert• Fairly optimized, more is possible
![Page 131: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/131.jpg)
Size 1 Absolute
0
50000
100000
150000
200000
250000
300000
350000
400000
450000
compress jess raytrace db javac mpegaudio mtrt jack checkit
None
RefCount
Both
CGC
Spec benchmark effectiveness
![Page 132: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/132.jpg)
size 10 Absolute
0
100000
200000
300000
400000
500000
600000
700000
800000
900000
compress jess raytrace db javac mpegaudio mtrt jack checkit
None
RefCount
Both
CGC
Spec benchmark effectiveness
![Page 133: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/133.jpg)
raytrace
1
2
3
4
5
6--10
>10
javac
1
2
3
4
5
6--10
>10
mpegaudio
1
2
3
4
5
6--10
>10
jack
1
2
3
4
5
6--10
>10
db
1
2
3
4
5
6--10
>10
jess
1
2
3
4
5
6--10
>10
Exactness of Equilive Sets
![Page 134: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/134.jpg)
jess
0
1
2
3
4
5
>5
raytrace
0
1
2
3
4
5
>5
db
0
1
2
3
4
5
>5
javac
0
1
2
3
4
5
>5
mpegaudio
0
1
2
3
4
5
>5
jack
0
1
2
3
4
5
>5
Distance to die in frames
![Page 135: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/135.jpg)
Speed of CGC
1.05
1.24 1.21.14 1.17
1.04
1.21.1
0
0.2
0.4
0.6
0.8
1
1.2
1.4
com
pres
sjes
s
raytr
ace db
javac
mpe
gaud
iom
trtjac
k
over JDK bigheapover JDK sameheap
![Page 136: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/136.jpg)
Speedups of Mark-Free Approaches
0.000.200.400.600.801.001.201.40
com
pres
sjes
s
raytr
ace db
javac
mpe
gaud
iom
trtjac
k
RefCount
CGC
![Page 137: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/137.jpg)
Future Plans
• VHDL simulation of more efficient buddy allocator
• VHDL simulation of garbage collection methods
• Better buddy defragmentation• Experiment with informed allocation• Comparison/integration with other IRAM-
based methods (with Krishna Kavi)
![Page 138: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/138.jpg)
Informed Storage Management
• Evidence that programs allocate many objects of the same size
![Page 139: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/139.jpg)
Benchmark jack20% fragmentation
0
5
10
15
20
25
0 5 10 15 20 25
Buddy block-size (log)
Num
ber
of r
eque
sts
(log)
![Page 140: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/140.jpg)
Benchmark raytrace12% fragmentation
0
5
10
15
20
25
0 5 10 15 20 25
Buddy block-size (log)
Nu
mb
er o
f req
ues
ts (l
og
)
![Page 141: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/141.jpg)
Benchmark compress34% fragmentation
0
5
10
15
20
25
0 5 10 15 20 25
Buddy block-size (log)
Num
ber
of r
eque
sts
(log)
![Page 142: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/142.jpg)
Informed Storage Management
• Evidence that programs allocate many objects of the same size
![Page 143: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/143.jpg)
Informed Storage Management
• Evidence that programs allocate many objects of the same size
• Not surprising, in Java
same type same size
![Page 144: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/144.jpg)
Informed Storage Management
• Evidence that programs allocate many objects of the same size
• Not surprising, in Java
same type same size• In C and C++ programmers brew their own
allocators to take advantage of this• What can we do automatically?
![Page 145: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/145.jpg)
Informed Storage Management
•Capture program malloc requests by phase
•Generate a .class file and put it in CLASSPATH
•Load the .class file and inform the allocator
![Page 146: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/146.jpg)
Different phasesdifferent distributions
-0.2
0
0.2
0.4
0.6
0.8
1
8 16 24 32
Block size
Req
uest
s (fr
actio
n)
Phase 1
Phase 2
Phase 3
Phase 4
Phase 5
Phase 6
raytrace
![Page 147: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/147.jpg)
How long is a phase?
500091000
461000
1903
Phases1-3
Phase 4
Phase 5
Phase 6
•Phases 1-3 are common to all programs—JVM startup
•Phases are keyed to allocations, not time, for portability
![Page 148: Getting Real, Getting Dirty (without getting real dirty) Ron K. Cytron Joint work with Krishna Kavi University of Alabama at Huntsville April 2001 Dante.](https://reader038.fdocuments.in/reader038/viewer/2022110404/56649e875503460f94b8b945/html5/thumbnails/148.jpg)
Questions?