1

Dynamic Memory Allocation: Basic Concepts

Andrew Case

Slides adapted from Jinyang Li, Randy Bryant and Dave O’Hallaro

2

Topics Basic concepts & challenges Implicit free lists

3

Why dynamic allocator? We’ve discussed how exactly systems handle two types of

data allocation so far: Data segments (global variables in .data/.bss) Stack-allocated (local variables)

Not sufficient! What about data whose whose size is only known during runtime? Need to control lifetime of allocated data?

E.g. a linked list that grows and shrinks

4

Dynamic Memory Allocation

Heap (via malloc)

Program text (.text)

Initialized data (.data)

Uninitialized data (.bss)

User stack

0

Holds local variables

Holds global variables

Holds global variables

Grown/shrunk by malloc library, holds dynamically-

allocated data

5

Example usage: a simple linked listtypedef struct item_t { int val; item_t *next;}item_t;

item_t *head = NULL;

void insert(int v) { item_t *x; x = (item_t *)malloc(sizeof(item_t)); if (x == NULL) { exit(1);} x->val = v; x->next = head; head = x;}

void delete(int v) { item_t *x; if (head && head->val == v) { x = head; head = head->next; free(x); }}

void printlist() { item_t *x; x = head; printf(“list contents: “); while (x!=NULL) { printf(“%d “, x->val); x = x->next; }

}

int main(){ while (1) { if (fscanf(stdin, “%c %d\n”, &c, &v)!=2) { break; } if (c == ‘i’) { insert(v); }else if (c == ‘d’) { delete(v); } } printlist();}

Returns a pointer to a memory block of size bytes,

Returns memory block to allocator, x must come from previous calls to malloc

linux% ./a.outi 5i 10i 20d 20qlist contents: 10 5

6

Dynamic Memory Allocation

Application

Dynamic Memory Allocator

OS

malloc free realloc

mmap sbrk

Allocate or free data of arbitrary sizes

Request for or give back large chunk of page-

aligned address space

Dynamic memory allocator is part of user-level library. Why not implement its functionality in the kernel?

7

Types of Dynamic Memory Allocator

Allocator maintains heap as collection of variable sized blocks, which are either allocated or freed

Two types of allocators…

Explicit allocator (used by C/C++): application allocates and frees space

malloc and free in C new and delete in C++

Implicit allocator (used by Java,…): application allocates, but does not free space

Garbage collection in Java, Python etc.

This class

8

Challenges facing a memory allocator

Applications Can issue arbitrary sequence of malloc and free requests free request must be to a malloc’d block

Achieve good throughput malloc/free calls should return quickly

Achieve good memory utilization Apps issue arbitrary sequence of malloc/free requests of arbitrary sizes Utilization = sum of malloc’d data / size of heap

9

Challenges facing a memory allocator

Constraints Allocators Can’t control number or size of allocated blocks Must respond immediately to malloc requests

i.e., can’t reorder or buffer requests Must allocate blocks from free memory

i.e., can only place allocated blocks in free memory Must align blocks so they satisfy all alignment requirements

8 byte alignment for GNU malloc (libc malloc) on Linux boxes Can manipulate and modify only free memory Can’t move the allocated blocks once they are malloc’d

i.e., compaction is not allowed

10

Good Throughput Given some sequence of malloc and free requests:

R0, R1, ..., Rk, ... , Rn-1

Goals: maximize throughput and peak memory utilization These goals are often conflicting

Throughput: Number of completed requests per unit time Example:

5,000 malloc calls and 5,000 free calls in 10 seconds Throughput is 1,000 operations/second

11

Good Utilization Given some sequence of malloc and free requests:

R0, R1, ..., Rk, ... , Rn-1

Pk :Aggregate payload malloc(p) results in a block with a payload of p bytes After request Rk has completed, the aggregate payload Pk is the sum of

currently allocated payloads

Hk: Current heap size - Assume Hk is nondecreasing i.e., heap only grows when allocator uses sbrk

Def: Peak memory utilization after k requests Uk = ( maxi<k Pi ) / Hk

12

Challenge #1

Design an algorithm that maximizes throughput.

Design an algorithm that maximizes usage.

Designing an algorithm that balances both.

13

Assumptions made in these lectures Memory is word addressed (each word can hold a

pointer)

Allocated block(4 words)

Free block(3 words) Free word

Allocated word

14

Allocation Example

p1 = malloc(4)

p2 = malloc(5)

p3 = malloc(6)

free(p2)

p4 = malloc(2)

15

Fragmentation Poor memory utilization caused by fragmentation

internal fragmentation external fragmentation

16

Internal Fragmentation Malloc allocates data from ``blocks’’ of certain sizes. Internal fragmentation occurs if payload is smaller than block size

May be caused by Limited choices of block sizes Padding for alignment purposes Other space overheads

100 byte Payload Internal fragmentation

Block of 128-byte

17

External Fragmentation Occurs when there is enough aggregate heap memory,

but no single free block is large enough

Depends on the pattern of future requests Thus, difficult to measure

p1 = malloc(4)

p2 = malloc(5)

p3 = malloc(6)

free(p2)

p4 = malloc(6) Oops! (what would happen now?)

18

Challenge #2 How do we know how much memory to free given just a

pointer?

How do we keep track of the free blocks?

What do we do with the extra space when allocating a structure that is smaller than the free block it is placed in?

How do we pick a block to use for allocation -- many might fit?

How do we reinsert freed block into available memory?

19

Knowing How Much to Free Standard method

Keep the length of a block in the header field preceding the block Requires an extra word for every allocated block

p0 = malloc(4)

p0

free(p0)

block size data

5

20

Keeping Track of Free Blocks Method 1: Implicit list using length—links all blocks

Method 2: Explicit list among the free blocks using pointers

Method 3: Segregated free list Different free lists for different size classes

Method 4: Blocks sorted by size Can use a balanced tree (e.g. Red-Black tree) with pointers within each

free block, and the length used as a key

5 4 26

5 4 26

21

Topics Basic concepts Implicit free lists

22

Method 1: Implicit List Heap is divided into variable-sized blocks Each block has a header containing size and allocation status

some low-order address bits are always 0 Instead of storing an always-0 bit, use it as a allocated/free flag When reading size word, must mask out this bit

Size

1 word

Format ofallocated andfree blocks

Payload

Size: block sizea = 1: Allocated block a = 0: Free block

Payload: application data(allocated blocks only)

a

Optionalpadding

header

23

Implicit List: Allocating in Free Block Allocating in a free block: splitting

Since allocated space might be smaller than free space, we might want to split the block

malloc(3)

4 4 26

4 24

p

24

space allocatedsplit free space

free space being allocated

p

24

Implicit List: Freeing a Block Simplest implementation:

Need only clear the “allocated” flag Problem?

Leads to “false fragmentation”

4 24 24

free(p) p

4 4 24 2

malloc(5) Oops!

There is enough free space, but the allocator won’t be able to find it

25

Implicit List: Coalescing Join (coalesce) with next/previous blocks, if they are free

Coalescing with next block

4 24 2

free(p) p

4 4 2

4

6 2

logicallygone

4 4 26

Check if next block is free

How to coalesce with a previous block?

26

Implicit List: Bidirectional Coalescing Boundary tags

Replicate size/allocated word at “bottom” (end) of free blocks Allows us to traverse the “list” backwards, but requires extra space

What does this look like? Doubly Linked List.

Size

Format ofallocated andfree blocks

Payload andpadding

a = 1: Allocated block a = 0: Free blockSize: Total block size

Payload: Application data(allocated blocks only)

a

Size aBoundary tag(footer)

4 4 4 4 6 46 4

Header

27

Constant Time Coalescing

Allocated

Allocated

Allocated

Free

Free

Allocated

Free

Free

Block beingfreed

Case 1 Case 2 Case 3 Case 4

28

m1 1

Constant Time Coalescing (Case 1)

m1 1n 1

n 1m2 1

m2 1

m1 1

m1 1n 0

n 0m2 1

m2 1

29

m1 1

Constant Time Coalescing (Case 2)

m1 1n+m2 0

n+m2 0

m1 1

m1 1n 1

n 1m2 0

m2 0

30

m1 0

Constant Time Coalescing (Case 3)

m1 0n 1

n 1m2 1

m2 1

n+m1 0

n+m1 0m2 1

m2 1

31

m1 0

Constant Time Coalescing (Case 4)

m1 0n 1

n 1m2 0

m2 0

n+m1+m2 0

n+m1+m2 0

32

Additional requirements: alignment

Start of

heap

Double word aligned

8/0 16/1 16/132/0 0/1

Headers: labeled with size in bytes/allocated bit Allocated blocks: shaded

Free blocks

Unusable

33

Implicit List: Finding a Free Block First fit:

Search from beginning, choose first free block that fits: Time taken?

Linear time for number of blocks (both allocated and free)

Next fit: Like first fit, except search starts where previous search finished Faster than the first?

Should be faster: avoids re-scanning unhelpful blocks Some research suggests that fragmentation is worse

Best fit: Search the list, choose the best free block: fewest wasted bits Keeps fragments to a minimum (best utilization) Speed?

Will usually run slower than either

34

Key Allocator Policies Placement policy:

First-fit, next-fit, best-fit, etc. Trades off lower throughput for less fragmentation

Splitting policy: When do we go ahead and split free blocks? How much internal fragmentation are we willing to tolerate?

Coalescing policy: Immediate coalescing: coalesce each time free is called Deferred coalescing: try to improve performance of free by deferring

coalescing until needed. Examples: Coalesce as you scan the free list for malloc Coalesce when the amount of external fragmentation reaches

some threshold

35

Implicit Lists: Summary Implementation: very simple Allocate cost:

linear time worst case Free cost:

constant time worst case Memory usage:

will depend on placement policy first-fit, next-fit or best-fit

Not used in practice for malloc/free because of linear-time allocation used in many special purpose applications