Algorithm Ch6 Heapsort

download Algorithm Ch6 Heapsort

of 41

Transcript of Algorithm Ch6 Heapsort

  • 7/30/2019 Algorithm Ch6 Heapsort

    1/41

    AlgorithmsChapter 6 Heapsort

    Assistant Professor: ChingChi Lin

    [email protected]

    Department of Computer Science and Engineering

    National Taiwan Ocean University

  • 7/30/2019 Algorithm Ch6 Heapsort

    2/41

    Outline` Heaps

    ` Maintainingtheheapproperty

    ` Buildingaheap

    ` Theheapsort algorithm

    ` Priorityqueues

    2

  • 7/30/2019 Algorithm Ch6 Heapsort

    3/41

    The purpose of this chapter` Inthischapter,weintroducetheheapsort algorithm.

    ` withworstcase runningtimeO(nlgn)

    `

    an

    in

    place

    sorting

    algorithm:

    only

    a

    constant

    number

    of

    array

    elementsarestoredoutsidetheinputarrayatanytime.

    ` thus,requireatmostO(1)additionalmemory

    ` Wealso

    introduce

    the heap data

    structure.

    ` anusefuldatastructureforheapsort

    ` makesanefficientpriorityqueue

    3

  • 7/30/2019 Algorithm Ch6 Heapsort

    4/41

    Heaps` The(Binary) heap datastructureisanarray objectthatcanbe

    viewedasanearlycompletebinarytree.

    ` Abinary

    tree

    with

    n nodes

    and

    depth

    kis

    complete iff its

    nodes

    correspondtothenodesnumberedfrom1ton inthefullbinary

    treeofdepthk.

    16

    14 10

    8 7 9 3

    2 4 1

    1

    2 3

    4 5 6 7

    8 9 10

    1423978101416

    8 9 107654321

    4

  • 7/30/2019 Algorithm Ch6 Heapsort

    5/41

    Binary tree representations

    5

    1

    2

    4

    8 9

    5

    10 11

    3

    6

    12 13

    7

    14 15

    1

    2

    4

    8 9

    5

    3

    6 7

    10

    Afullbinarytree ofheight3. Acompletebinarytreewith10nodes

    andheight

    3.

  • 7/30/2019 Algorithm Ch6 Heapsort

    6/41

    Attributes of a Heap` AnarrayAthatpresentsaheapwithtwoattributes:

    ` length[A]: thenumberofelementsinthearray.

    ` heapsize[A]: the

    number

    of

    elements

    in

    the

    heap

    stored

    with

    arrayA.

    ` length[A] heapsize[A]

    length[A]=heapsize[A]=10

    16

    14 10

    8 7 9 3

    2 4 1

    1

    2 3

    4 5 6 7

    8 9 10

    1423978101416

    8 9 107654321

    6

  • 7/30/2019 Algorithm Ch6 Heapsort

    7/41

    Basic procedures1/2` Ifacompletebinarytreewithn nodesisrepresented

    sequentially,thenforanynodewithindexi,1 i n,wehave

    `

    A[1]

    is

    the

    root of

    the

    tree` theparentPARENT(i)isat i/2 ifi 1

    ` theleftchildLEFT(i)isat2i

    ` therightchildRIGHT(i)isat2i+1

    7

    16

    14 10

    8 7 9 3

    2 4 1

    1

    2 3

    4 5 6 7

    8 9 10

    1423978101416

    8 9 107654321

  • 7/30/2019 Algorithm Ch6 Heapsort

    8/41

    Basic procedures2/2` TheLEFT procedurecancompute2iinoneinstructionbysimply

    shiftingthebinaryrepresentationofileftonebitposition.

    ` Similarly,the

    RIGHT procedure

    can

    quickly

    compute

    2i+1

    by

    shiftingthebinaryrepresentationofileftonebitpositionand

    addingina1astheloworderbit.

    ` ThePARENT procedure

    can

    compute

    i/2 by

    shifting

    iright

    one

    bitposition.

    8

  • 7/30/2019 Algorithm Ch6 Heapsort

    9/41

    Heap properties` Therearetwokindofbinaryheaps:maxheapsandminheaps.

    ` Inamaxheap,themaxheapproperty isthatforeverynodei

    other

    than

    the

    root,

    ` thelargestelementinamaxheapisstoredattheroot

    ` thesubtree rootedatanodecontainsvaluesnolargerthanthat

    contained

    at

    the

    node

    itself` Inaminheap,theminheapproperty isthatforeverynodei

    otherthantheroot,

    ` thesmallest

    element

    in

    amin

    heap

    is

    at

    the

    root

    ` thesubtree rootedatanodecontainsvaluesnosmallerthanthat

    containedatthenodeitself

    9

    A[PARENT(i)] A[i] .

    A[PARENT(i)] A[i] .

  • 7/30/2019 Algorithm Ch6 Heapsort

    10/41

    Max and min heaps

    10

    14

    12 7

    10 8 6

    9

    6 3

    5

    30

    25

    2

    7 4

    10 8 6

    10

    20 83

    50

    11

    21

    MaxHeaps

    MinHeaps

  • 7/30/2019 Algorithm Ch6 Heapsort

    11/41

    The height of a heap` Theheight ofanodeinaheapisthenumberofedgesonthe

    longestsimpledownwardpathfromthenodetoaleaf,and

    theheightoftheheaptobetheheightoftheroot,thatis

    (lgn).

    ` Forexample:

    ` theheightofnode2is2

    ` theheight

    of

    the

    heap

    is

    3

    11

    16

    14 10

    8 7 9 3

    2 4 1

    1

    2 3

    4 5 6 7

    8 9 10

  • 7/30/2019 Algorithm Ch6 Heapsort

    12/41

    The remainder of this chapter` Weshallpresentssomebasicproceduresintheremainder

    ofthischapter.

    `

    The

    MAX

    HEAPIFY procedure,

    which

    runs

    in

    O(lgn)

    time,

    is

    the

    keytomaintainingthemaxheapproperty.

    ` TheBUILDMAXHEAP procedure,whichrunsinO(n)time,

    producesamaxheapfromanunorderedinputarray.

    ` TheHEAPSORT procedure,

    which

    runs

    in

    O(n lgn)

    time,

    sorts

    an

    arrayinplace.

    ` TheMAXHEAPINSERT,HEAPEXTRACTMAX,HEAPINCREASEKEY,

    andHEAPMAXIMUM procedures,whichruninO(lgn)time,

    allowthe

    heap

    data

    structure

    to

    be

    used

    as

    apriority

    queue.

    12

  • 7/30/2019 Algorithm Ch6 Heapsort

    13/41

    Outline` Heaps

    ` Maintainingtheheapproperty

    ` Buildingaheap

    ` Theheapsort algorithm

    ` Priorityqueues

    13

  • 7/30/2019 Algorithm Ch6 Heapsort

    14/41

    The MAXHEAPIFY procedure1/2` MAXHEAPIFY isanimportantsubroutineformanipulatingmax

    heaps.

    `

    Input:

    an

    array

    A

    and

    an

    index

    i` Output:thesubtree rootedatindexibecomesamaxheap

    ` Assume:thebinarytreesrootedatLEFT(i) andRIGHT(i) are

    maxheaps,butA[i]maybesmallerthanitschildren

    ` Method:let

    the

    value

    at

    A[i]

    float

    down in

    the

    max

    heap

    14

    16

    4 10

    14 7 9 3

    2 8 1

    1

    2 3

    4 5 6 7

    8 9 10

    16

    14 10

    8 7 9 3

    2 4 1

    1

    2 3

    4 5 6 7

    8 9 10

    MAXHEAPIFYi

  • 7/30/2019 Algorithm Ch6 Heapsort

    15/41

    The MAXHEAPIFY procedure2/2MAXHEAPIFY(A,i)

    1. l LEFT(i)

    2. r RIGHT(i)

    3. ifl heapsize[A]andA[l]>A[i]

    4. then largest l

    5. else largest i

    6. ifr heapsize[A]anda[r]>A[largest]

    7. then largest r

    8. iflargest i

    9. then exchangeA[i] A[largest]

    10. MAX

    HEAPIFY (A,

    largest)

    15

  • 7/30/2019 Algorithm Ch6 Heapsort

    16/41

    An example of MAXHEAPIFY procedure

    16

    16

    4 10

    14 7 9 3

    2 8 1

    1

    2 3

    4 5 6 7

    8 9 10

    16

    14 10

    4 7 9 3

    2 8 1

    1

    2 3

    4 5 6 7

    8 9 10

    16

    14 10

    8 7 9 3

    2 4 1

    1

    2 3

    4 5 6 7

    8 9 10

    i

    i

    i

  • 7/30/2019 Algorithm Ch6 Heapsort

    17/41

    The time complexity` Ittakes(1)timetofixuptherelationshipsamongthe

    elementsA[i],A[LEFT(i)],andA[RIGHT(i)].

    `

    Also,

    we

    need

    to

    run

    MAX

    HEAPIFY on

    a

    subtree rooted

    at

    one

    ofthechildrenofnodei.

    ` Thechildrenssubtrees eachhavesizeatmost2n/3

    ` worstcaseoccurswhenthelastrowofthetreeisexactlyhalffull

    ` TherunningtimeofMAXHEAPIFY is

    ` solveit

    by

    case

    2of

    the

    master

    theorem

    ` Alternatively,wecancharacterizetherunningtimeofMAX

    HEAPIFY onanodeofheighthasO(h).

    17

    T(n)=T(2n/3)+(1)= O(lg n)

  • 7/30/2019 Algorithm Ch6 Heapsort

    18/41

    Outline` Heaps

    ` Maintainingtheheapproperty

    ` Buildingaheap

    ` Theheapsort algorithm

    ` Priorityqueues

    18

  • 7/30/2019 Algorithm Ch6 Heapsort

    19/41

    Building a Heap

    19

    ` WecanusetheMAXHEAPIFY proceduretoconvertanarray

    A=[1..n]intoamaxheapinabottomupmanner.

    `

    The

    elements

    in

    the

    subarray A[(n/2+1)n ]

    are

    all

    leaves of

    thetree,andsoeachisa1elementheap.

    ` TheprocedureBUILDMAXHEAP goesthroughtheremaining

    nodesofthetreeandrunsMAXHEAPIFY oneachone.

    BUILDMAXHEAP(A)

    1. heapsize[A] length[A]

    2. for i length[A]/2 downto 1

    3. do MAXHEAPIFY(A,i)

  • 7/30/2019 Algorithm Ch6 Heapsort

    20/41

    An example

    20

    A 7814109162314

    8 9 107654321

    4

    1 3

    2 16 9 10

    14 8 7

    1

    2 3

    4 5 6

    8 9 10

    i7

  • 7/30/2019 Algorithm Ch6 Heapsort

    21/41

    4

    1 3

    2 16 9 10

    14 8 7

    [1]

    [2] [3]

    [5][4]

    [7][6]

    [8] [9] [10]

    4

    1 3

    2 16 9 10

    14 8 7

    [1]

    [2] [3]

    [5][4]

    [7][6]

    [8] [9] [10]

    4

    1 3

    14 16 9 10

    2 8 7

    [1]

    [2] [3]

    [5][4]

    [7][6]

    [8] [9] [10]

    4

    1 10

    14 16 9 3

    2 8 7

    [1]

    [2] [3]

    [5]

    [7][6]

    [8] [9] [10]

    4

    16 10

    14 1 9 3

    2 8 7

    [1]

    [2] [3]

    [5]

    [7][6]

    [8] [9] [10]

    4

    16 10

    14 7 9 3

    2 8 1

    [1]

    [2] [3]

    [5]

    [7][6]

    [8] [9] [10]

    [4] [4][4]

    MAXHEAPIFY(A,5) MAXHEAPIFY(A,4) MAXHEAPIFY(A,3)

    MAXHEAPIFY(A,1) MAXHEAPIFY(A,2)

  • 7/30/2019 Algorithm Ch6 Heapsort

    22/41

    16

    4 10

    14 7 9 3

    2 8 1

    [1]

    [2] [3]

    [5]

    [7][6]

    [8] [9] [10]

    [4]

    16

    14 10

    4 7 9 3

    2 8 1

    [1]

    [2] [3]

    [5]

    [7][6]

    [8] [9] [10]

    [4]

    16

    14 10

    8 7 9 3

    2 4 1

    [1]

    [2] [3]

    [5]

    [7][6]

    [8] [9] [10]

    [4]

    maxheap

  • 7/30/2019 Algorithm Ch6 Heapsort

    23/41

    Correctness1/2

    23

    ` ToshowwhyBUILDMAXHEAP workcorrectly,weusethe

    followingloopinvariant:

    ` Atthestartofeachiterationoftheforloopoflines23,each

    nodei+1,i+2,,n istherootofamaxheap.

    ` Weneedtoshowthat

    ` thisinvariant

    is

    true

    prior

    to

    the

    first

    loop

    iteration

    ` eachiterationoftheloopmaintainstheinvariant

    ` theinvariantprovidesausefulpropertytoshowcorrectness

    whentheloopterminates.

    BUILDMAXHEAP(A)

    1. heapsize[A] length[A]

    2. for i length[A]/2 downto 1

    3. do MAXHEAPIFY(A,i)

  • 7/30/2019 Algorithm Ch6 Heapsort

    24/41

    Correctness2/2

    24

    ` Initialization: Priortothefirstiterationoftheloop,i=n/2.n/2+1,n isaleafandisthustherootofa

    trivialmaxheap.

    ` Maintenance:Bythe loopinvariant,thechildrenofnodeiarebothrootsofmaxheaps.Thisispreciselythe

    conditionrequiredforthecallMAXHEAPIFY(A,i)

    tomake

    node

    ia

    max

    heap

    root.

    Moreover,

    the

    MAXHEAPIFY callpreservesthepropertythat

    nodesi+1,i+2,...,nareallrootsofmaxheaps.

    ` Termination: Attermination,i=0.Bytheloopinvariant,eachnode1,2,,n istherootofamaxheap.

    Inparticular,node1is.

  • 7/30/2019 Algorithm Ch6 Heapsort

    25/41

    Time complexity1/2` Analysis1:

    ` EachcalltoMAXHEAPIFY costsO(lgn),andthereareO(n)such

    calls.

    ` Thus,therunningtimeisO(n lgn).Thisupperbound,through

    correct,isnotasymptoticallytight.

    ` Analysis2:

    ` Forannelementheap,heightislgn andatmostn /2h+1

    nodesofanyheighth.

    ` ThetimerequiredbyMAXHEAPIFY whencalledonanodeof

    heighth is

    O(h).

    ` Thetotalcostis

    .2

    )(2

    lg

    0

    lg

    01

    =

    ==

    +

    n

    h

    h

    n

    h

    h

    hnOhO

    n

    25

  • 7/30/2019 Algorithm Ch6 Heapsort

    26/41

    Time complexity2/2` Thelastsummationyields

    ` Thus,therunningtimeofBUILDMAXHEAP canbeboundedas

    ` Wecanbuildamaxheapfromanunorderedarrayinlinear

    time.

    2)2/11(

    2/1

    2 20

    =

    =

    =hh

    h

    )(2)(2 0

    lg

    01 nO

    h

    nOhO

    n

    h

    h

    n

    h

    h =

    =

    ==+

    26

  • 7/30/2019 Algorithm Ch6 Heapsort

    27/41

    Outline` Heaps

    ` Maintainingtheheapproperty

    ` Buildingaheap

    ` Theheapsort algorithm

    ` Priorityqueues

    27

  • 7/30/2019 Algorithm Ch6 Heapsort

    28/41

    The heapsort algorithm` Sincethemaximumelementofthearrayisstoredattheroot,

    A[1]wecanexchange itwithA[n].

    `

    If

    we

    now

    discard A[n],

    we

    observe

    that

    A[1...(n

    1)]

    can

    easily

    bemadeintoamaxheap.

    ` ThechildrenoftherootA[1]remainmaxheaps,butthenew

    rootA[1]elementmayviolatethemaxheapproperty,sowe

    needto

    readjust the

    max

    heap.

    That

    is

    to

    call

    MAX

    HEAPIFY(A,

    1).

    28

    HEAPSORT(A)

    1. BUILDMAXHEAP(A)

    2. for i length[A] downto 2

    3. do exchangeA[1] A[i]

    4. heapsize[A] heapsize[A]1

    5. MAXHEAPIFY(A,1)

  • 7/30/2019 Algorithm Ch6 Heapsort

    29/41

    9

    An example

    29

    16

    14 10

    8 7 9 3

    2 4 1

    1

    2 3

    4 5 6 7

    8 10i

    A 1614109874321

    8 9 107654321

  • 7/30/2019 Algorithm Ch6 Heapsort

    30/41

    1

    14 10

    8 7 9 3

    2 4 16

    [1]

    [2] [3]

    [5]

    [7][6]

    [8] [9] [10]

    [4]

    Initialheap

    16

    14 10

    8 7 9 3

    2 4 1

    [1]

    [2] [3]

    [5]

    [7][6]

    [8] [9] [10]

    [4]

    Exchange

    Heapsize=10

    Sorted=[16]

    1

    14 10

    8 7 9 3

    2 4 16

    [1]

    [2] [3]

    [5]

    [7][6]

    [9] [10]

    Discard

    Heapsize=9

    Sorted=[16]

    [8]

    14

    8 10

    4 7 9 3

    2 1 16

    [1]

    [2] [3]

    [5]

    [7][6]

    [9] [10][8]

    ReadjustHeapsize=9

    Sorted=[16]

    1

    8 10

    4 7 9 3

    2 14 16

    [1]

    [2] [3]

    [5]

    [7][6]

    [9] [10][8]

    ExchangeHeapsize=9

    Sorted=[14,16]

    1

    8 10

    4 7 9 3

    2 14 16

    [1]

    [2] [3]

    [5]

    [7][6]

    [9] [10][8]

    DiscardHeapsize=8

    Sorted=[14,16]

    [4]

    [4][4][4]

  • 7/30/2019 Algorithm Ch6 Heapsort

    31/41

    [8]

    Heapsize=7

    Sorted=[10,14,16]

    8

    7 3

    4 2 1 9

    10 14 16

    [1]

    [2] [3]

    [5]

    [7][6]

    [9] [10]

    Heapsize=6

    Sorted=[9,10,14,16]

    [8]

    Heapsize=5

    Sorted=[8,9,10,14,16]

    Heapsize=4

    Sorted=[7,8,9,10,14,16]

    Readjust

    Heapsize=8

    Sorted=[14,16]

    10

    8 9

    4 7 1 3

    2 14 16

    [1]

    [2] [3]

    [5]

    [7][6]

    [9] [10][8]

    9

    8 3

    4 7 1 2

    10 14 16

    [1]

    [2] [3]

    [5]

    [7][6]

    [9] [10]

    [4][4] [4]

    7

    4 3

    1 2 8 9

    10 14 16

    [1]

    [2] [3]

    [5]

    [7][6]

    [9] [10][8]

    [4]

    4

    2 3

    1 7 8 9

    10 14 16

    [1]

    [2] [3]

    [5]

    [7][6]

    [9] [10][8]

    [4]

    3

    2 1

    4 7 8 9

    10 14 16

    [1]

    [2] [3]

    [5]

    [7][6]

    [9] [10][8]

    [4]

    Heapsize=3

    Sorted=[4,7,8,9,10,14,16]

  • 7/30/2019 Algorithm Ch6 Heapsort

    32/41

    Time complexity

    32

    ` TheHEAPSORT proceduretakesO(n lgn) time

    ` thecalltoBUILDMAXHEAP takesO(n)time

    ` eachofthen1callstoMAXHEAPIFY takesO(lgn)time

  • 7/30/2019 Algorithm Ch6 Heapsort

    33/41

    Outline` Heaps

    ` Maintainingtheheapproperty

    ` Buildingaheap

    ` Theheapsort algorithm

    ` Priorityqueues

    33

  • 7/30/2019 Algorithm Ch6 Heapsort

    34/41

    Heap implementation of priority queues

    34

    ` Heapsefficientlyimplementpriorityqueues.

    ` Therearetwokindsofpriorityqueues:maxpriorityqueues

    andminpriorityqueues.

    ` Wewillfocushereonhowtoimplementmaxpriorityqueues,

    whichareinturnbasedonmaxheaps.

    ` Apriorityqueue isadatastructureformaintainingasetS of

    elements,each

    with

    an

    associated

    value

    called

    akey.

  • 7/30/2019 Algorithm Ch6 Heapsort

    35/41

    Priority queues

    35

    ` Amaxpriorityqueue supportsthefollowingoperations.

    ` INSERT(S,x): insertstheelementxintothesetS.

    ` MAXIMUM(S): returnsthe

    element

    of

    S with

    the

    largest

    key.

    ` EXTRACTMAX(S): removesandreturnstheelementofS with

    thelargestkey.

    ` INCREASEKEY(S,

    x,

    k):

    increases

    value

    of

    element

    xs key

    to

    the

    newvaluek.Assumek xs currentkeyvalue.

  • 7/30/2019 Algorithm Ch6 Heapsort

    36/41

    Finding the maximum element

    ` MAXIMUM(S): returnstheelementofS withthelargestkey.

    ` Gettingthemaximumelementiseasy:itstheroot.

    ` TherunningtimeofHEAPMAXIMUM is(1).

    HEAP

    MAXIMUM(A)1. returnA[1]

    36

  • 7/30/2019 Algorithm Ch6 Heapsort

    37/41

    Extracting max element

    ` EXTRACTMAX(S):removesandreturnstheelementofS with

    thelargestkey.

    ` Analysis: constanttime

    assignments

    +time

    for

    MAX

    HEAPIFY.

    ` TherunningtimeofHEAPEXTRACTMAX isO(lgn).

    HEAPEXTRACTMAX(A)

    1. ifheapsize[A]

  • 7/30/2019 Algorithm Ch6 Heapsort

    38/41

    Increasing key value

    ` INCREASEKEY(S,x,k):increasesvalueofelementxs keytok.

    Assumek xs currentkeyvalue.

    ` Analysis: thepath

    traced

    from

    the

    node

    updated

    to

    the

    root

    haslengthO(lgn).

    ` TherunningtimeisO(lgn).

    HEAP

    INCREASE

    KEY (A,

    i,

    key)1. ifkey1and

    A[PARENT(i)]