1 Jeff Edmonds York University COSC 2011 Lecture 3 Contracts Assertions Loop Invariants The Sum of...

1

Jeff Edmonds

York University COSC 2011Lecture 3

ContractsAssertionsLoop InvariantsThe Sum of ObjectsInsertion and Selection SortBinary Search Like ExamplesBucket (Quick) Sort for HumansReverse Polish Notation (Stack)Whose Blocking your View (Stack)Parsing (Stack)Data Structure InvariantsStack and Queue in an ArrayLinked Lists

Contracts, Assertions, and Invariants

2

Precondition Postcondition

On Step At a Time

I implored you to not worry about the entire computation.

next

It can be difficult to understand where computation go.

i-1 i

ii

0 T+1

Trust who passes you the baton

and go around once

3


• Clear pre and post conditions separate out the responsibility of a routine or class and its user.

• Assertions state assumptions about what is true at a point in a computation.

• System, Data Structure, and Loop Invariants draw a picture of what must be maintained through out time while making progress.

4

A Routine, Class, or Data Structure:

Implementer:Coder of the routine.

Three Players

User:Coder using routine.

End User:Runs the program

Not part of this course.

This is us inCSE 3101

Complicated details are

hidden from him. Correctness & efficiency.

Contracts

5

Max( a,b,c )

if( b>m ) m = bendif

m = a

if( c>m ) m = cendif

return(m)

At least tell me what the algorithm is supposed to do.

• Preconditions: Any assumptions that must be true about the input instance.

• Postconditions: The statement of what must be true when the algorithm/program returns.

“preCond: Input has 3 numbers.”

“postCond: return max in {a,b,c}”

Contracts

ContractsModularity• Each module has a well-specified job.

specified by pre & post conditions of the abstract interface

• Allows the programmer to implement it as she wishes.

• Allows the user to reuse the module.

preCond: Input has n numberspostCond: returns max

or

class Node<E> { E element; Node<E> next; }Node<int> age = Node(5);Node<string> name = Node(“Jeff”’);





Hierarchical Design• Each level depends on but does not

micro manage the lower level





Hierarchical Design• Each level depends on but does not

micro manage the lower level Encapsulation• Internal details are kept private.• And lack of trust.

private class Node<E> { E element; Node<E> next; }Outside of the stack ADT no one can touch the nodes.

Goal of C is to write efficient code• In some ways close to machine code• But highly prone to hard to find bugs.• And computer security can be compromised by

programs intentionally accessing memory in unintended ways.

Goal of Java is encapsulation (and lack of trust)• the modal validates everything the user sends.

9

Contracts, Assertions, and Invariants• Clear pre and post conditions separate out the

responsibility of a routine or class and its user.


10

Paradigm Shift

Is the black the form and the green the background?

Is the green the form and the black the background?

It is helpful to have different ways of looking at it.

11

A Sequence of Actions

Max( a,b,c )


m = a


return(m)

If you are describing an alg to a friend, is this what you say?

Do you think about an algorithm as a sequence of actions?

Do you explain it by saying:“Do this. Do that. Do this”?

Do you get lost about where the computation is

and where it is going?

What if there are many paths through many ifs and loops?

How do you know it works

for every path on every input?

12


Max( a,b,c )


m = a


return(m)

How can you possibly understand this algorithm without knowing

what is true when the computation is here?



13


Max( a,b,c )


m = a


return(m)

How can you possibly understand this algorithm without knowing

what is true when the computation is here?

Tell me!

“assert: m is max in {a,b}”



“assert: m is max in {a}”

“assert: m is max in {a,b,c}”

14



“assert: m is max in {a}”

“assert: m is max in {a,b}”

“assert: m is max in {a,b,c}”

Max( a,b,c )


m = a


return(m)


vs

A Sequence of Assertions

It is helpful to have different ways of

looking at it.

15

Purpose of Assertions

Useful for– thinking about algorithms– developing– describing– proving correctness

16

Definition of Assertions

An assertion is a statement about the current state of the data structure that is either true or false.

eg. the amount in your bank account is not negative.

17


It is made at some particular point during the execution of an algorithm.

It should be true independent ofthe path followed through the code

and independent of the input.If it is false, then something has gone

wrong in the logic of the algorithm.

18


An assertion is not a task for the algorithm to perform.

It is only a comment that is added for the benefit of the reader.

19

Definition of Assertions• The current state of the computation

– Everything that is true at a particular instant in time.– Imagine flying in from Mars,

what do you need to know about the present to be able to continue.

– It does not say anything about the past,i.e. how the computation got here.

– Eg <x=8, y=3, current line of code = line 84>

20

Definition of Assertions• The current state of the computation• An assertion, A

– is a function • that takes as input the current state• and that outputs

– Eg A = “x is odd”• A(<x=5, y=3, line84> ) = True

• A(<x=8, y=3, line84> ) = False

True or False

Computation is on the path

Something has gone wrong!

21

Algorithm TerminationYou need to define some

Measure of progressto prove that your algorithm

eventually terminates.

• An Measure of Progress, M– is a function

• that takes as input the current state• and that outputs

– Eg Measure = “value of y”• M(<x=5, y=3, line84> ) = 3

– We must prove that this number goes down (up) each iteration.

a real number

22

Don't Be Frightened

An assertion need not consist of formal mathematical mumble jumble

Use an informal description

and a picture

23

• Preconditions: Any assumptions that must be true about the input instance.

• Postconditions: The statement of what must be true when the algorithm/program returns.

Specifying a Computational Problem

Max( a,b,c )“preCond: Input has 3 numbers.”


m = a


return(m)


24

Definition of Correctness

<PreCond> & <code> Þ <PostCond>

If the input meets the preconditions, then the output must meet the

postconditions.

If the input does not meet the preconditions, then nothing is required.

25

<assertion0>

if( <condition1> ) then

code<1,true>

else

code<1,false>

end if

<assertion1>

<assertionr-1>

if( <conditionr> ) then

code<r,true>

else

code<r,false>

end if

<assertionr>

…

<assertion0>any <conditions>code

<assertionr>

How is this proved?



26

<assertion0>


code<1,true>

else

code<1,false>

end if

<assertion1>

<assertionr-1>


code<r,true>

else

code<r,false>

end if

<assertionr>

…

Must check 2r different • settings of <conditions>• paths through the code.

Is there a faster way?


<assertion0>any <conditions>code

<assertionr>


27

<assertion0>


code<1,true>

else

code<1,false>

end if

<assertion1>

<assertionr-1>


code<r,true>

else

code<r,false>

end if

<assertionr>

Step 1

<assertion0><condition1>code<1,true>

<assertion1>

<assertion0>¬<condition1>code<1,false>

<assertion1>

…A Sequence of Assertions

28

<assertion0>


code<1,true>

else

code<1,false>

end if

<assertion1>

<assertionr-1>


code<r,true>

else

code<r,false>

end if

<assertionr>

…

Step 2


<assertion2>


<assertion2>


29

Max( a,b,c )“preCond: Input has 3 numbers.” m = a “assert: m is max in {a}”if( b>m ) m = bendif“assert: m is max in {a,b}”if( c>m ) m = cendif“assert: m is max in {a,b,c}”return(m)“postCond: return max in {a,b,c}”

A Sequence of AssertionsExample

<preCond>code <assertion0>

30



<assertion1>


<assertion1>


31



<assertion2>


<assertion2>


32



<assertion2>code <postCond>

33




• Loop Invariants draw a picture of what must be maintained through out time while making progress.

34

• A good way to structure many programs:

• Store the key information you currently know in some data structure.

• In the main loop, take a step forward towards destination

• by making a simple change to this data.

Iterative Algorithmsloop (until done) take stepend loop

35

Loop InvariantsMax( A )“preCond: Input is array A[1,n] of n values.” i = 1m = A[1] loop

exit when (i=n) i = i + 1 m = max(m,A[i]) endloop return(m)“postCond: m is max in A[1,n]”

Any assumptions that must be true about the state of computation every time it is at the top of the loop.

36

Loop InvariantsMax( A )“preCond: Input is array A[1,n] of n values.” i = 1m = A[1] loop “loop-invariant: m is max in A[1,i]” exit when (i=n) i = i + 1 m = max(m,A[i]) endloop return(m)“postCond: m is max in A[1,n]”

Any assumptions that must be true about the state of computation every time it is at the top of the loop.

37

• Not just to please the prof.• Not just to prove correctness.• Use them to think about your

algorithm!• Before you start coding.• What do you want to be

true in the middle of your computation? – You need to know.– Let your reader know.

Loop InvariantsMax( A )“preCond: Input is array A[1,n] of n values.” i = 1m = A[1] loop “loop-invariant: m is max in A[1,i]” exit when (i=n) i = i + 1 m = max(m,A[i]) endloop return(m)“postCond: m is max in A[1,n]”

38

• Not just to please the prof.• Not just to prove correctness.• Use them to think about your

algorithm!• Before you start coding.• What do you want to be

true in the middle of your computation? – You need to know.– Let your reader know.

Loop InvariantsMax( A )“preCond: Input is array A[1,n] of n values.”

loop “loop-invariant: m is max in A[1,i]” exit when (i=n)

“postCond: m is max in A[1,n]”

39

<preCond> codeA loop <loop-invariant> exit when <exit Cond> codeBendloop codeC<postCond>

Definition of Correctness<preCond><any conditions> code

<postCond>

How is this proved?

Iterative Algorithms Loop Invariants

We don’t know how manytimes it will iterate.

40


Max( A )“preCond: Input is array A[1,n] of n values.” i = 1m = A[1] loop “loop-invariant: m is max in A[1,i]” exit when (i=n) i = i + 1 m = max(m,A[i]) endloop return(m)“postCond: m is max in A[1,n]”

Step 0

<preCond> codeA

<loop-inv>

Example

41

Step 1

<loop-invariantt> ¬<exit Cond> codeB

<loop-invt+1>



Step 2 Step i

Example

42

Last Step

<loop-invariant><exit Cond> codeC

<postCond>



Example

43

• Your job is only to

• Trust who passes you the baton

• Go around once

• Pass on the baton

i-1 i

ii

0T+1

A Relay Race


44

<preCond> codeA loop <loop-invariant> exit when <exit Cond> codeBendloopcodeC<postCond>

0

Person 0: Carries baton region to safe region

<preCond>codeA

<loop-invariant>

Establishing Loop Invariant

45

Person i: Running around the track

i-1i

<preCond> codeA loop <loop-invariant> exit when <exit Cond> codeBendloopcodeC<postCond>i

i

<loop-invarianti-1>¬<exit Cond>codeB

<loop-invarianti>

Exit Maintaining Loop Invariant

46

T+1

<preCond> codeA loop <loop-invariant> exit when <exit Cond> codeBendloopcodeC<postCond>

<loop-invariant><exit Cond>codeC

<postCond>Exit

Clean up loose endsLast Person: Carries baton from safe region to finish.

47

OkI know you knew Insertion Sort

But hopefully you are beginning to appreciate

Loop Invariants

for describing algorithms

48

Maintain Loop Invariant

• Can we be assured that the computation will always be in a safe location?

• By what principle?

49

Maintain Loop Invariant• By Induction the computation will

always be in a safe location.

S

iS i S i

iS i

( )

( ) ( )

( )

0

1

50

No Assumptions:

Suppose a Martian jumped

into the top of the loop.

All you would know is that <loop-invariant> was true.

It alone must provide enough information

so that, after going around the loop, you can establish that the loop invariant is again true.

51

Ending The AlgorithmExitDefine Exit Condition

Termination:0 km Exit

With sufficient progress, the exit condition will be met.

Exit

from these we must establish the post conditions.

Exit

When we exit, we know• exit condition is true• loop invariant is true

52

Let’s Recap

53

Iterative Algorithm with Loop Invariants• Precondition:

What is true about input

• Post condition: What is true about output.

54

Iterative Algorithm with Loop Invariants

Goal: Your goal is to prove that • no matter what the input is,

as long as it meets the precondition, • and no matter how many times your algorithm

iterates, as long as eventually the exit condition is met,

• then the post condition is guarantee to be achieved.Proves that IF the program terminates then it works


55

Iterative Algorithm with Loop Invariants• Loop Invariant:

Picture of what is true at top of loop.

56

Iterative Algorithm with Loop Invariants• Establishing the Loop Invariant.

• Our computation has just begun.

• All we know is that we have an input instance that meets the Pre Condition.

• Being lazy, we want to do the minimum work.

• And to prove that it follows that the Loop Invariant is then made true.

<preCond>codeA

<loop-invariant>


57

Iterative Algorithm with Loop Invariants• Maintaining the loop invariant

(while making progress)79 km 75 km

Exit

• We arrived at the top of the loop knowing only

• the Loop Invariant is true

• and the Exit Condition is not.

• We must take one step (iteration) (making some kind of progress).

• And then prove that the Loop Invariant will be true when we arrive back at the top of the loop.

<loop-invariantt>¬<exit Cond>codeB

<loop-invariantt+1>

Maintaining Loop Invariant

Exit

58


<postCond>

Obtain the Post Condition

Exit

• We know the Loop Invariant is true because we have maintained it. We know the Exit Condition is true because we exited.

• We do a little extra work.

• And then prove that it follows that the Post Condition is then true.

• Obtain the Post Condition: Exit

Iterative Algorithm with Loop Invariants

59

Designing an Algorithm Define Problem Define Loop

InvariantsDefine Measure of Progress

Define Step Define Exit Condition Maintain Loop Inv

Make Progress Initial Conditions Ending

km

79 km

to school

Exit

Exit

79 km 75 km

Exit

Exit

0 km Exit

Is this sufficient?

60

Consider an Algorithm

km Exit0 km Exit

Exit

79 km 75 km

Exit

61

Loop Invariant Maintained

Exit

62

Computation can always proceed

63

Computation always makes progress

79 km 75 km

79 km 75 km

Exit

64

Computation Terminates

km

79 km 75 km

0 km

0 km

0 km Exit

Exit

65

Computation Terminates

Exit

Exit

66

Consider an Algorithm

This is sufficient!

km Exit0 km Exit

Exit

79 km 75 km

Exit

67

Partial Correctness

Proves that IF the program terminates then it works


<preCond>codeA

<loop-invariant>



<postCond>

Clean up loose ends

Exit


<loop-invariantt+1>


Exit

68

General Principle • Do not worry about the entire computation.

• Take one step at a time!

next

69

• Some locations are too confusing for algorithm

• Algorithm too lazy to define step for every location.

Computation Stuck

70

Safe Locations• Algorithm specifies from which locations

it knows how to step.

71

Loop Invariant• “The computation is presently in a safe

location.”• Maybe true and maybe not.

If not something has gone wrong.

72

Picture from the Middle

Leap into the middle of the algorithm.

What would you like your data structure to look like when you are half done?

73

Flow SmoothlyThe loop invariant should flow smoothly from

the beginning to the end of the algorithm. – At the beginning, it should follow easily from

the preconditions. – It should progress in small natural steps. – Once the exit condition has been met, the

postconditions should easily follow.

74

Ask for 100% A good philosophy in life is to ask for 100%

of what you want,

but not to assume that you will get it.– What would you like to be true in the middle of

your computation?

75

Ask for 100% Pretend that a genie has granted your wish.– You are now in the middle of your computation

and your dream loop invariant is true.

76

Ask for 100%

Maintain the Loop Invariant:– From here, are you able to take some

computational steps that will make progress while maintaining the loop invariant?

77

No Assumptions:

Suppose a Martian jumped

into the top of the loop.

All you would know is that <loop-invariant> was true.

It alone must provide enough information

so that, after going around the loop, you can establish that the loop invariant is again true.

78

Know What a LI Is Not

``the LI is ...''– code– A precondition– A postcondition– A statement that is always true

Eg: “1+1=2”– The steps taken by the algorithm

79

Differentiating between Iterations

x=x+2 – Meaningful as code– False as a mathematical statement

x’ = xi = value at the beginning of the iteration

x” = xi+1 = new value after going around the loop one more time.

x” = x’+2– Meaningful as a mathematical statement

80

This completes One Higher Level Abstract View

of Iterative Algorithms

81

Simple Example

A sum of objects

You MUST do it my way!

82





km

79 km

to school

Exit

Exit

79 km 75 km

Exit

Exit

0 km Exit

83

The Sum Problem• Precondition: The input is an integer I.• Postcondition: The output is S = 12 + 22 + 32 + … + I2

• What is the loop invariant?• Each iteration add in a new object.

Action not picture

If you fight me, I will fail you

84


• What is the loop invariant?• What do you want to be true in the

middle of the computation?• To have a partial sum:

s = 12 + 22 + 32 + … + i2 (and 1 ≤ i ≤ I)

85


• Loop Invariant: s = 12 + 22 + 32 + … + i2 (0 ≤ i ≤ I)

• Measure of Progress• i = # of objects added.

79 km 75 km

Exit

Exit

79 km

to school

• Step to make progress• i = i+1

• Exit Condition• i = I

86

The Sum ProblemIn order to make progress,we may head in one direction.

79 km 75 km

Exit


87

The Sum ProblemIn order to make progress,we may head in one direction.

79 km 75 km

Exit


88

The Sum Problem

In order to maintain the loop invariant,we must adjust to come back to the path.

Exit

89

The Sum Problem

This may be how we determine the step in then next iteration.

90


• Loop Invariant: s = 12 + 22 + 32 + … + i2 (1 ≤ i ≤ I)• Partial Step: i = i+1


<loop-invariantt+1>


Exit

Let it and st be the values of i and s at the beginning of the iteration and let it+1 and st+1 be their after going around again.

91


• Loop Invariant: s = 12 + 22 + 32 + … + i2 (1 ≤ i ≤ I)• Partial Step: i = i+1 Use the fact that you

already have the LI trueand take a small step.

<loop-invariantt>

¬<exit Cond>codeB:

<loop-invariantt+1>

Make the loop Invariant true!!!Solve the entire algorithm!!!Panic!!

next

92



st = 12 + 22 + 32 + … + it2

it < I

it+1 = it + 1

st+1

=12 + 22 + 32 + … + it+12

<loop-invariantt>

¬<exit Cond>codeB:

<loop-invariantt+1>

(codeB in loop)

(math)

Write down LHS/RHSof what we must prove

93



st = 12 + 22 + 32 + … + it2

it < I

it+1 = it + 1

st+1

=[12 + 22 + 32 + … + it2 ]+ it+1

2

=12 + 22 + 32 + … + it+12

= st + it+12

<loop-invariantt>

¬<exit Cond>codeB:

<loop-invariantt+1>

st+1 = st + it+12

94



s = s + i2

i = i + 1

it+1 = it + 1

st+1 = st + it+12

i = i + 1 s = s + i2

95


• Loop Invariant: s = 12 + 22 + 32 + … + i2 (1 ≤ i ≤ I)• Step: i = i+1, s = s + i2


<postCond>

Clean up loose ends

Exit

96



s = 12 + 22 + 32 + … + i2 i = I

S = s

S =12 + 22 + 32 + … + I2

<loop-invariant>

<exit Cond>codeC:

<postCond>

I2

97



<preCond>codeA

<loop-invariant>


i s

0123 1+4+9=14

1+4=51

0i = 0s = 0

98





km

79 km

to school

Exit

Exit

79 km 75 km

Exit

Exit

0 km Exit

99

Simple Example

Insertion Sort Algorithm

100

Reconsidering Simple Examples

A good martial arts student will attentively repeat each fundamental technique many times.

In contrast, many college students tune out when a concept (e.g., depth first search) is taught more than once.

The better students listen carefully in order to refine and develop their understanding.

Repetition Is An Opportunity Rudich www.discretemath.com

101

Code Representation of an Algorithm

class InsertionSortAlgorithm extends SortAlgorithm {

void sort(int a[]) throws Exception {

for (int i = 1; i < a.length; i++) {

int j = i;

int B = a[i];

while ((j > 0) && (a[j-1] > B)) {

a[j] = a[j-1];

j--; }

a[j] = B;

}}

102

Higher Level Abstract ViewRepresentation of an Algorithm

9 km

5 km

103

Problem Specification• Precondition: The input is a list of n values

with the same value possibly repeated.

• Post condition: The output is a list consisting of the same n values in non-decreasing order.

8814

982562

52

79

3023

31 14,23,25,30,31,52,62,79,88,98

104

• “State” of computation determined by values of all variables.

Location of Computation

105

Location of Computation

88 14

9825 62

52

79

3023

31 14,23,25,30,31,52,62,79,88,98

• “State” of computation determined by values of all variables.

88 14<5225<62

79<98

3023

31

106

88 14

9825 62

52

79

3023

31 14,23,25,30,31,52,62,79,88,98

23

• Location too confusing for algorithm

• Algorithm too lazy to define step for every location.

Computation Stuck

88 14<5225<62

79<98

3031

107

• Algorithm specifies from which locations it knows how to step.

Safe Locations

88 14

9825 62

52

79

3023

31 14,23,25,30,31,52,62,79,88,98

108

88 14

9825 62

52

79

3023

31 14,23,25,30,31,52,62,79,88,98

14

9825 62

79

3023,31,52,88

Define Loop Invariant• Loop Invariant: • Input numbers split to left and right.• Numbers on left are sorted.

109

14

9825 62

79

3023,31,52,88

23

9825 88

31

3014,52,62,79

Location Not Determined

Which subset of the elements are sorted, is not specified.

110

Defining Measure of Progress

14

9825 62

79

3023,31,52,88

6 elements

to school

111

Define Step

9825 62

79

23,31,52,881430

112

Define Step• Select arbitrary element from side.• Insert it where it belongs.

9825 62

79

23,31,52,88

14

9825

79

3023,31,52,62,88

1430

113

Making progress while Maintaining the loop invariant

9825 62

79

23,31,52,88

14

9825

79

3023,31,52,62,88

1430

79 km 75 km

5 elements

to school

6 elements

to school

Exit

Sorted sub-list

114

88 14

9825 6

2

52

79

3023

31

88 14

9825 6

2

52

79

3023

31n elements

to school

14,23,25,30,31,52,62,79,88,98

14,23,25,30,31,52,62,79,88,980 elements

to school

Beginning & Ending

km Exit0 km Exit

115

Explaining Insertion Sort We maintain a subset of elements sorted within a

list. The remaining elements are off to the side somewhere. Initially, think of the first element in the array as a sorted list of length one. One at a time, we take one of the elements that is off to the side and we insert it into the sorted list where it belongs. This gives a sorted list that is one element longer than it was before. When the last element has been inserted, the array is completely sorted.

But don’t do this. I want separate paragraphs for the separate steps.

116

Running Time Inserting an element into a list of size i

takes (i) time.

Total = 1+2+3+…+n

9825 62

79

23,31,52,88

14

9825

79

3023,31,52,62,88

1430

= ?

117

1 2 . . . . . . . . n

= number of white dots1 + 2 + 3 + . . . + n-1 + n = S

Adding Made Easy

118

1 + 2 + 3 + . . . + n-1 + n = S

n + n-1 + n-2 + . . . + 2 + 1 = S

1 2 . . . . . . . . n

= number of yellow dots

n . . . . . . . 2 1

= number of white dots

Adding Made Easy

119

1 + 2 + 3 + . . . + n-1 + n = S

n + n-1 + n-2 + . . . + 2 + 1 = S

= number of white dots

= number of yellow dots

n+1 n+1 n+1 n+1 n+1

n

n

n

n

n

n

= n(n+1) dots in the grid

2S dots

S = n (n+1)

2

Adding Made Easy

120

Adding Made Easy

121

Running Time Inserting an element into a list of size i

takes (i) time.

Total = 1+2+3+…+n

9825 62

79

23,31,52,88

14

9825

79

3023,31,52,62,88

1430

= (n2)

122

Algorithm Definition CompletedDefine Problem Define Loop




km

79 km

to school

Exit

Exit

79 km 75 km

Exit

Exit

0 km Exit

123

14

9825 62

79

3023,31,52,88

• Loop Invariant: • Input numbers split to left and right.• Numbers on left are sorted.• Select the smallest

Insertion SortSelection

What changes? This was on an exam and most got it

Action not picture

WRONG

124

88

9852 62

79

3114,23,25,30

• Loop Invariant: • Input numbers split to left and right.• Numbers on left are sorted.• All left ≤ All right

Insertion SortSelection

What changes? This was on an exam and most got it

≤

WRONG

125

Define Step

88

9852 62

79

3114,23,25,30 ≤

126

Define Step

88

9852 62

79

3114,23,25,30 ≤

88

9852 62

79

14,23,25,30,31 ≤

Select the smallest on right and put it at the end of the left.

127


88

9852 62

79

3114,23,25,30 ≤

88

9852 62

79

14,23,25,30,31 ≤


Exit

• Numbers now on left are sorted.

• Numbers on left were sorted.

• New number added to end came from right.

• All left ≤ All right <loop-invariantt+1>

128


88

9852 62

79

3114,23,25,30 ≤

88

9852 62

79

14,23,25,30,31 ≤


Exit

• Now

All left ≤ All right

• Was All left ≤ All right• New number is smallest

from right

<loop-invariantt+1>

129

OkI know you knew Insertion Sort

But hopefully you are beginning to appreciate

Loop Invariants

for describing algorithms

130

Define Problem: Binary Search

• PreConditions– Key 25– Sorted List

• PostConditions– Find key in list (if there).

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

131

Define Loop Invariant

• Maintain a sub-list• Such that

key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

132

Define Loop Invariant

• Maintain a sub-list.• If the key is contained in the original list,

then the key is contained in the sub-list.

key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

133

Define Step

• Make Progress• Maintain Loop Invariant

key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

134

Define Step

• Cut sub-list in half.• Determine which half the key would be in.• Keep that half.

key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

135

Define Step


key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

If key ≤ mid,then key is inleft half.

If key > mid,then key is inright half.

136

Define Step

• It is faster not to check if the middle element is the key.

• Simply continue.

key 43

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95



137

Make Progress

• The size of the list becomes smaller.

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

79 km

75 km

79 km 75 km

Exit

138

Initial Conditionskm

key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

• If the key is contained in the original list,


• The sub-list is the entire original list.

n km

139

Ending Algorithm



• Sub-list contains one element.

Exit

Exit

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

0 km


then the key is at this location.

key 25

Check this location.• If it’s the key, then you have found it.• If it’s not, then it was not in the original list.

140

Running Time The sub-list is of size n, n/2, n/4, n/8,…,1

Each step (1) time.

Total = (log n)

key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95



141

Code

142





km

79 km

to school

Exit

Exit

79 km 75 km

Exit

Exit

0 km Exit

143

Study:

• Many experienced programmers were asked to code up binary search.

80% got it wrong

Good thing is was not for a nuclear power plant.

144

Make Precise Definitions

• Maintain a sub-list with end points i & j.

key 25

i j

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

145


• Maintain a sub-list with end points i & j.

key 25

i j

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

Does not matter which,but you need to be consistent.

146

• If the sub-list has even length,

which element is taken to be mid?

key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95


mid or mid ?2 2

i j i j

147

• If the sub-list has even length,

which element is taken to be mid?

key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

Should not matterChoose right.


mid2

i j

mid

148

Common Bugs


key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95



149

Common Bugs

• If the middle element is the key,

it can be skipped over.

key 43

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95



Exit

150

Common Bugs• Fix the bug.

key 43

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95



151

Common Bugs• Second fix,

by making the left half slightly bigger.

key 43

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95



•New bug?

152

Common Bugs• Second fix,

by making the left half slightly bigger.

key 43

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

•New bug?

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

No progress is made.Loop for ever!

79 km 75 km

Exit

153

Why is Binary Search so Easy to Get Wrong?

?R 2

Oi j

OR > ?

mid2

i j 1

if key L(mid)2

• How many possible algorithms?

• How many correct algorithms?

• Probability of success by guessing?

3i = mid

else

j = mid -1

OR

i = mid + 1

else

j = mid

154

Moral

• Use the loop invariant method to think about algorithms.

• Be careful with your definitions.• Be sure that the loop invariant is always

maintained.• Be sure progress is always made.

155

Loop Invariants for

Iterative Algorithms

A second

Binary Search like

example

156

38

25

17

4 21

31

28 35

51

42

40 49

63

55 71

Binary Search Tree• <preCond>

– Key 17– A binary search tree.

• <postCond>– Find key in BST (if there).

key 17?

157

38

25

17

4 21

31

28 35

51

42

40 49

63

55 71

Binary Search Tree

≤ ≤

Its left children ≤ Any node ≤ Its right children

Data Structure Invariant

key 17?

158

38

25

17

4 21

31

28 35

51

42

40 49

63

55 71

Binary Search Tree

≤ ≤

• We have maintained a sub-tree.• If the key is contained in the original tree,

then the key is contained in this sub-tree.

Search Loop Invariant

key 17?

159

• Cut sub-tree in half.• Determine which half the key would be in.• Keep that half.

38

25

17

4 21

31

28 35

51

42

40 49

63

55 71

If key < root,then key is in left half.

If key > root,then key is in right half.

If key = root,then key is found

Binary Search Tree

≤ ≤key 17?

160

38

25

17

4 21

31

28 35

51

42

40 49

63

55 71

Binary Search Tree

key 17?

• We have maintained a sub-tree.• If the key is contained in the original tree,

then the key is contained in this sub-tree.

Search Loop Invariant

161





km

79 km

to school

Exit

Exit

79 km 75 km

Exit

Exit

0 km Exit

162

Loop Invariants for

Iterative Algorithms

A third

Binary Search like

example

163

• A volunteer, please.

Card Trick

164

Pick a Card

Done

165

Loop Invariant:The selected card is one of

these.

166

Which column?

left

167


these.

168

Selected column is placedin the middle.

169

I will rearrange the cards.

170

Relax Loop Invariant:I will remember the same about

each column.

171

Which column?

right

172


these.

173


174

I will rearrange the cards.

175

Which column?

left

176


these.

177


178

Here is your card.

Wow!

179

Bucket (Quick) Sort for Humans Input: A pile of things to sort. (Ex: Exams)Output: Sort themRequirements:• Easy to execute by humans• Only can have 7 piles on desk• Can move one exam (or pile)

at a time.• Fast – O(nlog(n)) time.

Likely the only algorithm in this course which you willexecute by hand yourself.

180

Bucket (Quick) Sort for Humans

[A-Z]

Denotes an unsorted pile of examswith last names starting in the range [A-Z]

Input:

181


[A-Z]

Psychology study: Humans can only think about 5 things at once.

Is the first letterof the name on the first exam

[A-E][F-K][L-O][P-T][U-Z]

in

182


[A-Z]

[A-E] [F-K] [L-O] [P-T] [U-Z]

Bucket Sort• Put exams one at a time

in the correct bucket.

183



A stack of piles• each pile unsorted• all in one pile before all in next

A sorted pile • before all the rest

(upside down)

[]

sorted

184


[]

sorted

[A] [B] [C] [D] [E]

Bucket Sort• Put exams one at a time

in the correct bucket.


185



(upside down)

[]

sorted

[F-K][L-O][P-T][U-Z]

[A][B][C][D][E]


186

[]

sorted

[F-K][L-O][P-T][U-Z]

[A][B][C][D][E]


[AA-AE][AF-AK][AL-AO][AP-AT][AU-AZ]

187

[]

sorted

[F-K]

[U-Z]

[B]

[E]


……

[AA-AE]

[AU-AZ]…



(upside down)

188

[]

sorted

[F-K]

[U-Z]

[B]

[E]


……

[AA-AE]

[AU-AZ]…

[AA-AE]

189

[]

sorted

[F-K]

[U-Z]

[B]

[E]


……

[AF-AK]

[AU-AZ]…

[AA-AE]

sorted

When sufficiently smallsort by hand

190

[AA-AE]

sorted

[F-K]

[U-Z]

[B]

[E]


……

[AF-AK]

[AU-AZ]…



(upside down)

191

[AA-AE]

sorted

[F-K]

[U-Z]

[B]

[E]


……

[AF-AK]

[AU-AZ]…

[AF-AK]

sorted

192

[AA-AK]

sorted

[F-K]

[U-Z]

[B]

[E]


……

[AL-AO]

[AU-AZ]…



(upside down)

193

[AA-AK]

sorted

[F-K]

[U-Z]

[B]

[E]


……

[AL-AO]

[AU-AZ]…

[AL-AO][AP-AT]

[AU-AZ]sorted

sortedsorted

194

[F-K]

[U-Z]

[B]

[E]


……[AA-AZ]

sorted

195

[F-K]

[U-Z]

[B]

[E]


……

[A]



(upside down)

sorted

196

[A]

[F-K]

[U-Z]

[B]

[E]


……

[BA-BE][BF-BK][BL-BO] [BP-BT][BU-BZ]

sorted

197

[A]

[F-K]

[U-Z]

[C]

[E]


……

[BA-BE]

[BU-BZ]…



(upside down)

sorted

198

[A-ZT]


[ZU-ZZ]

[ZU-ZZ]

sorted

sorted

199

[A-Z]


Exit

sorted

200

Bucket (Quick) Sort for Humans Fast – O(nlog(n)) time.• Each time you touch an exam,

the size of its pile goes down by a factor of 5• Hence I touch it only log5n times. (Assuming names distributed randomly through alphabet)

[A-Z]

[A-E] [F-K] [L-O] [P-T] [U-Z]

[A-Z]

[A-E] [F-K] [L-O] [P-T] [U-Z]

201

Reverse Polish Notation CalculatorMany calculators in 1970-1980 used what we called Reverse Polish Notation

Input: A string such as 3 1 4 + 2 Output: Determine the output of the calculator.

Numbers are pushed on the stack.

Operations pop two numbers and push back the result.

3 1 4 + 2 =Stack

3 1 45 21030

3 1 4 2+

30

202

Whose Blocking Your View?Input: A string buildingsOutput: For each, who is blocking its left view.

For each building, must search back.May have to search back O(n) buildings.Time = O(n2).We need a better loop invariant / stack algorithm!

203


A prefix of the buildings are done.The stack contains the sorted list of candidates for the next buildings.Pop all building that are too short for the next building,finding his blocker.Then push this next building.

Stack

204


A prefix of the buildings are done.The stack contains the sorted list of candidates for the next buildings.Pop all building that are too short for the next building,finding his blocker.Then push the next building.

Stack

205


Stack

For each building, must pop the stack searching back.May have to pop O(n) buildings.Time = O(n2).

Oops. No better!

But each building, is popped at most once.Time = O(n).

Computing Spans (not in book)• Using a stack as an auxiliary data

structure in an algorithm• Given an array X, the span S[i] of

X[i] is the maximum number of consecutive elements X[j] immediately preceding X[i] and such that X[j] X[i]

• Spans have applications to financial analysisExample: stock at 52-week high

Andranik Mirzaian 206

6 3 4 5 2

1 1 2 3 1X

S

01234567

0 1 2 3 4

Whose Blocking Your View?

Quadratic Algorithm

Last Update: Oct 1, 2014 Andranik Mirzaian 207

Algorithm spans1(X, n) // O(n2) time Input: array X of n integersOutput: array S of spans of X # of stepsS new array of n integers nfor i 0 .. n 1 do n

s 1 nwhile s i and X[i - s] X[i] 1 + 2 + …+ (n 1)

s s + 1 1 + 2 + …+ (n 1)

S[i] s nreturn S 1


Computing Spans with a Stack• Stack holds indices of the

elements visible when “looking back”

• Scan X left to right i current index pop stack until top index

j satisfies X[j] > X[i] set S[i] i j push X[i] onto stack


01234567

0 1 2 3 4 5 6 7


Linear Time Algorithm


Algorithm spans2(X, n) // O(n) time S new array of n integers

A new empty stack

for i 0 .. n 1 dowhile (!

A.isEmpty() && X[A.top()]

X[i] ) doA.pop()

if A.isEmpty()

then S[i]

i + 1else S[i] i -

A.top()A.push(i)

return S

Analysis:• Each index of X

o Is pushed into the stack exactly once

o Is popped from the stack at most once

• while-loop iterates at most n times, since each iteration has to pop.

• Algorithm spans2 runs in O(n) time.


210

Parsing with a StackInput: A string of brackets.

Output: Each “(”, “{”, or “[” must be paired with a matching “)”, “}”, or “[”.

[( )(( ))}{([( )])}]

incorrect

[( ) (( ) ){([()])}]

211



Loop Invariant: Prefix has been read.• Matched brackets are matched and removed.• Unmatched brackets are on the stack.

[()(()

Stack

[(

212



Loop Invariant: Prefix has been read.• Matched brackets are matched and removed.• Unmatched brackets are on the stack.

Stack

[ (

Opening Bracket: • Push on stack.

Closing Bracket: • If matches that on stack

pop and match.• else return(unmatched)

[()(()){

( ({

}])

public static boolean isMatched (String expression) { final String opening = "({["; // opening delimiters final String closing = ")}]"; // respective closing delimiters Stack<Character> buffer = new LinkedStack<>( ); for (char c : expression.toCharArray( )) { if (opening.indexOf(c) != −1) // this is a left delimiter buffer.push(c); else if (closing.indexOf(c) != −1) { // this is a right delimiter if (buffer.isEmpty( )) // nothing to match with return false; if (closing.indexOf(c) != opening.indexOf(buffer.pop( ))) return false; // mismatched delimiter } } return buffer.isEmpty( ); // were all opening delimiters matched?}

Andranik Mirzaian 213

Parsing with a Stack

214

begin Algx = 5begin loop

x = ( [ { 3 } ] )exit loopreturn xend Alg

Linked


215

Op

If

if (Boolean) then Op

Equ=Equ

Term Term+Equ

FactorTerm TermFactor

Factor Factor

y x 73


216




• System, Data Structure, and Loop Invariants draw a picture of what must be maintained through out time while making progress.

217

Dude! You have been teaching 3101 too long.This is not an course on Algorithms,

but on Data Structures!

Data Structure Invariants

The importance of invariants is the same.

Differences:1. An algorithm must terminate with an answer,

while systems and data structures may run forever.

2. An algorithm gets its full input at the beginning, while data structures gets a continuous stream of instructions from the user.

• Both have invariants that must be maintained.

218

Data Structure InvariantsA Data Structure: • Has data organized in ways dictated by its invariants.• Has allowed operations on that data.

Implementer:Coder of the data structure.

Three Players

User:Coder using data structure.

End User:Runs the program

Not part of this course.

This is us inCSE 2011

Complicated details are

hidden from him. Correctness & efficiency.

219

Data Structure InvariantsAn Abstract (Public) Data Structure: The minimal needed by the User so he can• understand how best to visualize the data• and what each operation does.Eg of Public Invariants:• Bank: The amount is not negative. • Stack: Contains an ordered list of objects which cannot

be changed except at the top via the explicit push and pop operations.

• Drug Allocation Data Base: A patient is not simultaneously prescribed a set of drugs that interact poorly with each other.

The implementer must make sure that each operationmaintains these invariants.

User

Implementer

Important in recursion and parsing.

220

Data Structure InvariantsAn Implemented (Private) Data Structure: • The details need to be worked out so that operations

are correct and efficient. • This is all hidden from the User.Eg of Private Invariants:• Bank: • Stack:

• Drug Allocation Data Base:

User

Implementer

Charge extra fees.

For each patient give name address and list of drugs.

or

221

ConstructorpreCondConstructor

InvariantsData Struc

Establishing Loop InvariantData Structure Invariants

When a user wants to construct an instanceof data structure the implementer must make sure that its constructor establishes all of its loop-invariants.

Initially the stack contains no objects.

222

InvariantsData StrucDestructor preCondDestructor

postCondData Struc

Clean up loose ends

Exit


A destructor for the data structuremust clean up loose ends.

223


Assume we fly in from Mars and InvariantsData Struc t is true:

InvariantsData Struc t+1

postCondPush

Maintaining Loop InvariantExit

InvariantsData Struc tPush OperationpreCondPush

Assume the user correctly calls the Push Operation: preCondPush The input is info for a new element.Implementer must ensure: postCondPush The element is pushed on top of the stack. InvariantsData Struc t+1

224

Data Structure Invariants• Do not worry about the entire computation.

• Take one step at a time!

top = top + 1;A[top] = info;

225


postCondPop The element is popped off top of the stack. InvariantsData Struc t+1

info = A[top];top = top - 1;

226

Data Structure InvariantsStack: Add and Remove from same end.

Queue: Add and Remove from opposite ends.

227

Data Structure InvariantsQueue: Add and Remove from opposite ends.

How would we removean element from the front end?

Shifting all these elements to the beginning of the array

time # of elementsinstead of constant

228



Leave the elements in place.

Change the loop invariant!

229



Leave the elements in place.

Change the loop invariant!

230



Change the loop invariant!Algorithm dequeue()

if isEmpty() then

throw EmptyQueueException

else

info A[bottom]

bottom (bottom + 1) mod N

return e

231


• Limitations of an array implementation.– The maximum size must be

defined a priori and cannot be changed

ie could be too much or too little.– Trying to push a new element into

a full stack causes an exception

or you need to create a bigger array and copy everything over.

232

Data Structure InvariantsAn Implemented (Private) Data Structure: • The details need to be worked out so that operations

are correct and efficient. • This is all hidden from the User.Eg of Private Invariants:• Bank: • Stack:

• Drug Allocation Data Base:

User

Implementer

Charge extra fees.

For each patient give name address and list of drugs.

or

233

ConstructorpreCondConstructor

InvariantsData Struc

Establishing Loop InvariantData Structure Invariants

When a user wants to construct an instanceof data structure the implementer must make sure that its constructor establishes all of its loop-invariants.

Initially it contains no objects.

234




postCondPush


InvariantsData Struc tPush OperationpreCondPush

Assume the user correctly calls the Push Operation: preCondPush The input is info for a new element.Implementer must ensure: postCondPush The element is pushed on top of the stack. InvariantsData Struc t+1

235

Data Structure InvariantsInvariantsData Struc t

preCondPush

postCondPushInvariantsData Struc t+1

Don’t panic. Just draw the pictures and move the pointers.

236


preCondPush


237


preCondPush


Special Case: Empty

238




postCondPop


InvariantsData Struc tPop OperationpreCondPop

Assume the user correctly calls the Pop Operation: preCondPop No inputs

Implementer must ensure: postCondPop The top element is Popped off stack and returned. InvariantsData Struc t+1

239


preCondPop

postCondPopInvariantsData Struc t+1

Don’t panic. Just draw the pictures and move the pointers.

240


preCondPop


241


preCondPop


Special Case: Empty

242


preCondAdd Rear

postCondAdd RearInvariantsData Struc t+1

How about adding a new element to the rear?

Not hard.Left as homework.

243


preCondRemove Rear

postCondRemove RearInvariantsData Struc t+1

How about removing an element from the rear?

Is it so easy???

last must point at the second last element.

How do we find it?

You have to walk there from first!

time # of elementsinstead of constant

244


Front Rear

Add Element Time Constant Time Constant

Remove Element Time Constant Time n

Stack: Add and Remove from same end.

Actually, for a Stack the last pointer is not needed.

245


Front Rear


Remove Element Time Constant Time n

Stack: Add and Remove from same end.

Queue: Add and Remove from opposite ends.

246


Front Rear


Remove Element Time Constant Time nTime Constant

trailerheader nodes/positions

elements

Doubly-linked lists allow more flexible list

247


preCondFindSuppose you want to find where the new element 6 would go?

InvariantsData Struc t

postCondFind

We want the location sandwiched.

How do you get there?

Walk!

248


preCondFind

InvariantsData Struc t

postCondFind


What would you like to be true in the middle of your computation?

249

Data Structure InvariantsHow do you

initialize this walk?

Sandwich the location before the first node.

250


Just take one step.

How do you maintain this while making progress?

Exit

251


Exit

252

Data Structure InvariantspreCondInsert

postCondFind


The location for the new element has been found.

PostCondInsert The new element has been inserted where it belongs.

253

Data Structure InvariantspreCondInsert The location for the new

element has been found.

PostCondInsert The new element has been inserted where it belongs.

postCondInsertInvariantsData Struc t+1

254

Data Structure InvariantspreCondDelete The element to be deleted

has been found.

PostCondDelete The found element

has been deleted

postCondDeleteInvariantsData Struc t+1

255

Learning Outcomes

• From this lecture, you should be able to:– Use the loop invariant method to think about iterative algorithms.– Prove that the loop invariant is established.– Prove that the loop invariant is maintained in the ‘typical’ case.– Prove that the loop invariant is maintained at all boundary

conditions.– Prove that progress is made in the ‘typical’ case– Prove that progress is guaranteed even near termination, so that

the exit condition is always reached.– Prove that the loop invariant, when combined with the exit

condition, produces the post-condition.– Trade off efficiency for clear, correct code.

256

End


1 Jeff Edmonds York University COSC 2011 Lecture 3 Contracts Assertions Loop Invariants The Sum of...

Documents

Transcript of 1 Jeff Edmonds York University COSC 2011 Lecture 3 Contracts Assertions Loop Invariants The Sum of...