Introduction to Computer Science I Topic 6: Generative Recursion

Telecooperation/RBG

Technische Universität Darmstadt

Copyrighted material; for TUD student use only

Introduction to Computer Science ITopic 6: Generative Recursion

Prof. Dr. Max MühlhäuserDr. Guido Rößling

Dr. G. RößlingProf. Dr. M. Mühlhäuser

RBG / Telekooperation©

Introduction to Computer Science I: T6

2

Outline

• Introduction to generative recursion• Sorting with generative recursion• Guidelines for the design of generative

recursive procedures• Structural versus generative recursion• Backtracking: Traversing graphs




3

Generative Recursion• So far, we have used structural recursion to

process structurally recursive data– We have decomposed the data to their immediate

structural components– We have processed these components and combined the

results• However:

1. Not all problems can be solved using structurally recursive functions

2. Even if there is a structurally recursive solution for a problem, it might not be optimal

• Now we will consider a new programming style: Generative recursion




4

Generative Recursion• Divide and Conquer is an important principle

underlying generative recursion – If the problem is trivial to solve, the solution for the

trivial case is returned– Otherwise:

• Divide the problem in new smaller problems (generate new sub-problems)

• Conquer the sub-problems by applying the same technique recursively

• Combine the solutions for the sub-problems into a solution for the original problem

• The design of generative recursive programs is more an ad-hoc activity as compared to the design of the structural recursive programs that needs an insight – a “Eureka!"




5

Modeling a rolling ball on a tableTask Description:

– The ball rolls at a constant speed until it drops off the edge of the table

– We can model the table as a canvas/surface with a pre-defined length and width

– The ball can be represented as a disc that moves across the canvas

– The disc movement can be represented by repeating the following steps:

• Draw the disc at the current position on the canvas• Wait for a certain pre-defined time• Erase the disc at the current position• Move it to a new position




6

Ball Structure and Operations;;TeachPack: draw.ss

;; structure: (make-ball number number number number)(define-struct ball (x y delta-x delta-y))

;; draw-and-clear : a-ball -> true(define (draw-and-clear a-ball) (and (draw-solid-disk (make-posn (ball-x a-ball) (ball-y a-ball)) 5 'red)

(sleep-for-a-while DELAY)

(clear-solid-disk (make-posn (ball-x a-ball) (ball-y a-ball)) 5 'red)))




7

Ball Structure and Operations;; move-ball : ball -> ball(define (move-ball a-ball) (make-ball (+ (ball-x a-ball) (ball-delta-x a-ball)) (+ (ball-y a-ball) (ball-delta-y a-ball)) (ball-delta-x a-ball) (ball-delta-y a-ball)))

;; Dimension of surface (define WIDTH 100)(define HEIGHT 100)

;; Delay constant(define DELAY .1)




8

Ball Structure and Operations

To move the ball multiple times we can write:

This gets tedious after a while. We need a function that moves the ball until it is out of bounds.

(define the-ball (make-ball 10 20 -5 +17)) (and (draw-and-clear the-ball) (and (draw-and-clear (move-ball the-ball)) ...))




Rolling the BallDetermine whether a-ball is outside of the bounds:

Template for moving the ball until it is out of bounds:

;; out-of-bounds? : a-ball -> boolean(define (out-of-bounds? a-ball) (not (and (<= 0 (ball-x a-ball) WIDTH) (<= 0 (ball-y a-ball) HEIGHT))))

;; move-until-out : a-ball -> true(define (move-until-out a-ball) (cond [(out-of-bounds? a-ball) ... ] [else ...]))

The trivial case: return true

?true




10

Rolling the Ball

After drawing and moving the ball, we apply move-until-out again, which means the function is recursive:

We can now test the function as follows: The code creates a canvas of proper size and a ball that moves to the bottom left of the canvas.

(start WIDTH HEIGHT) (move-until-out (make-ball 10 20 -5 +17)) (stop)

(define (move-until-out a-ball) (cond [(out-of-bounds? a-ball) true] [else (and (draw-and-clear a-ball) (move-until-out (move-ball a-ball)))]))




11

A New Type of Recursion

The procedure move-until-out uses a new type of recursion:– Conditions have nothing to do with the input data– Recursive application in the body does not use part of the

input • move-until-out generates an entirely new and

different ball structure and uses it for the recursion We do not have a design recipe for this

(define (move-until-out a-ball) (cond [(out-of-bounds? a-ball) true] [else (and (draw-and-clear a-ball) (move-until-out (move-ball a-ball)))]))




12

Outline






13

Sorting: Quicksort and Mergesort• We are once more concerned with sorting the

elements of a list …

– We have seen insertion sort:• A structurally recursive procedure

– Now we will see two other algorithms for sorting: Quicksort and Mergesort

• Classic examples of generative recursion• Based on the idea of “divide and conquer”




14

[Reminder: insertion sort];; insertion-sort: list-of-numbers -> list-of-numbers ;; creates a sorted list of numb. from numbers in alon (define (insertion-sort alon) (cond [(empty? alon) empty] [else (insert (first alon) (insertion-sort (rest alon)))]))

anan

sortedunsorted




15

Quicksort: The Idea• An arbitrary intermediate step: sorting an arbitrary

sub-list:L0 =(list elp…elr)– Divide: Partition L0 in two (possibly empty) lists, L1 = (list elp…elq-1) and L2 = (list elq+1…elr), such that

• each element of L1 is smaller than elq, and • the latter is in turn smaller than each element in L2

– Conquer: Apply the same procedure recursively to sort L1 and L2

– Combine: simply concatenate the sorted lists L1 and L2

<= elq > elqelq

Turning point




16

Quicksort: The Idea• Two open questions so far:

– How do we select the pivot element?• We always use the first element

– When do we stop? That is: what is the trivial case of Quicksort?• The empty list is always sorted!




Quicksort: Approach1. Select the first element from the list as pivot item2. Determine the first sub-list with elements <= pivot3. Sort this sub-list recursively using Quicksort4. Determine the second sub-list with elements > pivot5. Sort this sub-list recursively using Quicksort6. Concatenate the sorted sub-lists to a new list

17




Quicksort @ Work

18

Sort (list 11 8 7 14): 1. Select the first element from '(11 8 7 14) as pivot item:112. Determine the first sub-list <= 11: '(8 7)3. Sort this sub-list

1. Select the first element from '(8 7) as pivot item: 82. Determine the first sub-list with elements <= 8: '(7)3. Sort this sub-list

1. Select the first element from '(7) as pivot item: 72. Determine the first sub-list with elements <= 7: empty3. Sort this sub-list Result empty4. Determine the second sub-list with elements > 7: empty5. Sort this sub-list Result empty6. Concatenate (empty 7 empty) to a new list -> (list 7)

4. Determine the second sub-list with elements > 8: empty5. Sort this sub-list Result empty6. Concatenate((list 7) 8 empty) to a new list (list 7 8)

4. Determine …




Quicksort at Work

19

(list 8 7)

(list 7)

empty 7

empty

(list 7)

8

empty

(list 7 8)

11

(list 14)

empty 14

empty

(list 14)

(list 7 8 11 14)




20

Quicksort schematically

List to be sorted Sorting process for the partition with elements smaller than the pivot element

sorting process for the partition with elements larger than the pivot element

Sorted list

pivot item




21

Quicksort Algorithm• quick-sort distinguishes two cases:

– If the input is empty, it returns empty. – Otherwise, it performs recursion.

• Each sub-list is sorted separately using quick-sort • The sorted versions of the two lists are then combined using

append

;; quicksort2: (listof number) -> (listof number)(define (quicksort2 alon) (cond [(empty? alon) empty] [else (append (quicksort2 (less-or-equal (rest alon) (first alon))) (list (first alon)) (quicksort2 (greater-than (rest alon) (first alon)))) ]))




22

Auxiliary Functions of Quicksort• greater-than filters out the items that are larger

than threshold:

• less-than filters out the items that are smaller than threshold:

(define (greater-than alon threshold) (filter1 > alon threshold)))

(define (less-than alon threshold) (filter1 < alon threshold)))




23

Quicksort Evaluation Example (quick-sort (list 11 8 14 7))

= (append (quick-sort (list 8 7)) (list 11) (quick-sort (list 14)))

= (append (append (quick-sort (list 7)) (list 8) (quick-sort empty)) (list 11) (quick-sort (list 14)))

= (append (append (append (quick-sort empty) (list 7) (quick-sort empty)) (list 8) (quick-sort empty)) (list 11) (quick-sort (list 14)))= ...




24

Quicksort Evaluation Example= (append (append (append empty (list 7) empty) (list 8) empty) (list 11) (quick-sort (list 14)))

= (append (append (list 7) (list 8) empty) (list 11) (quick-sort (list 14)))

= (append (list 7 8) (list 11) (quick-sort (list 14)))

= ...




25

mergesort: The IdeaIdea:1. split the list in the middle2. apply the function to the sub-lists recursively3. merge the two sorted lists into a new sorted list




26

Merging two ordered lists• Given two ordered lists ls1 and ls2• How do we merge them into a new ordered list?

1 2 4 7 3 5 6 8

ls-1 ls-2

1 2 4 73 5 6 8

sorted-list

Compare and copy the smaller element, then continue




27

Merging two ordered lists

(define (merge ls1 ls2) (cond [(empty? ls1) ls2] [(empty? ls2) ls1] [(<= (first ls1) (first ls2)) (cons (first ls1)

(merge (rest ls1) ls2)) ] [else (cons (first ls2)

(merge ls1 (rest ls2)))] ))




28

mergesort: algorithm in Scheme(define (merge-sort alon) (local ((define (merge-step left right) (cond [(>= left right) alon] [else (local ( (define mid (floor (/ (+ left right) 2)))

(define left-list (merge-sort (extract alon left mid)))

(define right-list (merge-sort (extract alon (+ mid 1) right)))) (merge left-list right-list) ) ] ))) (merge-step 1 (length alon))))




29

mergesort: algorithm in Scheme

(define (extract alon left right) (cond [(empty? alon) empty] [(> left right) empty] [(> left 1) (extract (rest alon) (- left 1) (- right 1))] [else (cons (first alon) (extract

alon (+ left 1) right))]))




30

Outline






31

Guidelines for DesigningGenerative Recursive Procedures

• Understand the nature of the data of the procedure

• Describe the process in terms of data, creating a new structure or partitioning a list of numbers

• Distinguish between those input data– which can be processed trivially, – and those which cannot

• The generation of problems is the key to algorithm design

• The solutions of the generated problems must be combined




32

A Return to the Six Stages of Structural Design

1. Data analysis and design: – Analyze and define data collections representing the

problem2. Contract, purpose, header

- Specify what the function does- Explain in general terms how it works

3. Function examples: – Illustrate how the algorithm proceeds for some given

input4. Template:

– Follow a general template5. Definition:

– Answer the questions posed by the template6. Test

- Test the completed function- Eliminate the bugs




33

General Template forGenerative Recursive Functions

(define (generative-recursive-fun problem) (cond [(trivially-solvable? problem) (determine-solution problem)] [else (combine-solutions ... problem ... (generative-recursive-fun (generate-problem-1 problem)) ... (generative-recursive-fun (generate-problem-n problem)))]))




34

Procedure Definition1. What is a trivially solvable problem, and the

pertaining solution?

2. How do we generate new problems that are easier to solve than the initial problem?• Is there one new problem to generate, or are there

more?

3. Is the solution for the given problem the same as for (one of) the new problems?• Or do we have to combine solutions to create a solution

for the initial problem?• If so, do we require parts of the data constituting the

initial problem?




35

Termination of Structurally Recursive Procedures

• So far, each function has produced an output for a valid input Evaluation of structurally recursive procedures has

always terminated.• Important characteristic of the recipe for

structurally recursive procedures:– Each step of natural recursion uses an immediate

component of the input, not the input itself• Because data is constructed in a hierarchical

manner, the input shrinks at every stage – The function sooner or later consumes an atomic piece

of data and terminates.




36

Termination of Generative Recursive Procedures

• This characteristic is not true for generative recursive functions. – The internal recursions do not consume an

immediate component of the input, but some new piece of data, which is generated from the input.

• This step may produce the initial input over and over again and thus prevent the evaluation from ever producing a result– We say that the program is trapped in an infinite

loop.




37

Non-Terminating Procedures• What happens if we place the following three

expressions at the bottom of the DrScheme Definitions window and click execute?

• Does the second expression ever produce a value so that the third expression is evaluated and the canvas disappears?

(start WIDTH HEIGHT) (move-until-out (make-ball 10 20 0 0)) (stop)




38

Non-Terminating Procedures

;; less-or-equal: (list-of-numbers) number -> (list-of-numbers)(define (less-or-equal alon threshold) (cond [(empty? alon) empty] [else (if (<= (first alon) threshold) (cons (first alon) (less-or-equal alon threshold)) (less-or-equal (rest alon) threshold))]))

Instead of(rest alon)

(quick-sort (list 5))= (append (quicksort (less-or-equal 5 (list 5))) (list 5) (quicksort (greater-than 5 (list 5))))= (append (quicksort (list 5)) (list 5) (quicksort (greater-than 5 (list 5))))

The slightest mistake in process definition may cause an infinite loop:

Quicksort does not terminate with the new function




39

Termination Argument• The termination argument is an additional step

in the design recipe of generative recursive procedures

• Termination argument explains – why the process produces an output for every input – how the function implements this idea– when the process possibly does not terminate




40

Termination Argument for Quicksort

At each step, quick-sort partitions the list into two sublists using less-or-equal and greater-than. Each function produces a list that is smaller than the input (the second argument), even if the pivot element (the first argument) is an item on the list. Hence, each recursive application of quick-sort consumes a strictly shorter list than the given one.Eventually, quick-sort receives an empty list and returns empty.




41

New Termination CasesTermination argument may reveal additional termination cases.

This knowledge can be added to the algorithm:

So liefert (less-or-equal N (list N)) und(greater-than N (list N)) immer empty

;; quick-sort : (listof number) -> (listof number)(define (quick-sort alon) (cond [(empty? alon) empty] [(empty? (rest alon)) alon] [else (append (quick-sort (less-or-equal (rest alon) (first alon))) (list (first alon)) (quick-sort (greater-than alon (first alon)))) ]))




42

Guidelines for Algorithm DesignPhase Goal ActivityExamples Characterize the input-

output relationshipand the computational process via examples

• Create and show examples of trivially solvable problems

• Create and show examples that require recursive processing

• Illustrate how to work through the examples

Body Define an algorithm • Formulate tests for trivially solvable problems

• Formulate answers for the trivial cases

• Determine how to generate new problems from the given problem

• Determine how to combine the solutions of these problems into a solution for the given problem

…Termination

Argue that the algorithm terminates for all possible inputs

Show that the inputs to the recursive applications are smaller than the given input




43

Outline






44

Structural Recursion as aSpecial Case of Generative

Recursion(define (generative-recursive-fun problem) (cond [(trivially-solvable? problem) (determine-solution problem)] [else (combine-solutions problem (generative-recursive-fun (generate-problem problem)))]))

(define (generative-recursive-fun problem) (cond [(empty? problem) (determine-solution problem)] [else (combine-solutions

problem(generative-recursive-fun (rest problem)))]))

Template for generative recursion

Template for list processing

trivially-solvable? empty?generate-problem rest




45

Structural vs. Generative Recursion• Is there a difference between structural and

generative recursion?– Structurally recursive functions seem to be just special

cases of generative recursion – But this "it's all the same" attitude does not improve the

understanding of the design process• Structurally and generative recursive functions are designed

using different approaches and have different consequences.

Structural Recursion Generative RecursionRelies on a systematic data analysis

Requires a deep insight into the problem-solving process

Leads to naturally terminating functions

Requires a termination argument




46

Greatest Common Denominator (GCD)• Examples

– 6 and 25 are both numbers with several denominators: • 6 is evenly divisible by 1, 2, 3, and 6; • 25 is evenly divisible by 1, 5, and 25. • The greatest common denominator of 25 and 6 is 1.

– 18 and 24 have many common denominators : • 18 is evenly divisible by 1, 2, 3, 6, 9, and 18; • 24 is evenly divisible by 1, 2, 3, 4, 6, 8, 12, and 24. • The greatest common denominator is 6.




47

GCD Based on Structural RecursionTest for every number i = [min(n,m) .. 1] whether it divides both n and m evenly and return the first such number.

;; gcd-structural : N[>= 1] N[>= 1] -> N;; structural recursion using data definition of N[>= 1] (define (gcd-structural n m) (local ((define (first-divisor i) (cond [(= i 1) 1] [(and (= (remainder n i) 0) (= (remainder m i) 0)) i] [else (first-divisor (- i 1))] ) ) ) (first-divisor (min m n))))

Inefficient for large numbers!




48

Analysis of Structural Recursion• gcd-structural simply tests every number whether it

divides both n and m evenly and returns the first such number – For small natural numbers, this process works just fine

• However, for (gcd-structural 101135853 450146405) = 177 the procedure will compare 101135853 – 177 = 101135676 numbers! – Even reasonably fast computers spend several

minutes on this task.

• Enter the definition of gcd-structural into the Definitions window and evaluate the following expression… in the Interactions window … and go get a coffee (time (gcd-structural 101135853 450146405))




49

GCD Euclidean Algorithm• The Euclidean algorithm to determine the greatest

common denominator (GCD) of two integers is one of the oldest algorithms known– Appeared in Euclid’s Elements around 300 BC– However, the algorithm probably was not discovered by

Euclid and it may have been known up to 200 years earlier.

Insight: For two natural numbers n and m, n > m, GCD(n, m) = GCD(m, remainder(n,m))

(gcd larger smaller) = (gcd smaller (remainder larger smaller))

Example: GCD(18, 24) = GCD(18,remainder(24/18)) = GCD(18,6) = GCD(6,0) = 6




50

GCD Generative Algorithm

clever-gcd is based on generative recursion:– The trivially solvable case is when smaller is 0– The generative step calls clever-gcd with smaller and (remainder larger smaller)

;; gcd-generative : N[>= 1] N[>=1] -> N(define (gcd-generative n m) (local ((define (clever-gcd larger smaller) (cond [(= smaller 0) larger] [else (clever-gcd smaller (remainder larger smaller))])) ) (clever-gcd (max m n) (min m n))))

(gcd-generative 101135853 450146405) needs only 9 iterations!




51

GCD Euclidean AlgorithmLet n = mq + r, then any number u which divides both n and m (n = su and m = tu), also divides r

r = n – qm = su – qtu = (s – qt)u

Any number v which divides both m and r (m = s’v and r = t’v ), also divides nn = qm + r = qs´v + t´v = (s´q + t´)v

• Therefore, every common denominator of n and m is also a common denominator of m and r.

• gcd(n,m) = gcd(m,r) • It is enough if we continue the process with m and r • Since r is smaller in absolute value than m, we will reach r

= 0 after a finite number of steps.




52

Which one to use?• Question: can we conclude that generative

recursion is better than structural recursion?

• Answer: no, not automatically.– Even a well-designed generative procedure is not

always faster than structural recursion.• For example, quick-sort wins over insertion sort

only for large lists– Structural recursion is easier to design

• Designing generative recursive procedures often requires deep mathematical insight

– Structural recursion is easier to understand• It may be difficult to grasp the idea of the generative

step.




53

Which one to use?

Start by using structural recursion.

If it is too slow, try to design generative recursion.

Document the problem generation with good examples, give a good termination argument.




54

Outline






55

Traversing Graphs

• A graph is a collection of nodes and edges. • The edges represent one-way connections between the

nodes.• Can be used to describe

– a plan of one-way streets in a city, – relationships between persons,– connections on the Internet, etc.

A

B

E

C

F

D

G

Scheme - list representation

(define Graph '((A (B E)) (B (E F)) (C (D)) (D ()) (E (C)) (F (D G)) (G ())))




56

Traversing Graphs;; find-route : node node graph -> (listof node);; to create a path from origin to destination in G;; false, if there is no path, (define (find-route origin destination G) ...)

(find-route 'C 'D Graph) = (list 'C 'D)(find-route 'E 'D Graph) = (list 'E 'C 'D)(find-route 'C 'G Graph) = false

A

B

E

C

F

D

G




57

Backtracking AlgorithmsA backtracking algorithm follows a specific template:1. Pursue a (possible) path to a solution until

• the solution is found (success! terminate), or• the path cannot be continued.

2. If the path cannot be continued:• Walk back along the path unto the last branch where

alternatives exist that have not yet been chosen,• Choose such an alternative and proceed with step 1.

3. If the starting point is reached again and there are no more alternatives: failure! terminate

Usage examples: – "N Queens Problem", finding paths in graphs




58

Traversing Graphs - Example• Find the path from node A to G!

A

B

E

C

F

D

G

BACKTRACK !




59

Traversing Graphs• If the origin is equal to the destination, the

problem is trivial; the answer is (list destination).

• Otherwise, try to find route from all neighbors of the origin.

(define (find-route origination destination aGraph) (cond [(symbol=? origination destination) (list destination)]

[else ... (find-route/list (neighbors origination aGraph) destination aGraph) ...]))




60

Neighbor nodesneighbors is similar to the function contains-doll?

;; neighbors : node graph -> (listof node);; to lookup the neighbors of node in graph(define (neighbors node graph) (cond [(empty? graph)

(error 'neighbors "empty graph")] [(symbol=? (first (first graph)) node) (second (first graph))] [else (neighbors node (rest graph))]))




61

Traversing Graphs• find-route/list

– Processes a list of nodes – Determine, for each of them, whether a path to the

destination node in this graph exists

;; find-route/list : ;; (listof node) node graph -> (listof node) or false(define (find-route/list lo-origins destination G) ...)

The result of find-route depends on that of find-route/list, which can be one of these:

– a path from a neighbor node to the destination– false, if no path from one of the neighbors could be

found




62

Traversing Graphs(define (find-route origination destination aGraph) (cond [(symbol=? origination destination) (list destination)]

[else (local ((define possible-route

(find-route/list (neighbors origination aGraph) destination aGraph))) (cond [(boolean? possible-route) ...] [else (cons? possible-route) ...]))]))




63

Traversing Graphs(define (find-route origin destination aGraph) (cond [(symbol=? origin destination)

(list destination)] [else (local ((define possible-route

(find-route/list (neighbors origin aGraph)

destination aGraph))) (cond

[(boolean? possible-route) false] [else (cons origin possible-route)]))]))




64

Traversing Graphs - Example

A

B

E

C

F

D

GNei

ghbo

rs o

f A

• Find the path from node A to G!




65

Traversing Graphs(define (find-route/list lo-Os dest aG) (cond [(empty? lo-Os) false] [else (local ((define possible-route (find-route (first lo-Os) dest aG)))

(cond [(boolean? possible-route)

(find-route/list (rest lo-Os) dest aG)] [else possible-route])

) ]))

A

B

E

C

F

D

G

A

B

E

C

F

D

G




66

Traversing Graphs

The function fails to terminate in a graph with a cycle:

A

B

E

C

F

D

G

(find-route 'B 'D Cyclic-graph)= ... (find-route/list (list 'E 'F) 'D Cyclic-graph) ...= ... (find-route 'E 'D Cyclic-graph) ...= ... (find-route/list (list 'C 'F) 'D Cyclic-graph) ...= ... (find-route 'C 'D Cyclic-graph) ...= ... (find-route/list (list 'B 'D) 'D Cyclic-graph) ...= ... (find-route 'B 'D Cyclic-graph) ...= ...

B, E, C is a cycle




67

Summary• There are problems that cannot be solved, or can

only be solved sub-optimally, using structural recursion

• Generative recursion is based on the principle: “divide-and-conquer”

• The design recipe for generative recursive functions has to be adapted:– In particular, we need an argument for the termination

• Structurally recursive functions are a subset of generative recursive functions

• When both strategies are possible the choice depends on a case-by-case basis– one cannot say that one class is better than the other

Introduction to Computer Science I Topic 6: Generative Recursion

Documents

Transcript of Introduction to Computer Science I Topic 6: Generative Recursion