© DEEDS 2008 Graph Algorithms 1

Application: Process Relations

Some algorithms require that sub processes (tasks) are executed in a certain order, because they use data from each other

Example: get dressed

In which order should I don my clothes? Such an order is known as a topological sort

• Tee-shirt before shirt• Briefs before trousers• Socks before shoes• Pullover before coat• Trousers before shoes• Shirt before pullover• Trousers before coat

© DEEDS 2008 Graph Algorithms 2

Application: Process Dependencies

Are these the only rules?

Tee-shirt

Briefs

PulloverShirt

Trousers Shoes

Coat

Socks • Tee-shirt before shirt• Briefs before trousers• Socks before shoes• Pullover before coat• Trousers before shoes• Shirt before pullover• Trousers before coat

© DEEDS 2008 Graph Algorithms 3

Application: Process Relations

Transitivity

Tee-shirt

Briefs

PulloverShirt

Trousers Shoes

Coat

Socks Relation??

© DEEDS 2008 Graph Algorithms 4

Partial Order

A partial order is a relation that (informally) expresses:

For any two objects holds that either... The object relate with a “greater/smaller than”,

“before/after” type relation ...or the two objects are incomparable

(shirt or socks first?)

So what is so partial about the ordering?

© DEEDS 2008 Graph Algorithms 5

Acyclic Graphs and Partial Order

Definition: A partial Order (M, ) consists of a set M and a irreflexive, transitive, antisymmetric order relation ()

Cormen’s definition is reflexive, transitive, antisymmetric

In a partial order, some elements can be incomparable

Acyclic graphs specify partial orders : Let

G = (V,E) be an acyclic, directed graph G+ = (V,E+) be the corresponding transitive closure

The partial order ( V, G ) is defined as: vi G vj ( vi, vj ) E+

p. 1075

© DEEDS 2008 Graph Algorithms 6

Transitive Closure

There are unfortunately multiple definitions for the transitive closure… There is another definition, where G* is called the transitive closure

(Cormen for instance) G* = (V,E*), where E* = E+ { (v,v) | v V } G* sometimes also known as the reflexive transitive closure

We will use: G+ = transitive closure (Wegegraph in German)

For all definitions: use the one on the slides! Also for the exam!

© DEEDS 2008 Graph Algorithms 7

Maths…

Let be a relation (on for example )

Reflexive (=,≤,≥)

Irreflexive (<)

Antisymmetric (≤,≥)

Transitive (=,<, ≤,≥)

baabba

aa

cacbba

)( aa

© DEEDS 2008 Graph Algorithms 8

Example

E+ = {

(v1, v3), (v1, v4), (v1, v5), (v1, v6), (v1, v7),

(v2, v3), (v2, v4), (v2, v5), (v2, v6), (v2, v7),

(v3, v4), (v3, v5), (v3, v6), (v3, v7),

(v4, v6),

(v5, v6), (v5, v7)

}

Graph G

© DEEDS 2008 Graph Algorithms 9

Deadlock

Test: find cycles = test for partial order How do we do this?

Example (fictive :-)

Assume: “The coat should be put on before the shirt”

Tee-shirt

Briefs

PulloverShirt

Trousers Shoes

Coat

Socks

© DEEDS 2008 Graph Algorithms 10

Topological Sort

Topological sort order: a sequence of objects that adhere to a partial order

Example:

Tee-shirt Shirt Pullover Briefs Trousers Socks Shoes Coator

Briefs Tee-shirt Trousers Shirt Socks Shoes Pullover Coat

Tee-shirt

Briefs

PulloverShirt

Trousers Shoes

Coat

Socks

p. 549

© DEEDS 2008 Graph Algorithms 11

Topological Sort

Definition: The arrangement of elements mi of a partially ordered set (M,) in a sequence m1, m2, ..., mk is called topological sort if for any two indices i and j with 1 i < j k the following holds: Either mi mj or mi and mj are incomparable

Topological Sort is a “linearization” of elements, that is compatible with partial order

Visualized as a graph: Arrangement of vertices on a horizontal line so that all edges go from left to right

v1

v3

v4

v2 v5

v6

v7

v1 v2 v3v5 v4 v7 v6

© DEEDS 2008 Graph Algorithms 12

Finding of a Topological Sort

Finding a topological sort can be done using graph algorithms

Starting point: Determine the acyclic graph G that corresponds to (M,).

Two possible approaches: (top down): As long as G is not empty...

1. Find a vertex v with incoming degree 02. Delete v from G together with its corresponding edges3. Save the deleted vertices in this order

(bottom up): 1. Starting from the vertex with incoming degree 0, start a DFS traversal. 2. Assign reverse “visit numbers” beginning with the “deepest” vertex.

© DEEDS 2008 Graph Algorithms 13

Example: top down

1. Find a vertex with d-(v) = 0

2. Delete from G vertex v and all incident edges

3. Save the deleted vertices in this order

4. Repeat steps 1 to 3 until G is empty

Graph G

Possible topological sorts:

1, 2, 3, 4, 5, 6, 7 2, 1, 3, 5, 4, 6, 7 1, 2, 3, 4, 5, 7, 6

2, 1, 3, 5, 7, 4, 6 ...

1

3

2

4 5

6 7

© DEEDS 2008 Graph Algorithms 14

Cost: top down

In worst case you always find the next vertex with in-degree 0 last

O(|V|2)

Example:

v4 v3 v2 v1

© DEEDS 2008 Graph Algorithms 15

Example: bottom up (DFS)

Given: acyclic Graph G = ( V, E )

Find: topological sort TS[v] for all v V

for ( all v V ) { v.unvisit( ); } /* initialization */

z = |V|; /* maximum number in TS */

for ( all v V ) {if ( d-(v) == 0 ) v.topSort( ); /* all starting vertices */

}

void topSort( ) { /* DFS Traversal */

this.visit( );

for ( all k‘ this.neighbors( ) ) if ( k‘.unvisited( ) ) k‘.topsort();

TS[ v ] = z; /* Assignment of TS at the end*/

z--;

}

© DEEDS 2008 Graph Algorithms 16

Example

Vertices are named “backwards” while ascending from DFS recursion.

Sorting order: 2 1 3 5 7 4 6

7

6

3

5

4

12 1

3

2

4 5

6 7

© DEEDS 2008 Graph Algorithms 17

Cost: bottom up

Consists of Search for nodes with in-degree 0

O (|V|) DFS on the corresponding subgraphs

O (|V|+|E|)

Total cost O(max (|V|,|E|)

A bit more complicated, but “better” than top down

© DEEDS 2008 Graph Algorithms 18

Side kick: Total Order

There is not only partial order Total order is a partial order where all elements in the

set M are pair wise comparable For example: ‘≤’ and ‘≥’ But not: “is a child of” over all Germans

Where could these orders be useful? Deadlock detection in concurrent programs Message ordering and consistency in distributed computing

© DEEDS 2008 Graph Algorithms 19

Application: Cable Routing

The possible cable channels are defined by the channels

Goal: every computer should be reachable

© DEEDS 2008 Graph Algorithms 20

Solution: Spanning Tree

Model the problem as a graph

Create a spanning tree As long there is a cycle

left: Remove an edge from a cycle

1 32

45

6 7 8

© DEEDS 2008 Graph Algorithms 21

Spanning Trees

Definition: A subgraph B is a spanning tree of a graph G, if there are no cycles in B and B contains all vertices in G

It is not difficult to build a spanning tree of a connected graph: If G is acyclic, then G is a spanning tree itself. If G contains cycles, delete an edge from the cycle (the graph

stays connected) and repeat the procedure until there are no more cycles.

Ch. 23

© DEEDS 2008 Graph Algorithms 22

Spanning Trees (2)

Edges in G and B are called branches Edges in G and not in B are called chords A graph with n vertices and e edges has n-1 branches

and e - n + 1 chords The spanning tree of a graph is not unique All spanning trees of a graph can be systematically

constructed by means of an acyclic exchange: Find any spanning tree B of G Add a chord to B (that leads to a cycle in B) Delete another edge from B

© DEEDS 2008 Graph Algorithms 23

Example

B1, B2, B3 - all spanning trees of G

Chords (edges in G and not in B)Branches (edges in B)

Graph G

B1 B2B3

© DEEDS 2008 Graph Algorithms 24

Application: Cable Routing

The green numbers indicate the lengths of the cables

How do we find the cheapest (shortest) routing?

© DEEDS 2008 Graph Algorithms 25

Minimum Spanning Trees

We look at spanning trees of labeled graphs where edges are labeled with c.

Definition: The weight c of a spanning tree B = (V,E) is the sum of the weights of its branches, i.e.

c(B) = eE c(e) .

Definition: A spanning tree B = ( V, E ) is called the minimum spanning tree of graph G, if for any B‘ = ( V, E‘ ) of G the following holds:

c(B) c(B‘)

© DEEDS 2008 Graph Algorithms 26

Example

c(B1) = 12, c(B2) = 15, c(B3) = 16

B1 is minimum spanning tree of G

Chord

Branch

labeled graph G

5

3

7

4

B1 B2 B3

© DEEDS 2008 Graph Algorithms 27

Finding Minimum Spanning Trees

Idea: greedy-principle, i.e. decide on the basis of the knowledge currently available and never revise the decision.

Efficient in contrast to methods that sample solutions, evaluate and revise them if necessary.

Given: Graph G = ( V, E ) labeled with cFind: minimum spanning tree T‘ = (V, E‘) of G.

E‘ = { };while (not yet finished) {

choose an appropriate edge e E;E‘ = E‘ { e }

}

© DEEDS 2008 Graph Algorithms 28

Kruskal‘s Algorithm

Sampling of edges according to the principle of the lowest weight. Sort edges according to their weights. Choose the edge with the lowest weight. If E‘ { e } results in a cycle, drop e.

e

Observation: If there is a cycle, it is pointless to leave e in the tree and to drop another edge e‘, because c(e‘) < c(e).

e‘

© DEEDS 2008 Graph Algorithms 29

Example: Kruskal

Sorting:(1, 2)(3, 4)(2, 3)(3, 6)(4, 6)(1, 5)(5, 6)(5, 7)(4, 8)(7, 8)(2, 7)(1, 7)

1 32

45

6 7 8

5 6

5229

11

7

8

1414

19

16

Needed length: 60m

© DEEDS 2008 Graph Algorithms 30

Kruskal‘s Algorithm: Complexity

The complexity comes from Cost for sorting the edges

(possible with O(|E| log |E|)) Cost for testing for cycles

(also O(|E| log |E|))

Total: O(|E| log |E|)

© DEEDS 2008 Graph Algorithms 31

Prim‘s Algorithm

Enhancement of Kruskal‘s Algorithm by Prim(sometimes Dijkstra is declared as the author):

Given: G = ( V, E ) labeled with cFind: minimum spanning tree T = ( V, E‘ ) of G

V‘ = { arbitrary start vertex v V }for (int i = 1; i |V| - 1; i++ ) {

e = edge (u,v) E with u V‘ and v V‘ and minimum weight cV‘ = V‘ { v };E‘ = E‘ { e }

}

© DEEDS 2008 Graph Algorithms 32

Example

1 32

45

6 7 8

5 6

5229

11

7

8

1414

19

16

© DEEDS 2008 Graph Algorithms 33

Analysis

(Efficient) cycle testing is an integral part of Kruskal‘s algorithm: Interconnecting two different connected components of a graph

by an edge does not lead to a cycle The vertices belonging to the new edge must be located in

different components Upon the assembly, the smaller component is added to the larger

one Works with appropriate data structures in

O( |E| log |V| ) time

© DEEDS 2008 Graph Algorithms 34

Path Finding Problems

Finding certain paths in a graph Distinct boundary conditions lead to different problems:

(un)directed graph Type of weights (only positive or also negative weights, bounded

weights?) Shortest vs. longest path Path existence (Number of Paths) Start and end vertex:

• Paths between a specified pair of vertices• Paths between a start vertex and all others• Paths between every pair of vertices

Ch. 24

© DEEDS 2008 Graph Algorithms 35

Application: Route Planning

© DEEDS 2008 Graph Algorithms 36

Abstraction: Model as a Graph

© DEEDS 2008 Graph Algorithms 37

Refinement

© DEEDS 2008 Graph Algorithms 38

1. Approach: Brute force

Brute force: The simplest way Try all possible paths

Cost: normally exponential

Worst case: Complete graph n cities (n-1)! Different paths with (n-1) edges each

complexity O(nn!)

© DEEDS 2008 Graph Algorithms 39

1. Approach: Brute force

5 Nodes: 96 paths 10 Nodes : 3.265.920 paths 100 Nodes :

92392953289504711154882246467704033485808808581737805253907034256265423993297616452852049336394953103391160941618951520668673358807695360000000000000000000000 paths

All 12903 communities in Germany:

474381088,9

© DEEDS 2008 Graph Algorithms 40

988032780296517663436322076701095155700770024537890245495168073394410613749907903334908537127328540766793107809246348588724025167815276417017650390124486547249379172139679071211781763482703236042541349862318218136576551103758850603937190932371732941574515443140722211275244104835452316865803709346211824565400461322044268326721434035983838111178657891154164927556643178656171474660541364955846571135269566089067209050083842063966215125519481522183823585218692212743733202669715476272895547369627756913232675761878439204981953372613356827581804169522488746469163622604760967219252795485480375527583094186163733617471275178950304495062900336879944902153498925439763269150239710115653805503370058652307975224136372148701221581438236021839242848128851546825711228911202757947505784345873803895182007675155066858898693737648891744866579496575892260026300294887734157699863935792018007700633163234414762019639622776185890245821775621611386383897803543489896010396675700309744434876784840088465489343324134115523556680571964252563179124505009237822399060108114358568757406253755902528547047697975637947254426283182107816551488579830240507465718633120296507369068830031103226529254096912888355394582488701708312346659355342807395570422354578245246938061310209543661790640824248864515698258916783063862709184192639909885673222152979676570965226247673732187517713838924480328589155031996896876630274080083237499467671885548338229554799947277310507760197919131028896513288374654041642193935067734583293056077975165566070718798016703959586297899227493229109065658198379525446761936503813125987314217476912540920958691501786781633016193952420445818637943831285237097681218386907496815085128670158136918224743013470148417525928758794125736083444423449517158184585748244991925372752352883903196664446123035911353975285681385557607950025161830060127387144386770890182283129433279229878044001420477388322367001685489721693216961358741984895809910057530410237766907408298984317251170185725001911792002356449462337432259915811855680138431420900271721621227629171082975740974612055073676688305916188061569085712379422149608668571756702872714717887242081282688937644017036781926791886892850699160723701808477393242501975526406247528322685597081253628903644703870145482094042563403342726628505879061682285383030772566219038403276226525576463824761999416389357888909877646081196344931614459530305303529216037971446335469336304933789696590465986538192098788718867231866722821972411202608920215463998261375226177037293246241414617886441005846443741362574589796763866647308280944923725543427360293500292357816587680690786981644763480822493581171066514641833375350953690889267118124738752384665657758279740313428551789738503345388500169962568116376501762419896564889448468299648444141841066258146163041111965515673358814164304045545169389544640456926558980709540435412249915976314003081122455954626450715899677907566902565738992731601501551892143697443236936018991215182673945313156849126844651391733253422369214395206355345371819267478091898022034500176275236986899261341308560337891814734904904095606613036861273053573490027296196740110765950770284215762304864858219405620645776247712410673816673388898455767444875353302980656538334614922173712939929382251274859196638408735513634978587075918632129610820121916947146012885960919517262526170571569545433930157932578249720066991682399275974750539324680010204734952204498414639509378962190058420679748993065899071872607045028486161438381779765629600634030830519503930480455581589235296566877393283962355565514153608617960190376427814930409939360840428953331227888642550077797950421825318235323633591523629287742602115081427067548146397849652716631746875548436220463519025875710571074416560542680266847237162982060791093099202937028778389396785579005279700409864140859641831581824389250925834691192115339791116725689447438970597317542515057363180498144026411604942744501989229304393298136629453231141812343783702413620634977019376630935205574478496273999737731565337853302481322103319295832635199091984223776614780993204045913654979139512626636535892344852600449877590541418661886923150246125633153760853409749658566380281475740120783813531809124147535433842147650806325969306425896289306258816067502103665246960074413499574332516755046107787139896686076116794787042761654341632880793384181647767253070357669481538612787731131419150022741705645646195886568331437573603682151453178761386416165152068754415078583846174012911182038337485709651924293887252362445912534437928494150975429264621005806512435045473792118831327687762824273343681145088031134009280785177950763387146501570739875072669207916022499284795689694418645469722682945507431826148379772319905972125433287055862273411891154379344506115400718996013423920293338929554843496264060069725868597164936043777557992066080293665341619550555569627806920340136483758552894707491434189865560074389908441788339034753190731362879116889273970057384244439592117251632036499360693179282953883272990501859405679435447769374160422381566623005851005999053697002552443576649735678983421516794583623454779405740274066897714110701101220388696120241269161793243130063402669177999893905252449993490729671360694698687811678700557487828520360912741855357733179540069657225768956679116405108105512772646029186261384326038422677899012589231989032333949759209992805719018714389948233473772229947524042391313292384699936213124863058560239754531694008072170962537403609125573154455224845583557874725166083947788886218817900571969902890266292923804989473982203817928217460794608293990073275755721495162223152626698507171416870748872482674239813533012940345500207082429687262161568556229858964015387763496671061890693990070602321532645396281808295496726822293511431024353505212462583756499293711574644502025540582439362382442709708040820720785306473852408425590877003045327494820078952292121765490042462300654721510962470907134479672710303215643121393792386856209991474352926024891632625911963912949760698950675785831879475331221806902023361413032724035894093608382406265646363075631840150290517680308655443891490655288584020334662252563563480232553478074003841772401043233499592980816073211103947924365003986709399355971513534579882181899224890660776147671196431079475756820250084300075966485713329896926608303268831195892023403390398201571956480739010924768475399057820310111562151832013984753479172661838243598706138112674900562277196436063702853916074454806256993211883181023010394103392204755749657217437059270699347723514366120404569826845375776357911833579006893901311020237987759426816892863346405243396486671729903709794557839038983869920701150292654535501352688647321550887477859932242159009018283207759419306998429676960247737419767268294287045224967496736554249489217942627112296625074267609629797170434356918751367330714174279615443713631038198286858979871874379934230505931480714215652572561128797518546876476647719694229519750659928176175947078135827573771830590551888363617181602532322304025441036971458937405452689583381058346598163093475461736188465611273232321263402473062885324002928680650038382449251629066689206424546771086625840288832003506763497691942032011243199214567395688362396131092129396538367456149724969170892603376840514648387321523873182682454045445638054237377459869623971425719745291960389291186598164838856133824982134315186971136160730903692192507575544951984609258319380176078686844876101949538881827257739983970316859707832074296708409948128191320865248832404816812478798673620514459354937110461331280927407051769380131550951008176222223468852541963172165865845560401793584971078384602997722900284031517819518755817404321056601424525253830584365741458632121970688160520070245189027181452190131019316944424755123067915033870052675998925476234803656560385064530519736581079453875155680244083097237666773453599814651412463303759981403737754310325380115726421703585365808129532546740694993030649545654616659748203748653884864574463322430175973529693593144370924743101825944539999533424833236794459469643732601644129783401784090869552980190302633244756382138009276788686975001957034692014158004591325414123243237892577921254977966757755838077556499093449597908470473702831048722949581723430018236809660183703719010595338608353401602305433405758470205269939757477022479613812835588929130175190092535657628325654959465075675703898262959686089032248029017714718386821673535538639264920562147171035266877039940982774185772576200558036213401066063061521600536323867176297347804263266198498698462761253817344582351270743507384640781197881322204248766526962884568279066072912733124055200693517702303669530504087231143630048100569563239738485884517422724943524264879102865381187206095961324143232692240433829445643057891542542921666797560830065071891594201604067312943657831170053048543404021182834230142711690128664776464313449666758853310977446276614333705940489350552653347749407801689009434049583625246269889769787756807987123475766789231515418890396930956743512555252972668420700156162678705486715195582806763473828179589341973822709904325983687202597536936741565553664386422712213424853031862124585251661750070508585897603803225056705517628705543945502286396346037908977248492761718998384781666563587846931160613171352511882554964493389379889185202100222662412356744314112047775423352057330756177091660771465222748562015619666650202388524047869857689586544006650409816057599659844415805603266212748474376464466977419744879701394320804990215785407712875080043130257776603411539183923340689850869715164912778237021074563986010544113682476797668900958609600430728990407331279588014337498584948058690215191832408232854214203134399617079664852458709754228660966318799807662836416934358632663692227260919187747450712347228915993614290878542292697832270368566381132277941262949296964292954283437514331515429409641204749743109012927533127763177467869898014365005754122875436659269033532755390624285034527707655342634266582209528279922113915197716479667795480433311081821252118744904524882499421361524498136699095244785942622676238750387200032737714735751017930454541332478741745363424932434030769771813581017285088662765207636043088831048835809255713327157201566864590640914013046325821926065822405759963200301106385655227249729637666701989589459407851972698421356579719454258057849958458627612166676019521804099636708887243225498506113270263800490832122689575494219686447093755252942398815682855236622454661172960863816429406116414061202191895514558390111768085751185236250731729234457468761433805141049220426761339681268680333504905612306214001964127697118097873631861172108490593685384924221132606157773164097652730989825681094721360576086652340389683516937672916138353275740135360029666778244242414560052445432011901952420822949588385982458786711722337992347637331746118797553744870716404365708562636808718884533589154058843493897340555102736783762967346995044872659713728722566751162490783900024723877040971387666820437829784466696792720985350845680104071358376986827622226195771256149302256114718902225059630452072555629690817670517473015044078146979592816090205592277956059271675552356992127856807540496349622794676168679715371937108583411189326680658825113828836354543388561439115960388530231638526226760516827053552374542833261848791831399149720945714453445699150575955112219940715228747554872258615303059155161490837381943680235804617831576030903503383322729187044111712116083001637516540871989472602179097872958825049911448067110392394987359257109464441712230324032432496777044472079462863773131538947574340914437902631884962822302973643689139002359121543748380224292212825430616216963313311975758109461647478162197534573560480522225979786091194528593909122527782886796307040736844261196132852173125067743329407365537842978056185518968496733411989711793416170812477743150664429163780480891770056253943054700632029530536294031284829899665563325106919726957801971422089883185050114762046633616503362337091304205480873436905396404091918067146510436868294865594650351135115729462869297112163090861468617241253076933549894270710238085438939800700781635052235462457775417727833210415251457180713753960480375826288185379098908266169086559883170829681071075530666205316146456803991917526572254347319903305490624971895786985012413576618668181877815739037255001596392253775490988568606899611933843019154880200156043438076078394422424088767076094704920705962578479728543764796352951011392311453297287517010918319636685083341648248203229848328576769717443453927150469654300132787983523046541710787463296935089578549413732334131888779218459234861637779701243172154788810597501621146656498378670799330418980665406082650609849386668186660711550279676724112454493260947414222537050311082447907831379090239303946407949854778837411885412725931806776123554073669251415795747611954033015280591086109368357425059056340317609912489732601781876291518746873773690528795925421174116501128887508838246773713687322642828310975975322849348942613072476784277840690913266550758805017044529389581546546050061763128537071835000318790974820036115006050965907629061526571607273935091401481071496825991951070943477536550730522827006553377270645793709209529119577316905839186323570723167834198146795301780747911411309028324132669355636287389846432665040637615565482105040115941130757590577063122879548504734507398524669122734990146306920988981865680319106150939285723954414086669573875067084031675073742991732356401395963198155207352871492231518112968858426995946050213454760412524790194980978396675119620335239973408593908577593930680601216794171641426275304989284770272031550786471337357240347688546046386316725164617102111544300485148549788117348074672157219545898535412892624085280324574748870789264755679368895824226685670906712114711568509450711266116913718620477223695350613890430492667479990297039277917753410937834381909975317851513125009557378050744879355438456086135693037875679555941746472870673314849639147160680220115231557067695370843706096810461954672077147205906428343696540454137873997377012019484849521736800717778517042700062513862701774217978637021321259923283282116874299171389758153889300634946774405778633371305448938019485022289345337447142570987863158910839511596136764799302979200653378604009432782656381540668324176678842387490882475482946051914391190466960105166312390431401166035669871704170252704917928191528224176214455126535058659283499839881829064642123939678880191709101139700381103688040678838016132444268946927572114709996483908148932750110684599117449230051410703123127433884971661035743707723581090224354993041638815761248891915272802309241808477771405913375482440491890774512171882099510602661383738043805214406259224267853462499173534023587702986314724715479380106013617393108953572394763442998024562020855883008880336654657182491912670198139545993397937838423662789812009649220242580131383872892470074715142122804716425646556773104966109877878914028847150785866591653883438860080546866646846181897796786364007772473532199565256222860817793055394680328739492828011807911542863480861150712102330001850148854780137584867912102043538086865956063065691048933999915485569909986869181752094305021707169011000286519028495616201343738956504107719747711849875545918582433626550716450935812416448670036785436699094192551004048129180737436841088889349698553137004975888080558649557198417384607021415382382738492291503234215007304427294325996752311232735401040360302093477117515818694728633959992224729282100985630305709724504333855096027018192645430719429852140720251178936062256591495066312127918199309878528076901650693049382168915700640444096940367244972698574240194898044586844507142463300770162411240430873838961883476224126978040735007580072085554824409189059031405166011370169192993345436560827314965448498952856662808405868788481610020068469728874614746145343843838245797394062648053034378216504023288428902445053029207536097823254459834114387116060781866961279245224790494755470839066500220967651811887837667895445099258991680359406757038726867848049106989464120912818309144263136721144306248226538867473659976464356102191936842502463340069440746262212029893505962849575218290814513719795951497900378366186861127980250767785986353739430916412059465844980637865165138141912151519832808268005455813332533615831612512808887498579302602647933582382604382361001221370816584929035882707022387300794039435312150112045380136382725032638347234492137755405441414366407527454494103056593678706271483261769886141994676565671952595233669769064342085383642914886198555409676151083234623923641602193133365381505010245265273071629688007826550315529329374404548643315554133998074822461027536027956441912040115705101040376883578837021436894872809828925850230582560975829151957664910355144711044455243591671869834222913679338803687208303817663910319370425293490163986553855339299464006035543245669708554902525006588651995311666906282117471923111030710753001476172377795449248078742868418801972628888426837686243677508633625117119816462167785638849758374311449582679390995961760458983703293922100661783770576571930686755329361447414136795713510855179118451796097938045450105288148009901268966632635242452607282078873943462857380766546410572212742716135296661932772631826781636106446105596889996059174092917192281153754045992818243335306492887103053742558244287434167102458996106380665633617304408683440051432082391131811826577627346778740083463868887845285305934548313486230634920213967451311625908295273048534238793221429867008312337684311521032157410775072316583639974722191384562273089318153137984916043118281369152450795784852208267064868093875437276931721425952686597736992878273478882148291921723431155553147528959935734605596939309103303343447821638437037515893093975529675051354317590549862167579433917189571665427863047574406828263031831378710692119723926689518233331992558430168046770022081216414034494382949661870181415672268759451373355067464107319320151092253240091525904076676791049761288461927075872154722215071152581353861019801291497201129991994824241306206843441657125767817797571137297521885481309782290348515162236264818955403405534059021840015537437915511815543842115693829511939471690294196133252894294421036846874440672365743536888820326539173939548314956869836891439002343632797894021522273089702890408094433348562685279840907945980569500671966610188367650084401589268338936396406993208360699565045427188707494167962065603455911892281455369336525371604557145117138354991309564153996009099258986113821380484829983225184881157732042717523638878915459479100909115639562097648788760673727373690543015516478541918204481834208980011181708450361021464095019865378740411651024127131686758542782074617794273714690464135821783852375982724499957511984271442700868563376825315135781371699589947215230002210851056134539123698288260533694188984106378441299975098622378545636063458497420228751576862833517592199170431057453565065708841201913069004447765912122175903339708461272040361738171069471084772918538752909095460954480532525312777231103697969187443286638267474624164252199840439027987066944163399069485677908221318622970553899887862388032329366876070076370385922200075787433505638471684199427910587784998717350107555051842220004433948835044720867069038582212974659470023863862891229125497541761552532241181251277578185835410284551411726315908641658100057227999451913420352086536849988886695397058222750414240095592523478243687012862085700363168532518348876766273854816392748154576643682310648165030660124130062638912547377761871375443205871053410947888202315905831681620135300070524928100684865674489303419652803025745854179688455385586264971398506112242592229689853518425585739612975173033633987838937430331978454478444804713909368303889374729443836289419052568055117607473104647069314013763271179332987137844408916445086220063579265375287245577736012523042184844570646952958909169657969187989772452812086997488219128391775456076225868914114016510623027335017633811497267923480640133063781676634498915994353483425253518615573756040583810319094553241220380880849383407339062328324852368957202339122929471762317417490704975105920367567942821891937353214725270173474365967856746775511136147678984710729040187234150723968570550055491017997609664260784346684097824809941217010980547010701348678017541802290268870892020482591910227225132499404543856060939265816740385612724722607026917702146957975510145650506861989411072818179092965069902450024385255941916237476323473006363384086672600844409846816448481986109433734471412710065808847260161675186639888312534562763868765702369730558401278975307632582832083159444992241241797606133901735461191105779964748847341613839718911348611720212563731089375969705411034651304688326166904409653077557470072072761305489436265792707169473075694217384184459641455948152122609401798233647749660562729076477835620580839881366579188787070479397978413011583089769739984234505693787942627070555615496485784557798015297826699902712795273792798656576297478683483909414190391039269050494897838859401621972962573300104769876523614631187863275372495405723039792355238027489385502879655089709373953128686057131491931181635182717725968646929863156873945945547616736054402883602194548527073944832369284081498230795992688164900372140371451828111329854455803853790099973531258879830344679458199695288344653529511104479249310029757999110494013907949429763148419434757965416330699223517992769304390949546627501286412519105423552689111278827846436126658566497340024393010034890064475857848931724748283999363251303692049970842509159373211784029855262443126595958526683037669146664274618177730852876882531030222945550469118390455092545363725680284666940436462814938405983550503710977717716879384713921311076418794384054521434154957862915389365891267671545977457484794469610370172358134555853644791320728305998716626429412013061028863646783423171204709069026042432437662226610183635372663815617559853895717150123801823164885359743792690590163402718614577852280568578099599471796387355714375430057266371759700259002695597296217098794899686157584126337655663555280176999369263144864827939248331030247105168134936037145432908235565624876414354126348102283806202130004364488158233810630295173520511798800368988562302567177528687553616348263986824874822948933829434789434067863341675354203634198330924106009524502017096777682324194274778357269840917275907068829781557205158435664528742474241747129830143266657937709501006573414305171303071803716215145053959516600067871570871689593995133140238338246468016679850840585481854714507087240223739013935987455899994229440659381788551058867934760335089193414386828118011308073836809125438440501124116893617421791310515493562659548785035325487643124860679645084774703769748619110766688750252240747950579559215229565797102800579104298045335357462839707089371963588705072791759661218480683976476264856951377028146807235348977640348319558624320374713799482466213623283854399521413951859492056972546459504702419757383830550693534024419968239453129453403062400331865893558210631779694931066460301449266265204052182718786432535959036438829425978964134155392411963538667089227891254861609349731666119688373753981263727728458366796185380957585866349319321046655106031963084085381888209405729736388892367781168240040866308715920681882136465859393804023886086720112260264853087770753080732497043552011039454915941787093135196695644942448036061740496916440466818271219539072827540017167811180346453678631504959516401454911823211202018527689119968954570455708939695776801490764313801483158498602577269634415479874715023674998423178359047783940573472413152226189330671433594104594379176902362214026960150510156999425254498559659680797766320157563015437169918840093315138966178830691559313068478044999485598624185118913474624908646258463660322229604397938242717649360231566796339992670194392709457396817227249338105171492096205890084305392568551058806390205770279604205330934402713801899985446711910701837803365717480380760196294740141579462022128795526691035648616821468490727657003800626793040969711747470414209491627611253836875260214097508350281310000544881816545376155656227592899971357318825150071172424143770345413721928308631453323603350259772846927589743234718822981631911360401060147619404317630483631265976607254759282058938458062797019707644247098663617872793624754941142511177915223282054192523138584354607263860313161628530666461291321097748522721715390402662170031825771427974140276325453782909641752866591327751224398151779175316612332251498847814113748946357235357826545777125095274038750085366606494714545519779512223221049557778069172026190485201475426404446047283618000180090153705649755325987162804660772651314504954963548607657060521482724491008143998397253768069412194903175414791316064308418884664009173290105585188616719444125986645727375117155092833295366335040702308291531770341826525294463130221677543415977633152571856529722077611249276784909399574859005600322816156356108033669422453575607232248732402299613023203837225797676648633090381891481754159002688360339665641728400890918745821544723850836594104041982774317378413851591214253108324513089581049048735777154213981662127263852752753790107981893594391172036997958222076264431419248991622761077507891462831681756922838805362819445094169721093022359145806235026531489551943765311319865895821162740318320775091662403634125342873166457923502483646967744089587755996585413678258553715687568689201392610571184839808406556210622525817949573345727136003198308838388117493079237817785698005064402496888107175579623684160712765660288510495558635256158598930187174047915171935180560226355478685178165242537177961755106596759454903884109421919506934818518650136354873771822605564836012320104422749916448599337284026816450377667156774850162222695074506225317391358063606643333507429084038126743268826727610490609839511916207524303352002796322056589342399673555960305744391317535602209845329728735788587927139810568000627809627285849542134686719382630124160173310724907802882281998593168455485106427989004629256384505377550084226106151316281154702734779515913582157894563233253078105893604302097212309220019877074277590596783985119962562997785505982692339714007103225918024517926908515345163507913165964470589568502631394778290312224645245210328585917925906609171027934252652449762790058571469331737938035073520214491313617241802496551848373635474097243234430534127675972228980830394374804137744231259283111731252607289433813609629413642617380981157882703324577601698471783969898280352966237259662153699340598342053393244793970576877122281630541724309726165450429190190928178847023223478023026767383441065105613926133386681249108510000601373011724983139107794674728156888954111052972163863909446474597908339557421744714569808479924166674120646768980349686837441595922759841030461218096712742016536407854460562638370377917910341262135461055956192775821653735213944746530554090965743685484120273109946424973181000513350958730674777419292355333082847199435867769906989052680363654827438596871817641218387413224933722962739908598696754892747608624983989517479825217326430682511949398939132322121330852179360671793748321589835052314998787262937197417986751792710981899406838342798856427468319859670910883673335114903980936080079529076086485449933329191345982943073982148841761905333960376883503920371506997949398943107894215796133428908135079797841738190066131854568744205035163540966631094307255143594807192910749942979483217733152430579387918807670749023462863945151689774589042050699873775108710337784389784261220763436566913741775660867401164533073873250550256367221332693749343768696983163279861242506336457865090122775949687080574459197382746872755458143460802004222013596768538358974651401053679020076987766094653753256120497781286429321116904906354389184992081770000513263820876148463817528592856261688833077653069326145111542557127421086587198446613103230000351246229972836611223917440398386957544983510918769431385745515776550930380695849792776427845274932535557453813770215433998952425331207677801655974531653331258105133784606115321630962789332722912988430345665541607470918035970123292845897846561432496152349026820294905056135241139711312810600783975528524101599551242477885976257412281103038921900473342420012463131591338481609931509018690094725721990085858303336656981329788305105669767844704952955228402658222560828129566310801654163220236525315464435912294253102429458439363280297042439896924855253165637266787424393277650122738212506430649563629659313981499527131193821793956544359563883731936935745546575653448279180287614383016329296168694325319990156896075728501329749054448523100625841989588315128133168543367916482017640981598618709872882080974765279474712836776957669816254811698609791759923211209398482040314660153644653311943751515661629243959461068326571557419023775959020948768521563994630939078451876913533837887016617131086499583842110277756604619626227250033822017622914883011637069154995955879087278822412361983378585407283542708645012429379377126882753826376713854984717323147409728514883530705710962263807092309447239470897075300416636715226693419231308552046146899580154937688567826003674667498845293560064403302377888099396078364484692383573642523618400187567368529308392537049856250442722110487711436254013341474344676950039361833594457395042782658933980914280864094664712236928137956984609322094170427255317726497979893444965166521326203685889375416771664237362965504001925363823006499793171722435818626373265315434817140794799897881315954507996269667515653998502246373853879261047654415131007020956332824004080873476008632038078908571692176610469430718987436390567758173038424758823264877417089516599723841776753481140924907137082103585712559026393483487044534808358627527643552565247554451874959074622201231329094148431147463149857188150579196073673243676086667555267058523638878300021089900121592431697314236149002273025609851012643724633400891906003773568053534756128200949668265529948235167029117108410788909403312273312839583222826614927371406457799063466782346379405968328375202393773820807210357515322626798991321777959415184910387382093431778009210321605506354017587026635371157757494113485541986137748429276639666347864724209532823346329942291525670520403959929408048480599630602471286761119605403643093795165452240572711340802733352242967327588920894986853855428695228123721515383372241468677676657690523495293991193089317631180401621542742144172479955727785822302523235218082574031735817953896040708276394655939154385201465486836296826369289028372746802164088832935968378376735959745107096150987973942911422182780997918635851055043101756513159623802274259910767636803740144494908735177630186599447257685338614252482048672429227311762468974925106959539790889155410962810166695278859917991221423269379338586649296931939422836226059168586858248644039055073049158874775356894150844561229068460596159498252319420048949460320707873190437521432125497124450812927473123233426112277141951169437831866251972967156030850237579207472345326122820951035140853577889069193220801277268727877462088477256728053533865379125515357874604763127646587058569449275459757946779249849668645546601682111071473490196532829054167058513717213002674859842367700971889541249200207275542206063741885115967391824802061029095778748636469234636581470557318980869354159403672935575524353178902973678630815462378998470254705860465012317151875156962180541352865250953160734816457763801035570929130202833755080099274542583432047644326117708773927117706285648517334888275025081906593432069147949245561537721980062513881595826962559270699751134974218339058877566106472400826141536944849548372026202951697896579870697551240173945154531908638275727407118177100308950492892357865845150788334764756639628320021084296795516659674021225966584349883360538948988695704332916916970968457753514626977394323235125937751596298414132821176502673559152982900907543880968559783585312849218903494924153956864751338725863748534892535808654881519748823747627172990832688322096315182171500878487858195545087822445823622172027985930981023200387487844105509751292486047537710357093034763191156956674550796847685124778495727889795469819809739211133103413434779822666526806424901626197887282974889325166305324367468486189534285230963976870315115910099770121128076828720568702348291333556553430367365014168771293106232000895389517502599856653293083823407123075654556535091268623667286509324586514944368465584850900253948687551001752212917105353667905456456778427115125964311241646439285849237135031736695597064250392688315961441452772944623933569725613347976756813338594643432630383392657599631682544985730742007858836915433604505719070855879189485197288719040021263719868370872603890471224865906247298557916928478221291167732539671661292887371135437981942779634825649139882012944572775856251041415094963377934028329521721036519200597694830700568471309673666443171259587901752003996381084283612192024173635727775849240013454410532914435555584921531777111497746804745730530095317761154643813569457717720566923931144038213744547355819912594121144910422694237154775487752005260871222729738810894646265432369005588661788919152163982140771630564900149673912974836304232189799743487901973846011704221189906187663415798978343397632890903018531294948170205920521810817079555037004170663721973573257622734385654541890774874197893194941181634994668457144887794875282066261847615572718799662464428738434438780825061929921484926915746942103848879619392172122284804702431010212519539969796788017846333873182922337676999777534812332012929564596104599508214149701145727495059306099523699510761420309048130272939065839261625847903097311300510506280343882615164671461555078160563909494052681747251161047160059699487066709910360984455010613051396660812504250678290008428852598035154540572827777776012255457008383846221905287787575146885791923472956169292751754532877537377210888668488850063956571258521640679392408898344931351638389606589417427473476507292012928999533182971060700410625023483788305882315410026995960591558583069277684801658721562872431626137159349073603480881241710404542763803861570206810973763804468626358414660740063357740456418981456129440496134941796810008809508788374342077093093032573221654574497024220255712560913428408088998336484628062602420972884881824235484997156686204850571815572289597037462797558743867558191988812172264459758021538210414051670339659254429158690879213474450218857384191333885465275086332763991715941519656460264069449678135453878298392932595928592735417547480435257809585105010218305181370464045159787719800844217944580559140987962634899939052790022722267608134919525507626667522702387995260462315343650118033684168651734549131231336977863781238099073974418036336404348251915526348716256148308366265710493939810433401574324606201851001108298603626032883931917102193786381778850221360149271446304437825797667215143815962299855108350685801519572698690875975167844559591576384803528784625173351945548820301273433263147108424812682126149704843457845771098879325188965254752647214632462274094327285939219625637908210569178906354323258068270638423718443370259173754959417280524241925413632488148322588627394310323948564813009199314311352803018221598147606559849449441930283542725117301958336300923349861886742605979336086963459746504767954407094475416276278164559258454739153194608040844670924770762639185547020009071394069165626147123832618188432405841403221478975186849192112360772169665002540901133189776403712705903170299557334238610160128352962483565719060124695974326763550084436607613043081353179500689647381215933656567158723361671674768998402424440050925248958944809638632578121251521517893047244620356007945685810267600703806696506811034616208402733862695041093087540495322984241333683745051037656990423172498225669542175893227464699035395611643202519094625652983036905179307827133463591466000260972584100751306203312729148981161886098078386987774631774628872511021482207910602594248338333653688375949911406183753947867431811617501231335840006434744147809675823583523243884079449048889645483549560682187901817756668038944630514566610136158705181573057034029514688051495727866153977779674901732493629430889653500150739041913862194790557138035834486563129700584827159926307467671039114095278994988619502912218185733880411659332876510621812940192686452614836391774232485554419353515389717178894105195097709882454365049668790154019450473729462062338175285757777220331479036285217420864441321256204989829615594911425708850538977391959700236803969002383361343180529263474633248641019728627355145215080950707142562724727379784353038457211655061801793612817865529403259118504030821365325832670300903746117708108940594398284068590231060393666902865542697619080827749742792528004130638723427802564336546966267216565663129834773737201814975996945261200646352492989680714370859969397714453559499705714752583076892375510657152127018886147639804363516993755298742787103891150700220714506301913945252708503844524188931607809291106650515823792227016427853235911041802235206685137365256401019284320832855069543469121237788941624920491426989683514552552778606044261894121946422080372039822028735770025697298646610412950573194135173996196487113793068038302423174540357048329843011515626843095296836221153859115570494563885850899357898998653920926127657581954002375699054777755793413346729815683937056235071096200948972224545441111976421949803863069791210596882786020829195672869731642969363371256235128286786494322015713897370274398431649588021987544820839753067235909062099592022980098971310059682850113483941431571980374626893492875138111858675995342924271335544904703625997990170746600001046381081394610407361858278066292733328733897503619800318544612501802739092404855756784873762845441839227696053945349373166520728784939803938687236087005436691027144039212027192562630220756241148178921471666791089733661158823965138545449464265042179903141071863860634820674693808091110040233166872929182145362742271621264848296839972006699787543846647166316832171981705345229155258853740389128472935464813551890718004522399908471460112191208294187733687208176809677440023938416823335680651720533001783199089241378188610648258925977302547087226001915784667848654569976939992800191947168992691640409363780754050616576642743116263619044683503524666271355940073703150798572195097116558255919875668443740312812885091538481696998640935296064309086938740436546158827565693737875765801134731144684944206505748478338068242223048451290109065051470567592280202113809190801118156966526164470029632830884067909579201197089984856313503391449665077302379902192608937637391519815701507521817915970925565410307046715458891394782906701634430859881733545355987595368111980381782949400354170629114341611410275358552893318152943729497103755285569915792834643963283215604286396508690418685242009489560348181982061208001219370843466155120956528522844796088562680021364428083022837355100476966979918400555873044818468991127698511726292216919632251034916167318089592699376357221088997669432517644047426564875012548960183953391444675733852462268899979284724389106189070437324879057100736377629819552912609494980475618795730599857379925133004171092622560599686218488385161665899200822216962034469209245898826036568843248370219818743511130685548894666351208734775343246725565184077878459401836822696554162390530919387039014905010977890797144485374663349962371959295723687708764704660682014885039050695390147723329421269448951708736676806237797469029409720437802364121825876745064652897513410542921498040988349394494877885632428277518099754513072624516259131382958192221132415190724677559649555157623571670834952021768305415231510484855287494795510035072937401010227794320452346929552871893336336223584538759375576318221613134409431248132145470384000289962963467709299649490623430936995709356315657925985837617281746436195424032995059682800056369803916023915466159537148171250303467970983929821699915191107892667621911098379575001256204267382073298228990277413518336720818690356696351344806740796578274286275015291663659121105329883773971968548288165166523288207939699675622974731460763361822451285568368078048908704324305227623770030375108460447355037246315094467328632985415839167120170443543183916632235096152270658581085852229519721841652317094238942912472684802688380481359624042116560637626307278223149308043230106131532057260973907559663650534631103818210924290889827147261854036977477246202584423647677302439047142421643526537980173548464474750146923183651253326207571103483020388998680803443082640787670965471378523813414600303679303337216562953584692073250178342914588457504645172582734587375353558146504645248699565474644967822969140393538048280668176906004311407520775308650197321856205962878713226540476853754333542667576418468845403155185619682605304950256773615921424671379181048784956470777782066891059790221567987244891740068853131532880945210793423034555334737660025453854812287915287367574722094737300651426629760463515778549995340105234193410522770511217230323794173125121888374664776382887409585186323113950315197022000762333919360401083374367592134434908777847625719650026220034047921321467154839628859083138802950513640019308317693786841141934540472435600534401656938443184214207263511469015649122624167982395710169899290195788102805758824444769972971040474296665346414939357948965452532159784638695555517922935930398351993771045173266825658611505177306100016129153354139892257741066374595266105688114107725172516146750920820579461545742450621314536355725031221832387628316939325052100656216406373342737248984858191108678881124576041543648200080422905476982207118772121330440288932827327761104662429036128396775803689324153970303045881083017237712821874296508805746288173415530913604617154265138730667559996605840157970824526624110875728581920378040730890453645184418402385847147461404906591404440898397760673869896656830962025631882947083824624788548861805576535186660290649531469898263555178246288102804855417041259939391122696152931052242377695915405245986156489168268299089118880311726307261356770064152744315062496052493578607239826475211106803645889616592397560778846598143629909177839671714674696881859978011676699975910610131845613802402144710006993653508488982447201590549756301533258720285122956767498690597114300602661524919958600736397491756568940198581230328419552319865354708103759124924745534158572658861446721035847887739435042246054791207348674592409522774858684924346794530034758631214817040083923430733516862901865304032040526531973673212341031144565850263327410813659547111964429870093450986111710015594788364079328252223075184368486843386067777124027372160341696042561211572266042687063746759346365928388489518829642574931002791505762214789626051856596068870227331271075371819097370083090460880663416335454807640435242289471406207247913154969701099505038724164891510218245363912903707654321966236612619581563678826136521012533151721432145118866559676620627539556685951070717238449568770629906364177302845468979642178711436187324055605425588395887773401922698736700077282274200630127966337944323612893104365337670626742723457611101848237098194971898311106575436410869149952902470469845340366098718526042450404150391366776082533569204648168467051311524402353306757499012005897084647952840926108403970191945040844688725239463949974646233136689432669059255549273794495502453404570213977395094532284190007603105269743659113235331074619837234351930892876362567606123434315167789930074131772609347015940106834280762980967616816096906230046585763659991594294849841834832414701118265758455907643539465020161044906274913352714319257921200253252062994563451532777598078633521953440917491886288492980182677564502673506001980200727765941546882052804008898327957141448899039287578436063431839085753539385850290426473722657304404328708283368990456004363462593693267233113610277404407054889020614210669381155236114153399888213821609030083962463390860046274893958743427825341822366879194590406703465400738425681537081956266857105006201896719074586131571527254876987459881263621475710581321754454449215825878353757799023776776983809184493044846165631691697022649924085999176358391111708814246789003903331956673648936259603158267583233701401520324078752883381963603200051670634155891666891140642244295146571350913425641178162777626492025465565295402949036390051839285379129822660331195521193124802122694215549198056895663010699228761735802388480000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

© DEEDS 2008 Graph Algorithms 41

2. Approach: Ants

How do ants find a short path from I to O?

Solution: search concurrently!

Instead of testing, find all ways at once

© DEEDS 2008 Graph Algorithms 42

The Dijkstra Algorithm

© DEEDS 2008 Graph Algorithms 43

Shortest Path

Definition: The weight of a path p = v1, v2, ..., vk in a graph G = ( V, E ) with edges labeled with c, is the sum of the weights of its constituent edges, e.g.:

c(p) = c(( vi, vi+1 ))

Definition: The shortest path between two vertices u and v in G is the path with minimum weight

The weight of the shortest path is called the distance between u and v

In non-weighted graphs, c(p) = |p|

k-1

i=1

© DEEDS 2008 Graph Algorithms 44

Example

All paths from a to d: p1 = a, b, c, d p2 = a, b, d

Shortest path from a to d: p2 (weight 9) Shortest path from e to f: p3 = e, f (weight 2) Shortest path from a to e?

5

3

7

4

a b

cd e

f

2

© DEEDS 2008 Graph Algorithms 45

Single Source Problem

single source shortest path: Given a graph G = ( V, E ), we want to find a shortest path from a source vertex s V to all other vertices v V.

BFS-based algorithm (by Moore): uses length attribute D of a vertex to keep the distance (as number of hops).

for ( all v V ) { v.D = } /* Initialization */Queue Q = new Queue( );Q.enqueue( s ); s.D = 0; i = 0; /* s is source vertex */while ( !Q.empty( ) ) {

w = Q.dequeue( );if ( w.D == i ) i++; /* next level in BFS */for ( all v w.neighbours( ) ) {

if ( v.D == ) { v.D = i; Q.enqueue(v); }}

}

© DEEDS 2008 Graph Algorithms 46

Example

Visited vertex i v1 v2 v3 v4 v5 v6 q

0 0 v1

v1 1 1 1 1 v2,v3,v6

v2 2 v3,v6

v3 2 2 v6,v4,v5

v6 v4,v5

v4 3 v5

v5 -

“Wave propagation” in the graph

© DEEDS 2008 Graph Algorithms 47

Analysis

Moore‘s Algorithm has the same time complexity as BFS, i.e. O( max( |E|, |V| )).

The algorithm does not work for weighted edges Enhancement by Dijkstra was published 1959!

© DEEDS 2008 Graph Algorithms 48

Dijkstra‘s Algorithm

single source shortest path with arbitrary positive edge weights, start vertex s

Correctness is based on the principle of optimality: For any shortest path p = v1, ..., vk from v1 to vk, each subpath p‘ = vi, ..., vj with 1 i < j k is a shortest path between vi and vj.

Proof: Assume that this does not hold, i.e. there is a shorter path p‘‘

p‘ between vi and vj. Then, there exists a shorter path between v1 and vk by

exchanging subpath p‘ with p“.

© DEEDS 2008 Graph Algorithms 49

Dijkstra‘s Algorithm (2)

Each vertex belongs to one of the categories M1, M2, M3. Extract vertex v from M2 to which there is a minimal edge

between M1 und M2 and insert v in M1 (principle of optimality). For each vertex that was moved to M1, move its successors to

M2.

s u v

not reached vertices

marginal vertices

selected vertices

M1

M2M3

© DEEDS 2008 Graph Algorithms 50

Dijkstra‘s Algorithm (3)

More precisely: Divide the vertices in the graph into three subsets: M1 contains all vertices v, for which shortest paths between s and v are

already known. M2 contains all vertices v, that are reachable via an edge from a vertex in

M1. M3 contains all other vertices.

Invariant: denote sp(x,y) as the shortest path from x to y. Then, the following holds: For all shortest paths sp(s,u) and all edges (u,v)

c(sp(s,u)) + c((u,v)) c(sp(s,v)) For at least one shortest path sp(s,u) and one edge (u,v)

c(sp(s,u)) + c((u,v)) = c(sp(s,v))

© DEEDS 2008 Graph Algorithms 51

Dijkstra‘s Algorithm

/* M3 implicitly defined as V \ ( M1 M2 ) */for (all v V) { if ( v s ) v.L = ; } /* Initialization */

M1 = { s }; M2 = { }; s.L = 0;for (all v s.neighbors( ) ) { /* fill M2 initially */

M2 = M2 { v };v.L = c((s,v));

}while ( M2 { } ) {

v = the vertex in M2 with minimum v.L;M2 = M2 \ { v }; M1 = M1 { v };/* move v */for (all w v.neighbours( ) ) {

if ( w M3 ) { /* move neighbours and update L */M2 = M2 { w }; w.L = v.L + c((v,w));

} else if ( w.L > v.L + c((v,w)) ) /* w in M2 (or M1)*/

w.L = v.L + c((v,w));}

}

© DEEDS 2008 Graph Algorithms 52

(7)

Example

v M1 M2 v1 v2 v3 v4 v5 v6

{ { } 0

v1 v2,v3,v6 3 2 6

v3 v3 v2,v4,v5,v6 6 7

v2 v2 v4,v5,v6 4

v6 v6 v4,v5 5

v4 v4 V5

v5 v5}

1 2

36

5 4

3

32

6

1

5

5

2

1

4

3(6)(2)

(3)

(6)

(4)

(5)