Introduction to Computer Science · Introduction to Computer Science CSCI 109 Andrew Goodney Fall...
Transcript of Introduction to Computer Science · Introduction to Computer Science CSCI 109 Andrew Goodney Fall...
Introduction to Computer ScienceCSCI109
AndrewGoodneyFall2019
China– Tianhe-2
ReadingsSt.Amant,1-4,8
Lecture6:FirstHalfReview October6th,2019
Where are we?
1
Review
u Lasttimewegotalittleahead
u Sowe’llreviewthefirsthalfofthesemester
2
Lecture #1
3
Lecture #1
4
Computational Thinking
u “thoughtprocessesinvolvedinformulatingproblemsandtheirsolutionssothatthesolutionsarerepresentedinaformthatcanbeeffectivelycarriedoutbyaninformation-processingagent” (Cuny,Snyder,Wing)
v wayofsolvingproblems,designingsystems,andunderstandinghumanbehaviorthatdrawsonconceptsfundamentaltocomputerscienceu Toflourishintoday'sworld,computationalthinkinghastobeafundamentalpart
ofthewaypeoplethinkandunderstandtheworldv creatingandmakinguseofdifferentlevelsofabstraction,tounderstandand
solveproblemsmoreeffectivelyv thinkingalgorithmically andwiththeabilitytoapplymathematicalconcepts
suchasinduction todevelopmoreefficient,fair,andsecuresolutionsv understandingtheconsequencesofscale,notonlyforreasonsofefficiency
butalsoforeconomicandsocialreasons
5
Humansthinking(i.e.,transforminginformation)todeviseproceduresforexecutionbyinformationtransformers(humanand/ormachine)
Before Mechanical Computers
Electroniccomputerswereprecededbymechanicalcomputersandmechanicalcomputerswereprecededby…
…looms
6
Discrete Machines: State
uHowdoestheloombehaveasafunctionoftime?u Atanygiventimeasetofthreadsisraisedandtherestarelowered
uWritingdownthesequenceofraised(andlowered)threadstellsusthestepsthemachinewentthroughtoproducethecloth/tapestry/whatever
u Thepatternofraised(andlowered)threadsiscalledthestate ofthemachine
7
CS Topic: State
u StateisaverycommonCSconcept
u Herewehavethestateofaphysicalmachine
u InCSwetalkaboutthe“state”ofanobjectv Ofadatabasev Ofarobotv Ofa“state-machine”(finite,Turing,etc…)v Ofasystem(physicalorvirtual)v …
u Thenweneedawaytodescribethestatev Givesusthenotionofanencoding
8
CS Topic: Discrete Machines, State and Encoding
u Choosingastaterepresentationtakesskill.Thestateshouldbev Parsimonious:itshouldbea“small”descriptorofwhatthemachineis
doingatanygiventimev Adequate:itshouldbe“bigenough”tocaptureeverything“interesting”
aboutthemachine
u Thesearesometimescontradictory.Theyarealsoqualitativeanddependonwhatbehaviorofthemachinewewanttodescribe
uUsuallyyouneedavocabulary(encoding)todescribestate.Inthecaseofaloom,statecanbeexpressedasabinarypattern(1forraised,0forlowered)
9
Discrete Machines: Abstraction
u Theloomisadiscretemachinev Stateisbinarypattern– i.e.discretev Thenotionoftimeisdiscrete– i.e.timeismodeledasproceedinginstepsor
finitechunks
uMoreprecisely,theloomcanbeusefullymodeledasadiscretemachinev Becauseofcoursebeingaphysicaldevicethereisvariation,nothingis
exactlyprecisev Butmodelingthemachineasdiscreetisgoodenoughandworksforthis
purpose
u Thisisanexampleofanabstraction – akeyconceptinComputerScience 10
CS topic: Abstraction
u Oneofthefundamental“things”wedoinCS
u Reducingordistillingaproblemorconcepttotheessentialqualitiesv Simplesetofcharacteristicsthataremostrelevanttotheproblem
u Many(most,all)ofwhatwedoinengineeringandcomputerscienceinvolvesabstractions
u Heretheabstractionismodellingtheloomasasimplediscreetstatemachinev Makesitpossibletounderstandv Andmakesitpossibleto“program”theloom
11
Lecture #1
u Whatmakesacomputer?v Lotsofthingscanhelpuscompute(informationtransformation)v Computersneed
u Memoryu Control-flow
u State
u Abstraction
12
Lecture #2
13
Motivation
u Whatdocomputersdo?v Mathwithbinarynumbers
u Sowhatdoweneedtobuildacomputer?v Placetostorebinarynumbersv Waytodomath
14
Arithmetic/Logic
u “math”weneedtodowithnumbersinmemoryv ADDv SUBTRACTv MULTIPLYv DIVIDEv AND,OR,XOR,NOTv Etc…
u Assumewecanbuildacircuitthatcandothisu Takesnumbersrepresentedasdigital(electrical)values,producesresultsasthesame
15
OP2
Start building a circuit…
16
bus
Instructing the CPU
u Nowwecanmakeinstructions…
u Instructionsarebinarynumbersthattellthecircuitwhattodo
u Selectthe1st operand,2nd operand,destinationandfunction
u Withaseriesofsuchinstructionsthecircuitcanperformarbitrarycomputations
17
Where to get the instructions?
u InstructionMemory
18Controller
ALU
How to compute?
u Fillinstructionmemorywithdesiredprogram
u Initializedatamemory
u Runaninstruction(givenbyprogramcounter)v Thenincrementprogramcounterv Runnextinstruction,incrementprogramcounter…
u Someearlycomputerswereprettymuchjustthis
19
The Central Processing Unit (CPU)
20
u Controller+ALU=CentralProcessingUnit(CPU)u CPUhasasmallamountoftemporarymemorywithinit
v Registersv Aspecialregistercalledtheprogramcounter(PC)
u CPUperformsthefollowingcyclerepeatedly
FetchInstruction
DecodeInstruction
ExecuteInstruction
Fetch-Decode-Excecute
u Fetchv Getthenextinstructionfrommemory
u Decodev SendthepropersignalsfromthecontrollertotheALUandRegisters
u Executev LettheALUdoitsworktoproducearesult
21
The Storage Hierarchy
22
Cheaper&larger
FasterRegisters
RAM(memory)
SecondaryStorage(DiskSpace)
Trade-offs
u Anaside…
u Identifyingtrade-offsisafundamentalengineeringskill
u Understandingandbalancingtrade-offsispartofdesignprocess
u Notalwayseasytomanage!
u Conflictinginterests
u Speedvs.spaceisverycommontradeoffinCSv Soifyouwantfasterexecutionyouneedmorememory
23
Cache
u Small(butbiggerthanregisters)u Volatileu Fast(notasfastasregisters,butfasterthanRAM)uWhattokeepinthecache?
v Thingsthatprogramsarelikelytoneedinthefuturev Localityprinciple:
u Lookatwhatitemsinmemoryarebeingusedu Keepitemsfromnearbylocations(spatiallocality)u Keepitemsthatwererecentlyused(temporallocality)
24
Modern Computer Architecture diagram
25
Controller
ALU
I/O Devices
(USB, etc)
CPU
Registers &
Program Counter
Memory
DRAM
Disk
L2
Cache
L3
Cache
Boot ROM
I/O Controller
Disk controller
On die, but not
part of "CPU"
Programming a CPU
u Howtocompute?
u Developaseriesoflow-levelinstructionsv Usingtheregistersand/ormainmemoryforstoragev Usingonlylow-leveloperationsmadeavailablebytheparticularCPU
u ”Assemblylanguage”v Ormaybeevenmachinecode(probablynot,though)
26
Typical Operationsu ADDRi Rj Rk AddcontentsofregistersRi andRj andputresultinregisterRku SUBTRACTRi Rj Rk SubtractregisterRj fromregisterRi andputresultinregisterRku ANDRi Rj Rk BitwiseANDcontentsofregistersRi,Rj andputresultinregisterRku NOTRi BitwiseNOTthecontentsofregisterRiu ORRi Rj Rk BitwiseORthecontentsofregistersRi,Rj andputresultinregisterRku SETRi value SetregisterRi togivenvalueu SHIFT-LEFTRi ShiftbitsofregisterRi leftu SHIFT-RIGHTRi ShiftbitsofregisterRi rightu MOVERi Rj CopycontentsfromregisterRi toregisterRju LOADMi Ri CopycontentsofmemorylocationMi toregisterRiu WRITERi Mi CopycontentsofregisterRi tomemorylocationMiu GOTOMi JumptoinstructionstoredinmemorylocationMiu COND_GOTORi Rj Mi IfRi >Rj,jumptoinstructionstoredinmemorylocationMi
27
Lecture #2 Summary
u Computersdotwothings:v BinaryMathv Movedata
u Sowebuildastatemachine(CPU):v Controller,Registers,ALUv Fetch-Decode-ExecuteCycle
u MemoryHierarchyandCaching
u AssemblyLanguageProgramming
28
Lecture #3
29
u “Thearchitecturelevelgivesusaverydetailedviewofwhathappensonacomputer.Buttryingtounderstandeverythingacomputerdoesatthislevelwouldbe…(insertanalogyaboutperspective).Ifallwecanseeisfinedetail,itcanbehardtograspwhat’shappeningonalargerscale.”
30
Problem Solving
u Architectureputsthecomputerunderthemicroscopev Imaginesolving*all*problemsbythinkingaboutthecomputeratthe
architecturelevel
u Earlycomputerscientists*had*todothisv Luckilywedon’t.
31
Problem Solving
u Computersareusedtosolveproblemsu Abstractionforproblems
v Howtorepresentaproblem?v Howtobreakdownaproblemintosmallerparts?v Whatdoesasolutionlooklike?
u Twokeybuildingblocksv Abstractdatatypesv Algorithms
32
Abstract Data Types
uModelsofcollectionsofinformationv Chosentohelpsolveaproblem
u Typicallyatanabstractlevelv Don’tdealwithimplementationdetails:memorylayout,pointers,etc.
“…describeswhatcanbedonewithacollectionofinformation,withoutgoingdowntothelevelofcomputerstorage.”[St.Amant,pp.53]
33
Motivation for Abstract Data Structures
u Thenatureofsomedata,andthewayweneedtoaccessesitoftenrequiressomestructure,ororganizationtomakethingsefficient(orevenpossible)
u Data:largesetofnames(maybeattendancedata)
u Problems:didJelenaattendon9/9?HowmanylecturesdidMarioattend?Whichstudentsdidn’tattend8/26?
34
Sequences, Trees and Graphs
35
u Sequence:alistv Itemsarecalledelementsv Itemnumberiscalledtheindex
uGraph
u TreeEric
Emily Jane
Terry Bob
Jim
Mike Chris
Bob
Sequences aka Lists
u Sequencesareourfirstfundamentaldatastructure
u Sequencesholditemsv Items=whateverweneed.It’sabstract.
u Sequenceshavethenotionoforderv Itemscomeoneafteranother
u Sequencescanbeaccessedbyindex,orrelativev Findthe5th itemv Ormovetonextorpreviousfromcurrentitem
u The“how”(implementation)isnotimportant(now)v Arrays(C,C++),Vectors(C++),ArrayList (Java),Lists(Python)…v Thesearealldifferentimplementationsofthisabstractdatastructure
36
Sequence Tasks
u Most“questions”(problems)thataresolvedusingsequencesareessentiallyoneoftwoquestions:
u IsitemAinsequenceX?
u WhereinsequenceYisitemB?
u Bothoftheseareansweredbysearchingthesequence
37
Sequences: Searching
u Sequentialsearch:startat1,proceedtonextlocation…
u Ifnamesinthelistaresorted (sayinalphabeticalorder),thenhowtoproceed?v Startinthe‘middle’v Decideifthenameyou’relookingforisinthefirsthalforsecondv ‘Zoomin’tothecorrecthalfv Startinthe‘middle’v Decideifthenameyou’relookingforisinthefirsthalforsecondv ‘Zoomin’tothecorrecthalfv …
u Whichismoreefficient(underwhatconditions)?38
bruteforce
divide-and-conquer
Sorting
u Ifsearchingasortedsequenceismoreefficient(persearch),thisimpliesweneedawaytosortasequence!
u SortingalgorithmsarefundamentaltoCSv UsedALOTtoteachvariousCSandprogrammingconcepts
u ComputerScientistslikecomingupwithbettermoreefficientwaystosortdatav Evenhavecontests!
u We’lllookattwoalgorithmswithverydifferentdesignsv SelectionSortv QuickSort
39
Sorting: Selection Sort
40
u Sorting:puttingasetofitemsinorderu Simplestway:selectionsort
v Marchdowntheliststartingatthebeginningandfindthesmallestnumber
v Exchangethesmallestnumberwiththenumberatlocation1v Marchdowntheliststartingatthesecondlocationandfindthesmallestnumber(overallsecond-smallestnumber)
v Exchangethesmallestnumberwiththenumberatlocation2v …
Sorting: Quicksort
41
u Picka‘middle’elementinthesequence(thisiscalledthepivot)
u Putallelementssmallerthanthepivotonitsleft
u Putallelementslargerthanthepivotontheright
u Nowyouhavetwosmaller sortingproblemsbecauseyouhaveanunsortedlisttotheleftofthepivotandanunsortedlisttotherightofthepivot
u Sortthesequenceontheleft(useQuicksort!)v Picka‘middle’elementinthesequence(thisiscalledthepivot)
v Putallelementssmallerthanthepivotonitsleft
v Putallelementslargerthanthepivotontheright
v Nowyouhavetwosmaller sortingproblemsbecauseyouhaveanunsortedlisttotheleftofthepivotandanunsortedlisttotherightofthepivot
v Sortthesequenceontheleft(useQuicksort!)
v Sortthesequenceontheright(useQuicksort!)
u Sortthesequenceontheright(useQuicksort!)v Picka‘middle’elementinthesequence(thisiscalledthepivot)
v Putallelementssmallerthanthepivotonitsleft
v Putallelementslargerthanthepivotontheright
v Nowyouhavetwosmaller sortingproblemsbecauseyouhaveanunsortedlisttotheleftofthepivotandanunsortedlisttotherightofthepivot
v Sortthesequenceontheleft(useQuicksort!)
v Sortthesequenceontheright(useQuicksort!)
Quicksort
42
Lecture #3 Summary
u Solvingaproblemwithacomputerusuallyinvolves:v Astructuredwaytostore(organize)datav Analgorithmthataccessesandmodifiesthatdata
u Algorithmshavecharacteristics,likebrute-force ordivide-and-conquer thathelpusunderstandhowtheywork
u Thinkingaboutabstractdatatypesandalgorithmsfreesusfromworryingabouttheimplementationdetails
u SequencesareafundamentalADTusedtoorganizedatainanorderedlist.u Sequencescanbesearched:
v Linearsearch(brute-force)v Binarysearch(divide-and-conquer),butrequiressortedlist
u Sequencescanbesorted:v Selectionsort(brute-force)v Quick-sort(divide-and-conquer
43
Lecture #4
44
Abstract Data Types
uModelsofcollectionsofinformationu Typicallyatanabstractlevel
“…describeswhatcanbedonewithacollectionofinformation,withoutgoingdowntothelevelofcomputerstorage.”[St.Amant,pp.53]
45
Sequences, Trees and Graphs
46
u Sequence:alistv Itemsarecalledelementsv Itemnumberiscalledtheindex
uGraph
u TreeEric
Emily Jane
Terry Bob
Jim
Mike Chris
Bob
Motivation for Abstract Data Structures (Graphs, Trees)
u Thenatureofsomedata,andthewayweneedtoaccessesitoftenrequiressomestructure,ororganizationtomakethingsefficient(orevenpossible)
u Data:largesetofpeopleandtheirfamilyrelationshipusedforgeneticresearch
u Problems:twopeopleshareararegenetictrait,howcloselyaretherelated?(motivatesforatree)
47
Motivation for Abstract Data Structures (Graphs, Trees)
u Dataset:roadsandintersections.
u Problem:howtotravelfromAtoB@5pmonaFriday?Howtoavoidtrafficvs.preferfreeways?(motivatesaweightedgraph)
u Dataset:freightenterscountryatbigport(LA/LongBeach).
u Problem:Howtoroutefreightgiventrainlines/connections?v Routefastest,vs.lowestcost?
u Dataset:airportlocations
u Problem:howtorouteanddeliverapackagetoanyaddressintheUSwithminimumcost?ThinkUPS,FedEx
48
Motivation for Abstract Data Structures (Graphs, Trees)
u Dataset:networkswitchesandtheirconnectivity(networklinks)
u Problem:Choseasubsetofnetworklinksthatconnectallswitcheswithoutloops(networksdon’tlikeloops).Motivatesgraphs,andgraph->treealgorithm
49
Motivation for Abstract Data Structures (Graphs, Trees)
u Dataset:potentialsolutionstoabigproblem
u Problem:howtofindanoptimalsolutiontotheproblem,withoutsearchingeverypossibility(solutionspacetoobig).Motivatesgraphsandgraphsearchtosolveproblems.
u Otherdata/problemsthatmotivategraphs/trees:v Financialnetworksandmoneyflows,socialnetworks,renderingHTML
code,compilers,3Dgraphicsandgameengines… andmore
50
Trees
u Eachnode/vertexhasexactlyoneparentnode/vertex
u Noloopsu Directed(links/edgespointinaparticulardirection)
u Undirected(links/edgesdon’thaveadirection)
u Weighted(links/edgeshaveweights)
u Unweighted (links/edgesdon’thaveweights)
51
Eric
Emily Jane
Terry Bob
Which of these are NOT trees?
52
1
2
3
5
6
7
4 8
Graph/Tree Traversal
u Traversingagraphoratree:“moving”andexaminingthenodestoenumeratethenodesorlookforsolutions
u Example:findalllivingdescendantsofXinourgeneticdatabase.
u Fortraversingagraphwepickastartingnode,thentwomethodsareobvious:v Depthfirst
u Goasdeep(farawayfromstartingnode)aspossiblebeforebacktracking
v Breadthfirstu Examineonelayeratatime
53
Tree Traversal
u DepthfirsttraversalEric,Emily,Terry,Bob,Drew,Pam,Kim,Jane
u BreadthfirsttraversalEric,Emily,Jane,Terry,Bob,Drew,Pam,KimEric,Jane,Emily,Bob,Terry,Pam,Drew,Kim
54
Eric
Emily Jane
Terry Bob
Drew Pam
Kim
Tree Traversal
u Depthfirstvs.Breadthfirsteventuallyvisitallnodes,butdosoinadifferentorder
u Usedtoanswerdifferentquestionsv Depthfirst:goodforgametrees,evaluatingdownacertainpathv Breadthfirst:lookforshortestpathbetweentwonodes(e.g for
computernetworks)
u Roughly:v Depthfirst:find‘a’solutiontotheproblemv Breadthfirst:find‘the’solutiontotheproblem
55
Graphs: Directed and Undirected
56
Tia
Jim
Mike
Chris
Bob
Joe
Sofie
Undirected
Tia
Jim
Mike
Chris
Bob
Joe
Sofie
Directed
Graph to Tree Conversion Algorithms
u Sometimesthequestionisbestansweredbyatree,butwehaveagraph
u Needtoconvertgraphtotree(bydeletingedges)
u Usuallywanttocreatea“spanningtree”
57
Spanning Trees
u Spanningtree:Anytreethatcoversallverticesv “Cover”=“include”ingraph-speak
u Example:graphofsocialnetworkconnections.Wanttocreatea“phonetree”todisseminateinformationintheeventofanemergency
u Example:networkofswitcheswithredundantlinksandmultiplepathsbetweenswitches(thereareloopsakacyclesinthegraph).Needtochoseasetoflinksthatconnectsallswitcheswithnoloops.
58
Minimum Spanning trees
uSpanningtree:Anytreethatcoversallvertices,notascommonastheMST
uMinimum spanningtree(MST):Treeofminimaltotaledgecost
u Ifyouhaveagraphwithweightededges,aMSTisthetreewherethesumoftheweightsoftheedgesisminimum
uThereisatleastoneMST,couldbemorethanoneu IfyouhaveunweightededgesanyspanningtreeisaMST 59
uWhycomputetheminimumspanningtree?v Minimizethecostofconnectionsbetweencities(logistics/shipping)
v Minimizeofcostofwiresinalayout(printedcircuit,integratedcircuitdesign)
60
Computing the MST
uTwogreedyalgorithmstocomputetheMSTv Prim’salgorithm:Startwithanynodeandgreedilygrowthetreefromthere
v Kruskal’salgorithm:Orderedgesinascendingorderofcost.Addnextedgetothetreewithoutcreatingacycle.
u ‘Greedy’meanssolutionisrefinedateachstepusingthemostobviousnextstep,withthehopethateventualsolutionisgloballyoptimal
61
Prim’s algorithm
u Initializetheminimumspanningtreewithavertexchosenatrandom.
u Findalltheedgesthatconnectthetreetonewvertices(i.euncovered,ordisconnected),findtheminimumandaddittothetree
u Keeprepeatingstep2untilallverticesareaddedtotheMST
(adaptedfrom:https://www.programiz.com/dsa )
62
Kruskal’s algorithm
u Sortalltheedgesfromlowweighttohigh
u Taketheedgewiththelowestweight,ifaddingtheedgewouldcreateacycle,thenrejectthisedgeandselecttheedgewiththenextlowestweight
u Keepaddingedgesuntilwereachallvertices.
(adaptedfrom:https://www.programiz.com/dsa )
63
Shortest path
u Foragivensourcevertex(node)inthegraph,itfindsthepathwithlowestcost(i.e.theshortestpath)betweenthatvertexandeveryothervertex.
u SayyoursourcevertexisMikeu LowestcostpathfromMiketoJimisMike– Bob- Tia– Jim(cost3)
u LowestcostpathfromMiketoJoeisMike– Bob– Tia– Jim– Joe(cost4)
v Veryimportantfornetworkingapplications!
64
Tia
Jim
Mike
Chris
Bob
Joe
Sofie1 1
2
4
3
1
1
3
41
1
Dijkstra’s algorithm: Basic idea
u Fanoutfromtheinitialnode
u Inthebeginningthedistancestotheneighborsoftheinitialnodeareknown.Allothernodesaretentativelyinfinitedistanceaway.
u Thealgorithmimprovestheestimatestotheothernodesstepbystep.
u Asyoufanout,performtheoperationillustratedinthisexample:ifthecurrentnodeA ismarkedwithadistanceof4,andtheedgeconnectingitwithaneighborB haslength2,thenthedistancetoB (throughA)willbe4+2=6.IfBwaspreviouslymarkedwithadistancegreaterthan6thenchangeitto6.Otherwise,keepthecurrentvalue.
65
Lecture 4 Summary
u TreesandGraphsv Sometimesneedtomodelinteractions,connectionsbetweendatav Vertices,edgesv Directed/undirectedv Weighted/unweighted
u GraphTraversalv BFS,DFS
u GraphtoTreev Spanningtrees,minimumspanningtrees
u Prim’s,Kruskal’s
u Shortestpath:Dijkstra’s
66
Lecture #5
67
Recursion
u Recursion,recursionrelations,recursivedatastructures,recursivealgorithms
u Definingadatastructureoralgorithmintermsofitself
u Manyproblemsareeasiertounderstand(implement,solve)asrecursivealgorithms
68
Recursion: abstract data types
uDefiningabstractdatatypesintermsofthemselves(e.g.,treescontaintrees)
u Soalistis:Theitematthefrontofthe
list,andthentherestofthelist(whichis,anitemandthentherestofthelist…)
69
[1,3,5,7,32,6,7,121,7…]
Recursion: abstract data types
uDefiningabstractdatatypesintermsofthemselves(e.g.,treescontaintrees)
u SoatreeisEitherasinglevertex,oravertexthatistheparent
ofoneormoretrees
70
Eric
Emily Jane
Terry Bob
Drew Pam
Kim
Recursion and algorithms
u Conceptofrecursionappliestoalgorithmsaswell
u Somealgorithmsaredefinedrecursively:v Fibonaccinumbers:
u Fib(n)=0(n=0),1(n=1),fib(n-1)+fib(n-2)
u Somecanbeexpressediteratively:v Factorial=n*(n-1)*(n-2)*(n-3)…*1
u Orrecursively:v Factorial=n*factorial(n-1)
71
Recursion and algorithms
u Ifanabstractdatatypecanbethoughtofrecursively(likealist)theseofteninspirerecursivealgorithmsaswell
u Listsum:v Sumofalist=valueoffirstitem+sumoftherestofthelist
72
Recursion: algorithms
u Definingalgorithmsintermsofthemselves(e.g.,quicksort)Checkwhetherthesequencehasjustoneelement.Ifitdoes,stopCheckwhetherthesequencehastwoelements.Ifitdoes,andtheyareintherightorder,stop.Iftheyareinthewrongorder,swapthem,stop.Chooseapivotelementandrearrangethesequencetoputlower-valuedelementsononesideofthepivot,higher-valuedelementsontheothersideQuicksorttheleftsublistQuicksorttherightsublist
73
Recursion: algorithms
uHowdoyouwriteaselectionsortrecursively?uHowdoyouwriteabreadth-firstsearchofatreerecursively?Whataboutadepth-firstsearch?
74
Recursive Selection Sort
u Howtodothis?
u Needtothinkabouttheprobleminrecursiveterms:v Thinkoftheprobleminawaythatgetssmallereachtimeyouconsider
it…v Alsoneedstohaveaterminatingcondition(basecase)
u Thinkingofselectionsortinthisway…
75
Recursive selection sort
u Selectionsortfindsminimumelement,swapstofront.Thenfindsnextsmallest,swapsto2nd… andsoon
u Observation:thefrontelementiseither:v Alreadytheminimumorv Theminimumisintherestofthelist
u Observation:oncewemovetheminimumtothefrontofthelist,wecancallselectionsortontherestofthelist
76
Recursive selection sort
u Weactuallyneedtworecursivealgorithms:v find_min(list):recursivelyfindtheindexoftheminimumitemv selection_sort(list):
u Ifthelengthofthelistisone,stop,thelistissortedu callfind_min()tofindtheminimumelement,swapwiththefrontofthelist
(ifnecessary)u Callselection_sort()ontherestofthelist
v Stopwhen”restoflist”isoneitem
77
Recursive DFS, BFS
u RecursiveDFSisprettyeasy:v foreachneighboruofv:
u Ifuis‘unvisited’:calldfs(u)
u RecursiveBFS…
78
Analysis of algorithms
uHowlongdoesanalgorithmtaketorun?timecomplexity
uHowmuchmemorydoesitneed?spacecomplexity
79
Estimating running time
uHowtoestimatealgorithmrunningtime?vWriteaprogramthatimplementsthealgorithm,runit,andmeasurethetimeittakes
vAnalyzethealgorithm(independentofprogramminglanguageandtypeofcomputer)andcalculateinageneralwayhowmuchworkitdoestosolveaproblemofagivensize
uWhichisbetter?Why?80
Analysis of binary search
u n=8,thealgorithmtakes3stepsu n=32,thealgorithmtakes5stepsu Forageneraln,thealgorithmtakeslog2nsteps 81
Big O notation
u Characterizefunctionsaccordingtohowfasttheygrow
u Thegrowthrateofafunctioniscalledtheorderofthefunction.(hencetheO)
u BigOnotationusuallyonlyprovidesanupperbound onthegrowthrateofthefunction
u Asymptoticgrowth
f(x)=O(g(x))asx->∞ifandonlyifthereexistsapositivenumberM suchthatf(x)≤M*g(x)forallx>x0
82
Conventions
uO(1)denotesafunctionthatisaconstantv f(n)=3,g(n)=100000,h(n)=4.7areallsaidtobeO(1)
u Forafunctionf(n)=n2 itwouldbeperfectlycorrecttocallitO(n2)or O(n3)(orforthatmatterO(n100))
uHoweverbyconventionwecallitbythesmallestordernamely O(n2)v Why?
83
What do they have in common?
u (Binary)searchofasortedlist:O(log2n)
u Selectionsort:O(n2)
u Quicksort:O(nlogn)
u Breadthfirsttraversalofatree:O(V)
u Depthfirsttraversalofatree:O(V)
u Prim’salgorithmtofindtheMSTofagraph:O(V2)
u Kruskal’s algorithmtofindtheMSTofagraph:O(ElogE)
u Dijkstra’s algorithmtofindtheshortestpathfromanodeinagraphtoallothernodes:O(V2)
84
Subset sum problem
u Givenasetofintegersandanintegers,doesanynon-emptysubsetsumtos?
u {1,4,67,-1,42,5,17}ands=24 Nou {4,3,17,12,10,20}ands =19 Yes {4,3,12}
u IfasethasN elements,ithas2N subsets.u CheckingthesumofeachsubsettakesamaximumofNoperations
u Tocheckallthesubsetstakes2NNoperationsu Someclevernesscanreducethisbyabit(2Nbecomes 2N/2,butallknownalgorithmsareexponential
85
Travelling salesperson problem
u Givenalistofcitiesandthedistancesbetweeneachpairofcities,whatistheshortestpossibleroutethatvisitseachcityexactlyonceandreturnstotheorigincity?
u Givenagraphwhereedgesarelabeledwithdistancesbetweenvertices.Startataspecifiedvertex,visitallotherverticesexactlyonceandreturntothestartvertexinsuchawaythatsumoftheedgeweightsisminimized
u Therearen! routes(anumberontheorderofnn - muchbiggerthan2n)
u O(n!)
86
Enumerating permutations
u Listallpermutations(i.e.allpossibleorderings)ofnnumbers
uWhatistheorderofanalgorithmthatcandothis?
87
u Sowehave:v Knapsack/Subsetsum:N*2N
v Setpermutation:n!v Travelingsalesman:n!
88
Analysis of problems
u Studyofalgorithmsilluminatesthestudyofclasses ofproblems
u Ifapolynomialtimealgorithmexiststosolveaproblemthentheproblemiscalledtractable
u Ifaproblemcannotbesolvedbyapolynomialtimealgorithmthenitiscalledintractable
u Thisdividesproblemsintothree groups:v Problemswithknownpolynomialtimealgorithmsv Problemsthatareproventohavenopolynomial-timealgorithmv Problemswithnoknownpolynomialtimealgorithmbutnotyetproventobeintractable
89
Tractable and Intractable
u Tractableproblems(P)v Sortingalistv Searchinganunorderedlistv Findingaminimumspanningtree
inagraph
90
u Intractablev Listingallpermutations(all
possibleorderings)ofnnumbers
u Mightbe(in)tractablev Subsetsum:givenasetof
numbers,isthereasubsetthataddsuptoagivennumber?
v Travellingsalesperson:ncities,n!routes,findtheshortestroute
Theseproblemshavenoknownpolynomialtimesolution
Howevernoonehasbeenabletoprovethatsuchasolutiondoesnotexist
Tractability and Intractability
u ‘Propertiesofproblems’(NOT ‘propertiesofalgorithms’)
u Tractable:problemcanbesolvedbyapolynomialtimealgorithm(orsomethingmoreefficient)
u Intractable:problemcannotbesolvedbyapolynomialtimealgorithm(allsolutionsareproventobemoreinefficientthanpolynomialtime)
u Unknown:notknowniftheproblemistractableorintractable(noknownpolynomialtimesolution,noproofthatapolynomialtimesolutiondoesnotexist)
91
Subset sum problem
u Givenasetofintegersandanintegers,doesanynon-emptysubsetsumtos?
u {1,4,67,-1,42,5,17}ands=24 Nou {4,3,17,12,10,20}ands =19 Yes {4,3,12}
u IfasethasN elements,ithas2N subsets.u CheckingthesumofeachsubsettakesamaximumofNoperations
u Tocheckallthesubsetstakes2NNoperationsu Someclevernesscanreducethisbyabit(2Nbecomes 2N/2,butallknownalgorithmsareexponential)
92
P and NP
u P:setofproblemsthatcanbesolvedinpolynomialtime
u Considersubsetsumv Noknownpolynomialtimealgorithmv However,ifyougivemeasolutiontotheproblem,itiseasyformetocheckifthesolutioniscorrect– i.e.Icanwriteapolynomialtimealgorithmtocheckifagivensolutioniscorrect
uNP:setofproblemsforwhichasolutioncanbecheckedinpolynomialtime
93
Easytosolve(implieseasytocheck)
Easytocheckifsolutionisgood
Easy to Solve vs. Easy to Check
u Easytosolve:sortingv Solve:sortthelistinO(nlogn)v Check:isthelistsorted?O(n)v ClearlysortingisinP
u Hardtosolve:sub-setsumv Solve:generateallsubsets:O(2n)v Check:sum-upsubset.O(n)
u Hardtosolve:integerfactorizationv Solve:checkallnumbersbetween2andsqrt(n)O(2w)v Check:isonenumberafactorofanother?DivideandcheckO(n2)
94
P=NP?
95
uAllproblemsinP arealsoinNPuArethereanyproblemsinNP thatarenotalsoinP?
uInotherwords,is
P =NP ?uCentralopenquestioninComputerScience
P vs. NP Example
u Publickeyencryptionusestwolargeprimenumbersp,qu Ifk=p*q, thenwecansendk intheclearneed pandq todecrypt
u WhyisthisPvs.NP?v p*q clearlyPalgorithmv Findingpandq givenjustk isO(2w)wherew=sizeofthenumber
(digitsorbits)
u IfP=NPthenpublickeyencryptionwouldbe“broken”u Sidenote:ascomputershavegottenfaster,keysizegoesup,makingproblemexponentiallyharderv Keysarenow>=2048bits->22048 isapreposterouslylargenumberv Check1Bkeys/second=1.7x10600 yearstocrack
96
Midterm Style Questions
97
1. Basedontheinformationpresentedinclassandthelectureslides,whichcomponentisnotpartofamodernCPU:A. Arithmetic/logicunitB. ProgramCounterC. CachememoryD. DiskcontrollerE. Registers
2. Whichchoiceforpivotalwaysallowsoptimalruntimeofthequicksortalgorithm?
A.MaximumelementB.MinimumelementC.AverageamongallelementsD.AveragebetweenmaximumandminimumelementsE.Noneoftheabove
3. Inordertofindthek-thsmallestelementinalistofnintegerswerunasmany
iterationsofSelectionSortasnecessaryandthenwestop.Whatisthecomplexityofthisalgorithmintermsofk,n?A. O(k*log(n))B. O(k*n*log(n))C. O(n*log(n))D. O(k*n)E. Notenoughinformationisgiventodeterminethecorrectanswer
4. WhichisTRUEaboutDFS(depthfirstsearch)vs.BFS(breadthfirstsearch)?
A.BFSonaveragehassmallerruntimethanDFSB.DFSonaveragehassmallerruntimethanBFSC.BFScannotdetectloops(i.ewouldnotworkongraphs)D.DFScannotdetectloopsE.Noneoftheaboveistrue
5. Whichisthecorrectorderingofmemoryfromfastestaccesstoslowestaccess?
A. Registers,RAM,Cache,DiskB. Disk,RAM,Cache,RegistersC. Registers,Cache,RAM,DiskD. Cache,Registers,RAM,DiskE. Disk,Cache,RAM,Registers
1. Basedontheinformationpresentedinclassandthelectureslides,whichcomponentisnotpartofamodernCPU:A. Arithmetic/logicunitB. ProgramCounterC. CachememoryD. DiskcontrollerE. Registers
2. Whichchoiceforpivotalwaysallowsoptimalruntimeofthequicksortalgorithm?
A.MaximumelementB.MinimumelementC.AverageamongallelementsD.AveragebetweenmaximumandminimumelementsE.Noneoftheabove
3. Inordertofindthek-thsmallestelementinalistofnintegerswerunasmany
iterationsofSelectionSortasnecessaryandthenwestop.Whatisthecomplexityofthisalgorithmintermsofk,n?A. O(k*log(n))B. O(k*n*log(n))C. O(n*log(n))D. O(k*n)E. Notenoughinformationisgiventodeterminethecorrectanswer
4. WhichisTRUEaboutDFS(depthfirstsearch)vs.BFS(breadthfirstsearch)?
A.BFSonaveragehassmallerruntimethanDFSB.DFSonaveragehassmallerruntimethanBFSC.BFScannotdetectloops(i.ewouldnotworkongraphs)D.DFScannotdetectloopsE.Noneoftheaboveistrue
5. Whichisthecorrectorderingofmemoryfromfastestaccesstoslowestaccess?
A. Registers,RAM,Cache,DiskB. Disk,RAM,Cache,RegistersC. Registers,Cache,RAM,DiskD. Cache,Registers,RAM,DiskE. Disk,Cache,RAM,Registers
Midterm Style Questions
98
6. Considertheabovegraph.WhatisthesequenceofnodesproducedbyrunningBreadthFirstSearch(BFS)startingfromv1?ForBFSonadirectedgraphonlyfollowoutgoingedges.Ifanodehasmorethanoneoutgoingedge,followtheedgetothevertexwiththelowestindexfirst(nodeindexiskforvertexVk).
A. v1 v2 v4 v5 v7 v6 v3B. v1 v2 v4 v5 v6 v7 v3C. v1 v2 v4 v3 v6 v5 v7D. v1 v2 v4 v5 v3 v6 v7E. v1 v2 v4 v5 v3 v7 v6
7. Forthesamegraphasabove,whatisthesequenceofnodesproducedbyrunningDepthFirstSearch(DFS)startingfromv1.ForDFSonadirectedgraphonlyfollowoutgoingedges.Ifanodehasmorethanoneoutgoingedge,followtheedgetothevertexwiththelowestindexfirst(nodeindexiskforvertexVk).
A. v1 v2 v4 v3 v5 v6 v7B. v1 v2 v4 v5 v3 v7 v6C. v1 v2 v4 v3 v6 v5 v7D. v1 v2 v4 v5 v7 v6 v3E.v1 v2 v4v5v3v7v6
8. WhichoftheproblemsdescribedCANNOTbesolvedoptimallywithanMST(minimumspanningtree)?
A.Buildtheshortest-lengthbridgenetworkbetweenasetofislands.
B.Eliminateloopsinacomputernetwork.C.Givenalistofcitiesandthedistancesbetweeneachpair,findtheshortestpossibleroutethatvisitseachcityandreturnstothestartingcity.
D.Eliminatemultiplepathsbetweenanytwoverticesinagraph.E.AlloftheaboveCANbesolvedoptimallywithaMST.
6
V1 V2
2
4 10 1 3
V3 V4 V5 2 2
V6 V7
5
1
8 4
9. ImaginethatitwasprovedthatP=NP.Considerthefollowingstatements:I) Wewillbeabletoverifyanycomputationalprobleminpolynomialtime.
II) Wewillbeabletosolveanycomputationalprobleminpolynomialtime.
III) WecanbuildhardwareimplementationofanondeterministicTuringmachine
IV) Allcomputationalproblemsthatcanbeverifiedinpolynomialtimewillbesolvable
inpolynomialtime
V) WewillbeabletosolveanyNPhardprobleminpolynomialtime
A. IIandVaretrueB. OnlyIIistrueC. OnlyIVistrueD. IandIVaretrueE. AllstatementsI–Varetrue
10. Whatistheroleoftheprogramcounter?A. ItisaspecialregisterintheCPUusedtostorehigh-prioritydataB. IttracksthenumberofinstructionsexecutedbythecurrentlyexecutingprogramC. Ittracksrecentlyaccessedmemorylocationstohelpimplementtheprincipleof
temporallocality
D. ItspecifiesthememorylocationfromwheretheCPUwillfetchthenextinstructionE. IttracksthenumberofrunningprogramsaskingforaccesstotheCPU
11. WhichofthefollowingisTRUEaboutbinarysearch?A. Consideringtheinputdata,binarysearchwillALWAYShaveasmallerruntimevs.sequentialsearchonthesamedata.
B. BinarysearchcanbeappliedtoanylistC. BinarysearchhasruntimecomplexityofO(2N)foranunsortedlistD. BinarysearchcanbeimplementedrecursivelyE. Noneoftheaboveistrue
12. WhichstatementisFALSE?A. EverytreeisagraphB. EverygraphhasacycleC. AtreehasnocyclesD. Everyconnected,undirected,weightedgraphhasatleastonespanningtreeE. Thenumberofverticesintheminimumspanningtreeofagraphisthesameasthenumberofverticesinthegraphitself
Midterm Style Questions
99
13. Whenconsideredintermsofbig-O,chosetheorderingofthefollowingfunctionsthatisinorderofincreasingasymptoticgrowthrates:A. 2n,2n-1,2n+1B. 2n-1,2n+1,2nC. 2n-1,2n,2n+1D. Allthreeofthesefunctionshavethesamebig-Ogrowthrate,soorderisnot
important.14. ThereexistsadatastructurecalledaMaxHeapthatwillreturnit’smaximumelement
(andthenremoveitfromtheheap)inO(logn).YoucanconvertanunsortedlisttoaMaxHeapinO(n).Thus,youcansortanunsortedlistusingaMaxHeap(resultinginareverseordescendingorderedsort).Whatisthetimecomplexityofthissortingalgorithm?A. O(n2log(n))B. O(n2)C. O(log(n)*log(n))D. O(nlog(n))E. O(n)
15. Whatbestdescribesthedefinitionofanalgorithm?
A. Anunambiguousandordered,stepbystepdescriptionofactionstosolveaproblemwrittentobeunderstandablebyhumans
B. sequenceofinstructionwrittentorundirectlyonacomputerC. codedlanguageusedbyprogrammerstowriteinstructionsthatacomputercan
understandtodowhattheprogrammerwantsD. Acodewhosetargetalphabetcontainsonlydigitsand/orstringsofdigitsE. Amathematicalcalculationaccordingtosomewell-knownformula
16. Youareinamazeandafriendsuggeststhatyouputyourrighthandonthewalland
followthewalluntilyoufindtheexit.This“righthandrule”representsanalgorithmforsolvingthemaze.Whichalgorithmdiscussedinclassdoestheapproachcorrespondto?A.BreadthFirstSearchB.DepthFirstSearchC.Kruskal’sAlgorithmD.BinarySearch
1. Basedontheinformationpresentedinclassandthelectureslides,whichcomponentisnotpartofamodernCPU:A. Arithmetic/logicunitB. ProgramCounterC. CachememoryD. DiskcontrollerE. Registers
2. Whichchoiceforpivotalwaysallowsoptimalruntimeofthequicksortalgorithm?
A.MaximumelementB.MinimumelementC.AverageamongallelementsD.AveragebetweenmaximumandminimumelementsE.Noneoftheabove
3. Inordertofindthek-thsmallestelementinalistofnintegerswerunasmany
iterationsofSelectionSortasnecessaryandthenwestop.Whatisthecomplexityofthisalgorithmintermsofk,n?A. O(k*log(n))B. O(k*n*log(n))C. O(n*log(n))D. O(k*n)E. Notenoughinformationisgiventodeterminethecorrectanswer
4. WhichisTRUEaboutDFS(depthfirstsearch)vs.BFS(breadthfirstsearch)?
A.BFSonaveragehassmallerruntimethanDFSB.DFSonaveragehassmallerruntimethanBFSC.BFScannotdetectloops(i.ewouldnotworkongraphs)D.DFScannotdetectloopsE.Noneoftheaboveistrue
5. Whichisthecorrectorderingofmemoryfromfastestaccesstoslowestaccess?
A. Registers,RAM,Cache,DiskB. Disk,RAM,Cache,RegistersC. Registers,Cache,RAM,DiskD. Cache,Registers,RAM,DiskE. Disk,Cache,RAM,Registers
Midterm Style Questions
100
17. Thealgorithmfrom#16canfailonsomemazes.Themazescanbethoughtofasgraphs.Ifthegraphrepresentationofthemazehascertainproperties,thenthe“righthandrule”mightfail.Whichpropertycancausefailure?A.Weighted(somepassagesaresmaller,orsomepassagesforceyoutocrawl)B.Cyclic(thereareloopsinthemaze) C.Planar(themazecanbedrawnasagraphwherenoedgescross) D.Acyclic(therearenoloopsinthemaze)
18. WhichofthefollowingisTRUEabouttreesandgraphs?
A.AlltreesaregraphsB.EverygraphhasexactlyoneMSTC.AlltreesaredirectedgraphsD.AllgraphsaretreesE.NotallgraphshaveaMST
19. TheJacquardLoom(andsimilarmachines)areconsideredinformationtransformers,
butnotcomputers.Whichanswerbestdescribeswhy:A.Programmingthesemachinesdoesn’tscaleB.Programmingthesemachinesrequirespunch-cardsC.MachineslikethesedonothavememoryorcontrolflowD.Machineslikethesearetoooldtobeconsideredcomputers
20. Whenaninstructionisloadedfrommemory,itisdesirabletoloadthecontentsofafewsucceedingmemoryaddressesintothecache.Whyisthat?A.TheCPUisunabletoonlyloadoneinstructionatatimeB.Thosecontentsarelikelytobeusefulintheimmediatefutureaccordingtothe
spatiallocalityprincipleC.Thosecontentsarelikelytobeusefulintheimmediatefutureaccordingtothe
temporallocalityprincipleD.Thecontentsstoredaftertheinstructionarethevaluesusedinthecomputationof
theinstructionandthereforemustbeloadedwiththeinstruction.
21. Thesubset-sumproblemhastimecomplexityO(N*2N).WheredoesthefactorNcomefrom?A:ThatishowmanysubsetsasetofsizeNhas.B:O(N)isthetimecomplexityrequiredtocheckeachpossiblesubsetsum.C:Thatisthetimecomplexityofthealgorithmthatgeneratesthesubsets.D:Noneoftheabove.
17. Thealgorithmfrom#16canfailonsomemazes.Themazescanbethoughtofasgraphs.Ifthegraphrepresentationofthemazehascertainproperties,thenthe“righthandrule”mightfail.Whichpropertycancausefailure?A.Weighted(somepassagesaresmaller,orsomepassagesforceyoutocrawl)B.Cyclic(thereareloopsinthemaze) C.Planar(themazecanbedrawnasagraphwherenoedgescross) D.Acyclic(therearenoloopsinthemaze)
18. WhichofthefollowingisTRUEabouttreesandgraphs?
A.AlltreesaregraphsB.EverygraphhasexactlyoneMSTC.AlltreesaredirectedgraphsD.AllgraphsaretreesE.NotallgraphshaveaMST
19. TheJacquardLoom(andsimilarmachines)areconsideredinformationtransformers,
butnotcomputers.Whichanswerbestdescribeswhy:A.Programmingthesemachinesdoesn’tscaleB.Programmingthesemachinesrequirespunch-cardsC.MachineslikethesedonothavememoryorcontrolflowD.Machineslikethesearetoooldtobeconsideredcomputers
20. Whenaninstructionisloadedfrommemory,itisdesirabletoloadthecontentsofafewsucceedingmemoryaddressesintothecache.Whyisthat?A.TheCPUisunabletoonlyloadoneinstructionatatimeB.Thosecontentsarelikelytobeusefulintheimmediatefutureaccordingtothe
spatiallocalityprincipleC.Thosecontentsarelikelytobeusefulintheimmediatefutureaccordingtothe
temporallocalityprincipleD.Thecontentsstoredaftertheinstructionarethevaluesusedinthecomputationof
theinstructionandthereforemustbeloadedwiththeinstruction.
21. Thesubset-sumproblemhastimecomplexityO(N*2N).WheredoesthefactorNcomefrom?A:ThatishowmanysubsetsasetofsizeNhas.B:O(N)isthetimecomplexityrequiredtocheckeachpossiblesubsetsum.C:Thatisthetimecomplexityofthealgorithmthatgeneratesthesubsets.D:Noneoftheabove.