PowerPoint Presentation

OverviewIntroduction to Parallel ComputingCIS 410/510Department of Computer and Information ScienceLecture 1 OverviewOutline Course OverviewWhat is CIS 410/510?What is expected of you?What will you learn in CIS 410/510?Parallel ComputingWhat is it?What motivates it?Trends that shape the fieldLarge-scale problems and high-performanceParallel architecture typesScalable parallel computing and performance2Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewHow did the idea for CIS 410/510 originate?There has never been an undergraduate course in parallel computing in the CIS Department at UOOnly 1 course taught at the graduate level (CIS 631)Goal is to bring parallel computing education in CIS undergraduate curriculum, start at senior levelCIS 410/510 (Spring 2014, experimental course)CIS 431/531 (Spring 2015, new course)CIS 607 Parallel Computing Course DevelopmentWinter 2014 seminar to plan undergraduate courseDevelop 410/510 materials, exercises, labs, Intel gave a generous donation ($100K) to the effortNSF and IEEE are spearheading a curriculum initiative for undergraduate education in parallel processinghttp://www.cs.gsu.edu/~tcpp/curriculum/

3Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewWhos involved?InstructorAllen D. Malonyscalable parallel computingparallel performance analysistaught CIS 631 for the last 10 yearsFaculty colleagues and course co-designersBoyana NorrisAutomated software analysis and transformationPerformance analysis and optimizationHank ChildsLarge-scale, parallel scientific visualizationVisualization of large data setsIntel scientistsMichael McCool, James Reinders, Bob MacKayGraduate students doing research in parallel computing4

Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewIntel PartnersJames ReindersDirector, Software ProductsMulti-core EvangelistMichael McCoolSoftware architectFormer Chief scientist, RapidMindAdjunct Assoc. Professor, University of WaterlooArch RobisonArchitect of Threading Building BlocksFormer lead developers of KAI C++David MacKayManager of software product consulting team5

Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewCIS 410/510 Graduate AssistantsDaniel Ellsworth3rd year Ph.D. studentResearch advisor (Prof. Malony)Large-scale online system introspectionDavid Poliakoff2nd year Ph.D. studentResearch advisor (Prof. Malony)Compiler-based performance analysisBrandon Hildreth1st year Ph.D. studentResearch advisor (Prof. Malony)Automated performance experimentation

6

Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewRequired Course BookStructured Parallel Programming: Patterns for Efficient Computation, Michael McCool,Arch Robinson, James Reinders,1st edition, Morgan Kaufmann,ISBN: 978-0-12-415993-8, 2012http://parallelbook.com/ Presents parallel programmingfrom a point of view of patternsrelevant to parallel computationMap, Collectives, Datareorganization, Stencil and recurrence,Fork-Join, PipelineFocuses on the use of sharedmemory parallel programminglanguages and environmentsIntel Thread Building Blocks (TBB)Intel Cilk Plus7

Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewReference TextbooksIntroduction to Parallel Computing, A. Grama,A. Gupta, G. Karypis, V. Kumar, Addison Wesley,2nd Ed., 2003Lecture slides from authors onlineExcellent reference list at endUsed for CIS 631 beforeGetting old for latest hardwareDesigning and Building Parallel Programs,Ian Foster, Addison Wesley, 1995.Entire book is online!!!Historical book, but very informativePatterns for Parallel Programming T. Mattson,B. Sanders, B. Massingill, Addison Wesley, 2005.Targets parallel programmingPattern language approach to parallelprogram design and developmentExcellent references8

Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewWhat do you mean by experimental course?Given that this is the first offering of parallel computing in the undergraduate curriculum, we want to evaluate how well it workedWe would like to receive feedback from students throughout the courseLecture content and understandingParallel programming learning experienceBook and other materialsYour experiences will help to update the course for it debut offering in (hopefully) Spring 20159Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewCourse PlanOrganize the course so that cover main areas of parallel computing in the lecturesArchitecture (1 week)Performance models and analysis (1 week)Programming patterns (paradigms) (3 weeks)Algorithms (2 weeks)Tools (1 week)Applications (1 week)Special topics (1 week)Augment lecture with a programming labStudents will take the lab with the coursegraded assignments and term project will be postedTargeted specifically to shared memory parallelism

10Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewLecturesBook and online materials are you main sources for broader and deeper background in parallel computingLectures should be more interactiveSupplement other sources of informationCovers topics of more priorityIntended to give you some of my perspectiveWill provide online access to lecture slidesLectures will complement programming component, but intended to cover other parallel computing aspectsTry to arrange a guest lecture or 2 during quarter11Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewParallel Programming LabSet up in the IPCC classroomDaniel Ellsworth and David Poliakoff leading the labShared memory parallel programming (everyone)Cilk Plus ( http://www.cilkplus.org/ )extension to the C and C++ languages to support data and task parallelismThread Building Blocks (TBB) (https://www.threadingbuildingblocks.org/ )C++ template library for task parallelism OpenMP (http://openmp.org/wp/ )C/C++ and Fortran directive-based parallelismDistributed memory message passing (graduate)MPI (http://en.wikipedia.org/wiki/Message_Passing_Interface )library for message communication on scalable parallel systems12Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewWOPR (What Operational Parallel Resource)WOPR was built from whole clothConstructed by UO graduate studentsBuilt Next Unit of Computing (NUC)cluster with Intel funds16x Intel NUCHaswell i5 CPU (2 cores, hyperthreading)Intel HD 4000 GPU (OpenCL programmable)1 GigE, 16 GB memory, 240 GB mSATA16x Logitech keyboard and mouse16x ViewSonic 22 monitorDell Edge GigE switchDell head node13

Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewOther Parallel Resources Mist ClusterDistributed memory cluster16 8-core nodes2x quad-core Pentium Xeon (2.33 GHz)16 Gbyte memory160 Gbyte diskDual Gigabit ethernet adaptorsMaster node (same specs)Gigabit ethernet switchmist.nic.uoregon.edu

14Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewOther Parallel Resources ACISS ClusterApplied Computational Instrument for Scientific SynthesisNSF MRI R2 award (2010)Basic nodes (1,536 total cores)128 ProLiant SL390 G7Two Intel X5650 2.66 GHz6-core CPUs per node72GB DDR3 RAM per basic nodeFat nodes (512 total cores)16 ProLiant DL 580 G7Four Intel X7560 2.266 GHz8-core CPUs per node384GB DDR3 per fat nodeGPU nodes (624 total cores, 156 GPUs) 52 ProLiant SL390 G7 nodes, 3 NVidia M2070 GPUs (156 total GPUs)Two Intel X5650 2.66 GHz 6-core CPUs per node (624 total cores)72GB DDR3 per GPU nodeACISS has 2672 total coresACISS is located in the UO Computing Center

15Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewCourse AssignmentsHomeworkExercises primarily to prepare for midtermParallel programming labExercises for parallel programming patternsProgram using Cilk Plus, Thread Building Blocks, OpenMPGraduate students will also do assignments with MPITeam term projectProgramming, presentation, paperGraduate students distributed across teamsResearch summary paper (graduate students)Midterm exam later in the 7th week of the quarterNo final examTeam project presentations during final period16Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewParallel Programming Term ProjectMajor programming project for the courseNon-trivial parallel applicationInclude performance analysisUse NUC cluster and possibly Mist and ACISS clustersProject teams5 person teams, 6 teams (depending on enrollment)Will try our best to balance skillsHave 1 graduate student per teamProject datesProposal due end of 4th week)Project talk during last classProject due at the end of the termNeed to get system accounts!!!

17Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewTerm Paper (for graduate students) Investigate parallel computing topic of interestMore in depth reviewIndividual choiceSummary of major pointsRequires minimum of ten referencesBook and other references has a large bibliographyGoogle Scholar, Keywords: parallel computingNEC CiteSeer Scientific Literature Digital LibraryPaper abstract and references due by 3rd weekFinal term paper due at the end of the termIndividual work18Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewGradingUndergraduate5%homework10%pattern programming labs20%programming assignments30%midterm exam35%projectGraduate15%programming assignments30%midterm exam35%project20%research paper19Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewOverviewBroad/Old field of computer science concerned with:Architecture, HW/SW systems, languages, programming paradigms, algorithms, and theoretical modelsComputing in parallelPerformance is the raison dtre for parallelismHigh-performance computingDrives computational science revolutionTopics of studyParallel architecturesParallel programmingParallel algorithms20Parallel performance models and toolsParallel applicationsIntroduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewWhat will you get out of CIS 410/510?In-depth understanding of parallel computer designKnowledge of how to program parallel computer systemsUnderstanding of pattern-based parallel programmingExposure to different forms parallel algorithmsPractical experience using a parallel clusterBackground on parallel performance modelingTechniques for empirical performance analysisFun and new friends21Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewParallel Processing What is it?A parallel computer is a computer system that uses multiple processing elements simultaneously in a cooperative manner to solve a computational problemParallel processing includes techniques and technologies that make it possible to compute in parallelHardware, networks, operating systems, parallel libraries, languages, compilers, algorithms, tools, Parallel computing is an evolution of serial computingParallelism is naturalComputing problems differ in level / type of parallelismParallelism is all about performance! Really?22Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewConcurrencyConsider multiple tasks to be executed in a computerTasks are concurrent with respect to each ifThey can execute at the same time (concurrent execution)Implies that there are no dependencies between the tasksDependenciesIf a task requires results produced by other tasks in order to execute correctly, the tasks execution is dependentIf two tasks are dependent, they are not concurrentSome form of synchronization must be used to enforce (satisfy) dependenciesConcurrency is fundamental to computer scienceOperating systems, databases, networking, 23Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewConcurrency and ParallelismConcurrent is not the same as parallel! Why?Parallel executionConcurrent tasks actually execute at the same timeMultiple (processing) resources have to be availableParallelism = concurrency + parallel hardwareBoth are requiredFind concurrent execution opportunitiesDevelop application to execute in parallelRun application on parallel hardwareIs a parallel application a concurrent application?Is a parallel application run with one processor parallel? Why or why not?24Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewParallelismThere are granularities of parallelism (parallel execution) in programsProcesses, threads, routines, statements, instructions, Think about what are the software elements that execute concurrentlyThese must be supported by hardware resourcesProcessors, cores, (execution of instructions)Memory, DMA, networks, (other associated operations)All aspects of computer architecture offer opportunities for parallel hardware executionConcurrency is a necessary condition for parallelismWhere can you find concurrency?How is concurrency expressed to exploit parallel systems?25Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewWhy use parallel processing?Two primary reasons (both performance related)Faster time to solution (response time)Solve bigger computing problems (in same time)Other factors motivate parallel processingEffective use of machine resourcesCost efficienciesOvercoming memory constraintsSerial machines have inherent limitationsProcessor speed, memory bottlenecks, Parallelism has become the future of computingPerformance is still the driving concernParallelism = concurrency + parallel HW + performance26Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewPerspectives on Parallel ProcessingParallel computer architectureHardware needed for parallel execution?Computer system design(Parallel) Operating systemHow to manage systems aspects in a parallel computerParallel programmingLibraries (low-level, high-level)LanguagesSoftware development environmentsParallel algorithmsParallel performance evaluationParallel toolsPerformance, analytics, visualization, 27Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewWhy study parallel computing today?Computing architectureInnovations often drive to novel programming modelsTechnological convergenceThe killer micro is ubiquitousLaptops and supercomputers are fundamentally similar!Trends cause diverse approaches to convergeTechnological trends make parallel computing inevitableMulti-core processors are here to stay!Practically every computing system is operating in parallelUnderstand fundamental principles and design tradeoffsProgramming, systems support, communication, memory, PerformanceParallelism is the future of computing28Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewInevitability of Parallel ComputingApplication demandsInsatiable need for computing cyclesTechnology trendsProcessor and memoryArchitecture trendsEconomicsCurrent trends:Todays microprocessors have multiprocessor supportServers and workstations available as multiprocessorsTomorrows microprocessors are multiprocessorsMulti-core is here to stay and #cores/processor is growingAccelerators (GPUs, gaming systems)29Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewApplication CharacteristicsApplication performance demands hardware advancesHardware advances generate new applicationsNew applications have greater performance demandsExponential increase in microprocessor performanceInnovations in parallel architecture and integration

Range of performance requirementsSystem performance must also improve as a wholePerformance requirements require computer engineeringCosts addressed through technology advancementsapplicationsperformancehardware30Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewBroad Parallel Architecture IssuesResource allocationHow many processing elements? How powerful are the elements?How much memory?Data access, communication, and synchronizationHow do the elements cooperate and communicate?How are data transmitted between processors?What are the abstractions and primitives for cooperation?Performance and scalabilityHow does it all translate into performance?How does it scale?31Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewLeveraging Moores LawMore transistors = more parallelism opportunitiesMicroprocessorsImplicit parallelismpipeliningmultiple functional unitssuperscalarExplicit parallelismSIMD instructionslong instruction works32Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewWhats Driving Parallel Computing Architecture?

von Neumann bottleneck!!(memory wall)33Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewMicroprocessor Transitor Counts (1971-2011)

34Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewWhat has happened in the last several years? Processing chip manufacturers increased processor performance by increasing CPU clock frequencyRiding Moores lawUntil the chips got too hot!Greater clock frequency greater electrical powerPentium 4 heat sink Frying an egg on a Pentium 4

Add multiple cores to add performanceKeep clock frequency same or reducedKeep lid on power requirements35

Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 Overview36Power Density Growth

Figure courtesy of Pat Gelsinger, Intel Developer Forum, Spring 2004Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewWhats Driving Parallel Computing Architecture?

37Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewWhats Driving Parallel Computing Architecture?

38power wallIntroduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewClassifying Parallel Systems Flynns TaxonomyDistinguishes multi-processor computer architectures along the two independent dimensionsInstruction and DataEach dimension can have one state: Single or MultipleSISD: Single Instruction, Single DataSerial (non-parallel) machineSIMD: Single Instruction, Multiple DataProcessor arrays and vector machinesMISD: Multiple Instruction, Single Data (weird)MIMD: Multiple Instruction, Multiple DataMost common parallel computer systems39Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewParallel Architecture TypesInstruction-Level ParallelismParallelism captured in instruction processingVector processorsOperations on multiple data stored in vector registersShared-memory Multiprocessor (SMP)Multiple processors sharing memorySymmetric Multiprocessor (SMP)MulticomputerMultiple computer connect via networkDistributed-memory clusterMassively Parallel Processor (MPP)40Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewPhases of Supercomputing (Parallel) ArchitecturePhase 1 (1950s): sequential instruction executionPhase 2 (1960s): sequential instruction issuePipeline execution, reservations stationsInstruction Level Parallelism (ILP)Phase 3 (1970s): vector processorsPipelined arithmetic unitsRegisters, multi-bank (parallel) memory systemsPhase 4 (1980s): SIMD and SMPsPhase 5 (1990s): MPPs and clustersCommunicating sequential processorsPhase 6 (>2000): many cores, accelerators, scale, 41Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewPerformance ExpectationsIf each processor is rated at k MFLOPS and there are p processors, we should expect to see k*p MFLOPS performance? Correct?If it takes 100 seconds on 1 processor, it should take 10 seconds on 10 processors? Correct?Several causes affect performanceEach must be understood separatelyBut they interact with each other in complex wayssolution to one problem may create anotherone problem may mask anotherScaling (system, problem size) can change conditionsNeed to understand performance space

42Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewScalabilityA program can scale up to use many processorsWhat does that mean?How do you evaluate scalability?How do you evaluate scalability goodness?Comparative evaluationIf double the number of processors, what to expect?Is scalability linear?Use parallel efficiency measureIs efficiency retained as problem size increases?Apply performance metrics43Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewTop 500 Benchmarking MethodologyListing of the worlds 500 most powerful computersYardstick for high-performance computing (HPC)Rmax : maximal performance Linpack benchmarkdense linear system of equations (Ax = b)Data listedRpeak : theoretical peak performanceNmax : problem size needed to achieve RmaxN1/2 : problem size needed to achieve 1/2 of RmaxManufacturer and computer typeInstallation site, location, and yearUpdated twice a year at SC and ISC conferences44

Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 Overview

45Top 10 (November 2013)Different architecturesIntroduction to Parallel Computing, University of Oregon, IPCCLecture 1 Overview46Top 500 Performance (November 2013)

Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 Overview#1: NUDT Tiahne-2 (Milkyway-2) Compute Nodes have 3.432 Tflop/s per node16,000 nodes32000 Intel Xeon CPU48000 Intel Xeon PhiOperations Nodes4096 FT CPUsProprietary interconnectTH2 express1PB memoryHost memory onlyGlobal shared parallel storage is 12.4 PBCabinets: 125+13+24 =162Compute, communication, storage~750 m247

Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 Overview

#2: ORNL Titan Hybrid System (Cray XK7)Peak performance of 27.1 PF24.5 GPU + 2.6 CPU18,688 Compute Nodes each with:16-Core AMD Opteron CPUNVIDIA Tesla K20x GPU32 + 6 GB memory512 Service and I/O nodes200 Cabinets710 TB total system memoryCray Gemini 3D Torus Interconnect8.9 MW peak power48

4,352 ft2Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 Overview

#3: LLNL Sequoia (IBM BG/Q)Compute card16-core PowerPC A2 processor16 GB DDR3Compute node has 98,304 cardsTotal system size:1,572,864 processing cores1.5 PB memory5-dimensional torus interconnection networkArea of 3,000 ft249Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 Overview#4: RIKEN K Computer80,000 CPUsSPARC64 VIIIfx640,000 cores800 water-cooled racks5D mesh/torus interconnect (Tofu)12 links between node12x higher scalability than 3D torus50

Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewContemporary HPC Architectures51DateSystemLocationCompCommPeak (PF)Power (MW)2009Jaguar; Cray XT5ORNLAMD 6cSeastar22.37.02010Tianhe-1ANSC TianjinIntel + NVIDIAProprietary4.74.02010NebulaeNSCS ShenzhenIntel + NVIDIAIB2.92.62010Tsubame 2TiTechIntel + NVIDIAIB2.41.42011K ComputerRIKEN/KobeSPARC64 VIIIfxTofu10.512.72012Titan; Cray XK6ORNLAMD + NVIDIAGemini2792012Mira; BlueGeneQANLSoCProprietary103.92012Sequoia; BlueGeneQLLNLSoCProprietary207.92012Blue Waters; CrayNCSA/UIUCAMD + (partial) NVIDIAGemini11.62013StampedeTACCIntel + MICIB9.552013Tianhe-2NSCC-GZ (Guangzhou)Intel + MICProprietary54~20Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewTop 10 (Top500 List, June 2011)Figure credit: http://www.netlib.org/utk/people/JackDongarra/SLIDES/korea-2011.pdf

52Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewJapanese K Computer (#1 in June 2011)

53Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewTop 500 Top 10 (2006)

54Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewTop 500 Linpack Benchmark List (June 2002)

55Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewJapanese Earth SimulatorWorlds fastest supercomputer!!! (2002)640 NEC SX-6 nodes8 vector processors5104 total processorsSingle stage crossbar~2900 meters of cables10 TB memory700 TB disk space1.6 PB mass storage40 Tflops peak performance35.6 Tflops Linpack performance

56Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewProf. Malony and colleagues at Japanese ES

Mitsuhisa SatoBarbara ChapmanMatthias Mller57Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewPerformance Development in Top 500

Figure credit: http://www.netlib.org/utk/people/JackDongarra/SLIDES/korea-2011.pdf58Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewExascale InitiativeExascale machines are targeted for 2019What are the potential differences and problems?

???59Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewMajor Changes to Software and AlgorithmsWhat were we concerned about before and now?Must rethink the design for exascaleData movement is expensive (Why?)Flops per second are cheap (Why?)Need to reduce communication and sychronizationNeed to develop fault-resilient algorithmsHow do with deal with massive parallelism?Software must adapt to the hardware (autotuning)60Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewSupercomputing and Computational ScienceBy definition, a supercomputer is of a class of computer systems that are the most powerful computing platforms at that timeComputational science has always lived at the leading (and bleeding) edge of supercomputing technologyMost powerful depends on performance criteriaPerformance metrics related to computational algorithmsBenchmark real application codesWhere does the performance come from?More powerful processorsMore processors (cores)Better algorithms61Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewComputational ScienceTraditional scientific methodologyTheoretical scienceFormal systems and theoretical modelsInsight through abstraction, reasoning through proofsExperimental scienceReal system and empirical modelsInsight from observation, reasoning from experiment designComputational scienceEmerging as a principal means of scientific researchUse of computational methods to model scientific problemsNumerical analysis plus simulation methodsComputer science toolsStudy and application of these solution techniques62Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewComputational ChallengesComputational science thrives on computer powerFaster solutionsFiner resolutionBigger problemsImproved interactionBETTER SCIENCE!!!How to get more computer power?Scalable parallel computingComputational science also thrives better integrationCouple computational resourcesGrid computing63Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewScalable Parallel ComputingScalability in parallel architectureProcessor numbersMemory architectureInterconnection networkAvoid critical architecture bottlenecksScalability in computational problemProblem sizeComputational algorithmsComputation to memory access ratioComputation to communication rationParallel programming models and toolsPerformance scalability64Introduction to Parallel Computing, University of Oregon, IPCCLecture 1 OverviewNext LecturesParallel computer architecturesParallel performance models65CIS 410/510: Parallel Computing, University of Oregon, Spring 2014Lecture 1 Overview