Post on 20-Dec-2015
Distributed ComputingDistributed Computing
Software based solutions to Software based solutions to Parallel ComputingParallel Computing
Distributed ComputingDistributed Computing
Cluster based computingCluster based computing Network of WorkstationsNetwork of Workstations
• Duality of standalone workstation Vs. Duality of standalone workstation Vs. parallel computing environment.parallel computing environment.
Distributed ApplicationsDistributed Applications• RC5 ClientsRC5 Clients• SETI ClientsSETI Clients
Issues of Distributed Issues of Distributed ComputingComputing
Homogeneous networksHomogeneous networks• Networks of similar machines. Requires only Networks of similar machines. Requires only
one version of executables. one version of executables. Heterogeneous networksHeterogeneous networks
• Different Architectures of machines. Different Architectures of machines. (Alpha,x86]. Each architecture requires its (Alpha,x86]. Each architecture requires its own version of executables.own version of executables.
Issues of data representation, system Issues of data representation, system level interface differences, and suitability.level interface differences, and suitability.
Support ModelSupport Model
Application support is critcal.Application support is critcal. Three basic modelsThree basic models
• Application is built with full awareness Application is built with full awareness and supportand support
• User land facilities are provided to User land facilities are provided to facilitate awareness and supportfacilitate awareness and support
• Kernel level supportKernel level support
Application SupportApplication Support
Application modeled as self-contained Application modeled as self-contained entity capable of client-server, peer-entity capable of client-server, peer-peer cooperation models.peer cooperation models.• RC5 clientRC5 client• SETI clientSETI client
Support can be streamlinedSupport can be streamlined Loss of transparency, load Loss of transparency, load
balancing,lower QoS.balancing,lower QoS.
User Level SupportUser Level Support
Provide basic parallel mechanismsProvide basic parallel mechanisms• Message/Data passingMessage/Data passing• SynchronizationSynchronization• Process HandlingProcess Handling
– Rough controlRough control
Provides greater transparencyProvides greater transparency• Less dependence on architecture, Less dependence on architecture,
transparency supports heterogeneous transparency supports heterogeneous solutionssolutions
Kernel SupportKernel Support
Provides system level supportProvides system level support• SchedulingScheduling
– Load BalancingLoad Balancing– Process MigrationProcess Migration
• Resource AllocationResource Allocation Doesn’t work well in heterogeneous Doesn’t work well in heterogeneous
configurations.configurations. Process Migration increases Process Migration increases
communication costscommunication costs
Three SolutionsThree Solutions
PVM - Parallel Virtual MachinePVM - Parallel Virtual Machine• User Land Parallel SolutionUser Land Parallel Solution• Message PassingMessage Passing
MPI - Message Passing InterfaceMPI - Message Passing Interface• StandardStandard• Several ImplementationsSeveral Implementations• User Land SupportUser Land Support
Three Solutions - Three Solutions - continuedcontinued
MOSIX - Linux Kernel ExtensionsMOSIX - Linux Kernel Extensions• Provide dynamic schedulingProvide dynamic scheduling• process migrationprocess migration• system level integrationsystem level integration
PVM - Parallel Virtual PVM - Parallel Virtual MachineMachine
Message Passing ParadigmMessage Passing Paradigm Heterogeneous ComputingHeterogeneous Computing
• Deals effectively with data Deals effectively with data representation issuesrepresentation issues
Supports variety of systems including Supports variety of systems including MPP’s, SMP’s, and vector machinesMPP’s, SMP’s, and vector machines
Uses a daemon to provide parallel Uses a daemon to provide parallel facilitiesfacilities
PVM - continuedPVM - continued
Tracks tasks using a unique Tracks tasks using a unique system assigned ID called a TIDsystem assigned ID called a TID
Supports grouping, and group level Supports grouping, and group level activities.activities.
MPI or LAMMPI or LAM
Message Passing InterfaceMessage Passing Interface MPI is the standard, several MPI is the standard, several
implementations exist. One is LAM implementations exist. One is LAM and is maintained by Notre Dame.and is maintained by Notre Dame.
Similar to PVMSimilar to PVM• HeterogeneousHeterogeneous• Uses a daemon, also includes a peer-Uses a daemon, also includes a peer-
peer mode.peer mode.
MPI - LAM - continuedMPI - LAM - continued
LAM environment must be started LAM environment must be started explicitlyexplicitly
Provides compiler shells to handle Provides compiler shells to handle program compilationprogram compilation
Nodes are dynamicNodes are dynamic Suite of utilities to maintain the Suite of utilities to maintain the
message passing virtual machine.message passing virtual machine. The parallelism is explicitly programmedThe parallelism is explicitly programmed
MPI - LAM - continuedMPI - LAM - continued
MPI addresses resource limitations. MPI addresses resource limitations. Uses a property called Guaranteed Uses a property called Guaranteed Envelope Resources to maintain Envelope Resources to maintain integrity of processes.integrity of processes.
MOSIX and Linux 2.2.7MOSIX and Linux 2.2.7
Linux, not much more to say.Linux, not much more to say. Mosix is developed to extend Mosix is developed to extend
several unix operating systemsseveral unix operating systems HomogeneousHomogeneous Transparent and Preemptive Transparent and Preemptive
process migrationprocess migration Dynamic process reassignmentDynamic process reassignment
Mosix - continuedMosix - continued
Global resource assignmentGlobal resource assignment Ideal for cluster based computingIdeal for cluster based computing Offers memory usheringOffers memory ushering
• prevents thrashingprevents thrashing Maintains home systemMaintains home system Requires that the nodes of the Requires that the nodes of the
cluster be networked wellcluster be networked well
Mosix - ContinuedMosix - Continued
Mosix maintains a given process’ Mosix maintains a given process’ association with a home node.association with a home node.• Unique home nodeUnique home node
Implements a bi-level approach to Implements a bi-level approach to maintain this association through maintain this association through migrationmigration• User Context (remote) - system User Context (remote) - system
independent contextsindependent contexts
Mosix - continuedMosix - continued
• Deputy - node dependent interface. Remains Deputy - node dependent interface. Remains when process migrates from home node.when process migrates from home node.
All system dependent calls are routed All system dependent calls are routed through remote to deputy at home through remote to deputy at home systemsystem• gettimeofdaygettimeofday
Initial assignment needs to be done by Initial assignment needs to be done by PVM or MPI. Mosix doesn’t handle this PVM or MPI. Mosix doesn’t handle this aspect.aspect.
Bad MosixBad Mosix
Overhead is increasedOverhead is increased• Delayed system callsDelayed system calls• I/O sockets (file, network) don’t route I/O sockets (file, network) don’t route
wellwell• Migration time adds overheadMigration time adds overhead
Needs further work in migratable Needs further work in migratable sockets and filessockets and files