PresentedBy

Kelly Technologies

www.kellytechno.com

AgendaWhat is distributed computingWhy distributed computingCommon ArchitectureBest PracticeCase study

CondorHadoop – HDFS and map reduce

www.kellytechno.com

What is Distributed Computing/System?Distributed computing

A field of computing science that studies distributed system.

The use of distributed systems to solve computational problems.

Distributed systemWikipedia

There are several autonomous computational entities, each of which has its own local memory.

The entities communicate with each other by message passing.

Operating System Concept The processors communicate with one

another through various communication lines, such as high-speed buses or telephone lines.

Each processor has its own local memory. www.kellytechno.com

What is Distributed Computing/System?Distributed program

A computing program that runs in a distributed system

Distributed programmingThe process of writing distributed program

www.kellytechno.com

What is Distributed Computing/System?Common properties

Fault tolerance When one or some nodes fails, the whole system

can still work fine except performance. Need to check the status of each node

Each node play partial role Each computer has only a limited, incomplete view

of the system. Each computer may know only one part of the input.

Resource sharing Each user can share the computing power and

storage resource in the system with other usersLoad Sharing

Dispatching several tasks to each nodes can help share loading to the whole system.

Easy to expand We expect to use few time when adding nodes.

Hope to spend no time if possible.

www.kellytechno.com

Why Distributed Computing?The nature of applicationPerformance

Computing intensive The task could consume a lot of time on computing. For

example, π Data intensive

The task that deals with a lot mount or large size of files. For example, Facebook, LHC(Large Hadron Collider).

RobustnessNo SPOF (Single Point Of Failure)Other nodes can execute the same task executed on

failed node.www.kellytechno.com

Common ArchitecturesCommunicate and coordinate works among

concurrent processesProcesses communicate by sending/receiving

messagesSynchronous/Asynchronous

www.kellytechno.com

Common ArchitecturesMaster/Slave architecture

Master/slave is a model of communication where one device or process has unidirectional control over one or more other devices Database replication

Source database can be treated as a master and the destination database can treated as a slave.

Client-server web browsers and web

serverswww.kellytechno.com

Common ArchitecturesData-centric architecture

Using a standard, general-purpose relational database management system customized in-memory or file-based data structures and access method

Using dynamic, table-driven logic in logic embodied in previously compiled programs

Stored procedures logic running in middle-tier application servers

Shared databases as the basis for communicating between parallel processes direct inter-process communication via message passing function

www.kellytechno.com

Best PracticeData Intensive or Computing Intensive

Data size and the amount of data The attribute of data you consume Computing intensive

We can move data to the nodes where we can execute jobs

Data Intensive We can separate/replicate data to difference nodes,

then we can execute our tasks on these nodes Reduce data replication when executing tasks

Master nodes need to know data location No data loss when incidents happen

SAN (Storage Area Network)Data replication on different nodes

SynchronizationWhen splitting tasks to different nodes, how can

we make sure these tasks are synchronized?

www.kellytechno.com

Best PracticeRobustness

Still safe when one or partial nodes failNeed to recover when failed nodes are online.

No further or few action is needed Condor – restart daemon

Failure detection When any nodes fails, master nodes can detect this

situation. Eg: Heartbeat detection

App/Users don’t need to know if any partial failure happens. Restart tasks on other nodes for users

www.kellytechno.com

Best PracticeNetwork issue

Bandwidth Need to think of bandwidth when copying files from one node

to other nodes if we would like to execute the task on the nodes if no data in these nodes.

ScalabilityEasy to expand

Hadoop – configuration modification and start daemon

OptimizationWhat can we do if the performance of some nodes is

not good? Monitoring the performance of each node

According to any information exchange like heartbeat or log Resume the same task on another nodes

www.kellytechno.com

Best PracticeApp/User

shouldn’t know how to communicate between nodes

User mobility – user can access the system from some point or anywhere Grid – UI (User interface) Condor – submit machine

www.kellytechno.com

Case study - CondorCondor

Computing intensive jobsQueuing policy

Match task and computing nodesResource Classification

Each resource can advertise its attributes and master can classify according to this

www.kellytechno.com

Case study - Condor

www.kellytechno.com

Case study - CondorRole

Central Manger The collector of information, and the negotiator

between resources and resource requestsExecution machine

Responsible for executing condor tasksSubmit machine

Responsible for submitting condor tasksCheckpoint servers

Responsible for storing all checkpoint files for the tasks

www.kellytechno.com

Case study - CondorRobustness

One execution machine fails We can execute the same task on other nodes.

Recovery Only need to restart the daemon when the failed

nodes are online

www.kellytechno.com

Case study - CondorResource sharing

Each condor user can share computing power with other condor users.

SynchronizationUsers need to take care by themselves

Users can execute MPI job in a condor pool but need to think of the issues of synchronization and Deadlock.

Failure detectionCentral manager can know when nodes fails

Based on update notification sent by nodesScalability

Only execute few commands when new nodes are online.

www.kellytechno.com

Case study - HadoopHDFS

Namenode: manages the file system namespace and regulates access

to files by clients. determines the mapping of blocks to DataNodes.

Data Node : manage storage attached to the nodes that they run on save CRC codes send heartbeat to namenode. Each data is split as a chunk and each chuck is stored on

some data nodes.Secondary Namenode

responsible for merging fsImage and EditLog

www.kellytechno.com

Case study - Hadoop

www.kellytechno.com

Case study - HadoopMap-reduce Framework

JobTracker Responsible for dispatch job to each tasktracker Job management like removing and scheduling.

TaskTracker Responsible for executing job. Usually tasktracker

launch another JVM to execute the job.

www.kellytechno.com

Case study - Hadoop

From Hadoop - The Definitive Guide From Hadoop - The Definitive Guide www.kellytechno.com

Case study - HadoopData replication

Data are replicated to different nodes Reduce the possibility of data loss Data locality. Job will be sent to the node where data are.

RobustnessOne datanode fails

We can get data from other nodes.One tasktracker failed

We can start the same task on different nodeRecovery

Only need to restart the daemon when the failed nodes are online

www.kellytechno.com

Case study - HadoopResource sharing

Each hadoop user can share computing power and storage space with other hadoop users.

SynchronizationNo synchronization

Failure detectionNamenode/Jobtracker can know when

datanode/tasktracker fails Based on heartbeat

www.kellytechno.com

Case study - HadoopScalability

Only execute few commands when new nodes are online.

OptimizationA speculative task is launched only when a task

takes too much time on one node. The slower task will be killed when the other one

has been finished

www.kellytechno.com

www.kellytechno.com