Download - Dist Sys Funda

Distributed Computing FundamentalsAnd

Distributed Computing paradigms

Distributed SystemDistributed System

• Collection of independent computers, interconnected via a network, that are capable of collaborating on a task.

• Computers are considered independent if they do not share the memory or program execution space.

• Loosely coupled configuration

Distributed ComputingDistributed Computing• It is computing performed in distributed

system.• Web based applications are good examples of

distributed computing– Network Services.

• Email, ftp, telnet etc.

– Network applications• Enterprise application such as online shopping,

electronic auction etc.

• Historically started with the development of Internet

Different Forms of ComputingDifferent Forms of Computing• Monolithic Computing

– Performed on PCs known as single user monolithic computing

– Mainframes providing timesharing concept to support multiple users

• Distributed Computing– Performed among multiple network connected computers,

each of having its own processor and resources

• Parallel Computing– Another form of distributed computing which uses more

than one processor simultaneously to executes a single program

Different Forms of ComputingDifferent Forms of Computing• Cooperative Computing

– Applied to projects such as SETI (Search for Extraterrestrial Intelligence) and distributed.net

– Parcels out large scale computing to workstations on Internet making use of surplus CPU cycles

Strengths and WeaknessesStrengths and Weaknesses

• Strengths:– The affordability of computers and availability of

network access– Resource sharing– Scalability– Fault tolerance

• Weaknesses:– Multiple points of failure– Security concerns

PrerequisitePrerequisite• Operating System

– Process, Process states– Concurrent Programming

• Concurrent processes running on multiple computers– Programming support is needed for communication among processes

• Concurrent processes running on single computer– Supported by operating system

• Concurrent programming in a process– Process Vs. Threads

• Network – Architecture, addressing, protocols etc.

• Software Engineering– Procedural Vs. Object Oriented programming– Modeling Language like UML

PrerequisitePrerequisite

– Architecture of Distributed Applications• Three Layers

– Presentation

– Application Logic

– Service Layer

– Toolkits, Frameworks and Components• Toolkits or frameworks are collections of classes, tools

and programming samples. Examples, JDK, .NET

• Components : EJB, COM, DCOM– Supports evolution of application by assembly of reusable

components

Interprocess CommunicationInterprocess Communication

• Backbone of the Distributed Communication• Provides the ability for separate independent

processes to communicate among themselves to collaborate on task

• Two independent processes running on different machines exchanges the data

• Unicast : One sender, One receiver• Multicast : One sender, Multiple receiver• IPC in form of API (Application Program Interface)

provides abstraction of the details and intricacies of the system level facilities

IPC Program InterfaceIPC Program Interface

• Send : – issued by sending process for the purpose of

transmitting data. – Allows identification of receiving process and

specified the data

• Receive :– Issued by receiving process to receive the data. – Allows the identification of sending process and

specified the space to store the data


• Connect :– For connection oriented IPC, allows the logical

connection to be established between processes.– One process issues a request-to-connect and other

issues a accept-connection

• Disconnect:– Previously created connection is deallocated on

both the sides


• Issuance of operation causes the occurrence of an event– Send results into data transmission to the receiver– Receive results into the delivery of data

• Example: – Communication between

Web-server and Web-browser Accept connection Make connection

Receive (Request) Send (Request)Send (Response) Receive (Response)Disconnect Disconnect

Event SynchronizationEvent Synchronization• Major difficulty is that processes runs independently

• HTTP needs that two sides involved in communication must issue IPC operations in specific order

• Event synchronization is provided by blocking, suspending the process until operation issued by process has been completed

• Blocking operations are also called synchronous operations

• Alternatively, IPC operations may be asynchronous or non-blocking

• Event synchronization is fundamental to distributed computing

• Example : Receive issued by web-browser is blocking, while send issued by web-server is non-blocking

Event SynchronizationEvent Synchronization

• Four different modes– Synchronous send and synchronous receive

• Required when logic of both processes requires that data sent must be received before further processing can start

Blockingreceive starts

Blockingsend returns

Blockingreceive ends

Transparentacknowledgement provided by the IPC facility

Process 1 Process 2

Blockingsend starts


– Asynchronous send and synchronous receive• Appropriate when sender’s logic does not depend on the

receiving of the data at the other end

Blockingreceive starts

Blockingreceive ends

Process 1 Process 2

Non Blocking Send

Event SynchronizationEvent Synchronization– Synchronous send and asynchronous receive

• Causes different scenarios for a protocol session. Depends on IPC implementation. Three scenarios:

– Data already arrived at the time when receive operation is issued. Immediate delivery

– Data requested by receive operation has not arrived; no delivery to the process. Receive process can use polling (loop) until data is received

– Data requested by receive operation has not arrived; When data arrives IPC facility notifies the process2.


Blockingsendissued

Nonblockingreceiveissued

TransparentAcknowledgeProvided by

The IPC facility

Process 1 Process 2

Blockingsendissued

Nonblockingreceiveissued andreturnedimmediately

Process 1 Process 2

Blockingsendissued

NonblockingReceive issuedAnd returnedimmediately

TransparentAcknowledgeProvided by

The IPC facility

Process isNotifed of the arrival of data

Process 1 Process 2


– Asynchronous send and asynchronous receive• Data retained by IPC facility and receiving process is

notified or

• Receiving process may poll for the data

Nonblocking receive issued and returnd immediately

Process is notified of the arrival of the data and the data may now be delivered to it

Process 1 Process 2

NonBlocking Send issued

Timeouts and ThreadingTimeouts and Threading

• Blocking may cause indefinite suspension

• Two possible solutions– Timeout, specified in a program– Program may create a child process or thread to

issue a blocking operation. Mainline process continues with other operation

• Care must be taken to avoid the deadlocks, which causes the indefinite suspension

Data RepresentationData Representation

• Different languages have different size of data types and different representation

• When heterogeneous systems are involved, there are three possibilities– Sending process converts to representation of receiving

process before sending– Receiving process converts the senders representation upon

receiving– Exchange the data in External Representation

• Data marshaling is supported by IPC performing– Serializing the data structure– Converting data values to external representation

• JAVA supports object serialization

Data EncodingData Encoding• General purpose distributed application needs platform

independent scheme of encoding the data exchanged. Three network data encoding standard exists– External Data Representation (XDR) from sun

– ANS.1 (Abstract Syntax Notation Number 1) from OSI

• Supports data tagging

• Each data is encoded with type,length,value and optinal tag to specify the interpretation

– XML (Extensible Markup Language)

<message>

<to>[email protected]</to>

<from>[email protected]</from>

<subject> Interprocess communication</subject>

<text> IPC is backbone of distributed computing </text>

</message>

ProtocolsProtocols

• Text based protocols– Data exchanged as stream of characters– FTP, HTTP, SMTP are text based

• Request-Response protocols– One side issues the request and awaits for the

response from other side– Examples : FTP, HTTP, SMTP

• Event diagrams and sequence diagrams are used to document the details of communication

IPC ParadigmsIPC Paradigms

• Connection oriented Vs. Connectionless

• Abstractions:Level of IPC Paradigms Example

Abstraction

RPC/RMI

Unix Socket API Winsock

Serial/parallel

communication

Remote Procedure/Method

Socket API

Data Transmission

AbstractionAbstraction

• Idea of hiding the details or encapsulation• We often use abstraction when it is not necessary to

know the exact details of how something works or represented, because we can still make use of it in its simplified form. Getting involved with the detail often tends to obscure what we are trying to understand, rather than illuminate it. … Abstraction plays very important role in programming because we often want to model, in software, simplified version of things that exist in the real world… without having to built the real things.

Distributed Computing ParadigmsDistributed Computing Paradigms

• Paradigm is a pattern, example or modelLevel of

Abstraction

Object space, Collaborative applications

Network Services, object request broker, mobile agent

Remote Procedure Call / Remote Method Invocation

Client-Server, peer-to-peer

Message Passing

An Example SystemAn Example System

Online Auctioning System

System handles only one auctioned item per session. During each auctioning session, an item is open for bids placed by the auction participants. At the end the auctioneer announces the outcome


• Message Passing Paradigm– Basic approach to Interprocess communication– Sender and receiver communicates by exchanging

the messages– Operations needed are send and receive. For

connection oriented IPC, connect and disconnect are also needed

– Similar to File I/O– Socket API is an example. Sender writes message

to socket, while receiver extracts the message from socket


• Client-Server Paradigm– Well known approach for network applications– Two processes have asymmetric role. Server

process provides services, while client process accesses the services by making requests

– Operations for server are listen and accept the requests and client process to issue the requests and accept the response

– Examples : HTTP,FTP,DNS etc.– Connection oriented Socket API, RPC and RMI

provides notion of clients and servers


• Example system – For session control

• As a server, participant waits to hear an announcement from auctioneer (1) when session starts, (2) whenever there is an update on the current highest bid, and (3) when the session ends.

• As a client, the auctioneer sends a request that announces the three types of events outlined as above

– For accepting bids• As a client, a participant sends a new bid to a server

• As a server, an auctioneer accepts new bids and updates the current highest bid


• Peer-to-Peer Paradigm– Participating processes play equal role, with

equivalent capabilities and responsibilities– Each participant may issue request to another

participant and gets response.– Well-known example is peer-to-peer file transfer

service provided by Napster.com– Appropriate for instant messaging, peer-to-peer

file transfers, video conferencing and collaborative tasks

– Well known examples: JXTA project and Jabber an XML based protocol


• Example System– A participant contacts the auctioneer directly to

register for the auction– Auctioneer subsequently contacts each participants

to initiate the auction session, during which individual participants may obtain the status and submit the bid

– At the end, winning participant is notified by the auctioneer.

– Other may get the status by contacting the auctioneer


• Message System Paradigm– Message System or Message Oriented Middleware

(MOM) is an elaboration of basic message passing system

– Message system acts as intermediary among separate independent processes

– Message system acts as a switch for messages through which processes exchange message asynchronously, in a decoupled manner.

– Sender deposits a message to message system, which forwards it to message queue associated with each receiver.


• Message System Paradigm

Message queue

Receiver

Sender

Message System


• Message System Paradigm (Two models)– Point-to-Point Message Model

• Message system forwards message from sender to the receiver’s message queue

• System acts as depository and allows sender and receiver to be decoupled.

• Sender deposits the message in receiver’s queue and receiver extracts it from queue.

• Same as basic message passing system, but provides additional abstraction for asynchronous operations.

• Implementation of auction system is same as message passing system. Difference is that messages are passes through Message middleware.


– Publish/Subscribe Model• Each message is associated with a specific topic or

event• Application subscribes for the message for the event• When the event occurs, the process publishes the

message announcing the event or topic• The middleware message system distributes the

message to all subscribers• Offers powerful abstraction for multicast or group

communication.• The publish operation allows process to multicast, while

subscribe operation allows process to listen for multicast


• Example system with Publish/Subscribe Model– Each participant subscribes to a begin-auction event message– The auctioneer signifies the beginning of the auctioning

session by sending a begin-session event message– Upon receiving the begin-auction event, a participant

subscribes to an end-auction event message– The auctioneer subscribes to message for new-bid events– A participant wishing to place a new bid issues a new-bid

event message, which will be forwarded to auctioneer– The auctioneer issues an end-auction event message to inform

participants about outcome. If desired additional message events can be added to allow participants to monitor the status of the auction


• Remote Procedure Call Paradigm– Message passing is well for network protocols– More abstraction is required for network

programming so that programmer can built the distributed software similar to conventional application

– RPC provides such an abstraction– Message passing is data oriented, while RPC is

action oriented.


• Example system with RPC– The auctioning program provides a remote

procedure for each participant to register and another procedure for a participant to make a bid

– Each Participant program provides following procedures

• To allow auctioneer to call a participant to announce the onset of the session

• To allow auctioneer to inform a participant of a new highest bid

• To allow auctioneer to announce the end of session


• Distributed Object Paradigms– Provides the Object Oriented abstraction1. Remote Method Invocation (RMI)

• Object Oriented extension to the RPC• Auction system is same as RPC except that object

methods replaces the procedures

2. Object Request Broker• Process issues the request to ORB which directs the

request to the appropriate object • ORB acts as middleware• Allows interaction among objects written in different

languages


• Implementation of auction system is same as RMI except that each object must register with ORB

• Each participant issues requests to the auctioneer object to register for the session and to make bids

• Example : OMG’s CORBA• Through the ORB, the auctioneer invokes the methods

of each participant to announce the start of the session, to update the bidding status and to announce the end of the session

Object Request Broker

ObjectRequestor

Object


3. Component-based Technologies• Microsoft’s COM, DCOM

• JAVA beans, EJBs


• Object Space Paradigm– Assumes the existence of the logical entity called

object space– The participants of an application converge in a

common object space– Provider places object as entries into object space,

and requestors who subscribe to the space may access the entries

– Provides virtual space or meeting room among providers and requesters


– Mutual exclusion is inherent as only one participant can access at an object at a time from space

– JavaSpaces is an example toolkit

read

readwrite

Requestor

Requestor Provider


• Example system– All the participants as well as service provider subscribes

to a common object space– Each participant deposits an object into space to register

for the session and to be notified when the session starts.– At the onset of the session, auctioneer deposits an object

into object space containing item info. And bid history– A participant wishing to place a bid retrieves the object

from the space and if chooses, places the new bid in the object before returning object to the space

– At the end of session, auctioneer retrieves the object from the space and contacts the highest bidder.


• Mobile Agent Paradigm– Mobile agent is transportable program or object– Agent is launched by an originating host, travels

autonomously from host to host, access the necessary resources and performs the necessary tasks to complete the mission

Agent

Host 2

Host 3

Host1

Host4


• Example System– Each participant launches the mobile agent to the

auctioneer containing identity (network address)

– Once the session starts, auctioneer launches a mobile agent that carries the info. about participants and current highest bid

– The mobile agent circulates among the participants and auctioneer until session ends

– At the end auctioneer launches the agent to make one more round among the participants to announce the outcome

• Commercial Packages : Concordia, Aglet system


• Network Service Paradigm– Each service provider registers with the directory servers

on a network

– Process desires to get the service contacts the directory service to get the reference to the service

– Using reference, process accesses the service

– Extension to RMI except registration with global directory service, provides location transparency

Service Requestor

Directory Service

Service Object

Distributed Computing ParadigmsDistributed Computing Paradigms– Java’s Jini technology is based on this paradigm

• Example system– Auctioneer registers itself with directory service,

allowing the participants to locate it and once the session starts , to make the bid

– Participant provides call back methods to allow auctioneer to announce the start and end of the session and to update the status of the session


• Collaborative Application (GroupWare) Paradigm– Processes participates in collaborative session as a

group– Each process may contribute input to part or all of

the group– Either multicasting is used to send data to part or

group or they may use virtual sketchpads or white-boards which allows each participant to read or write data


– Message Based and White-board based groupware

– Java multicast API and Java Shared Data ToolKit (JSDT) are message-based

– SMART board and Microsoft’s Netmeeting are based on white-board meeting

messagemessage

message


• Example system (Message-based)– Auctioneer initiates group, joined by all participants

– Auctioneer announces the start of session by multicast

– During session, each bid is multicast to all participants

– Finally, auctioneer terminates session by multicasting a message announcing the outcome

• Example system (Whiteboard-based)– Auctioneer starts bidding by writing announcement to

whiteboard

– Participant may write their bids on board

– Auctioneer write the outcome on the board

Trade-offsTrade-offs

• Level of abstraction Vs. Overhead– Higher the abstraction, higher the overhead

• Scalability– Complexity increases as no. of participants increases

– High level paradigms, complexity is handled by system

• Cross Platform Support– Java RMI, JavaSpaces runs on only JAVA

– COM, DCOM is only for Microsoft

– CORBA is cross platform

ReferenceReference

Distributed Computing : Principals and Applications

- M.L. Liu

- Pearson Addison-Wesley