95-702 Distributed Systems Coulouris 5’th Ed. 95-702 Distributed Systems Distributed File Systems…
ICS362 Distributed Systems
description
Transcript of ICS362 Distributed Systems
![Page 1: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/1.jpg)
ICS362 Distributed Systems
Dr. Ken Cosh
Week 3
![Page 2: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/2.jpg)
Review
Architectural Styles– Layered Architectures– Object-Based– Data-Centred– Event-Based
System Architectures– Centralised / Decentralised / Hybrid
![Page 3: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/3.jpg)
This Week
Processes– Threads– Virtualisation– Clients– Servers– Code Migration
![Page 4: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/4.jpg)
What is a Process?
OS– A program in execution
Key Issue– Managing and Scheduling Processes– Both in terms of OS and in terms of Distributed
Systems
![Page 5: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/5.jpg)
Processes
The OS creates virtual processors– Each for running different programs
OS has a process table– To keep track of the virtual processors– CPU register values, memory maps, open files, privileges,
etc.
OS manage these independent processes– One process can not affect another– i.e. concurrency transparency
![Page 6: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/6.jpg)
Achieving Concurrency Transparency
Computationally Expensive– Allocation
Create complete independent address space Initialising memory segments
– Potentially moving existing data
– Switching Saving CPU context Modify Memory Management Unit (MMU) Potentially swapping processes between main memory and
hard disk– If more processes than main memory
![Page 7: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/7.jpg)
Threads
Threads are like processes– Executing code independently from other threads– But with no attempt at concurrency transparency
Threads can share the same address space
– Computationally less expensive– But programming requires more intellectual effort
![Page 8: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/8.jpg)
Multithreading
Consider a spreadsheet program– Editing one cell can cause updates to formulae
throughout multiple sheets.– How do we allow user input concurrently with
calculations? And perhaps autosaving
![Page 9: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/9.jpg)
Thread Implementation
Normally in the form of a thread package– Containing operations to create and destroy
threads & synchronisation variables such as mutexes.
Implementation can be done in different ways– Executed entirely in user mode– Kernel responsible for scheduling– Hybrid
![Page 10: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/10.jpg)
User Level thread library
All thread administration is kept in user’s address space;
– Cheap to create & destroy threads
Switching between threads is also cheap
However, a blocking system call will block the process – and hence any other threads in the process.
– E.g. when waiting for input
![Page 11: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/11.jpg)
Kernel level threads
That problem is avoided by implementing threads in the OS kernel.
Then, every thread operation is carried out by the kernel, through system calls– Creation, deletion, synchronisation etc.
Switching threads becomes as expensive as switching processes
![Page 12: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/12.jpg)
Hybrid Implementation
Light Weight Processes (LWP)– LWPs run in user space in the context of a single
process, but several LWPs can run per process with a shared address space.
– Creation, deletion operations etc. run in user space without intervention by the kernel.
![Page 13: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/13.jpg)
Threads in DS
A convenient way of allowing blocking system calls without blocking an entire process.
In DS, threads can be separated to different processors (clients or servers)
![Page 14: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/14.jpg)
Multithreaded Clients
Example: A Web Browser– Setting up connection– Read incoming data– Pass to display component
– Potentially connecting to multiple load balanced replicated servers to transfer data in parallel streams
![Page 15: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/15.jpg)
Multithreaded Servers
Suppose a request comes into a single threaded server.
– The request needs to be completed before any more requests can come
With multithreads, a dispatcher thread can assign a worker thread to read data from a file system
– Normally a blocking operation
Meanwhile the dispatcher thread can deal with more reads, or responses from other worker threads.
![Page 16: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/16.jpg)
Multithreaded Servers
Operating System
Request from network
DispatcherThread
WorkerThread
![Page 17: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/17.jpg)
Virtualisation
Of course, we are still referring to a single processor, but giving the illusion of simultaneous execution.
– ‘Resource Virtualisation’
Renewed interest in virtualisation as distributed systems become popular
– Extending (or replacing) an existing interface to mimic the behaviour of another system
– E.g. to allow software to run on a different platform.
![Page 18: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/18.jpg)
Applications of virtualisation
Hardware & Low level systems software evolves quicker than application software.– Virtualisation allows older, more stable, software
to run on newer platforms
Networking is pervasive across heterogeneous computers– Virtualisation helps by letting each application run
on its own virtual machine.
![Page 19: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/19.jpg)
Virtual Machine Implementation
Virtualisation mimics the behaviour of various interfaces:
– Interface between Hardware and Software, which can be invoked by any program.
– Interface between Hardware and Software, which can be invoked by privileged programs such as the OS.
– Interface of system calls offered by an OS– Interface of library calls offered by an API (essentially hiding
the OS system calls).
![Page 20: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/20.jpg)
Computer Systems Interfaces
Hardware
Operating System
Library
Application
PrivilegedInstructions
System Calls
Library Functions
GeneralInstructions
![Page 21: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/21.jpg)
Process Virtual Machine
Hardware
Operating System
Runtime System
Application
Virtualisation is for single processes
![Page 22: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/22.jpg)
Virtual Machine Monitor
Hardware
Operating System
Virtual Machine Monitor
Application
Multiple different Operating Systems running concurrently on the same platform
![Page 23: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/23.jpg)
Clients
Provide means for users to interact with remote servers– 1) Fat Client
Application specific protocol E.g. Agenda application running on a PDA,
synchronising with a remote (possibly shared) agenda.
– 2) Thin Client Application neutral protocol Client is a terminal with no need for local storage Example – X Window
![Page 24: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/24.jpg)
X Window
Oldest, still widely used networked user interface.
The heart is the ‘X-kernel’ OS running on the client– Containing terminal specific device drivers for
monitor, keyboard & mouse.– Captures keyboard / mouse events and made
available through Xlib library, running elsewhere and communicating through X protocol.
![Page 25: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/25.jpg)
X Window System
X Kernel
Device Drivers
Local OSLocal OS
Window Manager
Application
Xlib Xlib
![Page 26: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/26.jpg)
X Window Manager
Application which dictates the “look and feel” of the display– How to display windows, what buttons etc.– Other applications adhere to these rules.
So.. Confusingly the X Kernel acts as a server where applications play the role of clients!
![Page 27: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/27.jpg)
Communication Efficiency
This takes a lot of data communication!– Video stream 30 frames per second– 320x240 screen (PDA)– 24bits per pixel– = 53 Mbps!
Compression techniques are necessary
![Page 28: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/28.jpg)
Increasing UI Complexity
Drag & Drop Functionality Consider a document which contains text &
images– Document contains multiple applications– Suppose we want to rotate an image
It needn’t reside on the client, but the client may require more processing power.– Or the Client needs to be fatter
![Page 29: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/29.jpg)
Servers
A process implementing a specific service on behalf of a collection of clients– It waits for an incoming request– Then ensures the request is taken care of– Returns to wait for the next request
![Page 30: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/30.jpg)
Iterative vs Concurrent
Iterative Server – Handles the request itself and returns response if
necessary to the user.
Concurrent Server– Passes the request to a separate thread or
another process.– The other process is responsible for responding
to the requesting client.
![Page 31: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/31.jpg)
Ports
Clients send requests to an endpoint (port) Servers listen to a particular port
– E.g. FTP on TCP port 21, HTTP on TCP port 80
Or a Daemon can listen to a port and contact the specific server when requests arrive.
![Page 32: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/32.jpg)
Stateless Servers
Does not keep information on the state of its clients. Can change its state without informing clients
Once an HTTP server delivers the files it forgets about the client.
– Technically the web server might log all client requests as useful information, but if the information is lost there is no damage to the service.
![Page 33: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/33.jpg)
Soft State Servers
Server which promises to maintain state on behalf of the client – for a limited time– E.g. server promises to inform client about
updates, but only for a limited time.
![Page 34: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/34.jpg)
Stateful Server
Server maintains persistent information on its clients.– E.g. server allows a client to keep a local copy of
a file (possibly for updating). Server then maintains a list of client/file entries.
This can improve performance, however if the server crashes it has to restore the list.
![Page 35: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/35.jpg)
Session State vs Permanent State
Bear in mind that Permanent State is generally stored in a database, such as customer information, purchases etc.
Maintaining Session State is part of Stateful Server design.
![Page 36: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/36.jpg)
Stateful vs Stateless
Should not affect services provided by the server.– If files have to be opened before they can be read
from or written to, then either design should mimic this behaviour.
![Page 37: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/37.jpg)
Server Clusters
A collection of machines connected through a network, where each machine runs one or more servers.
Logically organised into three tiers (layers)– Switch – through which requests are routed– Application – servers where programs run– File/Database – where the data is stored
![Page 38: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/38.jpg)
Server Clusters
Client Requests
LogicalSwitch
Application ServersDatabase / File Servers
First Tier Third TierSecond Tier
![Page 39: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/39.jpg)
Server Clusters
Some systems combine the data and application into a single server resulting in a two tiered system architecture– E.g. Streaming Media
![Page 40: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/40.jpg)
Server Clusters
Often Offer Multiple Services, with different machines running different services
– Switch needs to be able to distinguish requests and services.
– Often caused by limitations on available software & hardware, and different administrators not wanting to interfere with each other.
Leads to some overloaded servers, while others are idle.
– Possibility for code migration?
![Page 41: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/41.jpg)
Code Migration
So far we have been concerned with passing data. Sometimes it is useful to pass programs
– Even while they are running
Sometimes overall system performance can be improved by moving processes from overloaded machines to lightly loaded machines.
– Load in terms of CPU queue length or CPU utilisation
![Page 42: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/42.jpg)
Examples
1) A client application needs to perform many database operations– Why not move the operations closer to the data
and then only return the results?
2) A form needs to be filled in, processed and then translated into a series of database operations– Why not do the form processing on the client?
![Page 43: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/43.jpg)
Code Migration
Benefits– Flexibility– Dynamically configured distributed systems– Software need not be pre-installed
Concerns– Security
![Page 44: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/44.jpg)
Strong vs Weak Mobility
A process consists of 3 segments– Code Segment – set of instructions– Resource Segment – references to necessary
external resources (files, devices, printers etc.)– Execution Segment – current execution state of
the process
![Page 45: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/45.jpg)
Weak Mobility
The code segment is sent– In this case the code must always begin from a
predefined point (perhaps the start)– E.g. Java Applets– Only requires that the target machine is capable
of running the code
![Page 46: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/46.jpg)
Strong Mobility
The execution segment is transferred as well as the code segment.– Code can be stopped, moved to a new machine,
and restarted from where it was.– More general than weak mobility, but harder to
implement.
![Page 47: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/47.jpg)
The Resource Segment
Consider; you’ve designed your website on localhost, and then upload it to a webserver– Absolute vs Relative URLs?– Changing IP addresses
![Page 48: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/48.jpg)
The Resource Segment
Binding by Identifier– A precisely referenced resource, and nothing else!
Binding by Value– It doesn’t matter which resource is used, so long as the
correct value is returned.– Does it matter if I use my C++ libraries or yours?
Binding by Type– What type of device are we referring to – a printer? A
monitor?
![Page 49: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/49.jpg)
Kinds of Resource
Unattached Resources– Easily moved – e.g. data files associated with the program
Fastened Resources– Possible, but relatively high cost – e.g. moving a database –
not dependent on current machine, but infeasible to move to a new environment
Fixed Resources– Cannot be moved from environment or machine – e.g. local
resources, or a local comunication end point.
![Page 50: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/50.jpg)
Solutions?
With 3 kinds of process-to-resource bindings and 3 kinds of resource-to-machine bindings, there are 9 combinations to consider.
And 4 common solutions– Establish a Global System wide reference (GR)– Move the Resource (MV)– Copy the Value of the Resource (CP)– Rebind process to a locally available resource
(RB)
![Page 51: ICS362 Distributed Systems](https://reader036.fdocuments.in/reader036/viewer/2022062315/56814bce550346895db8a684/html5/thumbnails/51.jpg)
Solutions
Unattached Fastened Fixed
By Identifier MV (or GR) GR (or MV) GR
By Value CP (or MV,GR) GR (or CP) GR
By Type RB (or MV, CP) RB (or GR, CP) RB (or GR)